Cut XML by XQuery Markup

SoapUI from SmartBear Software is traditionally used to work with web services. Great tool and also free. But ... this is a tool of a developer, tester, architect, but in no way focused on the work of the end user.

As I already wrote, I am not a developer, and sometimes I need to get data from internal and external sources, preferably without involving "heavy artillery", and so that the result can be shown to another non-developer. Therefore, it is time to add a new module to your tools, in which we will access the web services, parse the received data and display it in a digestible form.


')
There are a lot of ways to access the web service. In Python, there are requests (articles on Habré 1 , 2 ), but I will use Qt tools, partly out of habit, partly to reduce dependencies, since PyQt5 is already connected, partly to reduce intermediate data transformations. Accordingly, to convert the received xml-response I use XPath and XQuery, also embedded in Qt.

We will connect to the Bank of Russia exchange rate service . The service by reference responds with the following XML:

<?xml version="1.0" encoding="windows-1251" ?> <ValCurs Date="16.09.2017" name="Foreign Currency Market"> <Valute ID="R01010"> <NumCode>036</NumCode> <CharCode>AUD</CharCode> <Nominal>1</Nominal> <Name> </Name> <Value>46,0614</Value> </Valute> ... </ValCurs> 

In the parameters, the date of the format DD / MM / YYYY is transmitted, it can be omitted, then the courses will be for the current day.

But first, let's define what we want to receive. It is not interesting to make a module only for the exchange rate, otherwise it will turn out that for each new service a new module will be needed. Therefore, we will fill the module with universality so that it can work with (almost) any service, and all differences should be determined by the configuration file. And for simple services there should be a simple file so that the creation of a new file would not be like the development of a new module.

First of all, we will write such a configuration file that defines the service, its parameters and the display method:

 [Input] ;       Date=,02/03/2017 [WebPage] ;       Url="http://www.cbr.ru/scripts/XML_daily.asp?date_req={Date}" ; ,     Transform=valcurs.xq 

It was possible to substitute the parameters in the url in different ways. The traditional way in Qt is QString :: arg () in Qt, but it is not available in PyQt5 due to the lack of a QString . In Python, there is a% operator and a str.format () method ( there is also an article about it in Habré). Since we need the parameters named, the substitution must also be named - this method is provided by str.format (). Hence the Date parameter in curly braces.

The resulting answer can also be displayed in a million different ways. You can parse XML into a table (and currency rates are essentially a table) and show it in QTableWidget. But then you have to describe the rules for matching XML tags and table columns - you can, probably, but this description seems rather complicated to me.

You can leave the tag hierarchy and display it in a QTreeWidget. Yeah, but only then will we get a “clear view” type like in the Win event viewer.



Exceptionally clear.

So I decided to kill all the hares in one fell swoop and display the result in the form of a web page, having previously run the answer through some template engine, and also bundle the template into the configuration file.

To display web pages, Qt has a QWebEngineView module - in fact, the built-in Chromium. It used to be WebKit, simpler and much more integrated into Qt, but under the influence of the fashion on HTML5, developers refused to support it in the framework. So we live with what we have.

We will assemble a working window by analogy with the SQL window described in the first part , only instead of QTableView we will add QWebEngineView. Get this window




While it does not know how to do, you need to learn. In general, creating and executing a query is done in PyQt quite simply.

  #     self.man = QNetworkAccessManager(self) #  Run def run(self): try: #    values = { kp[0]:self.inputs[kp[0]].text() for kp in self.params} url = self.url.format(**values) req = QNetworkRequest(QUrl(url)) #  GET reply = self.man.get(req) reply.finished.connect(self.replyFinished) 

In the last line, the response handler function is connected to the finished signal, in terms of Qt a “slot” in which the original QNetworkReply object is obtained from QObject.sender ().

  def replyFinished(self): reply = self.sender() 

By the time this function is called from reply, you can read the data as from a file.

I didn’t have to think too long over the choice of a template engine - although there are many of them ( wiki , review ), both in JS and Python, it is natural to use one already installed - Qt has QXmlQuery that supports XQuery and XSLT transformation tables, but with restrictions .

Write a template on XQuery is quite simple. Here’s what a currency exchange pattern might look like that converts XML into a table:

 xquery version "1.0" encoding "utf-8"; <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/> </head> <body> <div>   {string(/ValCurs/@Date)}</div> <table><tbody> <tr> <th> </th> <th> </th> <th></th> <th></th> <th></th> </tr> {for $i in /ValCurs/Valute return ( <tr> <td>{string($i/NumCode)}</td> <td>{string($i/CharCode)}</td> <td>{string($i/Nominal)}</td> <td>{string($i/Name)}</td> <td>{string($i/Value)}</td> </tr> ) } </tbody></table> </body> </html> 

It now remains to run the answer through the template and insert the result into the web page. Conveniently, you can pass the main document to QXmlQuery not with a buffer, but with a source based on a QIODevice . Our answer is quite suitable for this.

  #   lang = QXmlQuery.XQuery10 query = QXmlQuery(lang) query.setMessageHandler(XmlQueryMessageHandler()) query.setFocus(reply) #     templ = QFile(self.transformTemplate) templ.open(QIODevice.ReadOnly) query.setQuery(templ) #        - out = query.evaluateToString() self.page.setHtml(out) 

Let's put it all together and run it. Does it work? Alas, the miracle did not happen again, it works, of course.



You can use XSL tables instead of XQuery. Just need to change one line

  lang = QXmlQuery.XSLT20 

It turns out



And what if to give the resulting XML without a template directly to the page?

  b = reply.readAll() self.page.setContent(b, reply.header(QNetworkRequest.ContentTypeHeader), reply.url()) 

Chromium will be able to display this XML.



The coding was violated, because the Central Bank service did not register the code page in the HTTP response header, and Chromium did not begin to parse the XML header. Let's add the forced addition of the default code page.

  b = reply.readAll() cth = reply.header(QNetworkRequest.ContentTypeHeader) if len(cth.split(';')) == 1: cth = cth + ";charset=windows-1251" self.page.setContent(b,cth,reply.url()) 

Now everything is fine.

It remains only to add a new module to the common set of tools. The source code is on github .