"Entertaining XenAPI", or "New Adventures of Citrix XenServer"

Hello colleagues!
Today I would like to continue my narration about Citrix XenServer 5.6 and about various aspects of working with it. This time I had to solve a rather simple (seemingly!) Problem: executing commands in dom0 without using SSH . Exploring the possibilities for implementation led to the discovery of some of the fun nuances of the HTTP API of this OS: ways to get / etc / passwd , remotely run rsync, and sketches XenSource thin CLI protocol . Now I will tell you what is called the story of a single recourse ...



First of all, I would like to explain how this problem arose. In the previous series, I showed the public beta of the security guard for XenServer , which I “saw” in the hope of writing a coherent guide. One of the recommendations (similar to the security hardening guide for VMWare ESXi ) is disabling the SSH daemon. The motivation is that in the corporate version of Xen it is possible to use the RBAC system with authentication through Active Directory. According to the recommendations of the vendor, this way is preferable from the point of view of security. After some modification of the console startup scripts on dom0 , described in my manual , getting through it to the system without a password is excluded. Accordingly, to get directly to the dom0 console requires knowledge of not only the password of the user with pool-admin rights, but also the root account information.
')
OK. Now we are faced with the task of remotely auditing an operating system using automated tools. All that we have at our disposal is XML-RPC leading to XenAPI , its documentation and Xen-org sources in the beautiful OCaml language. We would like to execute commands in bash and receive their “exhaust” for further processing. How to do it?

First you need to understand why the normal way (through the console provided by the API) will not do this. Let us recall the client-side console call technology: do you connect to the console ( https://<xen_host>/console?ref=OpaqueRef:console_id ), having a valid session_id, and get to the vncterm RFB terminal . It is clear that this protocol allows you to send mouse actions and button presses to a remote server, and in response to receive raster screen images. Further is obvious: modern versions of the RFB protocol allow, among other things, to transfer files. It is enough to master the execution of commands as well - and there are no problems. However, that would be too easy. Citrix uses the RFB protocol version 003.003 in its vncterm terminals : file transfer is not yet implemented.

Given this sad news, our development team began to analyze possible methods for implementing vehicles through the 1998 RFB. There are two ideas. The first is integration with ABBYY FineReader (with text recognition on raster images obtained from dom0 ). The second is the emulation of mouse movements, which allow you to select text on the screen and send it to the clipboard that is available within the protocol. Both options on closer inspection resemble the ravings of a madman :)

The sad prospects made me go back to reading the documentation for XenAPI . And I drew attention to what I had not encountered before, the plug-in architecture, namely the ability to access my own executable files via the RPC call call_plugin . The modules are located in the /etc/xapi.d/plugins/ directory. Then everything is simple: the plugin created by us is invoked via XML-RPC and runs the corresponding Python script that implements the execution of the necessary commands by subprocess . Fine! The methodology for executing commands on dom0 and getting an answer from them has become clear.

The next problem appeared by itself: how should our plugin get to the server? When solving this problem, in fact, they discovered some XenAPI “underwater rakes” .

Of course, I was interested in the function, access to which can be obtained using the standard xe.exe utility, - patch-upload . It allows you to remotely upload files to XenServer and install them on the entire pool of servers. The format of the patch is quite simple: it is shar , packed in zip and signed (!) By Citrix. You can read how to look into the patch and install it manually in a Selectel article. When a patch is loaded, its signature is checked against the corresponding set of public keys in the gpg keyring . Thus, it is enough to add your own signature to the common bundle - and the problem of pouring the plug-in disappears. Assembling a similar design is not difficult, but to pour your key in the bundle you must have access to the console. It turned out a vicious circle. Therefore, I began to look for a way to flood my plugin bypassing standard methods.

Using this call, I noted that the https://<xen_host>/pool_patch_upload is not in the official API description. There is a logical explanation for this: it really is not part of the API. Natural curiosity prompts the question: “ What is it then? ". The way to find the answer is very simple: Wireshark . Perhaps you condemn me for such a straightforward approach, but the HTTP interface of the XenServer operating system API is described, unfortunately, a little less than nothing. And by the time of work on this problem, I still did not understand OCaml at such a level as to analyze the source code quite effectively.

Having used Wireshark's great opportunity to decrypt TLS and carefully left a certificate in / etc / xensource / , I received a dump of the xe.exe utility (from XenCenter) communicating with the server. I expected to see the XML-RPC communication described in the official documentation. But it was not there! Instead, “POST / cli HTTP / 1.0” was lit up in the log. The utility took the command, its attributes - and sent them to https://<xen_host>/cli . " It seems that something is missing in the soup ." From the decryption of the protocol it followed that there is a certain XenSource thin CLI protocol that the utility uses. All paths led on Github to the XenAPI source .

After spending some time reading the source codes of this excellent component of the hypervisor, I found out that this “ XenSource thin CLI protocol ” API version 0.2 exists, which implements the execution of xe.exe helper commands on a remote host. It is described in the file xapi / cli_protocol.ml . It is noteworthy that this is the “API of the future”, designed to make the xe.exe utility just a means for sending commands, and the handler to be embedded in XenAPI.

In general, the discovery of this CLI API was fateful: it showed that at port 80 \ 443 there is not only an XML-RPC receiver and a switch / console . What other modules are available through a similar call - you accidentally found it in one of the source code files ( xen-api / ocaml / idl / constants.ml ). As you can easily guess, there were a lot of calls that gave out very interesting information. I was challenged by the https://<xen-host>/syns_config_files : with sufficient rights ( pool-admin ) you get / etc / passwd (as I mentioned in your past articles, XenServer stores the password hash there.)

Another interesting challenge is implemented via “ CONNECT /remotecmd?cmd=rsync&arg=some_nice_arg &pool_secret=your_pool_secret ”. It allows, knowing the value of / etc / xensource / ptoken , to remotely execute the rsync command on the server with root privileges . This gives, in essence, full access to the file system. But you probably ask: "But how to get ptoken ?".

Here is still more trivial. Xensource developers have created the ability to remotely retrieve the contents of the pool as an XML file. If you make a request on the server like " GET /pool/xmldbdump?session_id= " then you will get a complete set of key-value pairs, among which you can easily find the necessary pool_token .

Well and, actually, the remote download of patches is carried out through the call " PUT /pool_patch_upload?session_id= ". In response, the server will write: "200, OK." And it will wait when you start to fill in the socket information. As soon as you upload the file, the validation of the patch will begin. But there is one feature: as long as you keep the connection, the API considers that you are still uploading the file and does not touch it (although the file has already been created in / var / patch ). I did not find checks on file length. Since / var / patch is in the root of the server, DoS is inevitable if / dev / urandom is sent there.

Of course, this is not all. Look at the challenges and the necessary rights here .
All code is perfectly documented, and it seems to me that if there is an accurately formulated question, it will not be difficult to find the answer in it.

In general, the composition of the methods I described was enough to successfully upload the patch to the system without verifying the signature. I will not make a detailed description of the methodology, because it borders on the concept of “ exploitation of vulnerability ”, but I think that you yourself understood everything :)
Thanks for attention!