Testing client-server API security

2 years ago I spoke at the CodeFest conference with the topic “Pentest on steroids. We automate the process ”and made a retelling of the performance as an article . This year, it was with great pleasure that I took part in the conference again and spoke on the subject “BlackBox Security Testing of Client-Server API” and, apparently already as a tradition, I also retell the presentation.

API vulnerabilities are found. True.

What are you talking about?

Talk about the API, but the API is different - the API of the operating system, site, desktop program, but at least something. In short, an API is an appeal to methods of something (for example, in the case of an OS, writing to a file) through a specific method. And launch any file with t.ch. The development view will occur in a similar way (through the OS method through the API), although the result of the command is completely different. In web technologies, many also implement API to their projects, and if on the fingers: you can send a message in the social. network access to the site, but you can - by creating a special HTTP request. And when implementing this alternative use of functionality, security errors are admitted (as elsewhere). The article is about specific errors in the implementation of such functionality in web projects.

From interface to API methods

Let's analyze the API vulnerability on youtube, which has not received wide publicity. Youtube prohibits the use of specials from the interface. characters like <and> (which are needed for html tags and carrying out an XSS attack) in the title of the videos. But! If you find the API and try to change the name of the same video, then everything will be successful. As a result: no, xss was not executed on the youtube page. But it was fulfilled in the letter on gmail (when the letter with a roller comes), which is even more critical.
')
Findings? Logically - the problem is in the different work methods. But why it happened is a more interesting question. Here I would highlight the following:

• Almost all big projects use OOP and MVC-like patterns. And the implementations of the API should simply be inherited from the method in the interface and pick up all its limitations. Whether or not such a template means and the method was written from scratch (nonsense? Who knows) or a custom "hack" in the interface;
• Custom hack? Developer problem, of course. But also the team leader, you need to constantly convey to the developers, where and what you need to implement. Those. it is clear for everyone to disassemble the eternal thread: where in the development process we validate the data (in the model, service, controller, somewhere else). These are the consequences of the problem of using complex patterns in development, which the tmlid understands normally, and the developers do not even hold out part of the validations in the project in one place, and some in the other. For the API, they were correctly inherited, but got a vulnerability.

What is for testers?

1. It is necessary to forget everything that we knew about the interface (its limitations) and test the project from scratch;
2. Checking the substitution of parameters, a vivid example with a screen about Facebook - from the interface it was impossible to replace the sender's id, but from the API - it is possible;
3. Check for “standard” attacks, such as sqli / xss;
4. If there are autotests, then that's cool. You can replace the standard payload of testValue1 type with various special characters like ', ",>, <and match them (we are looking for XSS).

Compression

Often, APIs are being developed for use on mobile devices. And, when implemented, add various compression before shipping and release after acceptance. And there is an old, bearded attack (various file hosting services were given to it) as ZIP bombs. Here is the question: what size can a file with 42 kb reach when unpacking? 4.5 petabyte . Download here and here . The essence is simple - the file is created clogged with zeros and compressed. Therefore, compression, or rather unpacking - a dangerous thing, be careful.

Evil json

Sometimes the API is provided not only for some end user, and sometimes for sending data within the project. This is often found on large, large sites with different domains. And somehow it is necessary to interact on the client side between domains, and then JSONP comes to the rescue. This is such a JSON with the necessary ones on domain 1, which turns into a callback. When accessing a domain, 1 user will send his cookies, and you can check whether he is authorized and give the necessary data. On the second domain is inserted a similar JS

<script type="application/javascript" src="http://server1.example.com/api/getUserInfo?jsonp=parseResponse"> </script> 

with the already defined parseResponse function. But the bottom line is that an attacker can also insert this script on his domain, define his callback and, if there is a sense of data, use it somehow. A perfect example of using such a vulnerability is shown in the article “Fighting anonymity . ”

Cryptography

Speaking about the API, signing requests immediately comes to them. Since access to the API is often granted to different users, they need a scheme for their identification. And most often they use the following scheme: everyone is given his API key, with which the developer signs his requests, something like this:

sign = sha*(... + DATA + ...)

Data - the data to be sent, instead of the ellipsis API key.
And the question is where to put the key, left or right? Just to the right (we are talking about signing requests with the use of such a very popular scheme). Why? There is an attack on line expansion. Let's take a closer look.
Imagine that we have data A = 1 & B = 2 & C = 3 , their signature is 07ce36c769ae130708258fb5dfa3d37ca5a67514 , the signature goes according to the erroneous scheme sign = sha1 (KEY + DATA) .
And now the situation: someone literally intercepted one request from the client to the server and now wants to change the data in the request, but he will need a new signature, and the key for the signature is not transferred (it is logical). What does he know? Original data and their signature. In short: there is a technical possibility to create an expand string (add your own data) and make a new signature (hash), without knowing N bytes at the beginning. In practice, new data with indented N bytes at the beginning of hashing look like this: A = 1 & B = 2 & C = 3 \ x80 \ x00 \ x00 ... \ x02 & C = 4 . Where

• A = 1 & B = 2 & C = 3 - initial data
• \ x80 \ x00 \ x00 ... \ x02 - spec. bytes for “indent” (signature key) when hashing
• & c = 4 - our new data

When sending the same parameter for example PHP - will take the second. Just what you need to attack. Since the first we can not affect

Evil won: you can sign requests, change the parameters without knowing the signature key. But an excerpt from the documentation on working with the API VK and Mail.RU

  Vkontakte: sig = md5 (name1 = value1name2 = value2api_secret) 
 Mail.RU sig = md5 (uid + params + private_key) 

As you can see - the key on the right.

I mentioned the theft of just one request. In practice, this is a little easier than it seems (story # 1, article )

Unsafe xml

I think everyone met with XML, an unremarkable data type.

 <recipe name="" preptime="5" cooktime="180"> <title> </title> <composition> <ingredient amount="3" unit=""></ingredient> </composition> ... </recipe> 

But less people met with entities in XML

DTD Example:

 <!ENTITY writer "Donald Duck."> <!ENTITY copyright "Copyright W3Schools."> 

XML example:

 <author>&writer;©right;</author> 

Which allow to define something, and then to reuse. The example above is just strings.
And even fewer people came across external XML entities ...

 <!DOCTYPE foo [ <!ELEMENT foo ANY > <!ENTITY xxe SYSTEM "file:///etc/passwd" >]> <foo>&xxe;</foo> 

Something external, a local file, an external http file, etc. can act as an entity. Those. this is standard. Originally conceived for good purposes, for example, you need to give the XML client to the client and take the current time from another server (another xml file available via HTTP).

Without arguing for a long time, the vulnerability is obvious. Using different wrappers - file: ///, http: // the attacker can get information about the system, network, etc. Sometimes this can lead to remote execution of commands, for example in PHP there is a expect: // wrapper, which first executes a command on the OS and returns the result of its work. So by default all standard XML parsers are subject to this attack. One solution comes like this: disable support for external entities. Then the attacker can apply the XML bomb:

 <?xml version="1.0"?> <!DOCTYPE lolz [ <!ENTITY lol "lol"> <!ELEMENT lolz (#PCDATA)> <!ENTITY lol1 "&lol;&lol;&lol;&lol;&lol;&lol;&lol;&lol;&lol;&lol;"> <!ENTITY lol2 "&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;&lol1;"> <!ENTITY lol3 "&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;&lol2;"> <!ENTITY lol4 "&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;&lol3;"> <!ENTITY lol5 "&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;&lol4;"> <!ENTITY lol6 "&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;&lol5;"> <!ENTITY lol7 "&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;&lol6;"> <!ENTITY lol8 "&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;&lol7;"> <!ENTITY lol9 "&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;&lol8;"> ]> <lolz>&lol9;</lolz> 

And "put" the server. The essence of the constant transfer of the entity to the entity.
It should be remembered that this may be a problem not only for the server, but also for the client (for example, client parsing XML).
The most striking example of exploiting this vulnerability is FaceBook hacking in November 2013 , which could have resulted in the execution of commands from the OS.

Summarizing:

• First of all, retest all “interesting” interface restrictions;
• Deal with compression;
• Closely following JS callbacs;
• Cryptography. There can be not only an attack on the extension of a string, but simply a weak (for example, short) key, or somehow predictable.
• And, you can also find data just wired into the mobile API application. By the way, they can be found through the service http://hackapp.com ;
• XML - XXE;
• Anything else.

Presentation:


Video de sha padding and XXE:


upd 04/25/0214: Recording a report