TLS decryption of Java application traffic using logs

Debugging SSL / TLS-protected integrations in Java applications sometimes becomes a very nontrivial task: the connection is not established / broken, and application logs can be scarce, access to editing source codes may not be available, interception of the Wireshark traffic and decryption by the server's private key ( even if there is one) it can fail if a cipher with PFS was used in the channel; a proxy server like Fiddler or Burp may not be suitable, because the application does not know how to go through a proxy or refuses to trust the certificate slipped onto the segment ...

Recently , a publication from ValdikSS appeared on Habré about how you can use Wireshark to decrypt any traffic from the Firefox and Chrome browsers without having a private server key, without replacing certificates and without a proxy. She suggested the author of the current article to the idea - is it possible to apply this approach to Java applications, using JVM debugging records instead of the session key file? It turned out - it is possible, today, dear odnakhabryane, I will tell you how to do it.

')

Recipe idea

From recent versions, the Firefox and Chrome browsers have learned to output enough data into a specially defined file to derive session keys that encrypt the traffic they send (as well as received traffic, because symmetric encryption is used inside SSL / TLS). Strictly speaking, it is not the browsers themselves that do this, but the NSS library in their composition; it is the one that sets the format of the recorded files. This format can be read and used by Wireshark to decrypt SSL-records encrypted with the appropriate keys. The idea of ​​our “dish” is to learn how to create such files independently for Java applications, relying as a source on debug logs that are written to the standard output with the javax.net option javax.net.debug .

Ingredients

We will need:

• A Java application for which we can set launch parameters (JVM options).
  For definiteness, we assume that the application acts as a client when establishing a connection.
  The JDK (JRE) version is desirable to have 1.6 or 1.7, on others (so far) it has not been tested;
• Wireshark version 1.6.0 or higher;
  
• A text editor , for example, Notepad ++;
• Pinch of patience, attentiveness and time.

Kneading

Launch parameters

Since logs will be one of the main sources of information for us, the first thing is to properly configure their receipt. The JVM javax.net.debug = ssl: handshake: data option will be the working version. Immediately make a reservation that it does not have to have such a value, you can (probably) get along with the universal javax.net.debug = all , but working with the results of this choice can be complicated (the volume of logs can be gigantic). Our choice is explained by the following:

1. ssl - to write to the log messages related only to SSL;
2. handshake - to see every message within the main stage for us - handshake ;
3. data - for the lazy, so as not to manually translate some values ​​from the decimal number system to hexadecimal (hex);

The presence of such an option should provide us with output to the log (or standard output) of debug information, to which we will return a little later.

Sniffer setup

Sniffing the traffic of the Wireshark target application is possible only after setting the above option, otherwise we will not have session keys. In addition, we must remember that the keys are “ephemeral” - they are only suitable for one SSL session, that is, the log from one session is not suitable for decrypting the traffic of another session. Well, in order to breathe quite easily, I recommend immediately when starting the sniffer to indicate the host with which you plan to exchange data; this will allow one more “on the coast” to reject unnecessary packets passing through the network interface being listened:

Take and write

After setting the desired application launch parameters and sniffer settings, you can start the application itself and enable packet capture in Wireshark. Then, when the application tries to connect to a (secure) connection with any server, our “saucepans” will begin to fill up. From the point of view of Wireshark, it will look something like this:


^{As you can see, Wireshark explicitly identifies some SSL records as encrypted; records of the type Application Data are the same.}

And from the standpoint of the standard output (logs) of the application - something like this:

... *** ClientHello, TLSv1 RandomCookie: GMT: 1427238714 bytes = { 246, 5, 6, 214, 168, 159, ... , 140, 141, 50, 196 } Session ID: {} Cipher Suites: [SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA, ..., SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA] Compression Methods: { 0 } ... 

In fact, many more things will be output to the log (the debugging output format is not specified anywhere and changes from Java version to version), but for now we need to see at least that.

"Proofing"

Possessing debug entries from an SSL / TLS session, we can generate a session key file in the NSS format. To do this, we first need to determine what method of session key distribution was used in our communication session: exchange method (aka RSA) or generation method (aka DH or PFS, although these are different things). What is their essence and differences can be found in the wonderful work of Sally Vandeven . We only need to know the method itself, and you can define it in at least two ways:

1. By the name of the cipher output to the log or defined by the sniffer. For example, according to this conclusion
   

    *** ServerHello, TLSv1 RandomCookie: GMT: 1037995915 bytes = { 168, 183, ... 204, 178 } Session ID: {141, 155, ... 214, 36} Cipher Suite: SSL_RSA_WITH_RC4_128_SHA ... 

   it can be seen that the cipher has no mention of DH in the name, but it explicitly calls itself RSA. However, such clarity is not always present, so you can use Plan B:
   
2. For the presence of SSL ServerKeyExchange messages in the interception of traffic in Wireshark (see the screenshot in the subsection “Removing and Recording”) - it is present for DH methods and absent for RSA (an explanation of why is beyond the scope of this article). Of course, the presence of this message can be determined by the same logs.

Having defined the method of distributing session keys, we turn to the description of the NSS file format , which instructs us to distinguish the lines of each file just by these two methods. Consider each of them in more detail.
To begin with, suppose that in our communication session we used some RSA-based cipher. According to the format description, the corresponding line of the file must begin with RSA text, followed by 16 bytes of the HEX-coded PreMasterSecret encrypted key, and another space - 96 bytes of the HEX-coded non- encrypted PreMasterSecret key . This key is the basis for generating the master key, MasterSecret, and is transmitted from the client to the server in a ClientKeyExchange message, being encrypted with the server's public key. This means that the first part of the string (the encrypted representation of this key) should be visible in Wireshark. We find the right message and make sure - yes, it is:



By the way, the same value can be obtained from the application logs, and here we just need the JVM option " : data ":



The found value (16 bytes) can be inserted into the generated NSS file. Now we check the same operation for the second element of the line - the eigenvalue of the PreMasterSecret key. Since it is obviously never transmitted over the network in the open form (actually, that's why it is called ... Secret ), it will only be necessary to get it out of the logs. Fortunately, with unequivocal prompts from the JVM, it is not particularly difficult to do this:



Now we need to add this value to the NSS file line that we form and “brush” the line so that we get something like this (comments in standard notation are quite acceptable) :

 # SSL/TLS secrets log file, generated by Toparvion RSA 75ff866e23beca1c 03012aede74befa88233253e3207bb1320935ab206696512674df5c6dee7dfaa2156932bc559631c8f3bb46ae38a71ff 

For those for whom RSA is “just the case” (as a rule, these are applications before Java 7), you can already go to the section “Tasting”. Those who have come across PFS (often Java 7 or higher) will have to read further ...

Like the RSA method, the string for decrypting records based on PSF should consist of three elements, separated by a space:

1. Record type; in this case, it should be equal to CLIENT_RANDOM;
2. 64 bytes of HEX-coded client random number Random ;
3. 96 bytes of HEX-coded MasterSecret master key;

Data sources for the second and third element are also similar to the previous method, but there are some subtleties. The client random number is the concatenation of the current time in milliseconds of GMT and the random value itself. In the case of accessing Wireshark it is clearly visible:



But when accessing the logs it is easy to make a mistake, but you can use the following hint:



Here also the entry ": data" in the meaning of the jVM option javax.net.debug saves us from having to manually convert the number systems. All we need is 64 bytes of a random number, then all of it is entirely (and not just the beginning, as was the case with RSA). It will also play the role of an index when Wireshark searches for a suitable entry in the NSS file.
The third element of the line - the MasterSecret master key - can also be extracted from the application logs:



After extracting the master key from the logs, we supplement the generated string with it, we “brush” it and get something like:

 # SSL/TLS secrets log file, generated by Toparvion CLIENT_RANDOM 551435582740bdc1386b20b7fcb51428fe3042e06c8e6e94c910786f577a2ada 976dc1d54dd74d3c2e715109c8a4fb8e743efc084614abc0e12fdb78e472c30e3590ac5eb383424b2d8fa3de84c8b0f5 

It is impossible not to notice that Wireshark is very sensitive to the format of the NSS-file, so it is better now to carefully double-check whether the number of bytes in each element of the line converges and whether there are any extra spaces anywhere; It can save time in the future.

Tasting

Now that all the “manual” steps have been completed, it’s time to give the word to Wireshark - we present the created file to it exactly as described in the article mentioned at the beginning:

1. Open the context menu in Wireshark on any SSL / TLS package;
2. Choose Protocol Preferences -> Secure Socket Layer Preferences ... ;
3. In the window that opens, in the (Pre-) Master Secret log filnename column, specify the path to the NSS file that we formed.

Click OK and look at the changes in the intercepted traffic:



If the decryption is successful, then packages that previously had Encrypted in their names will acquire specific names. This is what happened with the Finished teams shown in the figure above.
In addition (and this is probably the most pleasant moment) you can now select any packet with SSL or TLS protocol and click on Follow SSL Stream in its context menu - the result should not require comments:



As you can see, despite the HTTP S request, we see the transmitted traffic and can export it for further analysis.

If, however, did not work ...

Errors on this slippery slope can be made a lot. One of the most valuable sources of information is the log of Wireshark itself - it is kept if the path to the log file in the SSL debug file column is specified in the same SSL settings window.
It is important to note that no filtering is provided for the log, but Wireshark is very detailed, so if you keep the log constantly turned on, it will, firstly, grow very quickly, and secondly, it may lead to Wireshark’s deceleration (since apparently spelled synchronously). In this regard, it is best to specify the log file only before the application of the NSS file, and at the end of the analysis - to clean it (but not delete it).

Conclusion

The article considered another approach to decrypting SSL / TLS traffic of Java applications with a view to debugging them.
In the above form, the approach is hardly applicable in practice, as it requires a significant investment of time and the availability of certain knowledge and skills from the performer. However, the presented description will allow to formalize, and therefore, automate (program) this approach and, thus, put it in the service of people. Such work has already begun by the author. If this idea is interesting and you - welcome! Do not forget to tell about it on Habré.

Thank you for reading!

Update

Dear ones, for those who are interested in the approach described in the article, but finally can not be too lazy to use it manually, the NSS Java Maker utility is available on GitHub , which automates the steps described in the article to parse the JVM debug log and generate an output NSS file for Wireshark.
The syntax for its launch is simple, you only need to specify the source file of the JVM log:

 java -jar nssjavamaker.jar some/directory/java-ssl-debug.log 

At the output in the current directory, an NSS-file session-keys.nss will be created, ready for import into Wireshark. To change this path and other parameters, refer to the Readme file or run the utility with absolutely no parameters.
A ready-to-run JAR can be downloaded from the latest version page.
Suggestions / suggestions / comments / remarks on the utility are welcome in the application section of the project page , as well as at toparvion@gmx.com . Good luck in job!