There is no reception against scrap: OpenJDK hack vs. Class Encryption

The purpose of this article is to prevent developers from using class-file encryption tools for obfuscators to protect their applications and from wasting money on them.
Issues of protecting bytecode from reverse engineering and circumventing this protection are discussed in detail in Dmitry Leskov's fundamental work - Protect Your Java Code - Through Obfuscators And Beyond .
The class-file encryption mechanism assumes that the contents of the classes are stored in an encrypted form, and when the application starts via a specialized ClassLoader or JVMTI interface, the decrypted bytecode is loaded into the Java virtual machine.

Methods for circumventing such protection are mentioned in detail in the above article, fortunately or unfortunately, there are a number of products that include a native component that interacts with the JVM and tracks debug mode or the presence of foreign agents. But this, despite all the assurances of the developers of such products, does not protect your byte code at all.

In order to demonstrate the vulnerability of ALL obfuscators that encrypt class files, it is enough to launch the application protected by them with the -XX: + TraceClassLoading option and make sure that all encrypted class files are safely seen at this JVM trace level. We will go further, take the source OpenJDK and insert the unloading of the byte-code of the loaded class-files.

For the experiment, we will use Debian Linux 6.0.5 (Stable) and the OpenJDK7 source bundle. Instructions for installing JDK from source codes on other platforms are available here: OpenJDK Build README .
')
In order to minimize the number of changes to the OpenJDK source code, we will, when the -XX option is enabled: + TraceClassLoading, save the byte code of all loaded classes to the class.dump file relative to the working directory. The file structure is as follows:

{ int lengthClassName, byte[] className, int lengthByteCode, byte[] bytecode }, { next record … }, … 

Prepare the environment for the assembly:

 # apt-get install openjdk-6-jdk # apt-get build-dep openjdk-6 

Next, you need to download the OpenJDK source code that is convenient for you and our patch , which will add the following code to the ClassFileParser :: parseClassFile function, to the hotspot / src / share / vm / classfile / classFileParser.cpp file:

       // dumping class bytecode      // dump file format:      // length of the class name - 4 bytes      // class name      // length of the class bytecode - 4 bytes      // byte code      // ... next class ...  ClassFileStream* cfs = stream();  FILE * pFile;  int length = cfs->length();  int nameLength = strlen(this_klass->external_name());  pFile = fopen("classes.dump","ab");  // size of the class name  fputc((int)((nameLength >> 24) & 0XFF), pFile );  fputc((int)((nameLength >> 16) & 0XFF), pFile );  fputc((int)((nameLength >> 8) & 0XFF), pFile );  fputc((int)(nameLength & 0XFF), pFile );      // class name  fwrite (this_klass->external_name() , 1, nameLength, pFile );  // size of the class bytecode  fputc((int)((length >> 24) & 0XFF), pFile );  fputc((int)((length >> 16) & 0XFF), pFile );  fputc((int)((length >> 8) & 0XFF), pFile );  fputc((int)(length & 0XFF), pFile );      // class bytecode  fwrite (cfs->buffer() , 1 , length, pFile );  fclose(pFile); 

Make sure the JDK is going fine:

 # export LANG=C ALT_BOOTDIR=/usr/lib/jvm/java-6-openjdk ALLOW_DOWNLOADS=true # make sanity && make 

Apply the patch and run the build

 # cd $OPENJDK_SRC # patch -p1 < $PATH_TO_PATCH_FILE # make 

Next, go to the bin directory of the compiled JRE: $ OPENJDK_SRC / build / linux-i586 / j2re-image / bin /
To test the performance of java run with a single parameter -XX: + TraceClassLoading:

 # ./java -XX:+TraceClassLoading 

And look at classes.dump, it will contain all the class files that the JRE loads at startup.

And now the most interesting, let's take a Java-application with an encrypted byte-code, for example, you can use some kind of obfuscator with this function. For obvious reasons, I will not mention specific names, it is enough to search Google for the key “byte-code encryption”. Inside SomeClassGuard.jar, the com / **** / someclassguard / engine hierarchy contains encrypted class files; you can verify this by installing any decompiler yourself or look in the HEX viewer for the file header.

Now run SomeClassGuard.jar:

 # ./java -XX:+TraceClassLoading -jar SomeClassGuard.jar 

Next, we need to unpack the classes.dump file obtained after running SomeClassGuard.jar, for this we will write a small Java program:

 package openjdkmod; import java.io.DataInputStream; import java.io.EOFException; import java.io.File; import java.io.FileInputStream; import java.io.FileNotFoundException; import java.io.FileOutputStream; import java.io.IOException; /** * Classes dump format extractor class. * Author Ivan Kinash kinash@licel.ru */ public class ClassesDumpExractor {   /**    * Extract contents classes.dump to specified dir    */   public static void main(String[] args) throws FileNotFoundException, IOException {       if (args.length != 2) {           System.err.println("Usage openjdkmod.ClassesDumpExtractor <classes.dump file> <out dir>");           System.exit(-1);       }       File classesDumpFile = new File(args[0]);       if (!classesDumpFile.exists()) {           System.err.println("Source file: " + args[0] + " not found!");           System.exit(-1);       }       File outDir = new File(args[1]);       if (!outDir.exists()) {           outDir.mkdirs();       }       DataInputStream din = new DataInputStream(new FileInputStream(classesDumpFile));       while (true) {           try {               int classNameLength = din.readInt();               byte[] classNameBytes = new byte[classNameLength];               din.readFully(classNameBytes);               String className = new String(classNameBytes);               System.out.println("className:" + className);               int classLength = din.readInt();               byte[] classBytes = new byte[classLength];               din.readFully(classBytes);               File parentDir = className.indexOf(".")>0?new File(outDir, className.substring(0,className.lastIndexOf(".")).replace(".", File.separator)):outDir;               if(!parentDir.exists()) parentDir.mkdirs();               File outFile = new File(parentDir, (className.indexOf(".")>0?className.substring(className.lastIndexOf(".")+1):className)+".class");               FileOutputStream outFos = new FileOutputStream(outFile);               outFos.write(classBytes);               outFos.close();           } catch (EOFException e) {               din.close();               return;           }       }   } } 

And run it with the parameters:

 # java openjdkmod.ClassesDumpExractor classes.dump dump_directory 

At the output, we get a directory with decrypted class files.

Findings.
Protecting class files with encryption is absolutely pointless, dangerous, expensive (at least not free).
If you need to protect your bytecode:
1) Use bytecode compilers in native code.
2) The combination of the classic obfuscator with obfuscator with the function of string encryption.
For super-protection: use external devices that support secure storage and execution of byte-code inside.
The technique used above can be used to debug various applications, when you need to see which bytecode is loaded during operation.

Note1:
The same results can be achieved without modifying the source code of the JDK using the sun.misc.Unsafe class, although you need to dig a little in the format of storing classes inside the JVM.

Note2:
Well, of course, the author is not responsible for your use of the data contained in this article.

Note3: The original image is taken from here: it.wikipedia.org/wiki/File : Netbeans-Duke.png