“65K methods will be enough for everyone” or how to deal with the limit of DEX methods in Android

This happened suddenly. You just wrote the code for your Android application, you liked it, and you enjoyed the process. You have added a cool library to get additional features and write simpler code. But instead of a working application at the output, you get a terrifying inscription:

Unable to execute dex: method ID not in [0, 0xffff]: 65536
Conversion to Dalvik format failed: Unable to execute dex: method ID not in [0, 0xffff]: 65536

And you are in a stupor, you are unable to create a DEX file for an APK. You have no idea what this is and how to fix it. And whatever you do, it will lead you to the most logical state: PANIC .

Meet the enemy

After you have tried all your usual tricks (in the following order: cleaning the project, restarting the IDE, building through the command line, rebooting the laptop, calling a friend, and even a ten-minute break in the hope that the problem will disappear by itself), you have to admit the cruel truth : the problem is still here. Therefore, it would be nice to know what caused it.

There are many posts on the Internet about this problem: here is one of them , one more , and one more ; Oh, and this and this, too . So what happened? Essentially, it looks like you are faced with something that is often (and mistakenly) called Dalvik 65K methods limit. Briefly (thanks fadden ):
')
You can refer to a very large number of methods in the DEX file, but you can only call the first 65536, because this is all the memory you have for the method call instruction.
[...] the number of methods you can refer to is limited, and not the number of methods you define. In other words, if your DEX file contains only a few methods, but together they call 70,000 different externally-defined methods, you will exceed the limit.

What we have? Your application has too many methods written by you or enclosed in library JAR files. For this reason, the dx tool cannot write the addresses of some methods simply because they do not fit into the field defined for this in the DEX file (which in turn contains the compiled Java classes). And that is why the problem should be called DEX 65K methods limit.

And yes, you understood everything correctly. This problem will not disappear even when the android switches to a new ART runtime, until Google decides to “fix” the DEX format or replace it with something else.

Let's count

You are probably surprised how you managed to bury more than 65,000 methods in your precious APK. Or maybe not. In any case, there must be a way to count these methods and determine their source.

The DEX file header already contains information on the number of method references (for clarity, this number represents unique method references, not the sum of calls for each method). But we also want to see which package adds an inordinate number of these methods.

The best tool I found was made by Mihai Parparita , who wrote a simple bash script called dex-method-counts . The script is very smart and on output gives XML, which matches the name of the package with the number of its methods. Another solution belongs to Jake Wharton, it gives the exact same result, but it takes much longer because of its recursive nature (Jake also wrote an interesting article on this topic).

Now we have a good tool in our hands, so why don't we test it on a small test application? Let's call it SampleApp. Here is the code that is contained in our simple Activity:

 package com.whyme.example; import android.app.Activity; import android.os.Bundle; public class SampleActivity extends Activity { @Override public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); dummyMethod(); } private void dummyMethod() { System.out.println(“Oh gosh.”); } } 

I also included Google Play Services 5.0.77 in the app, although I don’t use this library at all. The meaning of my act is clarified in several lines below. Now, let's analyze the dex-method-counts output (compressed, for simplicity):

 Read in 20332 method IDs. <root>: 20332 : 2 android: 834 […] com: 18802 google: 18788 […] whyme: 14 example: 14 dalvik: 2 system: 2 java: 624 […] javax: 5 […] org: 63 apache: 24 […] json: 39 

Wait. WHAT? I just defined one method in my Activity and already have 20,000 methods in my DEX file?



The culprit is obvious.

Google Play Services: mixed feelings

There is a non-zero probability that you are familiar with Google Play Services . Google’s smart move to support the API of many of its services, up to Android 2.3 Gingerbread, with a new release every six weeks.

However, everything has its price, which in this case was in the form of a HUGE number of methods that Play Services carries in itself. We are talking about about 19,000 methods. And with a limit of 65536, this means that a third of all the methods that we can include have already been exhausted. Like this.

Now, before we get to the “solution” of this problem, I think a little reflection would be appropriate. Some developers, both inside and outside Google, share a common point of view, which says: “If you encounter a restriction, then you are a bad, very bad developer, you deserve punishment (and you must also burn in hell)”. From a more impartial point of view, this means that you have recklessly included too many libraries, either because you are too lazy, or because your application does too much. Personally, I do not share this view. I think these are just excuses for two simple things:

1. I do not want to admit the problem / I do not want to fix it.
2. I did not go beyond the limit, then I am a cool developer, and you are doing something wrong.

Maybe the limit was set for the following reasons: “This limit will define the line between a good and a bad programmer for all future generations. So be it. ”But I doubt it.

Throw away all unnecessary

In a world where justice prevails, Google realized the depth of the problem and decided to reduce the damage it does by dividing the giant Play Services into modules that can be included depending on the functionality your application uses. For example, if my application is not related to games, it makes absolutely no sense to include all the Play Games API in it. This is the most reasonable move that can be taken and which is possible to carry out.

However, Google does not fully understand the problem, so better we will find another way to solve it on our own. As it was said, we’ll throw out parts of google-play-services.jar, those parts of the API that we don’t need.

There are a couple of ways to do this. You can do it manually, but remember that you have to do it every six weeks.

You can rely on the JarJar library , which allows you to remove desired packages from the jar file. Clear and simple.

If you, like me, prefer the good old command line, then you can use the script that I use for your projects, which is called (you will not believe) strip_play_services.sh . The script can be controlled using the strip.conf configuration file that comes with it. Using it, you can select the Play Services Library modules that should be disabled. Essentially the script does the following:

1. Extract content from google-play-services.jar
2. Checks whether strip.conf exists, and if it exists, it uses it for configuration; if not, it creates a new one, writing all the modules from the Play Services Library there.
3. Based on strip.conf, removes unnecessary modules from the library
4. Repack the remaining modules in the google-play-services-STRIPPED.jar file

Here is a simple configuration for the strip.conf file:

actions = true
ads = true
analytics = true
appindexing = true
appstate = true
auth = true
cast = true
common = true
drive = false
dynamic = true
games = false
gcm = true
identity = true
internal = true
location = false
maps = false
panorama = false
plus = true
security = true
tagmanager = true
wallet = false
wearable = true

And that's all. Now for an example, let's calculate the number of methods of our test application again and see what has changed:

 Read in 11216 method IDs. <root>: 11216 [...] 

This is already something. By choosing the right configuration based on the needs of your application, you can control the “thickness” of Google Play Services and leave more room for the methods that you (probably) really need.

But you must also be very careful. Play Services Library is very well built (in terms of modularity), and you can safely remove the “games” module without affecting the “maps” module. But if you remove a module that is used by others (internal, in simple terms), then nothing will work. Use trial and error!

Is there any other way?

Of course have. Running ProGuard for your application can help, as it removes unnecessary methods from your code and in addition reduces the size of the APK. But do not expect a big reduction, this will not happen.

Another solution is to create a second DEX file, which contains a part of your code, which you will access through interfaces / reflections and which you will load through a custom ClassLoader ( more ). This solution may be more effective in some cases. However, this path is very nontrivial.

I found another solution from my good friend Dario Marcato . He created the Gradle task to remove unnecessary modules. If you use Gradle, be sure to try it! .

Afterword

This is a translation of the original article by Sebastiano Gottardo , [DEX] Sky's the limit? No, 65K methods is . If the translation seems to be “clumsy” or you have found spelling / syntactic / factual errors in the text, please write me in the LAN.