Kotlin vs. Java: compile speed

As all Android developers already know, Google recently announced Kotlin’s official support for Android. Many of the risks associated with using this wonderful language in Android projects have been removed. But topical, especially for very large projects like Badoo, is the issue of assembly speed. I was glad to discover that there are already studies on this topic in the network, and I want to share the translation of one of them.

So, if you are translating an application from Java to Kotlin, will it compile longer?

An earlier article discussed converting an Android application from Java entirely into Kotlin . The code on Kotlin turned out to be less, and it was easier to maintain than to Java, so I came to the conclusion that it was worth it. But some developers don't want to try Kotlin, for fear that it might compile more slowly than Java. And this concern is fair: no one wants to spend time converting the code if, as a result, the assembly will last longer. So let's examine the duration of compilation of the App Lock application before and after converting to Kotlin. I will not compare the speed of Kotlin and Java line by line, but instead I will try to answer the question whether the conversion of the entire code base from one language to another will affect the overall build time.

How I tested the build duration

I wrote shell scripts for repeated launches of Gradle builds for different scripts. All tests were performed sequentially ten times. Before each new scenario, the project was cleared. For scripts that use the Gradle daemon , the last one stopped before starting the benchmark.

All benchmarks were performed on a machine with an Intel Core i7–6700 operating at 3.4 GHz, equipped with 32 GB DDR4 memory, and a Samsung 850 Pro SSD drive. The source code was compiled using Gradle 2.14.1.

Tests

I wanted to drive off benchmarks for several common usage scenarios: clean builds with / without the Gradle daemon, incremental builds without changing files, incremental builds with a modified file.

The Java App Lock codebase contained 5491 methods and 12,371 lines of code. After converting to Kotlin, the number of methods decreased to 4987, and the number of lines to 8564. During the conversion to the architecture, no major changes were made, so measuring the duration of the compilation before and after converting should give a clear idea of ​​the difference in the length of assembly between Java and Kotlin .

Clean builds without the Gradle daemon

This is the worst scenario in terms of the build duration for both languages: running a clean build with a cold start. For this test, I disabled the Gradle daemon.

That's how much time it took all ten assemblies:

Ten consecutive clean builds without a Gradle daemon

The average Java build duration is [figures corrected based on the author’s source data - approx. translator] 24.5 seconds, Kotlin - 32.4 seconds: an increase of 32%. Not the best start for Kotlin, but most people compile their code for other scenarios.

More often than not, we compile the same code base several times as we make changes to it. It is for this scenario that the Gradle daemon was developed, so let's turn it on and see what happens.

Clean builds with the Gradle daemon enabled

One of the problems with JIT compilers like the JVM is that they spend time compiling the code executed in them, so that as it executes, the performance of the process increases. But if you stop the JVM process, the performance gain is lost. Every time you build Java code, you usually have to start and stop the JVM. As a result, he does the same work every time. To solve this problem, Gradle comes with a daemon that continues to function between builds and helps maintain the performance gains provided by JIT compilation. You can enable the daemon using the --daemon command --daemon , entered on the command line, or by adding org.gradle.daemon=true to the gradle.properties file.

Here is the result of running the same build series, but with the Gradle daemon enabled:

Ten consecutive builds with the Gradle daemon enabled

As you can see, the first run takes about the same time as in the script without a demon. In subsequent builds, performance increases up to the fourth run. In this scenario, it is more appropriate to estimate the average build duration after the third run, when the daemon has already warmed up. In this case, the pure Java build takes an average of 14.1 seconds, and on Kotlin it takes 16.5 seconds: an increase of 13%.

Kotlin is catching up with Java, but still lags behind. However, regardless of the language used, the Gradle daemon reduces the build duration by more than 40%. If you are not using it yet, then it's time to start.

So, complete builds on Kotlin are slightly slower than on Java. But usually we compile after making changes in just a few lines of code, so incremental assemblies should demonstrate different performance. Let's find out if Kotlin can catch up with Java where it matters.

Incremental builds

Using incremental compilation is one of the most important features of the compiler to improve performance. In a normal build, all project source files are recompiled, and in incremental one, it is monitored which files have changed since the previous build, and as a result, only these files and those that depend on them are recompiled. This can have a very strong impact on the duration of the compilation, especially in large projects.
Incremental assemblies appeared in Kotlin 1.0.2 , you can enable them by adding kotlin.incremental=true to the gradle.properties file, or via the command line .

So, how does the Kotlin compilation duration change compared to Java when using incremental compilation?

Here are the benchmark results provided there are no changes in the files:

Ten consecutive incremental builds without changing files

Now we will test the incremental compilation, provided that one source file is modified. To do this, I changed the Java file and its Kotlin equivalent before each build. In this benchmark, this is a file related to the user interface; other files do not depend on it:

Ten consecutive incremental builds with one separate modified file.

Finally, let's look at the results of incremental compilation with one modified source file, which is imported into many other project files:

Ten consecutive incremental builds, subject to change of one key file

As you can see, the Gradle daemon still has to warm up for two or three runs, but after that both languages ​​become very close in performance. In the absence of changes in the files, Java takes 4.6 seconds to warm up the build, while Kotlin takes 4.5 seconds. If we change the file, but it is not used by other files, then Java takes 7 seconds to execute the heated build, and Kotlin takes 6.1 seconds. Finally, if a modified file is imported into many other project files, then with a heated Gradle daemon, an incremental Java build takes 7.1 seconds, and for Kotlin it takes an average of 6 seconds.

Conclusion

We measured performance under several different scenarios to see if Kotlin could compete with Java in terms of compilation duration. With clean builds that are relatively rare, Java exceeds Kotlin by 10–15%. But most often, developers perform partial assemblies, in which a large time gain is achieved through incremental compilation. Thanks to the running Gradle daemon and the included incremental compilation, Kotlin is not inferior, or even slightly superior to Java. An impressive result that I did not expect. I express my respect for the Kotlin development team for creating a language that not only has excellent capabilities, but also compiles so quickly.

If you have not tried Kotlin for fear of increasing the length of the compilation, then you can no longer worry: it compiles as fast as Java.

The raw data I collected when running the benchmarks is here .