The whole truth about vertical scaling in PaaS and why you are overpaying for regular hosting

As you know, an application does not always use the same amount of resources, but thanks to the automatic vertical scaling function, Jelastic changes the size of the container for the application. Accordingly, the user does not need to overpay for reserved resources that are not used, as is the case with other PaaS. Indeed, the business model of the entire hosting industry and the old generation of PaaS solutions is based on overselling . Thus, the importance of fair pay for the actual consumption of resources is obvious.

It is clear that at the time when they developed the JVM, nobody knew about clouds or virtualization, and, moreover, nobody even thought about density in PaaS. Today, virtualization has changed the course of the game in the hosting industry, and this revolution will continue. Now we can use resources more efficiently. Michael Widstendt , one of the main architects of the JVM at Oracle, confirmed that the JVM is not designed for PaaS at all, but Oracle is doing everything possible to change this. Plus, the guys from IBM are moving in the same direction. Some notes on the dynamic behavior of the JVM can be found in the IBM JavaOne Keynote 2012 Highlights .

To be honest, “inventing” vertical scaling was not easy due to a number of technical limitations. One of these limitations is directly related to the JVM. In this article, we would like to share some information that may be useful for understanding the vertical scaling and the above limitations.
')
For a start, we created an application that itself controlled the consumption of resources and, through the API, when it reached the upper border, gave the command to the infrastructure to increase the limit, and vice versa, when resources were not needed, gave the command to take them back. Such an algorithm worked quite well, but in the process of testing one more thing came to light: Java does not work well with scaling, as it can start up with a small amount of memory, then take additional resources when necessary, but it does not return them back to OC . This was specifically thought out by JVM engineers to speed up the process of getting memory the next time it is needed. Of course, this is not acceptable for our cloud model.

Then we conducted another series of experiments on how Java behaves, more precisely garbage collectors, such as: Serial, Parallel, Concurrent Mark Sweep and G1.

How the Garbage Collection works

In Java, dynamic object allocation is achieved using a new operator. The object, as soon as it is created, uses a part of the memory, and the memory remains reserved until there is no access to the object. In the case when the appeal to the object does not occur, it is assumed that it is no longer needed, and the memory occupied by it can be freed. There is no explicit need to destroy the object itself, since Java can automatically move it in memory. Garbage collection is just the method that solves this problem. Programs that do not free up unnecessary parts of memory in time can cause an uncontrolled decrease in the amount of free memory - the so-called “memory leak”. In Java, garbage collection occurs automatically throughout the program’s life cycle, eliminating the need to free up memory and thus prevents leaks. More information about the Garbage Collection in Java can be found in the Java Enterprise Performance Book or on the Javarevisted blog.

Serial Garbage Collector (-XX: + UseSerialGC)

Let's start with Serial. It uses one stream to do all the garbage collection. Best suited for single-processor machines, sometimes it can be useful for multiprocessor machines that run applications with small data sets (up to about 100 MB).

This garbage collector showed a very good result regarding scaling. We found out that it possesses properties of compacting, that is, we can say that it does defragment the memory, and returns unused resources to OS.
Let's see this again by testing the Serial Garbage Collector on the JVM 6 and JVM 7.

Java 6

More specifically, we will test Java version 1.6.0_27.

We run the Visual JavaVM utility, which allows you to monitor all Java processes running on your computer, as well as track how much memory these processes consume.

For testing, we will use a program that dynamically allocates and frees memory in an infinite loop:

public class Memoryleak {
public static void main ( String [ ] args ) {
System . out . println ( "START ...." ) ;
while ( true ) {
System . out . println ( "next loop ..." ) ;
try {
int count = 1000 * 1024 ;
byte [ ] array = new byte [ 1024 * count ] ;

Thread . sleep ( 5000 ) ;
array = null ;
System . gc ( ) ;
System . gc ( ) ;
Thread . sleep ( 5000 ) ;
} catch ( InterruptedException ex ) {
}
}
}
}

Run the JVM with the following parameters:

-XX: + UseSerialGC -Xmx1024m -Xmn64m -Xms128m -Xminf0.1 -Xmaxf0.3 , where

-XX: + UseSerialGC - use Garbage Collector Serial;
-Xmx1024m - maximum use of RAM - 1024 MB;
-Xmn64m - heap-step equal to 64 MB;
-Xms128m - the minimum heap value equal to 128 MB;
-Xminf0.1 - this parameter determines the minimum free space in the heap and instructs the JVM to expand it if after the garbage collection there is no at least 10% free space;
-Xmaxf0.3 - this parameter determines how much the heap is expanded and instructs the JVM to compact it if the amount of free space exceeds 30%.

The default values ​​for Xminf and Xmaxf are 0.3 and 0.6, respectively, but for the experiment we significantly reduced these limits to increase the amplitude of the vertical scaling.

As a result, for a separately taken process of our program execution, we see that the memory is dynamically filled and released:



Let's also take a look at the graph of total memory consumption to make sure that there are no leaks:



Java 7

We will test jdk version 1.7.0_03

We use the same program and set the same parameters for the JVM. As you can see with JDK 7, everything is also smooth:


Good result, but, unfortunately, as we mentioned above, this garbage collector works only with single-processor machines and small amounts of memory, which is not enough for the functioning of large, cumbersome applications.

Parallel Garbage Collector (-XX: + UseParallelGC)

Parallel can perform minor garbage collection work in parallel, which can significantly reduce the consumption of resources and time. This is useful for applications with medium and large datasets that run on multiprocessor and multi-threaded machines.

Let's repeat our experiment for the Parallel Garbage Collector.

The result for total consumed memory for Java 6 :


And for Java 7 :


Parallel, of course, has many advantages over Serial, for example: it can work with multi-threaded applications, multiprocessor machines, and quickly copes with large amounts of memory. But, alas, as we see in the graphs, it does not return unused resources back to the operating system.

Concurrent Mark Sweep Garbage Collector (-XX: + UseConcMarkSweepGC)

Concurrent Mark Sweep performs most of the work while the application is still running. This allows you to significantly reduce the pause for garbage collection. It is designed for applications with medium and large data sets, for which response time is more important than total throughput.

Repeat the procedure again for the Concurrent Mark Sweep.

And again the situation is similar to the previous one:

Java 6 :



Java 7 :

G1 Garbage Collector (-XX: + UseG1GC)

And here comes the G1, which has all the advantages of Parallel and Concurrent Mark Sweep, and satisfies all our requirements.

We started testing it, and it turned out that with a long G1 operation, there is a stable memory leakage constant.

Java 6 :


Of course, we addressed this problem to Oracle representatives. Oracle employees Ivan Krylov and Vladimir Ivanov from St. Petersburg quickly responded to our request and as a result, this problem was fixed in JVM 7. Let's check:



As you can see, in Java 7 everything is really ok, the improved G1 fulfilled its mission completely.

As you understand, the ability to pay upon the use of resources is a very important factor for each client and company. No one should overpay. However, there are still a few blockers that prevent rapid development in this direction. We will continue to do everything possible to continue the revolution in the hosting industry.

The video below explains how automatic vertical scaling works in Jelastic:



* For unknown reasons, the official bug on our G1 memory leak report has been removed from the Oracle database. More information about this bug can be found here .