Highload to Java: what you need to remember

Highload is a theme that is both fashionable and rather hackneyed, especially since there is no clear definition of what Highload is. For clarity, let's call the “Highload” a network application that should handle 1000 requests per second. And the application processing 1 request per second, respectively, "not Highload". My experience shows that between the first and second there is a significant difference in architecture, design approaches and problems. In this article I will try to present these differences, as I understand them.



So…

Many frameworks have performance limitations.

With this you just have to accept. A very large amount of code is not able to work under a large load - due to excessive memory requirements, a processor, inefficient synchronization, outdated I / O mechanisms, or poor error handling due to lack of resources. Even if your favorite library is written by very smart people and has no flaws - it may not be suitable simply because its design has sacrificed efficiency or ease of use. Therefore - be ready to live with sopisopiki or look for the few solutions that pass the load tests for your specific application. Believe in this matter can not be anyone.
')

Caching, caching and caching again

A rare high-load system does without caches - especially over a DBMS. Proper caching in a distributed system is a big and complex topic, so it makes sense to think about the data right away: who will update and query them and how, and where and how you can sacrifice data integrity or relevance.

Simplicity

In general, the simpler the system, the faster it works. One should strive for maximum simplicity, often at the expense of clarity, conceptuality, or beauty of architecture.

Network bandwidth has limits

When choosing the serialization format, take care to multiply in advance the average packet size by the number of packets per second and compare it with the bandwidth of your network. In this regard, Json is better than XML, Protobuf is better than JSON, and sometimes you have to invent your own format, with a better degree of packaging.

Do not forget about off-heap

Some objects in Java require additional memory to work, so you do not need to expect that if -Xmx is set, then the application will definitely fit into the server. For example, each stream in Java requires from 256K to 2 megabytes of off-heap memory for its work. When multiplying by 1000 it turns out quite a lot, so watch the number of threads used by your application.

GC is not rubber

Even if there are no strict requirements for latency, you need to remember about garbage collection. Try to limit the number of allocations and, especially, the total amount of allocated memory for each request. All the necessary metrics are in the Java Mission Control Profiler.

More accurate with logging

The statement “a decent application should always log the input and output data” is true, but in a high-loaded application it will easily become a bottleneck of your system. Contention on the logger, insufficient speed of the hard drive, gigabytes of logs per hour is all a harsh reality. Therefore, it is necessary to choose the data for logging very carefully and remember about the spectra - they take a lot of space and with a large number of errors they are able to put an application. Often it is necessary not to write logs to disk at all, but to send them over the network to a specialized system (and remember that the network is also not rubber, yeah).

Resource Behavior

What if the application receives more requests than it can chew? If the base suddenly began to respond twice as slowly? If the network started losing packets? A good application should not hang tight, and answer "come back tomorrow." Moral - for all interactions with external systems, there should be timeouts, the application should limit the number of requests processed in parallel and the client should know what to do if the application is busy or does not respond.

Do normal monitoring

Another complex and extensive topic, but in short - if the application has outlined the machine, then there should be graphics of memory consumption, swap, disk, CPU, threads, system descriptors somewhere.

And always test the application under load.

The application can easily die under high load, quite decently behaving under low. If it worked for weeks at 1 request per second, at 1000 it may die due to negligible memory or resource leaks, network overload, overload or disk overflow, and another 1001 reasons. Therefore - always drive the build at full load before the release on the prod.

That's all. Comment, correct, share your experience.