Highly loaded java application
I would like to add about the high-load java systems regarding this article. In the comments to write too much, so I will write separately. The approach is not quite a programmer, but can someone come in handy. This is just my opinion, perhaps it is somewhere wrong.
Of course, your application will run faster on standard java, consume less memory, and you can make the architecture exactly the way you need it. But this is on the condition that you have an unlimited number of man-hours, and every day you write such nonsense as an application for 100k clients. Otherwise, it will take a lot of time and iterations, and in the end it will still get a little worse than how it is implemented in modern JEE servers. All misses will emerge only under real load, and the application architecture will have to be redone more than once.
Non-blocking input appeared in java.nio, which means that you do not need to start a stream for each new connection. One thread can serve several hundred and more connections without calling the frequent context switch and not loading memory. But this is not a panacea, this approach has a significant drawback: any delay in the flow affects requests in the queue. In some cases, the “one connection - one stream” approach may be preferable.
The Linux operating system requires a 2.6+ kernel to support epoll. For Solaris, java 8 and the -Djava.nio.channels.spi.Selector = sun.nio.ch.EventPortSelectorProvider parameter is needed for eventport support, otherwise / dev / poll, which is also not bad.
')
The more different pools to create for different operations, the easier it will be to scale the system and manage the load, but the more there will be overhead costs for servicing threads, transferring requests between them, and synchronizing.
At a minimum, there should be one pool that takes the connections accepted by the list and performs all actions with them; its size should be set to the maximum that the system can handle. Since in case of filling it, incoming connections will not be accepted. But it must be necessarily limited to prevent DOS from increasing the load or attack. It should not contain blocking operations, all blocking operations should be carried out in separate streams.
For example: A pool of 100 threads with an average execution time of 5ms can serve 20k requests per second. And if they have to wait for a response from the external service 50ms, the number of requests processed will be reduced to 1.8k. If the service call is not needed for all requests, it is better to come up with asynchronous processing.
Pools do not need to be created immediately with maximum, it is better to increase them as the load grows, and to decrease with a decrease in load. Since the threads, even idle, consume memory and slow down the entire system. Flow in java == flow in OS, too much pool size over CPU flow leads to performance degradation.
The point of using pools is to save time on setting up a connection and controlling the number of connections. If you write the pool yourself, and do not use ready in the case of JEE, you need to carefully set the timings. If the connection is expected to take too long, this leads to a delay in the execution flow and an increase in the size of the corresponding pool. When re-creating a pool in case of a failure, you need to make the timing to repeat the connection attempt, it may be worth making it progressive, otherwise you can accidentally make a DDoS attack on the server.
This applies not only to the database, but to any other external resource, even http.
They are needed for interaction between threads, in the implementation it is some LinkedList or LinkedBlockingQueue. You should not make them too large, since they consume a lot of memory and it may happen that the threads do not have time to process requests from the queue during tcp or http timeout. At the same time, they should be large enough to smooth out short-term surges in load. The approximate value is 10 times the size of the processing pool.
You should not use JSP for a highly loaded system, even with a cache it will be slower than servlets. To be able to scale the system horizontally without problems, you need to make services without storing the state on the server.
It is clear that you need to log the minimum necessary information. Important point: logging should not lead to system failure, unless there are special requirements for this. That is, if it was not possible to write the log, we write about it in System.err and work further, and do not complete the stream with the operation. The log can be rotated in size, in time, but it should not be infinite. Fortunately, all the java libraries for logging do a great job with this.
It is recommended to wrap debug logging and output in something like this, so as not to waste extra time:
To assess the load, it is important in real-time to look at the sizes of the heap, thread pools, connection pools to the database and other resources, sessions. It is also necessary to have historical data for evaluation. You can monitor via jcontrol or similar functionality, but this is not very convenient. It is better to collect and write these metrics to the database with a separate software or independently. The number of metrics should be minimal for assessing the state, as it also requires system resources, with a large number of connections, this can be noticeable.
For a normal JEE server, you should not use your own streams and thread pools. It is better to use server tools, even if they are not so convenient. The fact is that the server monitors its threads and catches deadlock, but it will not know anything about yours.
If you have to write your own, then you need to track deadlock and just lock and force them to complete. In the case of a large number of threads deadlock occurs in the most unexpected places.
JEE platform
Of course, your application will run faster on standard java, consume less memory, and you can make the architecture exactly the way you need it. But this is on the condition that you have an unlimited number of man-hours, and every day you write such nonsense as an application for 100k clients. Otherwise, it will take a lot of time and iterations, and in the end it will still get a little worse than how it is implemented in modern JEE servers. All misses will emerge only under real load, and the application architecture will have to be redone more than once.
Java.nio for web
Non-blocking input appeared in java.nio, which means that you do not need to start a stream for each new connection. One thread can serve several hundred and more connections without calling the frequent context switch and not loading memory. But this is not a panacea, this approach has a significant drawback: any delay in the flow affects requests in the queue. In some cases, the “one connection - one stream” approach may be preferable.
The Linux operating system requires a 2.6+ kernel to support epoll. For Solaris, java 8 and the -Djava.nio.channels.spi.Selector = sun.nio.ch.EventPortSelectorProvider parameter is needed for eventport support, otherwise / dev / poll, which is also not bad.
')
Thread pools
The more different pools to create for different operations, the easier it will be to scale the system and manage the load, but the more there will be overhead costs for servicing threads, transferring requests between them, and synchronizing.
At a minimum, there should be one pool that takes the connections accepted by the list and performs all actions with them; its size should be set to the maximum that the system can handle. Since in case of filling it, incoming connections will not be accepted. But it must be necessarily limited to prevent DOS from increasing the load or attack. It should not contain blocking operations, all blocking operations should be carried out in separate streams.
For example: A pool of 100 threads with an average execution time of 5ms can serve 20k requests per second. And if they have to wait for a response from the external service 50ms, the number of requests processed will be reduced to 1.8k. If the service call is not needed for all requests, it is better to come up with asynchronous processing.
Pools do not need to be created immediately with maximum, it is better to increase them as the load grows, and to decrease with a decrease in load. Since the threads, even idle, consume memory and slow down the entire system. Flow in java == flow in OS, too much pool size over CPU flow leads to performance degradation.
Database Connection Pooling
The point of using pools is to save time on setting up a connection and controlling the number of connections. If you write the pool yourself, and do not use ready in the case of JEE, you need to carefully set the timings. If the connection is expected to take too long, this leads to a delay in the execution flow and an increase in the size of the corresponding pool. When re-creating a pool in case of a failure, you need to make the timing to repeat the connection attempt, it may be worth making it progressive, otherwise you can accidentally make a DDoS attack on the server.
This applies not only to the database, but to any other external resource, even http.
Queues
They are needed for interaction between threads, in the implementation it is some LinkedList or LinkedBlockingQueue. You should not make them too large, since they consume a lot of memory and it may happen that the threads do not have time to process requests from the queue during tcp or http timeout. At the same time, they should be large enough to smooth out short-term surges in load. The approximate value is 10 times the size of the processing pool.
Jsp
You should not use JSP for a highly loaded system, even with a cache it will be slower than servlets. To be able to scale the system horizontally without problems, you need to make services without storing the state on the server.
Logging
It is clear that you need to log the minimum necessary information. Important point: logging should not lead to system failure, unless there are special requirements for this. That is, if it was not possible to write the log, we write about it in System.err and work further, and do not complete the stream with the operation. The log can be rotated in size, in time, but it should not be infinite. Fortunately, all the java libraries for logging do a great job with this.
It is recommended to wrap debug logging and output in something like this, so as not to waste extra time:
if(log.isDebugEnabled()){ StringBuilder sb = new StringBuilder(); sb.append(arg0); sb.append(arg1); log.debug(sb.toString()); } Java monitoring
To assess the load, it is important in real-time to look at the sizes of the heap, thread pools, connection pools to the database and other resources, sessions. It is also necessary to have historical data for evaluation. You can monitor via jcontrol or similar functionality, but this is not very convenient. It is better to collect and write these metrics to the database with a separate software or independently. The number of metrics should be minimal for assessing the state, as it also requires system resources, with a large number of connections, this can be noticeable.
Deadlock threads
For a normal JEE server, you should not use your own streams and thread pools. It is better to use server tools, even if they are not so convenient. The fact is that the server monitors its threads and catches deadlock, but it will not know anything about yours.
If you have to write your own, then you need to track deadlock and just lock and force them to complete. In the case of a large number of threads deadlock occurs in the most unexpected places.
Source: https://habr.com/ru/post/270269/
All Articles