Horizontal scaling of small web-based Java applications (interview questions)
This topic was raised in the course of several (3+) interviews that I have completed over the past month and a half - in different variations, but about one. It would seem well-known things - but having collected all the answers and explanations that I gave (and found something later in Google), I decided to keep them not in my Google Drive, but to write a short review.
It was about small and typical Enterprise / Web applications in Java, which are written a lot (well, such, for 10-100 thousand customers, a million visits, etc.). Let it be a generalized dialogue in the form of questions and answers.
')
Q: Suppose you have an application (the most common is JSP, Spring, Hibernate, for example) deployed on Tomcat (Apache Tomcat) and you notice once that the server with Tomcat is 80% loaded on average. What to do?
A: We put several tomkats on separate servers in a parallel. They will still use one base on one server.
Q: But how will the user log on to your multiple servers?
A: We use load-balancer, for example, tomkats may have Apache (Apache httpd) with mod_proxy - it will distribute (proxy) requests coming to it between all of our tomkats.
Q: But it can happen that the user logs in on one volume, and the next request load-balancer will send to another, where the user is not logged in!
A: We are talking about how to organize a session. For example, we do sticky sessions (for example, when load-balancer adds a cookie to the request indicating what volume he is proxying this request - and all subsequent requests with this cookie are always sent to the same server. Thus, each individual user will work with only one server.
Q: And if this particular server falls?
A: User session will be lost. Therefore, it is better to use cache session storage. Tomkat "out of the box" can store them in memcached for example. That is, we add a line to the config and run memcached on a separate server - now all tomkats store sessions on it and if the user gets another server to another server, he will not notice this - the session will still work.
Q: What are the benefits of cache for sessions?
A: For example, you can deploy a new version of the application to only one of several tomkats, so let's say 25% of users see a new login page and will have time to give us suggestions if they don’t like it, i.e. they unwittingly work as beta testers :)
Q: But if the versions of the application use the base differently?
A: We can design database changes so as to maintain backward compatibility between two neighboring versions. That's not difficult. To add columns, for example, it is necessary together with the new version, but to delete unnecessary ones only with the next release.
Q: Well, now our base is becoming a bottleneck. What will we do with increasing load on it?
A: First of all, it is useful to make a cache between the base and the tomkats. Even earlier, we probably use the cache at the ORM level (for example, the second cache level in Hibernate). The general sense is that during the session the user uses a limited set of data, so it is convenient to cache them.
Q: Well, still, let's say, even the cache does not save us. How can I reduce the load on the base?
A: We have several paths. For example, it is possible to allocate a part of the database (some special pump table is allowed) to another database on a separate server, maybe even in NoSQL storage or some special cache. Of course, it is better to do this separation while designing :)
Q: What other ways are there? What are the solutions at the level of the database itself?
A: You can use sharding - in this case the tables are divided into several servers and access to the necessary one occurs, for example, in terms of the id-person. In some cases, you can immediately divide, say, transactions, transactions, electronic documents, etc. relating to one user since the user usually does not work with other people's documents - which means that all his data can be conveniently stored on a single server.
Q: What is the disadvantage of this approach?
A: With such tables it will be harder to work later - join with a table on multiple servers will obviously be less effective - in general, indexing, queries by criteria, etc. become more complicated. In general, the design itself is significantly more complicated.
Q: Well, do you know more options?
A: The easiest is to set up replication, for example, so that the database contains copies on several servers, one of which is used for recording and the rest for reading. These latter quickly synchronize their content during updates. It turns out that the total number of requests to the database is now distributed across several machines. Of course, it is useful when reading is more than writing.
Q: What further ways of scaling would you suggest?
A: For example, message queues. Let's say the user saves a new transaction - but we do not write it to the database ourselves. Instead, we send the message to the queue (say RabbitMQ) that such and such data should be saved. This message will be issued to one of several servers that process and save to the database. It is generally very easy to increase the number of such servers (using a distributed / replicated database or cache). However, the architecture itself at this level already requires more attention and reflection - perhaps even this is the moment when the application should be rewritten entirely :)
Q: Well, with this clearly, let's talk about something else ... (and here they can start talking about garbazh-collectors, or ask to write a binary search in an array - a test for lice - but this is not important)
Having shared my “observations” in interviews, I will of course be happy with additions, corrections, etc. which may be useful to me and other colleagues :)
It was about small and typical Enterprise / Web applications in Java, which are written a lot (well, such, for 10-100 thousand customers, a million visits, etc.). Let it be a generalized dialogue in the form of questions and answers.
')
Q: Suppose you have an application (the most common is JSP, Spring, Hibernate, for example) deployed on Tomcat (Apache Tomcat) and you notice once that the server with Tomcat is 80% loaded on average. What to do?
A: We put several tomkats on separate servers in a parallel. They will still use one base on one server.
Q: But how will the user log on to your multiple servers?
A: We use load-balancer, for example, tomkats may have Apache (Apache httpd) with mod_proxy - it will distribute (proxy) requests coming to it between all of our tomkats.
Q: But it can happen that the user logs in on one volume, and the next request load-balancer will send to another, where the user is not logged in!
A: We are talking about how to organize a session. For example, we do sticky sessions (for example, when load-balancer adds a cookie to the request indicating what volume he is proxying this request - and all subsequent requests with this cookie are always sent to the same server. Thus, each individual user will work with only one server.
Q: And if this particular server falls?
A: User session will be lost. Therefore, it is better to use cache session storage. Tomkat "out of the box" can store them in memcached for example. That is, we add a line to the config and run memcached on a separate server - now all tomkats store sessions on it and if the user gets another server to another server, he will not notice this - the session will still work.
Q: What are the benefits of cache for sessions?
A: For example, you can deploy a new version of the application to only one of several tomkats, so let's say 25% of users see a new login page and will have time to give us suggestions if they don’t like it, i.e. they unwittingly work as beta testers :)
Q: But if the versions of the application use the base differently?
A: We can design database changes so as to maintain backward compatibility between two neighboring versions. That's not difficult. To add columns, for example, it is necessary together with the new version, but to delete unnecessary ones only with the next release.
Q: Well, now our base is becoming a bottleneck. What will we do with increasing load on it?
A: First of all, it is useful to make a cache between the base and the tomkats. Even earlier, we probably use the cache at the ORM level (for example, the second cache level in Hibernate). The general sense is that during the session the user uses a limited set of data, so it is convenient to cache them.
Q: Well, still, let's say, even the cache does not save us. How can I reduce the load on the base?
A: We have several paths. For example, it is possible to allocate a part of the database (some special pump table is allowed) to another database on a separate server, maybe even in NoSQL storage or some special cache. Of course, it is better to do this separation while designing :)
Q: What other ways are there? What are the solutions at the level of the database itself?
A: You can use sharding - in this case the tables are divided into several servers and access to the necessary one occurs, for example, in terms of the id-person. In some cases, you can immediately divide, say, transactions, transactions, electronic documents, etc. relating to one user since the user usually does not work with other people's documents - which means that all his data can be conveniently stored on a single server.
Q: What is the disadvantage of this approach?
A: With such tables it will be harder to work later - join with a table on multiple servers will obviously be less effective - in general, indexing, queries by criteria, etc. become more complicated. In general, the design itself is significantly more complicated.
Q: Well, do you know more options?
A: The easiest is to set up replication, for example, so that the database contains copies on several servers, one of which is used for recording and the rest for reading. These latter quickly synchronize their content during updates. It turns out that the total number of requests to the database is now distributed across several machines. Of course, it is useful when reading is more than writing.
Q: What further ways of scaling would you suggest?
A: For example, message queues. Let's say the user saves a new transaction - but we do not write it to the database ourselves. Instead, we send the message to the queue (say RabbitMQ) that such and such data should be saved. This message will be issued to one of several servers that process and save to the database. It is generally very easy to increase the number of such servers (using a distributed / replicated database or cache). However, the architecture itself at this level already requires more attention and reflection - perhaps even this is the moment when the application should be rewritten entirely :)
Q: Well, with this clearly, let's talk about something else ... (and here they can start talking about garbazh-collectors, or ask to write a binary search in an array - a test for lice - but this is not important)
Having shared my “observations” in interviews, I will of course be happy with additions, corrections, etc. which may be useful to me and other colleagues :)
Source: https://habr.com/ru/post/220327/
All Articles