Scaling is simple. Part Two - Caching
In the previous part, we talked about the basic architectural principles for building scalable portals. Today let's talk about optimizing a properly built portal. So: the first type of optimization is local cache.
As I mentioned in the previous part , we are talking, first of all, about B2C portals. These portals have a common property: the predominance of reading requests over writers. And often, the predominance of 10-fold or more. It is clear why caching is such a promising tool.
')
Caches are an interesting and almost endless topic. Therefore, I will try to describe it rather shortly for the first round of optimization, without going into details about level 1 and 2 cache, distributed cache, etc. Since we need to cache the object, we will skip the write cache ( write cache).
When we start local caching, we usually have a choice: query or object cache (call or object caching). Query cache (or method cache) is a cache that can be used “outside”: it records the results of method calls (query), and we assume that the same queries cause the same answers. If you repeat the same request several times, you can, starting from the second time, skip complex and lengthy processing, and simply return the previous result. The advantage of such caches is that they do not depend on either the architecture or the domain, that is, they are integrated as a utility (for example, aspect) into the finished product, without much changing it. In contrast to this advantage, they have a number of disadvantages:
The situation with object caches is quite different: they are harder to integrate, but they are much more efficient. The ideal implementation of an object cache is a collection (a list or set depends on requirements), which contains all the objects managed by this service in their object form. Ideally, the service should be able to respond to each request with a cache and without accessing external persistence (for example, a database). Unfortunately, this 100% cacheability is rarely achievable, but where it can be applied, it is always cost-effective. The best example for 100% cache is user accounts. They are usually small (consist of id, mail, name, registration time, etc.) and are constantly required in different places of the application.
Another useful form of cache is the so-called zero-cache, or negative cache. In most cases, object caches are filled while the application is running, so that a so-called “cold start” and the creation of new objects in the database are possible, while several instances of the same service exist simultaneously. As a result, appeals to the application can "break through" through the cache and reach the base. In case such “penetration” will occur constantly, it can lead to base overload. Then all our caches, built with such difficulty, will lose all effectiveness and protective function. To combat this overload, there are negative caches that remember once unsuccessfully polled objects, thereby giving the service the next request to stop processing them at an early stage.
Another subcategory of caches is ExpiryCaches. They are based on the postulation that the object or its parts will not change their state for a certain period of time. A typical example of such an object is a user profile on an online dating site. If user A edits his profile, then for user B, who views user A’s profile, it does not matter whether he sees these changes after 5, 30 or 60 seconds (especially if these users do not contact). Thus, we can fix the state of user A’s profile for a time that is insignificant for a person, but long from the point of view of the machine, and work with the fixed version. In this way, we prevent the processing of a significant number of requests, which, with a high probability, would return an identical result (because users change the profile much less often than, for example, they search or respond to messages), and save resources. Paying for this savings is just a risk to show the update in the profile a bit later than we could. This technique with particular popularity is used on the client (web server) to save network traffic (to a database or service) and, accordingly, time. But this is not the only example when using Expiry Cache is advisable. Another use case is when, in the course of processing a request, we assume that the same object will be requested many times, but from places so remote from each other in the code that it is impossible or impractical to store its value in a variable. A popular technology for implementing such intermediate caches is ThreadLocal (Java).
ExpiryCache is nothing more than resource sharing (trade off): we sacrifice the speed of displaying changes for the sake of application performance.
In general, scaling applications (and especially portals) is the exchange of resources, which we have a lot of, for those that are not enough: for example, RAM on the CPU, network traffic or IO.
Caching is a good method for optimizing a single component or service, but the potential for such optimization is limited. At some point we will have to recognize that the limit on what one instance can handle has been reached, and we need to scale the components of our architecture (i.e., run multiple copies of them). This is the third part .
Part two. Caching
As I mentioned in the previous part , we are talking, first of all, about B2C portals. These portals have a common property: the predominance of reading requests over writers. And often, the predominance of 10-fold or more. It is clear why caching is such a promising tool.
')
Caches are an interesting and almost endless topic. Therefore, I will try to describe it rather shortly for the first round of optimization, without going into details about level 1 and 2 cache, distributed cache, etc. Since we need to cache the object, we will skip the write cache ( write cache).
When we start local caching, we usually have a choice: query or object cache (call or object caching). Query cache (or method cache) is a cache that can be used “outside”: it records the results of method calls (query), and we assume that the same queries cause the same answers. If you repeat the same request several times, you can, starting from the second time, skip complex and lengthy processing, and simply return the previous result. The advantage of such caches is that they do not depend on either the architecture or the domain, that is, they are integrated as a utility (for example, aspect) into the finished product, without much changing it. In contrast to this advantage, they have a number of disadvantages:
- They are inefficient when we have rich interfaces that have several methods of reading the same object. The reason is that they cache the path to the object, not the object itself.
- Their size depends not on the number of possible objects (results), but on the number of possible requests, and therefore they are usually heavier than alternatives. Their size and amount of necessary memory is hard to plan.
- They are ugly.
The situation with object caches is quite different: they are harder to integrate, but they are much more efficient. The ideal implementation of an object cache is a collection (a list or set depends on requirements), which contains all the objects managed by this service in their object form. Ideally, the service should be able to respond to each request with a cache and without accessing external persistence (for example, a database). Unfortunately, this 100% cacheability is rarely achievable, but where it can be applied, it is always cost-effective. The best example for 100% cache is user accounts. They are usually small (consist of id, mail, name, registration time, etc.) and are constantly required in different places of the application.
Cache what is not
Another useful form of cache is the so-called zero-cache, or negative cache. In most cases, object caches are filled while the application is running, so that a so-called “cold start” and the creation of new objects in the database are possible, while several instances of the same service exist simultaneously. As a result, appeals to the application can "break through" through the cache and reach the base. In case such “penetration” will occur constantly, it can lead to base overload. Then all our caches, built with such difficulty, will lose all effectiveness and protective function. To combat this overload, there are negative caches that remember once unsuccessfully polled objects, thereby giving the service the next request to stop processing them at an early stage.
Cache what changes
Another subcategory of caches is ExpiryCaches. They are based on the postulation that the object or its parts will not change their state for a certain period of time. A typical example of such an object is a user profile on an online dating site. If user A edits his profile, then for user B, who views user A’s profile, it does not matter whether he sees these changes after 5, 30 or 60 seconds (especially if these users do not contact). Thus, we can fix the state of user A’s profile for a time that is insignificant for a person, but long from the point of view of the machine, and work with the fixed version. In this way, we prevent the processing of a significant number of requests, which, with a high probability, would return an identical result (because users change the profile much less often than, for example, they search or respond to messages), and save resources. Paying for this savings is just a risk to show the update in the profile a bit later than we could. This technique with particular popularity is used on the client (web server) to save network traffic (to a database or service) and, accordingly, time. But this is not the only example when using Expiry Cache is advisable. Another use case is when, in the course of processing a request, we assume that the same object will be requested many times, but from places so remote from each other in the code that it is impossible or impractical to store its value in a variable. A popular technology for implementing such intermediate caches is ThreadLocal (Java).
ExpiryCache is nothing more than resource sharing (trade off): we sacrifice the speed of displaying changes for the sake of application performance.
In general, scaling applications (and especially portals) is the exchange of resources, which we have a lot of, for those that are not enough: for example, RAM on the CPU, network traffic or IO.
Caching is a good method for optimizing a single component or service, but the potential for such optimization is limited. At some point we will have to recognize that the limit on what one instance can handle has been reached, and we need to scale the components of our architecture (i.e., run multiple copies of them). This is the third part .
Source: https://habr.com/ru/post/209352/
All Articles