Distributed systems. Design patterns. Book Review
Hello colleagues. Today we publish the translation of the next review from the site of Ben Neidel - this site will certainly interest you in the original. This time we will talk about the book " Distributed Systems. Design Patterns ", which complements the book " We Master Kubernetes ", which we published earlier this year and, in essence, is analogous to GoF for the design of distributed systems.
Enjoy reading.
In total over the weekend I read the book “Distributed Systems. Design Patterns ”, written by Brendan Burns . I really liked the book, although I have to admit that I expected to find some other material in it. So, in the description of the book containers are mentioned only in passing. At the same time, although the patterns described in the book are applicable not only for containerization, almost all the templates described here are given exactly in the container context, and then are considered in the Kubernetes deployment pipeline. It turned out to be the way. I'm just starting to get acquainted with container-oriented development and deployment, and the discussion of architectural patterns from just such a “container” point of view was a revelation to me. This approach helped me to get a good idea of ​​how small services are correctly distinguished in the microservice landscape, each of which implements a specific feature.
')
The author of the book Brendan Burns is the co-founder of the open-source project Kubernetes. Therefore, it is not surprising that all its application examples are built around the deployment of containers based on the Kubernetes configuration files. At the time of reading the book, I was slightly versed in Docker and did not know anything about Kubernetes. Thus, I tried to understand the “purpose” of the Kubernetes configuration files by simply reading them. However, I believe that the book will be more useful if the reader has at least some experience with Kubernetes.
Reading the book, I could not get rid of the associations with the work of " Patterns of integration of enterprise applications " by Gregor Hop and Bobby Wolfe. Many of the patterns considered by Burns are very reminiscent of message queuing patterns discussed by Hop and Wolfe. In fact, I should note that many of the patterns are even named the same in both books (for example, Scatter / Gather). This is logical, since the theme of both books is the division of complex monolithic systems into sets of small, tightly fitted reusable services. I think it can even be argued that Burns sets out specific approaches that will be useful when implementing Service-Producers and Service-Consumers participating in the workflow based on messages, the very ones described in the book "Enterprise Application Integration Patterns".
Again, I believe that the parallels traced between these two books testify to the strength of the design patterns. Both in how well they lead us to proven solutions, and in that design patterns facilitate technical communication, because they allow us to reason in a common language.
At the same time, one of the approaches I liked most is that described in the book Distributed Systems. Design Patterns ”just associated with the consumption of a message queue. Burns advises not to force the working container to pull out messages directly from the system (just as it does in the Simple Queue Service (SQS) system from Amazon), but to create an “ambassador” (Ambassador pattern). Such an “ambassador” container will be deployed with the working container and provide a generic API for manipulating the queues. The Ambassador container allows you to abstract the implementation details associated with the permanent storage of items in the message queue, so that the working container is completely independent of any specific technological solutions.
“Ambassador of the working queue source container” is just one example of a cross-cutting theme running through the entire book with a red thread: use sets of small containers so that each individual container can concentrate on a specific task and make it as reusable as possible. Burns explores this concept at the level of individual containers, talking about parameterizing containers using command line arguments and environment variables. Then he rises to a higher level, talks about the multi-container side-car and ambassador patterns in a single-node context. Finally, it shows how to create powerful microservice architecture using multi-container patterns.
I think that Burns managed to perfectly articulate the “dream” of the entire microservice landscape:
(p. 79-80 in Russian translation)
I’m talking about a “dream” because, as Burns himself admits, it’s difficult to design a microservice system and design its architecture. And monitoring and debugging such a system is much more difficult than monitoring and debugging a monolithic counterpart. Just so I can easily agree. From my own limited experience with microservices, I confirm that shared services quickly become strongly connected, and “reliable contracts” between services quickly turn into a moving target, undergoing new and new changes.
In conclusion, I would like to touch upon the concept of FaaS - “functions as services”. Systems like Amazon's Lambda Service give me mixed feelings. In an extremely abstract sense, I like such systems, but I have no idea how they manifest themselves in a particular application. Burns touches on FaaS in Part II on "Design patterns for service systems," but unfortunately, it does not fully clarify the Faas problem for me.
I really liked the fact that Burns recommends using FaaS to solve only a certain subset of known problems:
(p. 135 in Russian translation)
Moreover, many thanks to him for mentioning the difficulties encountered when using FaaS:
(p. 136-137 in Russian translation)
Again, I was surprised by the fact that most FaaS systems are not too good for solving problems that require active processing:
(p. 139-140 in Russian translation)
As a result, I did not understand: when is it better to use "functions as services"? I note that further Burns briefly describes work with event-oriented short-term tasks that do not heavily load the processor, such as two-factor authentication (2FA). However, given that we are talking about small short-term tasks with low costs, the question arises: why should they be scaled independently? Why not just include these features in another container service, closely related to the first?
I hope to better deal with these issues when I move to using FaaS-technologies in practice.
Apart from some confusion with FaaS, I really liked the book. Reads quickly, easily perceived. It reminds once again how great potential there is in weakening the connection between components at all levels of application development. Finally, now it will be much easier for me to maintain a conversation with colleagues on topics such as “Sidecar-containers.”
Enjoy reading.
In total over the weekend I read the book “Distributed Systems. Design Patterns ”, written by Brendan Burns . I really liked the book, although I have to admit that I expected to find some other material in it. So, in the description of the book containers are mentioned only in passing. At the same time, although the patterns described in the book are applicable not only for containerization, almost all the templates described here are given exactly in the container context, and then are considered in the Kubernetes deployment pipeline. It turned out to be the way. I'm just starting to get acquainted with container-oriented development and deployment, and the discussion of architectural patterns from just such a “container” point of view was a revelation to me. This approach helped me to get a good idea of ​​how small services are correctly distinguished in the microservice landscape, each of which implements a specific feature.
')
The author of the book Brendan Burns is the co-founder of the open-source project Kubernetes. Therefore, it is not surprising that all its application examples are built around the deployment of containers based on the Kubernetes configuration files. At the time of reading the book, I was slightly versed in Docker and did not know anything about Kubernetes. Thus, I tried to understand the “purpose” of the Kubernetes configuration files by simply reading them. However, I believe that the book will be more useful if the reader has at least some experience with Kubernetes.
Reading the book, I could not get rid of the associations with the work of " Patterns of integration of enterprise applications " by Gregor Hop and Bobby Wolfe. Many of the patterns considered by Burns are very reminiscent of message queuing patterns discussed by Hop and Wolfe. In fact, I should note that many of the patterns are even named the same in both books (for example, Scatter / Gather). This is logical, since the theme of both books is the division of complex monolithic systems into sets of small, tightly fitted reusable services. I think it can even be argued that Burns sets out specific approaches that will be useful when implementing Service-Producers and Service-Consumers participating in the workflow based on messages, the very ones described in the book "Enterprise Application Integration Patterns".
Again, I believe that the parallels traced between these two books testify to the strength of the design patterns. Both in how well they lead us to proven solutions, and in that design patterns facilitate technical communication, because they allow us to reason in a common language.
At the same time, one of the approaches I liked most is that described in the book Distributed Systems. Design Patterns ”just associated with the consumption of a message queue. Burns advises not to force the working container to pull out messages directly from the system (just as it does in the Simple Queue Service (SQS) system from Amazon), but to create an “ambassador” (Ambassador pattern). Such an “ambassador” container will be deployed with the working container and provide a generic API for manipulating the queues. The Ambassador container allows you to abstract the implementation details associated with the permanent storage of items in the message queue, so that the working container is completely independent of any specific technological solutions.
“Ambassador of the working queue source container” is just one example of a cross-cutting theme running through the entire book with a red thread: use sets of small containers so that each individual container can concentrate on a specific task and make it as reusable as possible. Burns explores this concept at the level of individual containers, talking about parameterizing containers using command line arguments and environment variables. Then he rises to a higher level, talks about the multi-container side-car and ambassador patterns in a single-node context. Finally, it shows how to create powerful microservice architecture using multi-container patterns.
I think that Burns managed to perfectly articulate the “dream” of the entire microservice landscape:
The microservice approach has many advantages, many of which are related to reliability and flexibility. Microservices divide the application into small parts, each of which is responsible for providing a specific service. By narrowing the scope of services, each service is able to develop and maintain a team that can be fed with two pizzas.
Reducing the size of the team also reduces the cost of maintaining its activities.
In addition, the emergence of a formal interface between microservices weakens the interdependence of commands and establishes a reliable contract between services. Such a formal contract reduces the need for close synchronization of commands, because the team providing the API understands the extent to which it is necessary to ensure compatibility, and the team consuming the API can rely on stable service without worrying about the implementation details of the consumed service. This decomposition allows teams to independently control the pace of development and the release schedule of new versions, which enables them to perform iterations, thereby improving the service code.
Finally, the division into microservices improves scalability. Since each component is allocated to a separate service, it can be scaled independently of the rest
(p. 79-80 in Russian translation)
I’m talking about a “dream” because, as Burns himself admits, it’s difficult to design a microservice system and design its architecture. And monitoring and debugging such a system is much more difficult than monitoring and debugging a monolithic counterpart. Just so I can easily agree. From my own limited experience with microservices, I confirm that shared services quickly become strongly connected, and “reliable contracts” between services quickly turn into a moving target, undergoing new and new changes.
In conclusion, I would like to touch upon the concept of FaaS - “functions as services”. Systems like Amazon's Lambda Service give me mixed feelings. In an extremely abstract sense, I like such systems, but I have no idea how they manifest themselves in a particular application. Burns touches on FaaS in Part II on "Design patterns for service systems," but unfortunately, it does not fully clarify the Faas problem for me.
I really liked the fact that Burns recommends using FaaS to solve only a certain subset of known problems:
As is the case with other tools for developing distributed systems, there may be a desire to use a particular solution (for example, event-oriented processing) as a universal tool. The truth is that a particular solution usually solves particular problems. In a certain context, it will be a powerful tool, but pulling it by the ears to solve common problems generates complex, fragile architectures.
(p. 135 in Russian translation)
Moreover, many thanks to him for mentioning the difficulties encountered when using FaaS:
As mentioned in the previous section, developing systems using the FaaS approach forces the system components to be loosely coupled. Each function is independent by definition. All interaction is carried out over the network. Instances of functions do not have their own memory, therefore, they require a shared storage to store the state. Forcing a weakening of the connectivity of system elements can increase the flexibility and speed of service development, but at the same time can significantly complicate its support.
In particular, it is rather difficult to see the comprehensive structure of the service, to determine how the functions integrate with each other, to understand what went wrong and why in the event of a failure. In addition, the query-oriented and serverless nature of the functions means that some problems will be difficult to detect.
(p. 136-137 in Russian translation)
Again, I was surprised by the fact that most FaaS systems are not too good for solving problems that require active processing:
... besides, due to serverless implementation of services, the execution time of the function instance is usually limited. This means that the FaaS approach is usually not suitable for applications requiring long background processing. (P. 138 in Russian translation)
Finally, I was pleased with the remark that FaaS is becoming economically inexpedient if it is impossible to ensure long uninterrupted operation of the processor:
But if you have so many requests that the function is constantly active, you are probably overpaying for the number of requests being processed.
... as the service grows, the number of requests processed grows to such a level that the processor is busy processing them all the time. At this point, the charge for the number of requests begins to become unprofitable. The cost per unit of processor time of cloud virtual machines decreases as kernels are added, as well as due to the reservation of resources and discounts for long-term use. The cost of paying the number of requests usually increases with the number of requests.
(p. 139-140 in Russian translation)
As a result, I did not understand: when is it better to use "functions as services"? I note that further Burns briefly describes work with event-oriented short-term tasks that do not heavily load the processor, such as two-factor authentication (2FA). However, given that we are talking about small short-term tasks with low costs, the question arises: why should they be scaled independently? Why not just include these features in another container service, closely related to the first?
I hope to better deal with these issues when I move to using FaaS-technologies in practice.
Apart from some confusion with FaaS, I really liked the book. Reads quickly, easily perceived. It reminds once again how great potential there is in weakening the connection between components at all levels of application development. Finally, now it will be much easier for me to maintain a conversation with colleagues on topics such as “Sidecar-containers.”
Source: https://habr.com/ru/post/442514/
All Articles