Common code problems in microservices
Hello!
Recently, at the PGConf conference in Moscow, one of the speakers demonstrated a “microservice” architecture, mentioning in passing that all microservices inherit from one common base class. Although there was no explanation for the implementation, it seemed that in this company the term “microservices” is not understood in the same way as the classics would have taught us. Today we will deal with one of the interesting problems - what can be the common code in microservices and whether it can be at all.
What is microservice? This is a separate application. Not a module, not a process, not something that is simply separately deployed, but a full-fledged, real, separate application. It has its own main function, its own repository in the gita, its tests, its API, its web server, its README file, its own database, its own version, its own developers.
Like containers, microservices began to be used when the computing power of HW and the reliability of networks reached such a level that you can afford the challenge of a function that lasts 100 times longer than before, you can afford the memory consumption 100 times higher, you can afford the luxury to settle each “grandmother” not just into a separate “apartment”, but into a separate “house”. As with any architectural solutions, microservice architecture once again sacrifices performance, winning in the maintainability of the code for developers. But since the person and the speed of his reaction remained the same, the systems continue to meet the requirements.
')
Why split into separate applications? Because we distribute part of the complexity of the system at the level of the system architecture. The programming process is generally speaking a phased “biting off” of a large initial “piece of complexity”, and decomposition (into classes, modules, functions, and in our case into entire applications) is the implementation of part of this complexity as a structure. When we broke the system into microservices, we made an architectural decision (good or not), which developers will no longer need to take in the implementation of specific parts of the functionality. It is known that this particular microservice is responsible for sending emails, and this one is for authorization, it’s already started, so all my new features “fall” on this pattern without discussion.
A key aspect of microservices is weak connectivity. Microservices should be independent of the word "absolutely." They do not have common data structures, and each microservice can / should have its own architecture, technologies, method of assembly (and so on). By definition. Because it is independent applications. Changes in the code of one microservice should not affect the others in any way, unless the API is affected. If I have N microservices written in Java, then there should not be any limiting factors in order not to write N + 1-th microservice in Python, if suddenly it is beneficial for some reason. They are loosely coupled, and therefore a developer who starts working with a specific microservice:
a) Very sensitively monitors its API, because it is the only component visible from the outside;
b) Feels completely free in refactoring;
c) Understands the purpose of microservice (we recall SRP here) and implements a new function accordingly;
d) Select the persistence method that is most suitable;
etc.
All this is good and sounds logical and coherent, like many ideologies and theories (and here the theoretical ideologist puts an end and goes to dinner), but we practice with you. The code has to be written at all on the site martinfowler.com . And sooner or later we are faced with the fact that all microservices:
and make it identical.
And at some point an ideologue-architect comes to work in the morning and discovers that at night a “library” has appeared in the system - a new repository with common code that is used in many microservices. Should an architect be horrified?
It depends.
To correctly assess the situation, you should return to the main idea: microservices are a set of independent applications that interact with each other through the (network) API. In this we see the main advantage and simplicity of architecture. And we do not want to lose this advantage under any circumstances. Does the common code that is placed in the “library” hinder this? Consider examples.
1. The class “user” lives in the library (or some other business entity).
Unfortunately, any business logic placed in a shared library will have this effect. Libraries of a common code tend to grow, eventually a logical “tumor” is formed in the middle of the system that does not belong to any specific microservice, and the architecture crashes. The “center of logical gravity” of the system begins to move into the repo with a common code, and we get a hellish mixture of monolith and microservices, but we do not need to go there at all.
2. The library placed the code for parsing the message format.
The parser of messages, or an improved logger, or a wrapped client for sending data to RabbitMQ is sort of like helpers, auxiliary components. They are on a par with standard libraries from NuGet, Maven or NPM. The developer of microservice is always king, he decides whether to use the standard library, or make his own new code, or use the code from the general library of helpers. How it will be more convenient for him, because he writes a SEPARATE AND INDEPENDENT APPENDIX. Can a specific helper develop? Maybe he will probably have a version. Let the developer refer in his service to a specific version, no one forces to update the service, when updating the helpers, this is a question to who supports the service.
3. Java interface, abstract base class, treit.
The team that starts working on a new product lays the foundation for the architecture and has the greatest influence on what trends the product will have. If the principles of SRP, successful decomposition, low connectivity, etc. are initially incorporated into the system, then it has a chance to continue to develop correctly. If not, then the centrifugal acceleration of the “time factors” (another team, little time, urgent patches, lack of documentation) will push the system further to the curb faster than it seems.
The question of a common code in microservices remains difficult, because it is associated with some sort of trade-off: we weigh what will be more profitable for us in the future - the degree of independence of microservices, fewer repetitions in the code, qualification of engineers, simplicity of the system, etc. Each time there are reflections and discussions that can lead to different specific architectural solutions. Nevertheless, let us summarize some recommendations:
Recommendation 0: Do not call microservices any piece that is divided into independently existing pieces. Not every table with columns is a matrix, let's use the terms correctly.
Recommendation 1: It is highly desirable that microservices do not have a common code at all.
Recommendation 2: If there is a common code, let it be an aggregate (library) of optional “helpers”. The developer of the service decides whether to use them or write your own code.
Recommendation 3: Under no circumstances should there be any business logic in the common code. All business logic is encapsulated in microservices.
Recommendation 4: Let the common code library be framed as a typical package (NuGet, Maven, NPM, etc), with the possibility of versioning (or, even better, several separate packages).
Recommendation 5: The “center of logical gravity” of the system should always remain in microservices themselves, and not in the general code.
Recommendation 6: If you decided to write in the format of microservices, then beforehand accept the fact that the code between them will sometimes be duplicated. To some extent, our natural “DRY instinct” should be suppressed.
Thank you for your attention and successful microservices.
Recently, at the PGConf conference in Moscow, one of the speakers demonstrated a “microservice” architecture, mentioning in passing that all microservices inherit from one common base class. Although there was no explanation for the implementation, it seemed that in this company the term “microservices” is not understood in the same way as the classics would have taught us. Today we will deal with one of the interesting problems - what can be the common code in microservices and whether it can be at all.
What is microservice? This is a separate application. Not a module, not a process, not something that is simply separately deployed, but a full-fledged, real, separate application. It has its own main function, its own repository in the gita, its tests, its API, its web server, its README file, its own database, its own version, its own developers.
Like containers, microservices began to be used when the computing power of HW and the reliability of networks reached such a level that you can afford the challenge of a function that lasts 100 times longer than before, you can afford the memory consumption 100 times higher, you can afford the luxury to settle each “grandmother” not just into a separate “apartment”, but into a separate “house”. As with any architectural solutions, microservice architecture once again sacrifices performance, winning in the maintainability of the code for developers. But since the person and the speed of his reaction remained the same, the systems continue to meet the requirements.
')
Why split into separate applications? Because we distribute part of the complexity of the system at the level of the system architecture. The programming process is generally speaking a phased “biting off” of a large initial “piece of complexity”, and decomposition (into classes, modules, functions, and in our case into entire applications) is the implementation of part of this complexity as a structure. When we broke the system into microservices, we made an architectural decision (good or not), which developers will no longer need to take in the implementation of specific parts of the functionality. It is known that this particular microservice is responsible for sending emails, and this one is for authorization, it’s already started, so all my new features “fall” on this pattern without discussion.
A key aspect of microservices is weak connectivity. Microservices should be independent of the word "absolutely." They do not have common data structures, and each microservice can / should have its own architecture, technologies, method of assembly (and so on). By definition. Because it is independent applications. Changes in the code of one microservice should not affect the others in any way, unless the API is affected. If I have N microservices written in Java, then there should not be any limiting factors in order not to write N + 1-th microservice in Python, if suddenly it is beneficial for some reason. They are loosely coupled, and therefore a developer who starts working with a specific microservice:
a) Very sensitively monitors its API, because it is the only component visible from the outside;
b) Feels completely free in refactoring;
c) Understands the purpose of microservice (we recall SRP here) and implements a new function accordingly;
d) Select the persistence method that is most suitable;
etc.
All this is good and sounds logical and coherent, like many ideologies and theories (and here the theoretical ideologist puts an end and goes to dinner), but we practice with you. The code has to be written at all on the site martinfowler.com . And sooner or later we are faced with the fact that all microservices:
- log information;
- contain authorization;
- appeal to message brokers;
- return correct error messages;
- should somehow understand the common entities in the system, if any;
- should work with the general format (and protocol) of messages;
and make it identical.
And at some point an ideologue-architect comes to work in the morning and discovers that at night a “library” has appeared in the system - a new repository with common code that is used in many microservices. Should an architect be horrified?
It depends.
To correctly assess the situation, you should return to the main idea: microservices are a set of independent applications that interact with each other through the (network) API. In this we see the main advantage and simplicity of architecture. And we do not want to lose this advantage under any circumstances. Does the common code that is placed in the “library” hinder this? Consider examples.
1. The class “user” lives in the library (or some other business entity).
- those. a business entity is not encapsulated in one microservice, but is spread over different ones (otherwise, why should it be placed in a shared code library?);
- those. Microservices become connected through this business entity, changing the logic of working with the entity will affect several microservices;
- it is bad, very bad, it is not microservices at all, although it is not “big ball of mud”, but very quickly the team’s worldview will lead to “big ball of distributed mud”;
- but after all, microservices in the system work with the same concepts, and concepts are often entities, or just structures with fields, what to do? To read DDD, it is exactly about how to encapsulate entities inside microservices so that they do not “fly” through the API.
Unfortunately, any business logic placed in a shared library will have this effect. Libraries of a common code tend to grow, eventually a logical “tumor” is formed in the middle of the system that does not belong to any specific microservice, and the architecture crashes. The “center of logical gravity” of the system begins to move into the repo with a common code, and we get a hellish mixture of monolith and microservices, but we do not need to go there at all.
2. The library placed the code for parsing the message format.
- The code is most likely in Java if all microservices are written in Java;
- If tomorrow I write a service in Python, I can’t use a parser, but it’s not at all a problem, I’ll write a Python version;
- The key point: if I write a new microservice in Java, am I obligated to use this parser? Yes, probably not. Perhaps that is not required, although I, as a developer of microservice, it can be very useful. Well, as if I had found something useful in the Maven Repository.
The parser of messages, or an improved logger, or a wrapped client for sending data to RabbitMQ is sort of like helpers, auxiliary components. They are on a par with standard libraries from NuGet, Maven or NPM. The developer of microservice is always king, he decides whether to use the standard library, or make his own new code, or use the code from the general library of helpers. How it will be more convenient for him, because he writes a SEPARATE AND INDEPENDENT APPENDIX. Can a specific helper develop? Maybe he will probably have a version. Let the developer refer in his service to a specific version, no one forces to update the service, when updating the helpers, this is a question to who supports the service.
3. Java interface, abstract base class, treit.
- Or another thing from the category of "torn out a piece of code";
- Those. I am here, independent and independent, and a piece of my liver lies somewhere else;
- This is where microservices are connected at the code level, so we will not recommend it;
- At the initial stages, this probably will not bring any tangible problems, but the essence of architectural design is, after all, a guarantee of comfort (or discomfort) for years to come.
The team that starts working on a new product lays the foundation for the architecture and has the greatest influence on what trends the product will have. If the principles of SRP, successful decomposition, low connectivity, etc. are initially incorporated into the system, then it has a chance to continue to develop correctly. If not, then the centrifugal acceleration of the “time factors” (another team, little time, urgent patches, lack of documentation) will push the system further to the curb faster than it seems.
The question of a common code in microservices remains difficult, because it is associated with some sort of trade-off: we weigh what will be more profitable for us in the future - the degree of independence of microservices, fewer repetitions in the code, qualification of engineers, simplicity of the system, etc. Each time there are reflections and discussions that can lead to different specific architectural solutions. Nevertheless, let us summarize some recommendations:
Recommendation 0: Do not call microservices any piece that is divided into independently existing pieces. Not every table with columns is a matrix, let's use the terms correctly.
Recommendation 1: It is highly desirable that microservices do not have a common code at all.
Recommendation 2: If there is a common code, let it be an aggregate (library) of optional “helpers”. The developer of the service decides whether to use them or write your own code.
Recommendation 3: Under no circumstances should there be any business logic in the common code. All business logic is encapsulated in microservices.
Recommendation 4: Let the common code library be framed as a typical package (NuGet, Maven, NPM, etc), with the possibility of versioning (or, even better, several separate packages).
Recommendation 5: The “center of logical gravity” of the system should always remain in microservices themselves, and not in the general code.
Recommendation 6: If you decided to write in the format of microservices, then beforehand accept the fact that the code between them will sometimes be duplicated. To some extent, our natural “DRY instinct” should be suppressed.
Thank you for your attention and successful microservices.
Source: https://habr.com/ru/post/454100/
All Articles