Microservices. Good, bad, evil
Luxoft Training invites you to get acquainted with the translation of the article by Sander Hugendorn “Microservices. Good, bad, evil.
SANDER HUGENDORN (NETHERLANDS)
Mentor, trainer, software architect, programmer, speaker and writer.
Sander has been working in the field of software development for more than 30 years; he wrote his first commercial program at 18 years old in Pascal.
')
Collaborates with major IT consulting companies for 20 years. Currently working in Capgemini. Numerous clients from different countries highly appreciate it as an “activator” of innovations in software development.
He is the author of the world-famous best-selling book "This is Agile", as well as many books on UML and Agile. He is currently working on two new books on software architecture, patterns and code, and on Agile anti-patterns.
Sander is a member of the Microsoft Visual Studio Advisory Board; member of the Advisory Board @Portunity (MDA provider), the editorial boards of Software Release Magazine and Tijdschrift voor IT Management, and the development expert at Computable magazine. Information about the training of sander in Moscow .
In 1988, when I first started working at a software company, the world was quite simple. We had a textual development environment, a database - integrated and viewed by cursors, and we created a completely new administrative system, covering all that is possible. It took us five years to complete the project, mainly because the client periodically changed his or her opinion and each change could violate the code elsewhere in the system. Unit testing has not yet been invented, and testing was conducted by end users. In production.
Enough for the monoliths. In 1994, I joined a company that developed desktop applications — the world wide web was only a few years old, and web applications had not yet been invented. We used a great tool called PowerBuilder, and now we had two components to worry about: a desktop application and a database on the server. The applications we created were used by company departments, and sometimes even by the entire company. They were not very complex, but not very scalable. Well, we had fun while the client-server paradigm continued to exist.
The world became more complex in the mid-90s. Companies were craving web applications that would work on the Intranet to get rid of desktop hosting. And applications had to serve several departments, and sometimes even go beyond the boundaries of the company. A new paradigm has been established - component-oriented development , also known as CBD. She promised us reuse, scalability, flexibility, and the ability to extract code (usually written in COBOL). We began to break our systems into large functional parts and tried very hard to get these components to communicate with each other. Java was invented, and everyone suddenly wanted to write Java code (obviously, some still want it). The components worked on incredible technologies, such as application servers and CORBA (see on Wikipedia what this means to impress colleagues). Good old days, object request brokers !
Creepy Thing - Object Request Broker Architecture
At that time, I worked at a large international bank, trying to create a methodology for component-oriented development. Even with the well-armed team of Andersen consultants, it took us three years to write this damn thing. As a result, both the paradigm and the technology turned out to be too complicated for writing decent and well-functioning programs. It just did not work.
At that moment, in the early years of the 21st century, I thought that we had got rid of distributed software development, and began to create web applications. It seems that everyone bravely ignored the first law of distribution of Martin Fowler objects - not to distribute objects. Gradually, we moved on to the next paradigm of distributed computing, repackaging the promises of component-oriented development into an updated set of technologies. Now we have begun to conduct business process modeling (BPM) and implement these processes on the enterprise service bus (ESB), and the components provide services. We were in an era of service-oriented architecture known as SOA.
After CBD, SOA seemed easier. Until the components - manufacturers - were connected to the company's service bus, we figured out how to scale up scalable and flexible systems. Now we had much smaller components that we could extract from existing systems (written not only in COBOL, but also in PowerBuilder, .NET and Java). The necessary books on the design patterns were written, and the world was ready to get down to business. This time we have to cope!
Enterprise Service Bus
This time I worked at an international transportation company, and we created software around SAP middleware, delivering tools for both the ESB and BPM. Now we needed not only Java and .NET developers, we also had middleware developers and SAP consultants. And although Agile was proposed to speed up the development (I know this is an incorrect argument), the projects were still too slow, moreover, when all the pieces of the puzzle fell into place, we began to realize that integration testing and deployment of new releases the day is more complicated.
I hope you will forgive me for such a long and intricate introduction to the topic of microservices. You might think: “why do we need another article on the topic of microservices, isn't there enough literature on this topic already?”. In general, yes, enough. But if you look closely at the stream of articles that can be found on the Internet, most of them describe only the benefits and capabilities of microservices (hallelujah), some of them describe few famous examples of innovators (Netflix, Amazon, and Netflix, and Amazon, and Netflix ...). And only a few articles actually dig a little deeper, and those, as a rule, consist of summing up the technologies that are used in the implementation of microservices. It's only the beginning.
And here it does not hurt a little history. Interestingly, the benefits and opportunities of the predecessors of microservices are still with us. Microservices seem to promise scalable and flexible systems based on small components that can be easily deployed independently, and thus help to choose the best technology option for the component. That is the same promises that we bought with CBD and SOA in the past. There is nothing new here, but this does not mean that microservices are not worthy of close examination.
So why is this time different? What will make microservices a victorious paradigm, if the predecessors were obviously not? What is special about them? As a developer, I’m certainly in awe of microservices, but I was delighted (more or less) when CBD and SOA appeared.
There are differences. For the first time, we have all the necessary technologies for creating architectures of this type. All the intricate and complex middleware has disappeared, and we rely solely on very simple and long-existing web protocols and technologies. Just compare REST and CORBA. In addition, we understand deployment better by learning how to carry out continuous integration, unit testing, and even continuous delivery. These differences show that this time we can succeed.
However, some skepticism cannot be avoided. Ten years ago, we also strongly believed that a service-oriented architecture would be technologically possible, solve all our problems, and we can quickly create reliable, reusable things. The fact that we believe that technology is ready is not a very strong argument. At the same time, the world has also become more complex.
Over the past year, I have worked with a company that is moving away from its mainframe (too expensive) and a number of old Java monoliths (too large and complex to support). The timing of implementation also play an important role. The IT industry must support the introduction of a new product within a few months. Therefore, we decided to be fashionable and chose microservices. Here is my summary of the “good, bad, and evil” aspects of microservice architecture.
Let's start with the good aspects. We create small components, each of which provides from two to six services. Good examples of such small components are the PDF component, which only generates PDF from a data template, and the Customer component, which allows users to search for existing customers. These components have the right size: the amount of code, the right size for understanding and documenting, as well as for testing and deployment.
Good
Our team began to evolve into a set of small groups that develop, implement and maintain individual components. We did not insist on it, but over time, small groups themselves took up work on a specific component and began to feel responsible for it.
After we made the general scheme of the basic architecture of our microservices, we established a number of principles. We create not only small components, but also small single-purpose web applications. Applications can talk to other applications and components. The components themselves manage their own stability and storage, and can also communicate with other components. Applications do not speak directly to the repositories. Components do not speak with repositories. For us, these principles work.
Microservices fulfill some of their promises. You can choose the necessary technologies and mechanisms of sustainability for each component. Some components are stored in relational databases (DB2 or SQLServer), others are stored in document databases (in our case, MongoDB). Here comes the fashionable term "polyglot persistence".
We also decided that each application and component will have its own domain model. We directly use the principles and patterns of domain-specific design. We have domain objects, value objects, pivot roots, vaults, and factories. Since our components are small, domain models are fairly simple and, therefore, easy to maintain.
And although we have long gone to the testing of our components and services, from Fitnesse to handwritten tests, we now turn to writing tests with testers in SoapUI. We carry out testing both separately and during assembly. We had to learn to understand REST, but we already did it. And testers really like what happened.
When we started our journey with microservices, the team I was in was in jet mode - tell us what to program and how, and we will do it. I think you should not explain how this affects the motivation of team members. However, for microservices there is no ready “recipe book” or a given architecture. These things simply do not exist. This means that we constantly needed to solve the microservice puzzle, from discovering how to design microservices (we used smart usage scenarios), how to use REST interfaces, how to work with a completely new set of environments, and how to create a completely new way to deploy components. It is this puzzle that makes the work on this architecture interesting. We learn every day.
But, as always, there is a downside. When you choose the path of microservices - for whatever reason you might consider it useful - you need to understand that all this is very new. Just think: if you are not working on Netflix or Amazon, then do you know someone who has actively deployed microservices in production under full load? Who can you ask?
Bad
You must understand that you really have to dig deep. You have to learn a lot by yourself. There are no standards yet. You will understand that any decisions you make about techniques, protocols, frameworks, and tools are likely to be temporary. When you study daily, new, better options become available or necessary, and you will have to change your decisions. So if you are looking for a ready-made IKEA designer for the correct implementation of microservices, you may have to wait another five to seven years. Just wait for the large companies, they will join soon enough, because the business is profitable.
In terms of design, you will have to start thinking differently. Designing small components is not as simple as it seems. What component size is correct? Yes, he has one business goal, no doubt, but how do you define the boundaries of your component? At what point do you decide to divide a working component into two or more separate components? We faced a number of such problems during the past year. And although we shared the existing components, there are no hard and fast rules when this should be done. We made decisions based on intuition, usually when we could not understand the structure of the component, or when we understood that it performs several business functions. This becomes even more difficult if you “cut off” components from large systems. In this case, you usually need to cut a lot of "wiring", and at the same time you will need to ensure the stable operation of the system. In addition, you will need a sufficiently large amount of knowledge about the subject area in order to carry out the componentification of existing systems.
In general, we found that the components are not as stable as we assumed. Sometimes we combined the components, but much more often we broke them up into smaller ones to ensure reuse and a shorter time to market. For example, we have isolated the Q & A component from the Product component, and now it delivers survey forms not only about products, but also for other purposes. And recently we have created a new component that deals exclusively with checking and storing the completed questionnaires.
You will also have to find answers to many technical questions. What is component architecture? Is the application also a component? Or a less obvious question, if you think about using REST as your communication protocol - you probably think: how exactly does REST function? What does “service interface correspond to REST conditions” mean? What exit codes to use and when? How to implement error handling at consumers, if something is not properly processed by one of your services? REST is not as simple as it seems. You will need to spend a lot of time and effort to find your method of working with service interfaces. We found out that in order for services to be called more or less unified, we better create a small framework that makes requests, and also deals with answers and errors. Thus, the connection is the same for each request, and if we need to change the main protocol (in our case, JAX-RS), then we will need to change it only in one place.
And this leads me to the next problem. Yes, microservices fulfill the promise that you can select the best technologies for each component. We recognize this in our projects. Some of our components use Hibernate, some MongoDB connector, some are based on additional frameworks, such as Dozer for mapping, or use some kind of PDF-generating framework. But with additional frameworks comes the need for additional knowledge. In this company, we can easily create about a hundred small components, maybe even more. If even a quarter of them use their own specific frameworks, in the end we will have from twenty-five to thirty different frameworks (did I mention that we work in Java?). We will need to know them all. And even worse, all their versions (if they are, of course, not "dead"), too.
In addition, there is a need to standardize parts of the code that you write. Freedom of technology is great, but if each component is literally implemented differently, then as a result you will have a virtually unsupported code base, especially since, most likely, no one follows all the code written. I highly recommend making sure that there is consistency between the components in your code base, which you can and should unify. Think about the components of the user interface (grids, buttons, pop-up windows, error messages), checks (domain models), how to communicate with databases and how requests from services will be formulated. In addition, although I am categorically opposed to a split domain model (please do not choose this path), in your domain models there will be elements that you may have to separate. We, for example, share a number of enumerations and value objects, such as CustomerId and IBAN .
If you are like us, your code will fall into a set of libraries (or framework, if you wish), which will be reused by your components. We learned that with each new release of this home-grown framework, we ultimately refactor parts of the code for our components. I rewrote the interface of our testing framework last week, which was necessary to get rid of the state it was maintaining (components do not have to keep state for reasons of scalability), and I don’t really want to build it back into the system when I return to work on Monday. Most of our components use it, and their code may not compile. I want to say that it’s good to have a homebrew framework. At a certain level of discipline, it will help make the code more understandable and a little more uniform, but you will have to convince yourself that you need to devote yourself to it and release its new versions.
So what about really unpleasant moments of microservices? Let's start with the deployment pipelines. One of the promises of microservices is that microservices can and should be individually deployed and released. If you are used to having one development and deployment pipeline for one system that you are creating or expanding, then you will probably like it. With microservices, you create separate pipelines for individual components.
Evil
Release the first version of the component is not so difficult. We started with a simple Jenkins pipeline, but now we are exploring TeamCity. We have four different environments. One for development, one for testing, one for adoption, plus, of course, the production environment. Now that we gradually began to release our components, most of which have their own databases, we realized that we could not do this without the support and cooperation of the operating team. We expect to gradually evolve into a continuous supply mode, with operational activities integrated into the team. Now we already have enough problems in order to get support from the operating team. At the moment, they are accustomed to the quarterly releases of the whole system and certainly have no desire to switch to individual component deployment.
Another cause for concern is version control. If it is already quite difficult to control the versions of several interacting applications, then what about hundreds of applications and components, each of which relies on a bunch of others to deliver the necessary services? Of course, your services are all independent, as the microservice paradigm promises. But only being together, your services become a system. How to avoid that all your beautiful microservice architecture does not become a versioned hell, which seems to be a direct descendant of the hell DLL?
Honestly, at the moment I can not give you good advice on this topic, but a sincere warning will also be useful to you. We started with a simple three-digit numbering scheme (such as 1.2.5). The number on the right changes when minor errors are eliminated. The average number changes when small new functions are added to the component, and the left one changes when we release a new version of the component with major changes in the interface. Not that we encourage regular change of interfaces, but it happens.
In addition, we test our services at build time using both coded tests and tests that we create in SoapUI. We also document requirements in smart usage scenarios and in the domain models of our applications and components using UML. I am sure that in the future we will need to take more precautions to keep our system in a “healthy” state, for example, add Swagger to document coded services, but it is too early to judge.
Too early to judge? Yes, we follow the path of microservices for about a year. And I still do not understand whether we are going up the stairs to heaven or on the way to hell. I believe that, as in the case of historical predecessors, we will find ourselves somewhere in the middle, although I really believe that we have the technology to make this paradigm work. And I mean not only Netflix, Amazon, or any fashionable mobile company, but also the average midsize companies that we work with.
But will it be appropriate? Will it reduce the time of implementation? Will all the wonderful things the paradigm promises us provide? Honestly, I still do not know - despite .., or even because of all the hype surrounding microservices. I noticed that given the complexity surrounding microservices, it takes a long time to launch the first services, and we just passed this line. A few weeks ago, we drank beer with Sam Newman, the author of the book Building Microservices. Sam confirmed my observations with his own examples and called this the hockey stick model.
You need to do a lot before you are ready to release your first service. Consider the infrastructure, figure out how to properly apply REST, set up deployment pipelines and, above all, change the way you think in software development. But as soon as the first service is up and running, new ones will appear faster and faster.
So, if I learned something in the last year, it was patience - this word was not in my dictionary before. Do not attempt to apply organization standards if they do not already exist. Understand the process. Allow yourself to learn. Do everything gradually. Just try to do things a little better than yesterday. And as always - have fun!
The master class of Sander Hugendorn will be held on September 25 in Moscow and will be devoted to the design, development, testing and deployment of microservice architecture.
SANDER HUGENDORN (NETHERLANDS)
Mentor, trainer, software architect, programmer, speaker and writer.
Sander has been working in the field of software development for more than 30 years; he wrote his first commercial program at 18 years old in Pascal.
')
Collaborates with major IT consulting companies for 20 years. Currently working in Capgemini. Numerous clients from different countries highly appreciate it as an “activator” of innovations in software development.
He is the author of the world-famous best-selling book "This is Agile", as well as many books on UML and Agile. He is currently working on two new books on software architecture, patterns and code, and on Agile anti-patterns.
Sander is a member of the Microsoft Visual Studio Advisory Board; member of the Advisory Board @Portunity (MDA provider), the editorial boards of Software Release Magazine and Tijdschrift voor IT Management, and the development expert at Computable magazine. Information about the training of sander in Moscow .
Microservices. Good, bad, evil
In 1988, when I first started working at a software company, the world was quite simple. We had a textual development environment, a database - integrated and viewed by cursors, and we created a completely new administrative system, covering all that is possible. It took us five years to complete the project, mainly because the client periodically changed his or her opinion and each change could violate the code elsewhere in the system. Unit testing has not yet been invented, and testing was conducted by end users. In production.
Enough for the monoliths. In 1994, I joined a company that developed desktop applications — the world wide web was only a few years old, and web applications had not yet been invented. We used a great tool called PowerBuilder, and now we had two components to worry about: a desktop application and a database on the server. The applications we created were used by company departments, and sometimes even by the entire company. They were not very complex, but not very scalable. Well, we had fun while the client-server paradigm continued to exist.
Component-oriented development
The world became more complex in the mid-90s. Companies were craving web applications that would work on the Intranet to get rid of desktop hosting. And applications had to serve several departments, and sometimes even go beyond the boundaries of the company. A new paradigm has been established - component-oriented development , also known as CBD. She promised us reuse, scalability, flexibility, and the ability to extract code (usually written in COBOL). We began to break our systems into large functional parts and tried very hard to get these components to communicate with each other. Java was invented, and everyone suddenly wanted to write Java code (obviously, some still want it). The components worked on incredible technologies, such as application servers and CORBA (see on Wikipedia what this means to impress colleagues). Good old days, object request brokers !
Creepy Thing - Object Request Broker Architecture
At that time, I worked at a large international bank, trying to create a methodology for component-oriented development. Even with the well-armed team of Andersen consultants, it took us three years to write this damn thing. As a result, both the paradigm and the technology turned out to be too complicated for writing decent and well-functioning programs. It just did not work.
Service Oriented Architecture
At that moment, in the early years of the 21st century, I thought that we had got rid of distributed software development, and began to create web applications. It seems that everyone bravely ignored the first law of distribution of Martin Fowler objects - not to distribute objects. Gradually, we moved on to the next paradigm of distributed computing, repackaging the promises of component-oriented development into an updated set of technologies. Now we have begun to conduct business process modeling (BPM) and implement these processes on the enterprise service bus (ESB), and the components provide services. We were in an era of service-oriented architecture known as SOA.
After CBD, SOA seemed easier. Until the components - manufacturers - were connected to the company's service bus, we figured out how to scale up scalable and flexible systems. Now we had much smaller components that we could extract from existing systems (written not only in COBOL, but also in PowerBuilder, .NET and Java). The necessary books on the design patterns were written, and the world was ready to get down to business. This time we have to cope!
Enterprise Service Bus
This time I worked at an international transportation company, and we created software around SAP middleware, delivering tools for both the ESB and BPM. Now we needed not only Java and .NET developers, we also had middleware developers and SAP consultants. And although Agile was proposed to speed up the development (I know this is an incorrect argument), the projects were still too slow, moreover, when all the pieces of the puzzle fell into place, we began to realize that integration testing and deployment of new releases the day is more complicated.
Finally: microservices!
I hope you will forgive me for such a long and intricate introduction to the topic of microservices. You might think: “why do we need another article on the topic of microservices, isn't there enough literature on this topic already?”. In general, yes, enough. But if you look closely at the stream of articles that can be found on the Internet, most of them describe only the benefits and capabilities of microservices (hallelujah), some of them describe few famous examples of innovators (Netflix, Amazon, and Netflix, and Amazon, and Netflix ...). And only a few articles actually dig a little deeper, and those, as a rule, consist of summing up the technologies that are used in the implementation of microservices. It's only the beginning.
And here it does not hurt a little history. Interestingly, the benefits and opportunities of the predecessors of microservices are still with us. Microservices seem to promise scalable and flexible systems based on small components that can be easily deployed independently, and thus help to choose the best technology option for the component. That is the same promises that we bought with CBD and SOA in the past. There is nothing new here, but this does not mean that microservices are not worthy of close examination.
Is there any difference this time?
So why is this time different? What will make microservices a victorious paradigm, if the predecessors were obviously not? What is special about them? As a developer, I’m certainly in awe of microservices, but I was delighted (more or less) when CBD and SOA appeared.
There are differences. For the first time, we have all the necessary technologies for creating architectures of this type. All the intricate and complex middleware has disappeared, and we rely solely on very simple and long-existing web protocols and technologies. Just compare REST and CORBA. In addition, we understand deployment better by learning how to carry out continuous integration, unit testing, and even continuous delivery. These differences show that this time we can succeed.
However, some skepticism cannot be avoided. Ten years ago, we also strongly believed that a service-oriented architecture would be technologically possible, solve all our problems, and we can quickly create reliable, reusable things. The fact that we believe that technology is ready is not a very strong argument. At the same time, the world has also become more complex.
Over the past year, I have worked with a company that is moving away from its mainframe (too expensive) and a number of old Java monoliths (too large and complex to support). The timing of implementation also play an important role. The IT industry must support the introduction of a new product within a few months. Therefore, we decided to be fashionable and chose microservices. Here is my summary of the “good, bad, and evil” aspects of microservice architecture.
Good ones
Let's start with the good aspects. We create small components, each of which provides from two to six services. Good examples of such small components are the PDF component, which only generates PDF from a data template, and the Customer component, which allows users to search for existing customers. These components have the right size: the amount of code, the right size for understanding and documenting, as well as for testing and deployment.
Good
Our team began to evolve into a set of small groups that develop, implement and maintain individual components. We did not insist on it, but over time, small groups themselves took up work on a specific component and began to feel responsible for it.
After we made the general scheme of the basic architecture of our microservices, we established a number of principles. We create not only small components, but also small single-purpose web applications. Applications can talk to other applications and components. The components themselves manage their own stability and storage, and can also communicate with other components. Applications do not speak directly to the repositories. Components do not speak with repositories. For us, these principles work.
Microservices fulfill some of their promises. You can choose the necessary technologies and mechanisms of sustainability for each component. Some components are stored in relational databases (DB2 or SQLServer), others are stored in document databases (in our case, MongoDB). Here comes the fashionable term "polyglot persistence".
We also decided that each application and component will have its own domain model. We directly use the principles and patterns of domain-specific design. We have domain objects, value objects, pivot roots, vaults, and factories. Since our components are small, domain models are fairly simple and, therefore, easy to maintain.
And although we have long gone to the testing of our components and services, from Fitnesse to handwritten tests, we now turn to writing tests with testers in SoapUI. We carry out testing both separately and during assembly. We had to learn to understand REST, but we already did it. And testers really like what happened.
When we started our journey with microservices, the team I was in was in jet mode - tell us what to program and how, and we will do it. I think you should not explain how this affects the motivation of team members. However, for microservices there is no ready “recipe book” or a given architecture. These things simply do not exist. This means that we constantly needed to solve the microservice puzzle, from discovering how to design microservices (we used smart usage scenarios), how to use REST interfaces, how to work with a completely new set of environments, and how to create a completely new way to deploy components. It is this puzzle that makes the work on this architecture interesting. We learn every day.
The bad
But, as always, there is a downside. When you choose the path of microservices - for whatever reason you might consider it useful - you need to understand that all this is very new. Just think: if you are not working on Netflix or Amazon, then do you know someone who has actively deployed microservices in production under full load? Who can you ask?
Bad
You must understand that you really have to dig deep. You have to learn a lot by yourself. There are no standards yet. You will understand that any decisions you make about techniques, protocols, frameworks, and tools are likely to be temporary. When you study daily, new, better options become available or necessary, and you will have to change your decisions. So if you are looking for a ready-made IKEA designer for the correct implementation of microservices, you may have to wait another five to seven years. Just wait for the large companies, they will join soon enough, because the business is profitable.
In terms of design, you will have to start thinking differently. Designing small components is not as simple as it seems. What component size is correct? Yes, he has one business goal, no doubt, but how do you define the boundaries of your component? At what point do you decide to divide a working component into two or more separate components? We faced a number of such problems during the past year. And although we shared the existing components, there are no hard and fast rules when this should be done. We made decisions based on intuition, usually when we could not understand the structure of the component, or when we understood that it performs several business functions. This becomes even more difficult if you “cut off” components from large systems. In this case, you usually need to cut a lot of "wiring", and at the same time you will need to ensure the stable operation of the system. In addition, you will need a sufficiently large amount of knowledge about the subject area in order to carry out the componentification of existing systems.
In general, we found that the components are not as stable as we assumed. Sometimes we combined the components, but much more often we broke them up into smaller ones to ensure reuse and a shorter time to market. For example, we have isolated the Q & A component from the Product component, and now it delivers survey forms not only about products, but also for other purposes. And recently we have created a new component that deals exclusively with checking and storing the completed questionnaires.
You will also have to find answers to many technical questions. What is component architecture? Is the application also a component? Or a less obvious question, if you think about using REST as your communication protocol - you probably think: how exactly does REST function? What does “service interface correspond to REST conditions” mean? What exit codes to use and when? How to implement error handling at consumers, if something is not properly processed by one of your services? REST is not as simple as it seems. You will need to spend a lot of time and effort to find your method of working with service interfaces. We found out that in order for services to be called more or less unified, we better create a small framework that makes requests, and also deals with answers and errors. Thus, the connection is the same for each request, and if we need to change the main protocol (in our case, JAX-RS), then we will need to change it only in one place.
And this leads me to the next problem. Yes, microservices fulfill the promise that you can select the best technologies for each component. We recognize this in our projects. Some of our components use Hibernate, some MongoDB connector, some are based on additional frameworks, such as Dozer for mapping, or use some kind of PDF-generating framework. But with additional frameworks comes the need for additional knowledge. In this company, we can easily create about a hundred small components, maybe even more. If even a quarter of them use their own specific frameworks, in the end we will have from twenty-five to thirty different frameworks (did I mention that we work in Java?). We will need to know them all. And even worse, all their versions (if they are, of course, not "dead"), too.
In addition, there is a need to standardize parts of the code that you write. Freedom of technology is great, but if each component is literally implemented differently, then as a result you will have a virtually unsupported code base, especially since, most likely, no one follows all the code written. I highly recommend making sure that there is consistency between the components in your code base, which you can and should unify. Think about the components of the user interface (grids, buttons, pop-up windows, error messages), checks (domain models), how to communicate with databases and how requests from services will be formulated. In addition, although I am categorically opposed to a split domain model (please do not choose this path), in your domain models there will be elements that you may have to separate. We, for example, share a number of enumerations and value objects, such as CustomerId and IBAN .
If you are like us, your code will fall into a set of libraries (or framework, if you wish), which will be reused by your components. We learned that with each new release of this home-grown framework, we ultimately refactor parts of the code for our components. I rewrote the interface of our testing framework last week, which was necessary to get rid of the state it was maintaining (components do not have to keep state for reasons of scalability), and I don’t really want to build it back into the system when I return to work on Monday. Most of our components use it, and their code may not compile. I want to say that it’s good to have a homebrew framework. At a certain level of discipline, it will help make the code more understandable and a little more uniform, but you will have to convince yourself that you need to devote yourself to it and release its new versions.
Evil
So what about really unpleasant moments of microservices? Let's start with the deployment pipelines. One of the promises of microservices is that microservices can and should be individually deployed and released. If you are used to having one development and deployment pipeline for one system that you are creating or expanding, then you will probably like it. With microservices, you create separate pipelines for individual components.
Evil
Release the first version of the component is not so difficult. We started with a simple Jenkins pipeline, but now we are exploring TeamCity. We have four different environments. One for development, one for testing, one for adoption, plus, of course, the production environment. Now that we gradually began to release our components, most of which have their own databases, we realized that we could not do this without the support and cooperation of the operating team. We expect to gradually evolve into a continuous supply mode, with operational activities integrated into the team. Now we already have enough problems in order to get support from the operating team. At the moment, they are accustomed to the quarterly releases of the whole system and certainly have no desire to switch to individual component deployment.
Another cause for concern is version control. If it is already quite difficult to control the versions of several interacting applications, then what about hundreds of applications and components, each of which relies on a bunch of others to deliver the necessary services? Of course, your services are all independent, as the microservice paradigm promises. But only being together, your services become a system. How to avoid that all your beautiful microservice architecture does not become a versioned hell, which seems to be a direct descendant of the hell DLL?
Honestly, at the moment I can not give you good advice on this topic, but a sincere warning will also be useful to you. We started with a simple three-digit numbering scheme (such as 1.2.5). The number on the right changes when minor errors are eliminated. The average number changes when small new functions are added to the component, and the left one changes when we release a new version of the component with major changes in the interface. Not that we encourage regular change of interfaces, but it happens.
In addition, we test our services at build time using both coded tests and tests that we create in SoapUI. We also document requirements in smart usage scenarios and in the domain models of our applications and components using UML. I am sure that in the future we will need to take more precautions to keep our system in a “healthy” state, for example, add Swagger to document coded services, but it is too early to judge.
Model "hockey stick"
Too early to judge? Yes, we follow the path of microservices for about a year. And I still do not understand whether we are going up the stairs to heaven or on the way to hell. I believe that, as in the case of historical predecessors, we will find ourselves somewhere in the middle, although I really believe that we have the technology to make this paradigm work. And I mean not only Netflix, Amazon, or any fashionable mobile company, but also the average midsize companies that we work with.
But will it be appropriate? Will it reduce the time of implementation? Will all the wonderful things the paradigm promises us provide? Honestly, I still do not know - despite .., or even because of all the hype surrounding microservices. I noticed that given the complexity surrounding microservices, it takes a long time to launch the first services, and we just passed this line. A few weeks ago, we drank beer with Sam Newman, the author of the book Building Microservices. Sam confirmed my observations with his own examples and called this the hockey stick model.
You need to do a lot before you are ready to release your first service. Consider the infrastructure, figure out how to properly apply REST, set up deployment pipelines and, above all, change the way you think in software development. But as soon as the first service is up and running, new ones will appear faster and faster.
Be patient
So, if I learned something in the last year, it was patience - this word was not in my dictionary before. Do not attempt to apply organization standards if they do not already exist. Understand the process. Allow yourself to learn. Do everything gradually. Just try to do things a little better than yesterday. And as always - have fun!
The master class of Sander Hugendorn will be held on September 25 in Moscow and will be devoted to the design, development, testing and deployment of microservice architecture.
Source: https://habr.com/ru/post/258257/
All Articles