Kernel test automation in microservice architecture
Hi, Habr! My name is Dmitry Himion, I lead the quality assurance department at Avito. Today I want to talk about testing automation in the framework of work with microservice architecture. What can we offer development in order to facilitate quality control? Read under the cut.
To my story was complete, let's start with the basics. If to simplify, microservice architecture is a way of organizing an application server. How does he work? In essence, this is simply a response from a service-oriented architecture to the emergence of a practice like DevOps. If the SOA does not regulate the size of the services and what they should do, then there are some speculative limitations within the microservice architecture. Microservice is an entity that contains one small functionality that it manages and provides some data to external services.
Sometimes information is given that microservice is 500 lines of code. But this is optional; the point is that these services are rather small and form the same business processes as the monolithic back-end, which works on many projects. Functionally - the same thing, the differences - in the organizational structure.
')
What are the problems here? There are a lot of services, their connectivity is increasing. This complicates testing: checking the changes of one microservice entails the need to raise a rather thick layer of infrastructure from the set of services with which the variable microservice interacts. At a certain point in the development of a project, the number of links and dependencies grows strongly and it becomes difficult to perform fast and isolated testing. We start working slower. Development is slowing. These and some other problems can be solved with the help of test automation.
To begin, consider the transition from the monolith to microservice architecture. In the case of a monolith in order to replace any piece in this system, we cannot roll it out separately. You need to re-build and completely update the back-end. It is not always rational and convenient. What happens during the transition to microservice architecture? We take a back-end and divide it into its components, dividing them according to their functionality. We determine the interaction between them, and it turns out a new system with the same front-end and the same databases. Microservices interact with each other and provide all the same business processes. For the user and system testing, everything remained the same, as before, the internal organization has changed.
What does this give? You can make a change in microservice X, which implements some of its functionality, and immediately roll it into work. It all sounds very good. But, as always, there is a nuance: you need to check how microservices interact with each other.
The interactions of microservices between themselves are realized through contracts. What does this mean for service-oriented architecture? A contract is some kind of agreement that service developers create for interaction between external users. The service developers themselves decide that you can request an “X” from them, and they will provide you with a “Y”. The service may provide external users with apples, tomatoes, plutonium, sell children's clothes or TVs - it does not declare a clear focus on functionality. Internal content is regulated only at the level of common sense. And accordingly, this service dictates to external users what they will receive and how it should be addressed.
If we increase the complexity of this task, when there are not dozens (10-20-50) of services, but hundreds (200, 400 ... 2000), then traditional, “classic” contracts will no longer work in a functional sense. And then there is the need to upgrade contracts for microservice architecture. A work template for developers has been developed, in which they and the “end users” of microservice are swapped — this approach is called consumer driven contacts. Now requests are made by external users. This can be thought of as this kind of conversation:
Thus, it turns out that users create contracts, set requirements and direct them to the provider that implements them. What is the plus? Since each service has a rather limited number of consumers, it is much easier to get three specifications from each one and implement them than to try to guess exactly which “apples” need to be provided in each specific case and what else they may need for these “fruits” in addition.
For this approach, we, as a quality assurance department, should provide a decent response in the field of testing so as not to slow down the process of fast rollout of microservices.
Without what you can not start working on the principles of CDC-testing? The first. We will not be able to help if we see that we do not follow the contract work process in our development. The second thing is more technical: this is a post-commit system (post-PR, if you will) of hooks, which handles this flow of communication between developers with the help of contracts and signals our testing system about updating, deleting, new ones. Accordingly, the corresponding tasks are put in Jira, so that the automatics can “digest” it all. Anything else can be added to this basic process - additional checks, process gadgets, but without control of changes in interaction contracts between microservices it will be quite difficult to live. Having completed these two points in some form, we can proceed to the implementation.
Our test automation system will be based on a solution such as PACT frameworks. What you need to know about them? This is the protocol interaction with the API: JSON over HTTP, there is nothing complicated about it. These solutions interact with our microservices and provide some additional functionality for isolation and testing organization. What more to say? I have seen how this is implemented in one form or another in seven programming languages ​​(Java, Javascript, Ruby, Python, Go, .NET, Swift). But if your not in this list, do not worry: you can take the base library and make your own bike, or write something similar to what has already been implemented.
What else awaits us in almost 100% of cases? The first is the mocking of external services. The problem is that it is difficult to maintain the relevance of how an external service behaves, and to maintain it in all stubs. How to solve this problem? It depends on you. Most likely, you will need to allocate more resources to this or limit test coverage. And second: sampled databases. It will be necessary to cut the combat databases in a lightweight form so that they contain all the necessary data for testing within our system. Then get a visual relevant result.
We go further. The first thing I immediately wanted to do in this project is informative logging. The representativeness of the test results is always necessary. The simple truth is: if the results cannot be read and understood, no one will look at them. I must say that the basic logging in the PACT frameworks is very poorly implemented.
I would prefer to put it in a separate module, make a wrapper and scoop what we need there. First of all, make a separation of errors according to the sources of their appearance. Further, all that you can implement, for example, solutions like Allure (a solution from Yandex for increasing the simplicity of analyzing test results). You can do anything, but the need for informative logs should be taken into account.
The next, perhaps, “captain” moment is the Config Reader. But as part of testing the microservice architecture, this can be a bit catchy thing. For Config Reader, we have two sources. The first is the PACT files, and the second is the State.
What is this .pact file? This is not some kind of encrypted binary, but a regular JSON file that has a specific structure. It stands out consumer, provider (ie, who exactly and with whom interacts within the framework of this contract and in what role). The following describes the interaction (this is done by the developers): I am a consumer, and I want from this service (provider) to give me “little green apples”; I'm waiting for such a response code, status, header, body, and so on. There is a description field - this is just a description, an occasion to remind the developers what was discussed in the contract, what was the meaning of it.
And the most interesting is the State Provider. What is it? In essence, this is the state in which the microservice to be tested should be at the time of accessing it for a specific test iteration, for a specific request. In the States, both SQL queries (or other mechanisms for bringing a service into a state) can be described, as well as the creation of some data in our sampled database. States is a complex module that can contain all kinds of entities that bring our service to the proper state.
It is important to note that the Suite runner appears here (see the diagram below). This is the entity that will be responsible for running and configuring tests in a form convenient for the developer / tester. It is possible not to write it, but I would nevertheless single out this moment, since it is difficult to foresee which tests will need to be run at one time or another on the project. As a result, we have the following test automation core under the microservice architecture:
Now the most important is implementation. We must provide this scheme with the requirements for the presence of the described contracts to developers. What do we get after?
We have information on each service: about what it issues and what it requests from its “neighbors”. Accordingly, with the help of our test automation system, with the help of PACT-files we isolate our microservice to ensure the isolation of its testing, regardless of external services with which it is integrated. And we provide states through mocks, stubs, sampled databases, or directly change the service in some way. We receive, accordingly, the isolated testing. Voila: we have the answer to the question of what to do with test automation during the transition from monolith to microservice architecture.
What should be considered when implementing? The first is the use of containerization and virtualization in the process of building / deploying microservice. And our test automation system is twisted into a container the same way. How you will ensure the interaction of microservice and the test automation system is not so important: as it is more convenient, so do it.
The second is the actualization of contract requirements files. There is such a problem that the requirements of consumers to the provider begin to accumulate, and the developers are doing only what they urgently need now. Here you need to set a task for developers at the level of test management or product management: for example, work with this “tail” an hour a week so that it does not grow. Of course, these tests are quick, they can take a few seconds. But if their number is measured in thousands, it will complicate the actualization of testing. And we, for our part, will receive information through the commit hook system and remove unnecessary autotests. In fact, the CDC is documentation-driven development, so to speak.
Third: it is necessary to introduce the state and data suites process of updating into this work. What is data suites? Developers in the development process write some basic interaction scenarios. For example: “I need such a request and such an answer”. And about everything else - in which frameworks it should exist, what values ​​of the parameters are possible, and what values ​​are not, they forget. It is necessary to check. And here we have to go in with our data-driven approach to testing, implementing it in our configurator: positive, borderline and negative tests, in order to ensure the fault tolerance of our micro-service architecture in production. Without this, it is unlikely to work, any step left / right - and an interaction error.
Accordingly, when new pact-files appear, through hook we get hacks and fill new / changed interactions with a large number of data sets in interactions, which is equal to a number of tests performed. We can fill and edit interactions manually, and we can make generators that will parameterize them and drive tests.
As for the States: if we take the automation of testing the web, then this is preconditions, setUps, FixTure; for microservices there is a problem - there is no ready mechanism: you need to think about how you will match the change of the name States in the PACT files in this module. The easiest mechanism is to use Aliases, in fact, KDT (keyword driven testing), speaking in the language of test automation. I am not ready to tell now about a more beautiful and elegant solution: I have not yet invented it.
If you like not to read more, but to watch a video, then this information is available in the report format by reference . And at the end of August we will have a thematic meeting for testing professionals in Avito. Let's talk about the vectors of development of automation systems in general, discuss the application of applied tools and their impact on changing the testing infrastructure. If interested - stay tuned to our page in FB , or write in the comments: let us know when there will be details about the event.
Instead of intro
".
Jonathan Bruce Postel, computer scientist
What is microservice architecture?
To my story was complete, let's start with the basics. If to simplify, microservice architecture is a way of organizing an application server. How does he work? In essence, this is simply a response from a service-oriented architecture to the emergence of a practice like DevOps. If the SOA does not regulate the size of the services and what they should do, then there are some speculative limitations within the microservice architecture. Microservice is an entity that contains one small functionality that it manages and provides some data to external services.
Sometimes information is given that microservice is 500 lines of code. But this is optional; the point is that these services are rather small and form the same business processes as the monolithic back-end, which works on many projects. Functionally - the same thing, the differences - in the organizational structure.
')
What are the problems here? There are a lot of services, their connectivity is increasing. This complicates testing: checking the changes of one microservice entails the need to raise a rather thick layer of infrastructure from the set of services with which the variable microservice interacts. At a certain point in the development of a project, the number of links and dependencies grows strongly and it becomes difficult to perform fast and isolated testing. We start working slower. Development is slowing. These and some other problems can be solved with the help of test automation.
Transition to microservices
To begin, consider the transition from the monolith to microservice architecture. In the case of a monolith in order to replace any piece in this system, we cannot roll it out separately. You need to re-build and completely update the back-end. It is not always rational and convenient. What happens during the transition to microservice architecture? We take a back-end and divide it into its components, dividing them according to their functionality. We determine the interaction between them, and it turns out a new system with the same front-end and the same databases. Microservices interact with each other and provide all the same business processes. For the user and system testing, everything remained the same, as before, the internal organization has changed.
What does this give? You can make a change in microservice X, which implements some of its functionality, and immediately roll it into work. It all sounds very good. But, as always, there is a nuance: you need to check how microservices interact with each other.
Contracts for microservice architecture
The interactions of microservices between themselves are realized through contracts. What does this mean for service-oriented architecture? A contract is some kind of agreement that service developers create for interaction between external users. The service developers themselves decide that you can request an “X” from them, and they will provide you with a “Y”. The service may provide external users with apples, tomatoes, plutonium, sell children's clothes or TVs - it does not declare a clear focus on functionality. Internal content is regulated only at the level of common sense. And accordingly, this service dictates to external users what they will receive and how it should be addressed.
If we increase the complexity of this task, when there are not dozens (10-20-50) of services, but hundreds (200, 400 ... 2000), then traditional, “classic” contracts will no longer work in a functional sense. And then there is the need to upgrade contracts for microservice architecture. A work template for developers has been developed, in which they and the “end users” of microservice are swapped — this approach is called consumer driven contacts. Now requests are made by external users. This can be thought of as this kind of conversation:
User 1: “I heard you deliver apples. I need small and green. ”
User 2: "And I need huge red apples."
User N: "And I need you to bring three tons of apples."
Thus, it turns out that users create contracts, set requirements and direct them to the provider that implements them. What is the plus? Since each service has a rather limited number of consumers, it is much easier to get three specifications from each one and implement them than to try to guess exactly which “apples” need to be provided in each specific case and what else they may need for these “fruits” in addition.
For this approach, we, as a quality assurance department, should provide a decent response in the field of testing so as not to slow down the process of fast rollout of microservices.
Predicates for CDC-testing (Consumer Driven Contracts)
Without what you can not start working on the principles of CDC-testing? The first. We will not be able to help if we see that we do not follow the contract work process in our development. The second thing is more technical: this is a post-commit system (post-PR, if you will) of hooks, which handles this flow of communication between developers with the help of contracts and signals our testing system about updating, deleting, new ones. Accordingly, the corresponding tasks are put in Jira, so that the automatics can “digest” it all. Anything else can be added to this basic process - additional checks, process gadgets, but without control of changes in interaction contracts between microservices it will be quite difficult to live. Having completed these two points in some form, we can proceed to the implementation.
Automation implementation under CDC
Our test automation system will be based on a solution such as PACT frameworks. What you need to know about them? This is the protocol interaction with the API: JSON over HTTP, there is nothing complicated about it. These solutions interact with our microservices and provide some additional functionality for isolation and testing organization. What more to say? I have seen how this is implemented in one form or another in seven programming languages ​​(Java, Javascript, Ruby, Python, Go, .NET, Swift). But if your not in this list, do not worry: you can take the base library and make your own bike, or write something similar to what has already been implemented.
What else awaits us in almost 100% of cases? The first is the mocking of external services. The problem is that it is difficult to maintain the relevance of how an external service behaves, and to maintain it in all stubs. How to solve this problem? It depends on you. Most likely, you will need to allocate more resources to this or limit test coverage. And second: sampled databases. It will be necessary to cut the combat databases in a lightweight form so that they contain all the necessary data for testing within our system. Then get a visual relevant result.
We go further. The first thing I immediately wanted to do in this project is informative logging. The representativeness of the test results is always necessary. The simple truth is: if the results cannot be read and understood, no one will look at them. I must say that the basic logging in the PACT frameworks is very poorly implemented.
I would prefer to put it in a separate module, make a wrapper and scoop what we need there. First of all, make a separation of errors according to the sources of their appearance. Further, all that you can implement, for example, solutions like Allure (a solution from Yandex for increasing the simplicity of analyzing test results). You can do anything, but the need for informative logs should be taken into account.
The next, perhaps, “captain” moment is the Config Reader. But as part of testing the microservice architecture, this can be a bit catchy thing. For Config Reader, we have two sources. The first is the PACT files, and the second is the State.
What is this .pact file? This is not some kind of encrypted binary, but a regular JSON file that has a specific structure. It stands out consumer, provider (ie, who exactly and with whom interacts within the framework of this contract and in what role). The following describes the interaction (this is done by the developers): I am a consumer, and I want from this service (provider) to give me “little green apples”; I'm waiting for such a response code, status, header, body, and so on. There is a description field - this is just a description, an occasion to remind the developers what was discussed in the contract, what was the meaning of it.
And the most interesting is the State Provider. What is it? In essence, this is the state in which the microservice to be tested should be at the time of accessing it for a specific test iteration, for a specific request. In the States, both SQL queries (or other mechanisms for bringing a service into a state) can be described, as well as the creation of some data in our sampled database. States is a complex module that can contain all kinds of entities that bring our service to the proper state.
It is important to note that the Suite runner appears here (see the diagram below). This is the entity that will be responsible for running and configuring tests in a form convenient for the developer / tester. It is possible not to write it, but I would nevertheless single out this moment, since it is difficult to foresee which tests will need to be run at one time or another on the project. As a result, we have the following test automation core under the microservice architecture:
Now the most important is implementation. We must provide this scheme with the requirements for the presence of the described contracts to developers. What do we get after?
We have information on each service: about what it issues and what it requests from its “neighbors”. Accordingly, with the help of our test automation system, with the help of PACT-files we isolate our microservice to ensure the isolation of its testing, regardless of external services with which it is integrated. And we provide states through mocks, stubs, sampled databases, or directly change the service in some way. We receive, accordingly, the isolated testing. Voila: we have the answer to the question of what to do with test automation during the transition from monolith to microservice architecture.
What to consider when implementing?
What should be considered when implementing? The first is the use of containerization and virtualization in the process of building / deploying microservice. And our test automation system is twisted into a container the same way. How you will ensure the interaction of microservice and the test automation system is not so important: as it is more convenient, so do it.
The second is the actualization of contract requirements files. There is such a problem that the requirements of consumers to the provider begin to accumulate, and the developers are doing only what they urgently need now. Here you need to set a task for developers at the level of test management or product management: for example, work with this “tail” an hour a week so that it does not grow. Of course, these tests are quick, they can take a few seconds. But if their number is measured in thousands, it will complicate the actualization of testing. And we, for our part, will receive information through the commit hook system and remove unnecessary autotests. In fact, the CDC is documentation-driven development, so to speak.
Third: it is necessary to introduce the state and data suites process of updating into this work. What is data suites? Developers in the development process write some basic interaction scenarios. For example: “I need such a request and such an answer”. And about everything else - in which frameworks it should exist, what values ​​of the parameters are possible, and what values ​​are not, they forget. It is necessary to check. And here we have to go in with our data-driven approach to testing, implementing it in our configurator: positive, borderline and negative tests, in order to ensure the fault tolerance of our micro-service architecture in production. Without this, it is unlikely to work, any step left / right - and an interaction error.
Accordingly, when new pact-files appear, through hook we get hacks and fill new / changed interactions with a large number of data sets in interactions, which is equal to a number of tests performed. We can fill and edit interactions manually, and we can make generators that will parameterize them and drive tests.
As for the States: if we take the automation of testing the web, then this is preconditions, setUps, FixTure; for microservices there is a problem - there is no ready mechanism: you need to think about how you will match the change of the name States in the PACT files in this module. The easiest mechanism is to use Aliases, in fact, KDT (keyword driven testing), speaking in the language of test automation. I am not ready to tell now about a more beautiful and elegant solution: I have not yet invented it.
Results
- As for post-commit-hooks, automation of monitoring, the emergence and modification, removal of contracts between services - all this needs to be automated, it is part of the system. I would say that this is a piece of the kernel that is at the repository level.
- CDC pattern for developers is required. This is a foundation, nothing will work without it.
- You will notice a decrease in the propensity of the system to the appearance of defects. In fact, we have all of its parts shrouded in isolation testing. We quickly localize defects, fix them quickly, we have a high test coverage. If defects appear at the system testing level, they are likely to be associated with a front-end or some kind of external services.
- You have the opportunity to escape from manual testing of changes. Key points: fast and isolated testing. Then we get a profit from the fact that we can well localize the problem or defect.
PS
If you like not to read more, but to watch a video, then this information is available in the report format by reference . And at the end of August we will have a thematic meeting for testing professionals in Avito. Let's talk about the vectors of development of automation systems in general, discuss the application of applied tools and their impact on changing the testing infrastructure. If interested - stay tuned to our page in FB , or write in the comments: let us know when there will be details about the event.
Source: https://habr.com/ru/post/333644/
All Articles