Automation of system integration testing
Hi, Habravchane!
I want to share with you personal experience in system integration testing. Our team is developing an integration layer through which all the systems in the bank are connected. We have a lot of tasks, there is not enough time, and the issue of integration testing has always been postponed.
How does integration testing work? The shortest answer is no way, although we have more than a hundred systems that interact through the Oracle Service Bus (OSB) integration bus. This product has an OSB Console tool that allows you to send a test request and displays the response received. After the developer implements a new service on the bus, the service is manually checked through the OSB Console. If the check is successful, the service is declared to be working and changes only if external system developers start complaining about it.
')
The support of the OSB used by us came to an end, and it became necessary to upgrade to the new version. Although the migration itself did not cause major problems, the question arose of how to test the performance of the migrated solution? And then our team once again thought about the implementation of automatic testing.
The picture was simple and everyone liked it right away.
In fact, we just need to automate what we do with our hands. So let's create a test that will store pairs of messages (request, response). After launch, our test will send a request, get an answer and compare it with the answer it has. If the answers match, then the test was successful.
The question arose, what should we use as a back-end system in our environment? Obviously, a real back-end system is not suitable for several reasons:
It became clear that we would need a services simulator, and the logical solution was to look at the finished products.
It turned out that there was no place to look, because there are only 5 such products on the market:
The first three products could not be looked at, simply because they cannot be downloaded and viewed. It was necessary to fill out long forms, leave telephones that salespeople would contact us with, in general, all this could drag on for months. HP Service Virtualization, in principle, also falls into this group, but it turned out that this product was already purchased from us. However, after a week of torment, it turned out that it would not work. There is no open API for this product, and it was impossible to create thousands of services through the visual editor. A spokesman for HP said that services could only be manually created, and they didn’t think about automation.
Great hopes were placed on Soap UI, which is able to launch mock-services as a web application. But it turned out that the SOAP UI is very “UI”. Firstly, it is not thread-safe, and secondly, it uses a lot of memory and is unstable.
In the process of research, it turned out that our bank already has as many as four (!) Self-written web services simulators. One of them turned out to be very much even nothing, was taken as a basis and, as necessary, added up. So in the test environment, we got a simulator - a web application that returns certain HTTP responses for certain HTTP requests.
Each virtual service is a maven project. In the project configuration (service-descriptor.xml, response-selector.xml) it is written how to determine on the basis of an HTTP request which service is called, which operation is needed and by what rule to return an HTTP response.
After changing the source code, the project is automatically built on Jenkins and, with the help of the maven wagon, is applied to a running simulator application.
The next step is to write a test, which will actually be a simulator of the front-end system. That is, we need to write a web-service client.
Our test is a maven project. Inside the project there are pairs of files (request, response), which are actually the source code. The project’s build consists in the fact that our self-written maven plugin calls the web service to send it a test request, receives a response and compares it with a test response.
Tests run automatically every night on Jenkins.
Since the main task was to test the migration, test requests and responses were exported from an audit database that works in production. According to this data, the corresponding maven-projects were generated.
In the case of the development of new services, data from living systems yet. For such cases, we wrote our Eclipse-based tool. In a few clicks we can create a new project, fill it with test data, put it in the version control system and make a new project on Jenkins.
Of course, we failed to completely cover the integration layer, which had been created for more than ten years, with tests.
The most important thing is that the implemented tests found errors that would occur during the migration. And also found some services that, in principle, did not work and were not used. So our experience is definitely positive!
We liked it and we will continue. In the near future we plan to add simulators for JMS, support business processes and figure out how to conduct performance tests.
Are you testing integration? Share your experience!
I want to share with you personal experience in system integration testing. Our team is developing an integration layer through which all the systems in the bank are connected. We have a lot of tasks, there is not enough time, and the issue of integration testing has always been postponed.
How does integration testing work? The shortest answer is no way, although we have more than a hundred systems that interact through the Oracle Service Bus (OSB) integration bus. This product has an OSB Console tool that allows you to send a test request and displays the response received. After the developer implements a new service on the bus, the service is manually checked through the OSB Console. If the check is successful, the service is declared to be working and changes only if external system developers start complaining about it.
')
The support of the OSB used by us came to an end, and it became necessary to upgrade to the new version. Although the migration itself did not cause major problems, the question arose of how to test the performance of the migrated solution? And then our team once again thought about the implementation of automatic testing.
How did we imagine it
The picture was simple and everyone liked it right away.
In fact, we just need to automate what we do with our hands. So let's create a test that will store pairs of messages (request, response). After launch, our test will send a request, get an answer and compare it with the answer it has. If the answers match, then the test was successful.
Service Virtualization (mocks, stubs)
The question arose, what should we use as a back-end system in our environment? Obviously, a real back-end system is not suitable for several reasons:
- For tests, we need the system to work on a specific set of data. Suppose we check how the service that returns the account balance works. For such a test, an account with this number must exist, and the service must return a certain value. Maintaining test data on a live system can be a time consuming process.
- The system may not be available at all in a test environment. Or at the time of integration, the service may still be in development.
- To check for exceptions, the system must return an error.
It became clear that we would need a services simulator, and the logical solution was to look at the finished products.
It turned out that there was no place to look, because there are only 5 such products on the market:
- Parasoft SoaTest
- CA Lisa Service Virtualization
- IBM Rational Service Tester
- HP Service Virtualization
- Soap UI
The first three products could not be looked at, simply because they cannot be downloaded and viewed. It was necessary to fill out long forms, leave telephones that salespeople would contact us with, in general, all this could drag on for months. HP Service Virtualization, in principle, also falls into this group, but it turned out that this product was already purchased from us. However, after a week of torment, it turned out that it would not work. There is no open API for this product, and it was impossible to create thousands of services through the visual editor. A spokesman for HP said that services could only be manually created, and they didn’t think about automation.
Great hopes were placed on Soap UI, which is able to launch mock-services as a web application. But it turned out that the SOAP UI is very “UI”. Firstly, it is not thread-safe, and secondly, it uses a lot of memory and is unstable.
In the process of research, it turned out that our bank already has as many as four (!) Self-written web services simulators. One of them turned out to be very much even nothing, was taken as a basis and, as necessary, added up. So in the test environment, we got a simulator - a web application that returns certain HTTP responses for certain HTTP requests.
Each virtual service is a maven project. In the project configuration (service-descriptor.xml, response-selector.xml) it is written how to determine on the basis of an HTTP request which service is called, which operation is needed and by what rule to return an HTTP response.
After changing the source code, the project is automatically built on Jenkins and, with the help of the maven wagon, is applied to a running simulator application.
And now we write the test
The next step is to write a test, which will actually be a simulator of the front-end system. That is, we need to write a web-service client.
Our test is a maven project. Inside the project there are pairs of files (request, response), which are actually the source code. The project’s build consists in the fact that our self-written maven plugin calls the web service to send it a test request, receives a response and compares it with a test response.
Tests run automatically every night on Jenkins.
Creating test data
Since the main task was to test the migration, test requests and responses were exported from an audit database that works in production. According to this data, the corresponding maven-projects were generated.
In the case of the development of new services, data from living systems yet. For such cases, we wrote our Eclipse-based tool. In a few clicks we can create a new project, fill it with test data, put it in the version control system and make a new project on Jenkins.
What happened
Of course, we failed to completely cover the integration layer, which had been created for more than ten years, with tests.
- Not all back-end systems work through web-services; adapters for other protocols are needed.
- There are requirements to test not one service, but a whole business process. In this case, it is necessary to store consistent data sets for several services.
- Write a simulator that supports everything that can back-end - quite a lot of work. We did not have time to support REST services, asynchronous messages, as well as various authentication methods.
The most important thing is that the implemented tests found errors that would occur during the migration. And also found some services that, in principle, did not work and were not used. So our experience is definitely positive!
Future plans
We liked it and we will continue. In the near future we plan to add simulators for JMS, support business processes and figure out how to conduct performance tests.
Are you testing integration? Share your experience!
Source: https://habr.com/ru/post/215363/
All Articles