JMeter as a relatively convenient and practical tool for testing API
The article will talk about testing in a short time using the JMeter tool, as well as how to successfully complete the work with the forced replacement of specialists on the project.
As a tester, I love it when everything is in order, but life is full of dirty hacks. I like to automate, tying Selenium to Python, but when I meet the problem of limited resources, I rush for the tool that allows you to do "all the same, but faster." In this post I will tell you that JMeter is a great tool for both load and functional testing.
I was assigned the task of testing the API for the mobile application. Its features were as follows:
')
Hands were drawn to Python as usual, but something switched in my head. “Dear friend,” I said to myself, “if you are the only person on the team who will write in Python, then who will understand what you wrote and continue to work if you are absent?”
It was necessary to clearly look for another tool. And then I looked at the beloved JMeter from a completely different side: JMeter, of course, leaves much to be desired externally, but instead of a thousand small scripts we will have a clear structure of tests. Therefore, I can ask colleagues to run random tests instead of me from any machine. Even programmers appreciated that this is much more convenient than “running hands” for debugging.
It has everything from the box (well, almost) what is needed:
The only thing that was a little embarrassing - Jenkins views JMeter exclusively as a load testing tool, so the history of the found bugs was in the Performance report tab.
Briefly about the testing process itself:
Instead of a thousand words. Here is a small piece of the finished dough:
One of many tests. Groups are divided into Simple Controller. Http request, Regular Expression Extractor, Response Assertion, some User Defined Variables, Random Variable and almost everything.
Almost any test begins with the creation of the necessary users. Here the only “hardcoded place” is the authorization of the administrator to create a new item in the database.
What is funny, a method was created to create a user, but the removal was only done manually through the admin panel. I decided not to complicate the logic of each individual case by adding separate tests on Selenium for cleaning users, and did not delete the created users inside the test itself. I received thanks from the programmers for this: when the Padzhinator started writing for admin pages, there were already thousands of test users in the database.
To check the result, after each request, there is a long series of assertions. It checks the accuracy of the incoming data: http codes of responses, messages.
I have disabled all charts and reports except for simple tree view for debugging. All the bugs can be viewed directly in Jenkins, and for load testing it was more convenient to upload directly to LoadSophia with their proprietary plug-in and pick it up already.
Here's how the programmer’s message looks like why the next build failed:
And in more detail. Waited for the 400 code - returned 404:
The result of this black magic and obscurantism :
Of the 1200 hours laid down for development, the total amount of writing and actual maintenance of autotests was 50 hours. To fix bugs - 70 hours. And about half of them was associated with a misunderstanding of the customer and developers. The result is very good. The alternative to checking everything manually would take almost more time than it took to program. Programmers saw right away how something was going wrong, in the whole system with each new build.
In my absence of problems with launching and modifying autotests, there were no programmers or testers. Sometimes tests were changed right in notepad, because * .jmx is also XML.
For load testing, I simply turned off unnecessary tests for a while, calculating where the weak points of the system are and which tests pass slowly. There were no problems with the use of new JMeter buns. Everything is documented and everything is clear.
Of the difficulties encountered:
PS A small lyrical digression. A recent article talked about a way to reduce the work of a tester through decomposition and, more importantly, the problems of the testing market. One of the market responses to the “vicious circle of testers,” described at the beginning of the article, is the standardization of testing skills. With it, the employer will become more confident that he will find in the market the necessary specialist who can solve the problem. The tester will know that with the knowledge gained he will not be lost in the market. In this case, making functional testing through JMeter is not at all an academic solution. But it reduced time costs and a pound of nerves with a similar result. I had a friend who knew only the most basic basis of JMeter and Selenium, but was taken with open arms into one very respected company. No one in my life interview asked me if I knew any rarely used library. They asked mainly about JMeter, Selenium (and what I write for these solutions). The approach “most importantly, know a few of the most important tools well and be able to apply” makes life easier for the tester, the employer, and the market as a whole.
As a tester, I love it when everything is in order, but life is full of dirty hacks. I like to automate, tying Selenium to Python, but when I meet the problem of limited resources, I rush for the tool that allows you to do "all the same, but faster." In this post I will tell you that JMeter is a great tool for both load and functional testing.
I was assigned the task of testing the API for the mobile application. Its features were as follows:
')
- Dates, more than ever, are compressed;
- There is a specification of all methods, but it changes literally on the fly;
- The project is confusing. If I die in the mountains or get sick, it will be very expensive and difficult to introduce a new person into the project;
- As the CI system we use Jenkins. Tests should get up easily without any rarely used, raw and overly complex plug-in;
- Mandatory load testing, but there is little time to conduct it;
- The very mobile application for this API is written by a completely different team from another city. The only way to test is to view responses to requests.
Hands were drawn to Python as usual, but something switched in my head. “Dear friend,” I said to myself, “if you are the only person on the team who will write in Python, then who will understand what you wrote and continue to work if you are absent?”
It was necessary to clearly look for another tool. And then I looked at the beloved JMeter from a completely different side: JMeter, of course, leaves much to be desired externally, but instead of a thousand small scripts we will have a clear structure of tests. Therefore, I can ask colleagues to run random tests instead of me from any machine. Even programmers appreciated that this is much more convenient than “running hands” for debugging.
It has everything from the box (well, almost) what is needed:
- Working with variables, regular expressions, parsing JSON, working with cookies and another huge bag of plushies;
- A simple and easy-to-understand plugin for Jenkins that allows you to connect everything;
- For load testing, I do not have to write new scripts, you can only slightly modify existing ones.
The only thing that was a little embarrassing - Jenkins views JMeter exclusively as a load testing tool, so the history of the found bugs was in the Performance report tab.
Briefly about the testing process itself:
- Standard Thread group for 1 Virtual user;
- Each group of tests is enclosed in Simple Controller (it has only one distinctive feature: all requests go one after another. This is what you need, just like in a test case);
- Each individual test case is also enclosed in a separate Simple Controller. Very simple test cases, for example, checking whether 401 code will return when trying to climb without authorization, are made by a separate test case.
Instead of a thousand words. Here is a small piece of the finished dough:
One of many tests. Groups are divided into Simple Controller. Http request, Regular Expression Extractor, Response Assertion, some User Defined Variables, Random Variable and almost everything.
Almost any test begins with the creation of the necessary users. Here the only “hardcoded place” is the authorization of the administrator to create a new item in the database.
What is funny, a method was created to create a user, but the removal was only done manually through the admin panel. I decided not to complicate the logic of each individual case by adding separate tests on Selenium for cleaning users, and did not delete the created users inside the test itself. I received thanks from the programmers for this: when the Padzhinator started writing for admin pages, there were already thousands of test users in the database.
To check the result, after each request, there is a long series of assertions. It checks the accuracy of the incoming data: http codes of responses, messages.
I have disabled all charts and reports except for simple tree view for debugging. All the bugs can be viewed directly in Jenkins, and for load testing it was more convenient to upload directly to LoadSophia with their proprietary plug-in and pick it up already.
Here's how the programmer’s message looks like why the next build failed:
And in more detail. Waited for the 400 code - returned 404:
The result of this black magic and obscurantism :
Of the 1200 hours laid down for development, the total amount of writing and actual maintenance of autotests was 50 hours. To fix bugs - 70 hours. And about half of them was associated with a misunderstanding of the customer and developers. The result is very good. The alternative to checking everything manually would take almost more time than it took to program. Programmers saw right away how something was going wrong, in the whole system with each new build.
In my absence of problems with launching and modifying autotests, there were no programmers or testers. Sometimes tests were changed right in notepad, because * .jmx is also XML.
For load testing, I simply turned off unnecessary tests for a while, calculating where the weak points of the system are and which tests pass slowly. There were no problems with the use of new JMeter buns. Everything is documented and everything is clear.
Of the difficulties encountered:
- The plugin for JMeter that implements oAuth authentication is very outdated and does not work with oAuth 2.0. We had to test a small part of the functionality related to authorization via social networks separately.
- Tying Selenium to JMeter is also a dubious pleasure. Admin UI was also tested separately.
- For people who are not confronted with the JMeter interface, the first acquaintance ends with
retinal detachment aslight disorder.
PS A small lyrical digression. A recent article talked about a way to reduce the work of a tester through decomposition and, more importantly, the problems of the testing market. One of the market responses to the “vicious circle of testers,” described at the beginning of the article, is the standardization of testing skills. With it, the employer will become more confident that he will find in the market the necessary specialist who can solve the problem. The tester will know that with the knowledge gained he will not be lost in the market. In this case, making functional testing through JMeter is not at all an academic solution. But it reduced time costs and a pound of nerves with a similar result. I had a friend who knew only the most basic basis of JMeter and Selenium, but was taken with open arms into one very respected company. No one in my life interview asked me if I knew any rarely used library. They asked mainly about JMeter, Selenium (and what I write for these solutions). The approach “most importantly, know a few of the most important tools well and be able to apply” makes life easier for the tester, the employer, and the market as a whole.
Source: https://habr.com/ru/post/243097/
All Articles