Once met JMeter and a stranger ...

Shot from the film "Lake House". Meeting (www.kinopoisk.ru)

Jim did not yet know how to approach her, where to start a conversation and in what language to speak. But he saw a lot, spoke languages ​​and had not one trump card up his sleeve. And being confident in the help of faithful friends (this we are with you) and forgetting about the doubts, I went to meet the fate.

Below is a story about how Jim conquered the heart of an unfamiliar system again and again. Do not think that there were several strangers. She was one, unique, but so different, and from that the stories will follow one after the other.

Foreword

')

Popular options for sending requests to an unfamiliar system

I used such options for sending requests to an unfamiliar system:

1. Created an intermediate web application that can send requests to the system under test, and sent commands to a web application using the HTTP Request component.
2. Created an intermediate web service that can send requests to the system under test, and sent web service commands using the SOAP / XML-RPC Request component.
3. Created a console client to the system under test, called the client from JMeter using the OS component Process Sampler .
4. I created a library (API) in Java, C # to work with the system under test, and called library methods using the JSR223 Sampler component.
5. Created a plugin for JMeter .

Web application and HTTP Request or web service and SOAP / XML-RPC Request

It is known that JMeter can send HTTP and SOAP requests. And for solving the integration task, the option of creating a web application that sends requests to the system under test and accepts commands from JMeter via HTTP protocol is often selected.

Example. Tested the wcf-service working under the SOAP / MSBin1 protocol over HTTP . JMeter does not know how to work with the SOAP / MSBin1 protocol , it will be able to send a small request in one package in a very unreadable form, but it will not be able to send a large request or receive a large response in a stream.
The first solution was to use Visual Studio to generate a client for the system under test (wcf-service), and created a test web service that could receive commands from JMeter via HTTP .

Or to the system under test there can be an API in Python . And to call the Python code, you can create a Python web application that will accept commands from JMeter and apply system load.

Low load web application

Web applications to create a small load on the system under test. It is important to keep a log in the web application.

When working with a web application there will always be a time lag. The JMeter interaction time with the intermediate web application will be non-zero even at low load. Therefore, the response time to requests of the system under test should be measured by the logs of the intermediate web application. Request processing time, fixed in JMeter, will be obviously long. Therefore, it is important to maintain two logs - a log in JMeter and a log in an intermediate web application. To be able to parse the web application log, build statistics on it.

Web application under heavy load

A web application under heavy load may become a point of failure if it is not mapped.

When creating a large load, it is often necessary to run several JMeters . Many requests are created, which the intermediate web application can not cope with. The time lag will grow. With a large load, even logging in the web application may be delayed, an additional time lag will appear when recording the response time of the system under test. I depicted this in the figure above, highlighting the intermediate web application in red, and the arrow to the log file is dotted.

Also often tested system includes a load balancer and several application servers. If load balancing is performed on clients' ip-addresses, then using a single intermediate web application, the load will be applied only to one of the nodes of the system under test. Other nodes will be idle. I figured it out in the figure above, highlighting one of the nodes of the system under test in red.

Before applying a large load is important:

• set requirements and determine the maximum performance of an intermediate web application;
• consider when testing the method of balancing on the tested system.

To create a large load, you need to use multiple JMeter and several intermediate web applications, and the load will be distributed evenly

If there are several load stations and the load is fed through an intermediate web application. It is convenient to run a separate web server at each load station with the web application providing the load.

Then the load will be distributed evenly, even if the balancer of the system under test uses client binding to the application servers by ip-addresses.

For processing multiple logs and generating statistics on them, it is convenient to use the Pandas library (from Python ). And write a log of web applications so that you immediately get a CSV file. Such CSV files can be downloaded, analyzed and immediately plotted using a loop using MatPlotLib . Jupyther , Pandas and MatPlotLib are faithful helpers in processing logs.

Console application and OS Process Sampler

The option of applying the load through the launch of applications is justified if:

• there is a ready-made convenient client as an application for working with the system under test;
• There are ready-made unit tests for the system under test and you just need to use JMeter to run the required number of threads that run these tests and measure their execution time;
• you need to use a lot of cryptography, you know C # / Java and know how to work with cryptography on C # / Java;
• there is an SDK on .NET / Win32 / Java / Python for the system under test and you know C # / C ++ / Java / Python and you can easily write a console client using the examples from the SDK;
• you need to download or send huge files and standard JMeter components consume too many resources when performing such requests;
• when working with the system under test, a complex script that is easier to implement and debug in IDE ( Intellij IDEA , Eclipse , KDevelop , Visual Studio , ...) than in JMeter .

Load load on the system under test through the launch of client applications

The first time I used the load through the console application, when in the test I had to download files larger than 100 MB and up to several gigabytes via HTTP . JMeter consumed all the available RAM when receiving such responses. Wget came to the rescue, this console tool easily downloads huge files, consuming a minimum of RAM, has settings that allow you to enable and disable gzip and keepalive . OS Process Sampler and wget allowed to pump the channel and perform testing, which could not be achieved using HTTP Request .

Application features:

• The application is a separate process, and it consumes a lot of RAM, to create a large load, you will need a lot of RAM, a fast processor or several load stations;
• each application will maintain its own separate log, you will need to take care that it is then convenient to process a lot of logs and get statistics on them;
• it is too wasteful to launch an application to perform only one short request to the system under test, it is convenient to implement an entire script in the application: authentication, several iterations of useful actions and exit.

Applications do not become a point of failure. When they work, the time lag is much less than when using an intermediate web service or web application. They are easy to develop and debug.

In order for a console application to skillfully execute a script, it is convenient to implement in it the reception of a script from standard input using your pseudo-code. For example, such:

#AuthByCertificate "  " #GetInboxDocuments #GetContagentList #SendDocument ".\data\Document1.txt" #SendDocument ".\data\Document2.jpg" #Exit 

And to write a parser for such a scenario: if the line starts with #, then this is a command, control is transferred to the handler of such a command. If necessary, the handler reads the parameters required for execution, for example, the name of the user's certificate or the path to the file.

If you don’t have time to develop such an interface, you can wrap the scripts in functions by passing the names of functions and their parameters simply via command line parameters — a simple and familiar way:

 ConsoleApp.exe /test1 "  " ".\data\Document1.txt" ".\data\Document2.jpg" ConsoleApp.exe /test2 "UserLogon1" "UserPassword1" ".\data\Document1.txt" 

The number of scenarios is usually limited and negotiated before testing. Therefore, this option is also convenient.

Using OS Process Sampler you can transfer any number of parameters. You can use JMeter variables, read parameters from a CSV file. But practice has shown that it is more convenient to write a batch file for launching a console application. And call the batch file from OS Process Sampler , not the application itself.

If the script should return some result to the test. It is convenient to implement the output to the console of this result as a string. And on the JMeter side, implement a post processor to handle the response parameters.

If you use a ready-made application and it does not return certain response codes for successes and errors. That minimally necessary post-processor is needed to set the OS status of the Process Sampler - whether the application was successfully executed or not.

Library and JSR223 Sampler

Although the applications work fine. Still, I want to spend less resources - create threads, not processes, because sometimes there are not a few load stations and gigabytes of RAM. I want to accept any JMeter variables as input. And also sometimes, I want more flexibility in scripting, I want instead of sending pseudocode to stdin applications to write simply:

 TestClient client = new TestClient(); String certificateCommonName = vars.get("certificateCommonName"); //"  " String documentPath1 = vars.get("documentPath1"); //".\data\Document1.txt" String documentPath2= vars.get("documentPath2"); //".\data\Document2.jpg" client.authByCertificate(certificateCommonName); client.getInboxDocuments(); client.getContagentList(); client.sendDocument(documentPath1); client.sendDocument(documentPath2); client.exit(); 

Use cycles for this, transfer the results of executing some methods to the input of others, use some interesting logic, without having to switch between JMeter and the application development environment.

Or authenticate and log out only once per virtual user. And within one request from JMeter, perform only one request to the system under test without additional time lags.

There is such a way - use JSR223 Sampler , from which it is convenient to call library methods.

The JSR223 Sampler has a number of handy features.

Can be divided into three JSR223 Sampler library initialization, client creation, connection to the system. And within one flow (virtual user JMeter) operations will be executed successfully, which will measure the time of each step.

Client initialization to the system under test (library for .NET, loaded using jni4net). It is enough to load libraries once for the test:

 import net.sf.jni4net.Bridge; import java.net.URL; Bridge.init(); Bridge.setDebug(false); Bridge.setVerbose(false); java.io.File clientDllFile = new java.io.File("TestClient.j4n.dll"); Bridge.LoadAndRegisterAssemblyFrom(clientFile); 

Separately, for each thread, the client is instantiated to the system under test:

 import testClient.Client; //   Client client = new Client(); //     "testClient" JMeter vars.putObject("testClient", client); //          //       client String responseData = client; 

And then you can call any methods of this client, for example, authByCertificate:

 import testClient.Client; //  ,   Client client = vars.getObject("testClient"); //      String certificateCommonName = vars.get("certificateCommonName"); //"  " //     client.authByCertificate(certificateCommonName); 

Using jni4net , you can call library methods on .NET. Using just jni , you can call methods in the Win32 libraries. If the library is already written in Java, then no additional layers are needed. Of course, you can't call the code in Python, PHP, bash, but this is not always necessary.

The Groovy language used in JSR223 Sampler allows you to write compiled, fast scripts.

And while it does not need to handle a lot of logs, as when using web applications or simple applications. There is no time lag due to the use of intermediate components.

Resource consumption is minimized. You just need to be able to write code on Groovy, and it is very similar to Java.

JMeter plugin

If you do not want to write code. And when the load is applied, do not need to perform dozens of methods, and it is enough to perform a single-type operation, but with hundreds of parameter values. It is convenient to develop a plugin for JMeter. It's like a library, but with a graphical interface.


Plugin for JMeter

There will also be a single log. There will be no time lag. It will be possible to set parameters without switching between applications.

The plugin is not very difficult to develop. Developed simple plugins, expanding the capabilities of existing ones, combining them. And the existing ones can be found in the project repositories:

• http://jmeter.apache.org/ ;
• https://jmeter-plugins.org/ .

A couple of days with a debugger will give an understanding of how to develop a plugin.

For example, the TailSampler plugin (see the TailSampler article - sending GET requests in parallel to Apache.JMeter ) is a combination of components:

• HTTP Request from jmeter.apache.org and
• Dummy Sample from jmeter-plugins.org.

And it contains only a couple of hundred lines of the original code.

Commercial plugins

Separately, there is the theme of developing and receiving commercial or closed plug-ins. The code does not always need to be written, sometimes it is easier to buy or ask. JMeter developers create their add-ons. Testing and development companies have dozens of plug-ins for different tools in their arsenal.

One of the developers of JMeter philmdot , sells four plugins on ubikloadpack.com, video plugins, GWT, Java Serialization, FLEX.

Colleagues from Performance Lab have made plug-ins working with Citrix , ActiveMQ, Documentum WebTop, for various services and protocols. On github, so far little code has been posted, just started.

Surely in other companies that perform load testing projects, there are also ready-tested plug-ins. I will not say who to contact. But as an example I did this:

1. I typed in the search engine the phrase "Tender Load Testing".
2. Found a link to the tender in Sberbank.
3. I looked at the list of companies that participated: Performance Lab, Advanced Transformation Consulting, Ai-Teco, Aplana, EPAM Systems and others.

I think if you contact any of the companies that participated or won the tender for testing a certain system or technology, then they will have a ready-made plug-in, utility, emulator for such a system or protocol. And, probably, you can download it, buy it, ask for it.

Free License Plugins

Github allows you to find hundreds of plugins. RabbitMQ , Cassandra , WebSocket , Ajax , ... various technologies, protocols, approaches are already covered in JMeter plugins.

Two projects with a description in Russian, search for the phrase "JMeter plugin".
168 search results for "JMeter plugin".
And hundreds of projects when searching for the phrase "JMeter".

It is necessary only to be not too lazy to find, download, collect, test, find out the license from the author, if it is not specified, and use.

Conclusion

Shot from the film "Lake House". Communication Method (www.movpins.com)

Jim can find a way to interact with the system, even if she speaks a different language, if she needs a special approach and a secret path.

There is no universal way. In one situation it is more convenient to write a plugin. In the other use the console application. Sometimes Groovy library and scripts. And, sometimes, I want to raise my web server.

Everything is possible, you just have to believe.

Tell us about the limitations that were found in JMeter , and how they were handled.

Update . He touched on the topic of commercial and third-party free plug-ins. Did not begin to give references to specific projects. If there is a positive experience of using some kind of plug-in that is not included in the standard delivery, please write about it in the comments.