Performance testing of the Docsvision platform with Visual Studio

On Habré already there are articles on creation of the load project in the Visual Studio. In this post I want to talk about the practical side of testing: what infrastructure do we use to run the load, how do we measure performance, what do we do with the data obtained. Detailed step-by-step instructions on project preparation and configuration in Visual Studio are also on MSDN, so I will not focus on this, I will only note the important, in my opinion, points. Since the development in the organization is based on the Microsoft technologies stack, we decided to use Visual Studio capabilities for testing, namely the MS Visual Studio Load Testing module, which is available in the Ultimate VS version.

Polygon

For testing, a separate test site has been prepared, which can be divided into three main parts:

1. Load generator
2. Docsvision server with a database server.
3. A separate machine on which the Docsvision client is installed.

')

To generate the load, 5 virtual machines are used, one of which is a controller and controls the distributed load from four load agents. The load from the agents is sent to the Docsvision (DV) server, which communicates with the DV database server. There is one more machine, with the DV client installed, connected to the server, and the time taken to perform typical operations at different loads is measured from it.



On the controller, Visual Studio Ultimate is installed, which includes the Visual Studio Test Controller, and there is a project with tests in C #. About the tests themselves later. Since we need to run more than 250 virtual users on the server, VS 2010 requires the Microsoft Visual Studio 2010 Load Test Feature Pack . For more recent versions of VS Ultimate, as far as I know, the number of virtual users is not limited. The load between agents can be distributed unevenly. It is possible to run the agent without any load, only for collecting Performance Monitor data.



Visual Studio Test Agent is installed on each test agent. If you want to run the load from only one machine, you do not need to install the Test Agent. Using the Test Agent Configuration Tool on each agent, you need to start a service that collects Windows Performance Monitor counters and runs load tests. When configuring a load project, you can choose from which computers and which ones we want to collect counters (processor, memory, data from SQL server, etc.). To do this, they must be included in the Performance Monitor Users group by the user under which the load is launched.



Similarly, on the controller, through the Test Controller Configuration Tool, a service is started that manages agents and collects data from all machines in one place. Agents connect to Test Controller.

Database

So that the base for testing and the nature of the load were close to real, we received statistics on the work of users and a data organization scheme in the DV database from several of our large customers. On this basis, we created a folder structure in the load database, filled it with a large number of cards, folders and files and made up the average, somewhat overestimated, load profile. Before testing, the database is updated on the build for which we want to take measurements. Measurements from the client machine are conducted on the same prepared cards, folders, search queries.

Tests

The load project consists of a set of tests that mimic the activities of the average user. Tests are written in C # and use the Docsvision API.

According to the project settings: we use the Load pattern "Step" and the Test Mix Type "Based on user pace".

For each virtual user, the ClientStart test is performed first, opening a user session to the% DVServerName% / DocsVision / StorageServer / StorageServerService.asmx web service. During the entire test, each user performs the number of tests specified per hour in accordance with the project settings. Upon completion of all tests, the ClientEnd test is executed, closing the user session.





The initial and final test is set in the Edit test Mix window: load profile setting window.



Test example:

[TestMethod] public void _() { string filePath = (string)this.TestProperties["FilePath"]; UserSession session = this.GetUserSession(); Assert.IsNotNull(session); Document doc = DocumentService.CreateDocument(); doc.MainInfo.Name = " "; doc.MainInfo.Registrar = StaffService.GetCurrentEmployee(); objectContext.SaveObject<Document>(doc); //add file and save DocumentService.AddAdditionalFile(doc, filePath); objectContext.SaveObject<Document>(doc); } 

FilePath is the path to the file that is attached to the Docsvision document card. The project has an xml configuration file in which the values ​​of variables used in tests are stored: guids of cards, search queries, paths to files, etc. The method receives the current user session, checks that it exists, creates a new document card, fills in the fields, adds a file from the file system, and saves the card in the database.

Testing process

Run the load. While the Windows performance counters are initialized on all machines, you can have time to pour tea.



First, be sure to check how the system works without load. Then under the small. This allows you to immediately see if there are any serious problems in the system or on the landfill.



After launching load tests, using MS Visual Studio, we observe the execution time of each individual test, the load on computers, the number of tests per second, the number of user sessions, the number of successful / failed tests, the execution time of each test, the average execution time of each test. We follow the counters, we catch the moment when, for example, the processor starts to load, the memory of servers. We look at the time for which tests are performed, this is also an important parameter, because even if the server processes all requests, but does so for a long time, the user does not like it either, it doesn’t matter what happens on the server. If at some point the values ​​of the parameters become high, we notice how many virtual users our server is currently loading. Sometimes it makes sense at this moment not to stop the load, but to see how the system behaves further, to failure.



If everything is OK, wait until all users log in and the system stabilizes. When launching a large number of users, we sometimes increase the intensity of user actions, not their number, this allows us to reduce the time to reach a stable load. After that, we begin to measure performance indicators: connect from a separate machine via the DV client to the server, see how long the typical Docsvision user actions are performed: opening cards, launching Navigator, displaying the contents of folders with a large number of cards (10000, 100000), opening directories and etc. The obtained values ​​are stored in a table and used for comparison with values ​​from previous measurements.

results

The obtained data is analyzed together with the head of the quality assurance testing department, the system architect and programmers. Bottlenecks are more thoroughly investigated with the help of Fiddler ʻa, queries in the database using the MS SQL Management Studio profiler. The results are then passed on to the production director and product manager in order to plan the optimization at the next iterations of the system development. At the moment it becomes clear that it makes sense to compare not only the results obtained when measuring the time taken to perform actions performed by the user and the time taken to fulfill server requests, but also the results of Windows performance counters and compare them with each other to see changes over time. The problem is that the indicators are influenced not only by new DV releases, but also by a lot of external parameters, like Windows updates, switching to the new version of .Net and other third-party components.

Read

In addition to the links in the post, here are a couple of articles that once helped me:

• A long, good guide to finding interaction problems between test agents and a controller: Troubleshooting Guide for Visual Studio Test Controller and Agent . In any incomprehensible situation, before delving into this article, check whether the agent and controller see each other :)
• Deleting old load test results - a script for clearing the database with the results of load testing. It may help if, after deleting the results via the Visual Studio interface, the error "Load Test results repository is out of space ..." remains.
• The post describes testing from the point of view of using Visual Studio, in more detail about the methods of load testing of Docsvision itself is written here .

I hope the post will be useful for someone. Questions and comments are welcome. It is interesting to share experiences and learn how testing is organized for you.



This article is a joint work of Natalia Budenna and Olga Trachuk.