The practice of automating the measurement of performance indicators SED
Electronic document management systems and ERP-systems are complex software products, most often consisting of many subsystems. Parts of any system work both directly in the dialogue with the user and in the background, performing a certain part of the tasks on the server.
In order to monitor the performance of the system in both directions, the system administrator needs convenient measurement tools to find the weak link in time and take measures to improve performance. In this article I will share my experience on how to solve the issue of automatic measurement of system performance indicators using the example of an electronic document management system *.
Unlike other software products, there are certain speed requirements for EDMS: this is one of the important parameters that you need to keep an eye on.
* The article will be especially useful to novice Docsvision administrators, but in general, the experience described is applicable to similar products.
')
Speed
In addition to functionality and design, for any software, the key parameter is the speed of work, and this factor is especially significant in the case of a complex system like electronic document management systems.
Imagine that a clerk user in the workflow needs to quickly perform a number of actions: open the navigator *, create new cards and view existing ones, attach documents / files to cards, search for a large number of documents, generate data for reports. And you need to do all this quickly.
Back to the origins: what causes the speed of the system?
In general, the speed of each of the links of the application and the speed of interaction between the links. In complex software, there may be a lot of links, then the speed of work will be proportional to the work of the slowest link.
For example, the Docsvision electronic document management system is an application with a client-server architecture, and we use:
1. IIS as a web server
2. MS SQL Server as a DBMS
3. Client application written in .NET and WCF
* Navigator - this is how Docsvision refers to the main workplace of the SED user.
If to simplify completely: a user working in the client application Docsvision Navigator, from his workplace, exchanges requests with the web server, receiving / sending certain data (this allows you to work in Docsvision both on the local network and through the Internet (http / https to choose from).
Immediately, I note that for applications that use IIS (Apache, NGINX) as a web server and MS SQL as a DBMS, the recommendations given in the article will also be relevant.
If we return to the listed tasks of the clerk, all of them (and not only them) are performed at different speeds, and all of them can be measured, if desired, to determine the speed of the system.
As a rule, software vendors initially provide data on key performance indicators. For example, a list of key indicators (and their values ​​for the corresponding version of Docsvision 5) is officially published by DoxVision in the “Docsvision 5 Installation and Administration Guide”.
These indicators include:
• Launch of the Navigator (“cold” start)
• Opening the UD document card (the first after launching the Navigator)
• Opening of a document management card (subsequent)
• Opening of the UD task card (first after launching the Navigator)
• Opening a card, a task for the DD (subsequent)
The values ​​of these indicators are obtained empirically at the loading range during load testing for 500 concurrent users. Details on the methodology for performing load testing of EDS can be found in the article on the DoksVizn website: www.docsvision.com/testirovanie-sed-docsvision-5
Measurement Methods
How to verify independently the compliance of the implemented system with the manufacturer’s stated indicators? Based on our own experience, as well as the experience of our partners, a number of tools and recommendations have been formed for the optimal accomplishment of this task.
You can arm yourself with a stopwatch to measure, for example, the opening time of a client application (“Navigator”), but this is not exactly a professional approach. We will use the utility familiar to each web developer:
Feeddler
As the client application accesses the web server, we will use the Fiddler utility ( http://www.telerik.com/fiddler )
In this case, fiddler acts as a proxy server through which all client requests pass (here we are interested in requests from the Docsvision Navigator application).
It is convenient enough to look at the speed of query execution, and, not least, this information can be used to diagnose performance problems. It is possible to say with great accuracy where there is a slowdown on the client (weak hardware, bad network channel, slow drawing of the interface) or on the server.
Fiddler allows you to figure out the following:
1. The number of server calls.
2. Duration of server calls.
3. The nature of the server logic called.
4. Distribution of server calls in time.
5. The distribution of the script's running time between the client and the server (it is very useful to find out exactly where the performance has subsided).
JetBrains dotTrace Performance
Our colleagues (in terms of software development) from JetBrains are developing quite a lot of useful tools for developers. One of the applications can be used to analyze system performance indicators.
DotTrace - proprietary profiler for tracking performance problems and memory bottlenecks in .NET applications
This tool is quite powerful and allows you to:
1. Use several profiling modes (including changing the run time of a subprogram thread).
2. Compare pictures (“traces”) taken from different systems (in some cases it is useful to compare pictures of two client workstations).
3. Keep statistics on the functions used and the time of their execution (find out what exactly worked non-optimally - search for a folder or preview a file).
4. To profile the memory occupied in the system.
In the context of Docsvision: during the search for performance problems, we recommend profiling and analyzing (with our help) both the client and server components of the platform. This tool is mainly used by developers to diagnose problem scenarios in their work.
Navigator log
If there is nothing at hand, then the application can be analyzed using its own tools. The application has the ability to enable its own diagnostic log.
Most often, the diagnostic log of the navigator is used to locate errors in the work, but in addition it contains information about all operations performed by the application "Docsvision Navigator". Those. only client operations of the specific application for which the log is being logged.
The log looks like this:
In the log should pay attention to:
• Opening the navigator
• The regular closing of the navigator (together with the operation “Opening the navigator” gives indirect information about the events of the Navigator crash)
• Opening time of the navigator
• Card opening time
• Duration of opening the folder node
• The duration of the construction of the presentation in the folder
Since reading the log using text editors is not always convenient, Docsvision administrators can use the Docsvision Navigator Log Parser utility. This is our own development, which can be requested from the technical support staff.
She is pleasant because:
• simplifies the process of analyzing the advanced log of the navigator, isolating only information about useful operations;
• knows how to separate the first (the first after the operation Navigator started) and the subsequent opening of the cards of documents and tasks;
• can select operations in a specified time / date range;
• The result of processing the extended log of the navigator is displayed in a separate csv file for further more detailed analysis.
Server part
During the research of the client application for performance, it may happen that, after analyzing the data of client applications and finding no problems, it is necessary to conduct an audit of the server part (DBMS). In this case, it will be necessary to conduct server-side profiling, focusing on typical MS SQL admin tools.
SQL SERVER profiler
The standard SQL Server Administrators utility allows you to view absolutely all requests to all databases in your possession.
With the use of certain profiling templates you can get a lot of useful information:
1. Find out what operations in the database and how long they are performed
2. Learn about the presence of dead locks (Deadlocks)
3. Learn the execution plan for queries and stored procedures
Of course, these methods are applicable not only to the Docsvision database, but also to any database in SQL. The data collected in this way helps not only to see the speed of certain operations, but also to push the Docsvision Administrator (along with the DBA) to take steps to optimize performance. In fact, this topic is quite extensive, and is suitable for a separate article. Administrators of MSSQL, armed with SQL Profiler and Database Tuning Advisor, can put their knowledge into practice, optimizing the work of the database.
Objectivity and representativeness of measurements
To obtain accurate results when measuring, I recommend to consider the following factors.
• Representativeness. It is enough to analyze the indicators of the user group without the need to analyze the indicators at all workplaces.
• Objectivity. Measurable indicators should be quantified and calculated without the influence of subjective factors in the automatic mode. This, for example, the failure of the "iron" part or problems in third-party software that interferes with the work (antivirus software, firewalls, point-to-point network routers).
• Selection of users.
When conducting measurements in the ERMS, it is sufficient to select active users (up to 10), satisfying the following (at least one) criteria:
EDS is the main working tool of the user;
At the same time: the group covers various scenarios of users in the system; the equipment of user workplaces meets the recommended requirements of a software developer; Users represent different working conditions with EDMS in terms of connection methods (local from different network segments, terminal).
Additions
No less important when measuring:
• pay attention to monitoring the conditions under which performance indicators are obtained (measurement of the load on the equipment of the ERMS): client computers of the group, network connection to the application server, server.
• analyze data on equipment loading when receiving unsatisfactory performance indicators.
• perform database profiling when detecting slow data operations on the servers.
Conclusion
Not only the actual tools developed (logging embedded in the platform), but also a wide range of familiar applications that will help in diagnosing workplaces in a short time are used to measure performance.
In certain situations, it is possible to use only embedded tools: this, for example, can be useful when working “in the field” on the customer side when there is no access to a wide range of tools and applications.
Third-party diagnostic tools should be used thoughtfully and commensurately assigned tasks.
Overlaid with numbers and graphs, you need not to break away from reality. It is desirable to keep the application running, so that the effect of optimization after measurements can be felt in combat conditions.
In order to monitor the performance of the system in both directions, the system administrator needs convenient measurement tools to find the weak link in time and take measures to improve performance. In this article I will share my experience on how to solve the issue of automatic measurement of system performance indicators using the example of an electronic document management system *.
Unlike other software products, there are certain speed requirements for EDMS: this is one of the important parameters that you need to keep an eye on.
* The article will be especially useful to novice Docsvision administrators, but in general, the experience described is applicable to similar products.
')
Speed
In addition to functionality and design, for any software, the key parameter is the speed of work, and this factor is especially significant in the case of a complex system like electronic document management systems.
Imagine that a clerk user in the workflow needs to quickly perform a number of actions: open the navigator *, create new cards and view existing ones, attach documents / files to cards, search for a large number of documents, generate data for reports. And you need to do all this quickly.
Back to the origins: what causes the speed of the system?
In general, the speed of each of the links of the application and the speed of interaction between the links. In complex software, there may be a lot of links, then the speed of work will be proportional to the work of the slowest link.
For example, the Docsvision electronic document management system is an application with a client-server architecture, and we use:
1. IIS as a web server
2. MS SQL Server as a DBMS
3. Client application written in .NET and WCF
* Navigator - this is how Docsvision refers to the main workplace of the SED user.
If to simplify completely: a user working in the client application Docsvision Navigator, from his workplace, exchanges requests with the web server, receiving / sending certain data (this allows you to work in Docsvision both on the local network and through the Internet (http / https to choose from).
Immediately, I note that for applications that use IIS (Apache, NGINX) as a web server and MS SQL as a DBMS, the recommendations given in the article will also be relevant.
If we return to the listed tasks of the clerk, all of them (and not only them) are performed at different speeds, and all of them can be measured, if desired, to determine the speed of the system.
As a rule, software vendors initially provide data on key performance indicators. For example, a list of key indicators (and their values ​​for the corresponding version of Docsvision 5) is officially published by DoxVision in the “Docsvision 5 Installation and Administration Guide”.
These indicators include:
• Launch of the Navigator (“cold” start)
• Opening the UD document card (the first after launching the Navigator)
• Opening of a document management card (subsequent)
• Opening of the UD task card (first after launching the Navigator)
• Opening a card, a task for the DD (subsequent)
The values ​​of these indicators are obtained empirically at the loading range during load testing for 500 concurrent users. Details on the methodology for performing load testing of EDS can be found in the article on the DoksVizn website: www.docsvision.com/testirovanie-sed-docsvision-5
Measurement Methods
How to verify independently the compliance of the implemented system with the manufacturer’s stated indicators? Based on our own experience, as well as the experience of our partners, a number of tools and recommendations have been formed for the optimal accomplishment of this task.
You can arm yourself with a stopwatch to measure, for example, the opening time of a client application (“Navigator”), but this is not exactly a professional approach. We will use the utility familiar to each web developer:
Feeddler
As the client application accesses the web server, we will use the Fiddler utility ( http://www.telerik.com/fiddler )
In this case, fiddler acts as a proxy server through which all client requests pass (here we are interested in requests from the Docsvision Navigator application).
It is convenient enough to look at the speed of query execution, and, not least, this information can be used to diagnose performance problems. It is possible to say with great accuracy where there is a slowdown on the client (weak hardware, bad network channel, slow drawing of the interface) or on the server.
Fiddler allows you to figure out the following:
1. The number of server calls.
2. Duration of server calls.
3. The nature of the server logic called.
4. Distribution of server calls in time.
5. The distribution of the script's running time between the client and the server (it is very useful to find out exactly where the performance has subsided).
JetBrains dotTrace Performance
Our colleagues (in terms of software development) from JetBrains are developing quite a lot of useful tools for developers. One of the applications can be used to analyze system performance indicators.
DotTrace - proprietary profiler for tracking performance problems and memory bottlenecks in .NET applications
This tool is quite powerful and allows you to:
1. Use several profiling modes (including changing the run time of a subprogram thread).
2. Compare pictures (“traces”) taken from different systems (in some cases it is useful to compare pictures of two client workstations).
3. Keep statistics on the functions used and the time of their execution (find out what exactly worked non-optimally - search for a folder or preview a file).
4. To profile the memory occupied in the system.
In the context of Docsvision: during the search for performance problems, we recommend profiling and analyzing (with our help) both the client and server components of the platform. This tool is mainly used by developers to diagnose problem scenarios in their work.
Navigator log
If there is nothing at hand, then the application can be analyzed using its own tools. The application has the ability to enable its own diagnostic log.
Most often, the diagnostic log of the navigator is used to locate errors in the work, but in addition it contains information about all operations performed by the application "Docsvision Navigator". Those. only client operations of the specific application for which the log is being logged.
The log looks like this:
In the log should pay attention to:
• Opening the navigator
• The regular closing of the navigator (together with the operation “Opening the navigator” gives indirect information about the events of the Navigator crash)
• Opening time of the navigator
• Card opening time
• Duration of opening the folder node
• The duration of the construction of the presentation in the folder
Since reading the log using text editors is not always convenient, Docsvision administrators can use the Docsvision Navigator Log Parser utility. This is our own development, which can be requested from the technical support staff.
She is pleasant because:
• simplifies the process of analyzing the advanced log of the navigator, isolating only information about useful operations;
• knows how to separate the first (the first after the operation Navigator started) and the subsequent opening of the cards of documents and tasks;
• can select operations in a specified time / date range;
• The result of processing the extended log of the navigator is displayed in a separate csv file for further more detailed analysis.
Server part
During the research of the client application for performance, it may happen that, after analyzing the data of client applications and finding no problems, it is necessary to conduct an audit of the server part (DBMS). In this case, it will be necessary to conduct server-side profiling, focusing on typical MS SQL admin tools.
SQL SERVER profiler
The standard SQL Server Administrators utility allows you to view absolutely all requests to all databases in your possession.
With the use of certain profiling templates you can get a lot of useful information:
1. Find out what operations in the database and how long they are performed
2. Learn about the presence of dead locks (Deadlocks)
3. Learn the execution plan for queries and stored procedures
Of course, these methods are applicable not only to the Docsvision database, but also to any database in SQL. The data collected in this way helps not only to see the speed of certain operations, but also to push the Docsvision Administrator (along with the DBA) to take steps to optimize performance. In fact, this topic is quite extensive, and is suitable for a separate article. Administrators of MSSQL, armed with SQL Profiler and Database Tuning Advisor, can put their knowledge into practice, optimizing the work of the database.
Objectivity and representativeness of measurements
To obtain accurate results when measuring, I recommend to consider the following factors.
• Representativeness. It is enough to analyze the indicators of the user group without the need to analyze the indicators at all workplaces.
• Objectivity. Measurable indicators should be quantified and calculated without the influence of subjective factors in the automatic mode. This, for example, the failure of the "iron" part or problems in third-party software that interferes with the work (antivirus software, firewalls, point-to-point network routers).
• Selection of users.
When conducting measurements in the ERMS, it is sufficient to select active users (up to 10), satisfying the following (at least one) criteria:
EDS is the main working tool of the user;
- Users create documents and tasks;
- Users process documents (for example, register) and tasks (for example, complete);
- Users reveal different folder nodes;
- Users open standard and virtual folders.
At the same time: the group covers various scenarios of users in the system; the equipment of user workplaces meets the recommended requirements of a software developer; Users represent different working conditions with EDMS in terms of connection methods (local from different network segments, terminal).
Additions
No less important when measuring:
• pay attention to monitoring the conditions under which performance indicators are obtained (measurement of the load on the equipment of the ERMS): client computers of the group, network connection to the application server, server.
• analyze data on equipment loading when receiving unsatisfactory performance indicators.
• perform database profiling when detecting slow data operations on the servers.
Conclusion
Not only the actual tools developed (logging embedded in the platform), but also a wide range of familiar applications that will help in diagnosing workplaces in a short time are used to measure performance.
In certain situations, it is possible to use only embedded tools: this, for example, can be useful when working “in the field” on the customer side when there is no access to a wide range of tools and applications.
Third-party diagnostic tools should be used thoughtfully and commensurately assigned tasks.
Overlaid with numbers and graphs, you need not to break away from reality. It is desirable to keep the application running, so that the effect of optimization after measurements can be felt in combat conditions.
Source: https://habr.com/ru/post/246691/
All Articles