IncrediBuild to check a large project with PVS-Studio

PVS-Studio is a static analyzer that detects errors in the source code of C / C ++ applications. Like the compiler, the analyzer traverses the file behind the file in the directory with the project sources, performing its task. But without the use of external tools, the assembly of some projects may take several hours. Static analysis of such a project will take even more time. To build large projects, some developers have resorted to using a distributed assembly using the IncrediBuild tool. This article will not discuss the details of the integration of PVS-Studio into IncrediBuild, but will be told about checking a large project, measuring time and other interesting facts.

About PVS-Studio and IncrediBuild tools

PVS-Studio performs analysis of C / C ++ code and prompts the programmer where possible errors are located, or points to areas of code that may cause problems in the future.



The analyzer consists of several parts, and the PVS-Studio.exe process (the core of the analyzer) performs the verification of each source file. Further, I will use the term “stream”, meaning that one process PVS-Studio.exe is launched in one thread to check one source project file. And there will be a lot of files ...



IncrediBuild is a distributed computing software that makes it easy to use multiple computers and speed up your applications. You do not need to worry about the dependencies that are present in parallelized processes.

')

We will use IncrediBuild to perform project analysis in multiple threads using additional computers.

About the checked project

The project under review wished to be unnamed, but it has about 7 million lines of source code. The size of the code base is 300 MB, and about 9000 files. Verification of this project with the PVS-Studio analyzer on a computer with an Intel Core i7-4770 3.40 GHz processor and 16 GB of RAM in 8 threads takes about 6 hours. Then I will tell you how with the help of the IncrediBuild tool and several computers it was possible to greatly accelerate the time of project analysis.

results

In all tests, computers of approximately one configuration were used, allowing analysis to be performed in 8 threads on each of them.



When you configure IncrediBuild on computers, one of them is configured as a coordinator, the rest are configured as agents that connect to the coordinator. Since The project check was started on the agent, then on this computer in the settings it was set to load only 1 stream, so that the excessive load did not interfere with the collection of results. For the coordinator, it was also set to perform calculations only in 1 thread.

First test

For the first test launch of PVS-Studio, 5 computers were available:

• Coordinator (1 thread);
• Agent N1 (1 thread, it launched an analysis);
• Agent N2 (8 threads);
• Agent N3 (8 threads);
• Agent N4 (8 threads).

Total 1 + 1 + 8 + 8 + 8 = 26 streams for parallel launch of the analyzer for 26 files simultaneously.



Figure 1 shows the verification of the project by the analyzer in 26 streams.







Figure 1 - Verification of the project by the analyzer in 26 threads



For readers who are not familiar with IncrediBuild, I will explain that the green bar indicates 1 process that has successfully worked on a computer with the name that is written on the left. The strip is the name of the process. And on the timeline below you can determine at what point in time the process started and ended, thus the duration of any distributed process is immediately visible.



The image above captures the scan of rather large source files. Parallelization is performed very evenly and efficiently when loading streams for a long time.



Figure 2 shows an analysis of source code files of different sizes.







Figure 2 - Checking small and large project files in 26 streams



As we see, the rapid termination of threads leads to the overhead of creating new threads and there is a slight idleness of some computers. But in general, the parallelization is very good. The project analysis was completed in 2 hours and 10 minutes instead of 6 hours, very well.

Second test

For the second test run of PVS-Studio, 7 computers were available:

• Coordinator (1 thread);
• Agent N1 (2 threads, the analysis was launched on it);
• Agent N2 (8 threads);
• Agent N3 (8 threads);
• Agent N4 (8 threads);
• Agent N5 (8 threads);
• Agent N6 (8 threads).

Total 1 + 2 + 8 + 8 + 8 + 8 + 8 = 43 threads for parallel launch of the analyzer for 43 files simultaneously.



Figure 3 shows the verification of the project by the analyzer in 43 threads.







Figure 3 - Verification of the project by the analyzer in 43 threads.



The list of 43 threads cannot be seen in its entirety, but it is clearly seen how the idleness of computers increased when checking small files. But it still looks very cool, considering that the total analysis time was already 1 hour and 24 minutes instead of 6 hours, and the only effort was to install IncrediBuild on the right computers. To configure PVS-Studio, it was necessary to change only 1 item in the settings - set the number of threads.

Conclusion

Using the IncrediBuild tool, which is usually present for assembling very large projects, we were able to speed up the project analysis time using PVS-Studio 4.2 times, using several development computers that were sitting nearby. All IncrediBuild settings were used by default, and there are quite a lot of them. Surely you can achieve even greater performance, but we were interested in a “quick start”.



Once again, it took us to quickly check the project with PVS-Studio:

1. Install IncrediBuild on the right computers.
2. Increase the number of threads in the settings of PVS-Studio.
3. Run the analysis from the IncrediBuild console.

Results:

1. Check on 1 computer - 6 hours.
2. Check in 26 threads - 2 hours 10 minutes.
3. Check in 43 threads - 1 hour 24 minutes.

You can read about how to use PVS-Studio and IncrediBuild in your project in the article: " Integration of PVS-Studio with the IncrediBuild distributed assembly system ".





If you want to share this article with an English-speaking audience, then please use the link to the translation: Svyatoslav Razmyslov. Using IncrediBuild to Assist Analysis of a Large Project by PVS-Studio .