Another testing of PVS-Studio
I have been interested in static code analyzers a long time ago, from the first acquaintance with Klocwork about six years ago. Besides him, I had some time to work with PCLint. And now I have tested PVS-Studio a little. (And next in line are Parasoft and Coverity).
I conducted testing exclusively for the love of art - it was interesting to try another analyzer.
PVS-Studio seems to be focused exclusively on working with Visual Studio. After installation, it is automatically integrated into the installed version of the studio. This completes all the necessary integration steps. It remains to launch the studio itself - and in it there will already be a new menu item of PVS-Studio. Such transparent integration is beyond praise!
To start the analyzer, it is enough to select, for example, Check Solution from this menu. The process, I must say, is not fast - on my not very big project it took 18 minutes (for comparison, the complete assembly of the project lasts about 2 minutes). Of the benefits - the analyzer shows an adequate progress bar and the remaining time!
')
But the fact that the code analysis is automatically added to the assembly, I was slightly surprised. Especially due to the fact that except for the small tray icon, this was no longer visible. At first glance, nothing happens, the assembly is completed, and in the PVS-Studio menu for some reason all the menu items turned gray. For some reason, there was no beautiful window with a progress bar.
Output of the analysis results is in a separate tab of PVS-Studio. Everything has been done quite comfortably: for each found (potential) problem, a description is displayed, a place (file and string), and a link to a detailed description of problems of this type: why it’s a problem, how it can be fixed, etc. The descriptions are made in great detail and with high quality!
Double-click the studio to correctly transfer you to the problem area. However, if the problem affects more than one place, then the second will have to be found manually.
Also, for each problem, you can indicate that in this case there was a false positive response, or turn off the display of problems of this class altogether.
The menu item “Incremental analysis after build”, in my opinion, does not quite what I would expect. If this item is included, the analysis will be done only for the changed part of the project (the unit of change is the file, ie, changing one line in one function in one file, you will see all the problems found in this file). I would expect to see a comparison of the states between “was - has become”, although maybe the above described mode of operation also makes sense.
It seems that there is no (at least I did not see) an opportunity to see the result of the next build / fix, like “there were 40 problems, after fixing 6 problems disappeared, 2 new ones were added”. Without this, it is very difficult to track the dynamics of the project.
You can save the report either to internal format or to plaintext. After saving it and opening it in a text editor, I was very surprised at first that there were many more problems in the report than was shown in the studio. But then I realized what was the matter - by default, the studio only displays “General analysis results”, and besides this there is also “64 bit analysis results”, which is turned off by default, but falls into the saved report.
Well, he returned to the studio and looked at the problems of readiness for the 64th bits. Impressed. I can definitely recommend to those who are planning to switch to a 64-bit platform, the analyzer finds many unobvious moments that you don’t even think about, programming in a 32-bit system, but which will lead to a problem when switching to 64 bits. For example, “V104. Implicit type conversion to memsize type in an arithmetic expression ”or“ V102. Usage of non-memsize type for pointer arithmetic ”, for a code that is quite harmless at first glance:
From the disliked - too tight integration with Visual Studio. The documentation was so explicitly written that working with cross-platform compilers is very complicated and not supported, which I didn’t even try, although I would like to. Theoretically, it is possible to start PVS-Studio from the command line, giving it compiler options and a list of files, but - firstly, it is not so easy if there are many files, and secondly, the documentation for gnu compiler is stated as missing, although it is possible With some effort, it will be possible to force the analyzer to output something on the project under the cross-platform compiler.
And finally, a brief comparison of PVS-Studio with PC-Lint and Klocwork.
PVS-Studio is much more convenient to use than PC-Lint. The main problem of PC-Lint is too many potential problems, on the same project with default settings it gives out more than 23 thousand messages! (PVS-Studio - only 42). At the same time, PC-Lint cannot hide recurring problems (if the same .h file is connected from two .c files, PC-Lint will show the problem in the .h file twice, and PVS-Studio only once). And in PC-Lint, it is impossible to mark a separately taken problem as false positive, you can only “turn off” the entire class of problems.
Moreover, if you delve into the grief of the “garbage” issued by PC-Lint, you can find there more real potential problems than PVS-Studio finds (which, for example, did not find the absence of virtual destructors in a couple of classes, as well as the absence of clearly defined copy constructors and default constructors). Another thing is that in PC-Lint, these useful nuggets usually “sink” among the other, not so critical messages.
At the same time, PC-Lint is more versatile, although much more complicated to use, although its integration into Visual Studio is also quite simple.
But in general, here PVS-Studio is the undisputed leader. For it is possible to simply start working with it, which cannot be said about PC-Lint, where it can take a couple of days just to see all the messages issued!
But compared to Klocwork, PVS-Studio, alas, loses. First of all, for ease of use (I’ll only note state tracking from build to build - how many problems were fixed, how many were left and how many were added, as well as convenient reports), universality and better finding of critical potential problems. In addition, Klocwork is a server-based multi-user system, with the ability to assign those responsible for each problem found and work with several projects at once, easily tracking the status of each.
If you compare prices, you’ll get the following layout (information from the developer’s sites):
Well, the price roughly corresponds to the functionality, PVS-Studio is somewhere in the middle between cheap, but heavy and inconvenient to use PC-Lint and convenient and beautiful, but expensive Klocwork.
And finally - a couple of examples of actually found errors in the project, over which both Klocwork and PC-Lint have already managed to “make fun”, and which is already at the exploitation stage.
By the way, I don’t provide screenshots due to the fact that I tested all this on a live commercial project, and the screenshots contain too much “closed” information.
For example, PVS-Studio is well aware of the problems of this type (and the above-mentioned analyzers do not do this, as far as I have noticed!):
V519. The 'x' variable is assigned values ​​twice successively. Perhaps this is a mistake
In my case, one of these identified problems led to this code:
Which, of course, should look like this:
And this, by the way, is on a fairly well-established, tested and long-commissioned product! On the whole, all the other identified similar problems, although they were not obvious errors, at least pointed to, so to speak, not the highest quality, understandable and beautiful code:
or
Also, PVS-Studio well detects such places in the code (although not an error, but I would prefer that there is no such thing in the code):
telling them that: "The 'then' statement is equivalent to the 'else' statement"
Or this (here m2 and m1 of type float):
saying that “an odd comparison. It is probably better to use the Epsilon or fabs (A - B)> Epsilon
In general, if you have about 30 developers in the department and you write mostly in Visual Studio, and you don’t have 30 thousand euros for Klocwork, then PVS-Studio is a very good choice!
I conducted testing exclusively for the love of art - it was interesting to try another analyzer.
PVS-Studio seems to be focused exclusively on working with Visual Studio. After installation, it is automatically integrated into the installed version of the studio. This completes all the necessary integration steps. It remains to launch the studio itself - and in it there will already be a new menu item of PVS-Studio. Such transparent integration is beyond praise!
To start the analyzer, it is enough to select, for example, Check Solution from this menu. The process, I must say, is not fast - on my not very big project it took 18 minutes (for comparison, the complete assembly of the project lasts about 2 minutes). Of the benefits - the analyzer shows an adequate progress bar and the remaining time!
')
But the fact that the code analysis is automatically added to the assembly, I was slightly surprised. Especially due to the fact that except for the small tray icon, this was no longer visible. At first glance, nothing happens, the assembly is completed, and in the PVS-Studio menu for some reason all the menu items turned gray. For some reason, there was no beautiful window with a progress bar.
Output of the analysis results is in a separate tab of PVS-Studio. Everything has been done quite comfortably: for each found (potential) problem, a description is displayed, a place (file and string), and a link to a detailed description of problems of this type: why it’s a problem, how it can be fixed, etc. The descriptions are made in great detail and with high quality!
Double-click the studio to correctly transfer you to the problem area. However, if the problem affects more than one place, then the second will have to be found manually.
Also, for each problem, you can indicate that in this case there was a false positive response, or turn off the display of problems of this class altogether.
The menu item “Incremental analysis after build”, in my opinion, does not quite what I would expect. If this item is included, the analysis will be done only for the changed part of the project (the unit of change is the file, ie, changing one line in one function in one file, you will see all the problems found in this file). I would expect to see a comparison of the states between “was - has become”, although maybe the above described mode of operation also makes sense.
It seems that there is no (at least I did not see) an opportunity to see the result of the next build / fix, like “there were 40 problems, after fixing 6 problems disappeared, 2 new ones were added”. Without this, it is very difficult to track the dynamics of the project.
You can save the report either to internal format or to plaintext. After saving it and opening it in a text editor, I was very surprised at first that there were many more problems in the report than was shown in the studio. But then I realized what was the matter - by default, the studio only displays “General analysis results”, and besides this there is also “64 bit analysis results”, which is turned off by default, but falls into the saved report.
Well, he returned to the studio and looked at the problems of readiness for the 64th bits. Impressed. I can definitely recommend to those who are planning to switch to a 64-bit platform, the analyzer finds many unobvious moments that you don’t even think about, programming in a 32-bit system, but which will lead to a problem when switching to 64 bits. For example, “V104. Implicit type conversion to memsize type in an arithmetic expression ”or“ V102. Usage of non-memsize type for pointer arithmetic ”, for a code that is quite harmless at first glance:
uint8 * pRecv; int cur = 0; // somewhere later... pRecv += cur; From the disliked - too tight integration with Visual Studio. The documentation was so explicitly written that working with cross-platform compilers is very complicated and not supported, which I didn’t even try, although I would like to. Theoretically, it is possible to start PVS-Studio from the command line, giving it compiler options and a list of files, but - firstly, it is not so easy if there are many files, and secondly, the documentation for gnu compiler is stated as missing, although it is possible With some effort, it will be possible to force the analyzer to output something on the project under the cross-platform compiler.
And finally, a brief comparison of PVS-Studio with PC-Lint and Klocwork.
PVS-Studio is much more convenient to use than PC-Lint. The main problem of PC-Lint is too many potential problems, on the same project with default settings it gives out more than 23 thousand messages! (PVS-Studio - only 42). At the same time, PC-Lint cannot hide recurring problems (if the same .h file is connected from two .c files, PC-Lint will show the problem in the .h file twice, and PVS-Studio only once). And in PC-Lint, it is impossible to mark a separately taken problem as false positive, you can only “turn off” the entire class of problems.
Moreover, if you delve into the grief of the “garbage” issued by PC-Lint, you can find there more real potential problems than PVS-Studio finds (which, for example, did not find the absence of virtual destructors in a couple of classes, as well as the absence of clearly defined copy constructors and default constructors). Another thing is that in PC-Lint, these useful nuggets usually “sink” among the other, not so critical messages.
At the same time, PC-Lint is more versatile, although much more complicated to use, although its integration into Visual Studio is also quite simple.
But in general, here PVS-Studio is the undisputed leader. For it is possible to simply start working with it, which cannot be said about PC-Lint, where it can take a couple of days just to see all the messages issued!
But compared to Klocwork, PVS-Studio, alas, loses. First of all, for ease of use (I’ll only note state tracking from build to build - how many problems were fixed, how many were left and how many were added, as well as convenient reports), universality and better finding of critical potential problems. In addition, Klocwork is a server-based multi-user system, with the ability to assign those responsible for each problem found and work with several projects at once, easily tracking the status of each.
If you compare prices, you’ll get the following layout (information from the developer’s sites):
- for PVS-Studio they ask for € 3,500 euro for 5 licenses or € 9000 for 30, for a year of use.
- for PC-Lint - $ 389 for one license or $ 3,500 - for 10, indefinitely.
- for Klocwork - € 30,000 for the package "server + 20 clients" for the year of use.
Well, the price roughly corresponds to the functionality, PVS-Studio is somewhere in the middle between cheap, but heavy and inconvenient to use PC-Lint and convenient and beautiful, but expensive Klocwork.
And finally - a couple of examples of actually found errors in the project, over which both Klocwork and PC-Lint have already managed to “make fun”, and which is already at the exploitation stage.
By the way, I don’t provide screenshots due to the fact that I tested all this on a live commercial project, and the screenshots contain too much “closed” information.
For example, PVS-Studio is well aware of the problems of this type (and the above-mentioned analyzers do not do this, as far as I have noticed!):
V519. The 'x' variable is assigned values ​​twice successively. Perhaps this is a mistake
In my case, one of these identified problems led to this code:
Year = Payload[1]; Month = Payload[2]; Year = Payload[3]; Which, of course, should look like this:
Year = Payload[1]; Month = Payload[2]; Day = Payload[3]; And this, by the way, is on a fairly well-established, tested and long-commissioned product! On the whole, all the other identified similar problems, although they were not obvious errors, at least pointed to, so to speak, not the highest quality, understandable and beautiful code:
txtL = _T("None"); txtL = _T("Not found"); or
POSITION pos = m_items.GetHeadPosition(); pos = m_ownedItems.GetHeadPosition(); Also, PVS-Studio well detects such places in the code (although not an error, but I would prefer that there is no such thing in the code):
if (m_sc) { result = DoubleToString(m_highLimit,3); } else { result = DoubleToString(m_highLimit,3); } telling them that: "The 'then' statement is equivalent to the 'else' statement"
Or this (here m2 and m1 of type float):
if ((m2 - m1) == 0.0) { doSomethingHere(); } saying that “an odd comparison. It is probably better to use the Epsilon or fabs (A - B)> Epsilon
In general, if you have about 30 developers in the department and you write mostly in Visual Studio, and you don’t have 30 thousand euros for Klocwork, then PVS-Studio is a very good choice!
Source: https://habr.com/ru/post/135397/
All Articles