Klocwork - two week test experience
I did not have time to share my impressions of PVS-Studio, as the advertising charges immediately followed. Well, I will tell about my first experience of using systems of static code analysis.
It so happened that the first system I encountered was Klocwork. It happened 8-9 years ago, when I worked in the Ukrainian branch of a Slovenian company. Then the analyzer was launched at the head office, we only got the results exported to Excel. But even in this form, I remember, I was pleasantly surprised by how well the potential (and sometimes real) problems were.
And so (already back in 2010), having seen the stand of Klocwork itself on the Embedded World, I came up to ask them about the trial license for “try”. After some time I received from them an e-mail with the question: "Are you interested, still want?".
')
And then I just got a “legacy” of a decent amount of code, which, to put it mildly, was ... a little crookedly written. Yes, and the project was in the “First Release Released” stage, there was just time to look for possible problems.
Therefore, willingly agreed. I received a trial installation and integration instructions. I will only note that if integration with VS went off with a bang, in one click, then the integration with FreeScale's CodeWarrior made me sweat both me and technical support (which, to their credit, tried their best to help me by e-mail, while that the product was a trial, without any guarantee that it will be bought!). As a result, after two weeks of attempts, integration was still done (manually - too, CodeWarrior turned out to be a thing in itself). And finally, a license was obtained for two weeks, allowing you to do something for which everything was started - static code analysis.
In a nutshell about the system itself - there is a central server on which the so-called are started “Builds”, and there are web clients that can view the results of each “build” analysis in the browser. The license must be purchased both on the server and on each client. And the license is limited in time. When purchasing 20 client licenses, the server one is free of charge, such a package was obtained at a price of about 30 thousand euros per year.
I was very pleasantly surprised by the results.
In the “first release released” stage, the analyzer found 73 potential problems in three projects!
There were only 6 false positives.
Here is a list of problems found (which were actually present in the code):
It is clear that we are not talking about the problems that really happen, in most cases we are talking only about potential problems.
Here is an example of (almost real) code in which the analyzer shows the possibility of a buffer overflow:
For the code above, the analyzer displays the message “Buffer overflow: array of data”. Array 'data' of size 260 may use index values ​​from 260..65534 "
Specifying a detailed path from the beginning of the problem function through all possible branches, along with what values ​​the variables should have at each branch point, so that
The specified problem happened.
With convenient syntax highlighting and problem areas, with an adequate response to a click almost everywhere, it is very easy to find and understand exactly where the problem is and how exactly it can happen.
Unfortunately, I can not post screenshots, because they are all made on a real project and a real code, which cannot be shown in the public domain: (While doing the same report for the management two years ago, I did not take care to stock up on screenshots of the dummy code for a possible open publication.
In the above example, no real buffer overflow occurred - in all function calls the correct value of the parameter len was passed. The analyzer found in this case
only a potential problem.
But - if you are not the first year in IT, then you certainly know that with each new potential problem like this, the likelihood that one of them after another code change
It turns from potential to real (having provided a couple of pleasant weeks of debugging of the project that suddenly fell on the final demonstration at the customer), very quickly reaches a value close to one.
And from the "goodies" of this code analyzer:
The analyzer supports C / C ++, C #, Java, although I only tested the C / C ++ part.
Can I do without such a utility? It is possible, if everyone who writes C / C ++ code has read and understood all Myers, while constantly keeping his recommendations in mind and are able to quickly identify violations of these recommendations in an already written code. In reality, it seems to me, oh, this is unrealistic. And the use of such a tool is very able to help identify errors and potential errors almost immediately after they occur.
The price ... Yes, the price is definitely high. But for a large company, simultaneously leading 20 or more projects, the price when buying the most profitable package, in terms of the project will not be so high. Yes, and certainly the originally named price is far from final and can be lowered by negotiation if it comes to a real purchase. If we also take into account that one such potential problem, which at one fine moment became real, can easily “eat” two or three weeks of the work of the lead programmer (or even not one), and even worse (if it “crawls out” after release) can lead to a very, very tangible loss, far exceeding the cost of the analyzer ...
In my case, the leadership failed to convince. Well, at least on one of the projects there were few potential problems, and I got an interesting experience :)
And yes, both this post and the post about PVS-Studio are written solely out of interest to the topic. Neither in this, nor in that case did I receive any material compensation from the authors for writing the review, if someone suddenly doubts my impartiality.
It so happened that the first system I encountered was Klocwork. It happened 8-9 years ago, when I worked in the Ukrainian branch of a Slovenian company. Then the analyzer was launched at the head office, we only got the results exported to Excel. But even in this form, I remember, I was pleasantly surprised by how well the potential (and sometimes real) problems were.
And so (already back in 2010), having seen the stand of Klocwork itself on the Embedded World, I came up to ask them about the trial license for “try”. After some time I received from them an e-mail with the question: "Are you interested, still want?".
')
And then I just got a “legacy” of a decent amount of code, which, to put it mildly, was ... a little crookedly written. Yes, and the project was in the “First Release Released” stage, there was just time to look for possible problems.
Therefore, willingly agreed. I received a trial installation and integration instructions. I will only note that if integration with VS went off with a bang, in one click, then the integration with FreeScale's CodeWarrior made me sweat both me and technical support (which, to their credit, tried their best to help me by e-mail, while that the product was a trial, without any guarantee that it will be bought!). As a result, after two weeks of attempts, integration was still done (manually - too, CodeWarrior turned out to be a thing in itself). And finally, a license was obtained for two weeks, allowing you to do something for which everything was started - static code analysis.
In a nutshell about the system itself - there is a central server on which the so-called are started “Builds”, and there are web clients that can view the results of each “build” analysis in the browser. The license must be purchased both on the server and on each client. And the license is limited in time. When purchasing 20 client licenses, the server one is free of charge, such a package was obtained at a price of about 30 thousand euros per year.
I was very pleasantly surprised by the results.
In the “first release released” stage, the analyzer found 73 potential problems in three projects!
There were only 6 false positives.
Here is a list of problems found (which were actually present in the code):
- no explicit assignment statement or copy constructor for classes using dynamic memory;
- buffer overflow;
- memory leaks;
- dereferencing an uninitialized pointer;
- use of an uninitialized variable;
- null pointer dereference;
- memory use after release.
It is clear that we are not talking about the problems that really happen, in most cases we are talking only about potential problems.
Here is an example of (almost real) code in which the analyzer shows the possibility of a buffer overflow:
void function (int len, char * buf) { char data[DATA_SIZE]; for (int i = 0; i < len; ++i) { data[i] = buf[i]; } } For the code above, the analyzer displays the message “Buffer overflow: array of data”. Array 'data' of size 260 may use index values ​​from 260..65534 "
Specifying a detailed path from the beginning of the problem function through all possible branches, along with what values ​​the variables should have at each branch point, so that
The specified problem happened.
With convenient syntax highlighting and problem areas, with an adequate response to a click almost everywhere, it is very easy to find and understand exactly where the problem is and how exactly it can happen.
Unfortunately, I can not post screenshots, because they are all made on a real project and a real code, which cannot be shown in the public domain: (While doing the same report for the management two years ago, I did not take care to stock up on screenshots of the dummy code for a possible open publication.
In the above example, no real buffer overflow occurred - in all function calls the correct value of the parameter len was passed. The analyzer found in this case
only a potential problem.
But - if you are not the first year in IT, then you certainly know that with each new potential problem like this, the likelihood that one of them after another code change
It turns from potential to real (having provided a couple of pleasant weeks of debugging of the project that suddenly fell on the final demonstration at the customer), very quickly reaches a value close to one.
And from the "goodies" of this code analyzer:
- Ability to track every problem found. Those. if the problem has been fixed, the analyzer will detect it in the next build and will not show it anymore. And it will also reflect this on a beautiful graph of the number of problems that have appeared, remained-corrected from build to build.
- The ability to change the status of each problem. Those. if you find that this is a false positive, you can mark it as “no problem”, and the analyzer will not show it to you anymore.
- The possibility of appointing a person responsible for each issue.
- Ability to build code metrics (and display trend graphs from build to build).
- The ability to export results to Excel. In this case, very convenient interactivity disappears, but all the necessary information in order to find and fix problems remains. Conveniently, if there is no “extra” license.
- Customizable reports - you can choose what to show in the reports.
The analyzer supports C / C ++, C #, Java, although I only tested the C / C ++ part.
Can I do without such a utility? It is possible, if everyone who writes C / C ++ code has read and understood all Myers, while constantly keeping his recommendations in mind and are able to quickly identify violations of these recommendations in an already written code. In reality, it seems to me, oh, this is unrealistic. And the use of such a tool is very able to help identify errors and potential errors almost immediately after they occur.
The price ... Yes, the price is definitely high. But for a large company, simultaneously leading 20 or more projects, the price when buying the most profitable package, in terms of the project will not be so high. Yes, and certainly the originally named price is far from final and can be lowered by negotiation if it comes to a real purchase. If we also take into account that one such potential problem, which at one fine moment became real, can easily “eat” two or three weeks of the work of the lead programmer (or even not one), and even worse (if it “crawls out” after release) can lead to a very, very tangible loss, far exceeding the cost of the analyzer ...
In my case, the leadership failed to convince. Well, at least on one of the projects there were few potential problems, and I got an interesting experience :)
And yes, both this post and the post about PVS-Studio are written solely out of interest to the topic. Neither in this, nor in that case did I receive any material compensation from the authors for writing the review, if someone suddenly doubts my impartiality.
Source: https://habr.com/ru/post/131245/
All Articles