PVS-Studio, CppCat. Interview with Andrei Karpov, technical director of these projects

From Habré from time to time there are articles devoted to the static analyzer of the PVS-Studio code (other analyzers are also considered) in which tests of a project are made every time with the publication of the problem areas found. Not so long ago, a new CppCat product was introduced - a light version for a small, relatively PVS-Studio, money. Some description of the PVS-Studio project for Visual C ++ can be found here and here , and a description of the new project can be found in the article “Alternative to PVS-Studio for $ 250” .

This article will not be focused on all sorts of technical details related to products, but on product positioning on the market on a particular platform, a little about the company, etc.

In many articles about the company's products in the comments you can find questions about the support of a platform such as Linux. Here this question is also touched upon and an exhaustive answer is received.
')
Andrey Karpov Andrey2008 - technical director of projects PVS-Studio and CppCat answers the questions I asked. Andrew also shares some thoughts about programming language, products of other companies and interaction with these companies.

Preamble

If you observe the development of the programmer, you can sometimes notice some pattern. At the beginning of the journey, the future developer studies some kind of literature, asks something on the forums. At a certain point, for some step forward, he gets a kick in the back of a question like: “Have you ever tried to open a debugger for yourself?”. Having discovered this one of the many, but at the same time, basic tools, the developer continues his path of improvement. At a certain point, he comes to the idea or he is told that it is possible to speed up his work by reducing the time to search for errors if you look for them at the initial stages using a static code analyzer, and not to look for errors in the finished product, setting WinDbg on it and doing all sorts of exercises in sophisticated debugging applications. At this point, an overview of available funds on the market, both paid and free, begins. This review may also include this material.

We are talking ...

I followed the link to “My Circle” specified in Andrew’s profile. Judging by the read information, the company is not big. Maybe this is its plus ... It is usually a minimum of bureaucracy, greater involvement in the process, more creative work, etc. Small companies have their own charm in comparison with large ones. On the positive aspects of small business there is a good article .

Andrew, please tell us a little about the company: how many employees are there at the moment, how many are involved in the development of the company's products directly, and a little more about what you consider necessary to indicate in general terms?

Andrei: Yes, we are a small company. Of course, we want to become big and rich. But this desire is fulfilled slowly. So to speak. We are satisfied with what we have, but really, really want more. Perspectives are visible, but they resist us reaching them.

There are also obvious advantages: a calm atmosphere and no need to adapt, for example, to the customer or the release date invented by the marketing department. Very quick response to user emails. Of course, over time, the company can become fat and acquire bureaucracy. But while our decisions and actions are swift. We have repeatedly surprised the user by giving him the distribution on the same day when he wrote to us about the detected error.

Now directly in the development and promotion of PVS-Studio / CppCat 4 people participate. Then I find it difficult. Employees or in combination, such as an accountant, or perform work remotely. I remember the term "half miner".

Some practices of companies show that one of the company's success is that they are focused primarily on the product, on its quality, and, as it were, on what I would like to have for myself. Tell me, in the company, its products are used to develop its own products, i.e. When developing a new version, PVS-Studio is used to search for problems the same PVS-Studio, but a stable, salable version? Could you give some interesting examples when the product helped in finding problem areas in the development version?

Andrey: I think that PVS-Studio and CppCat are very high-quality products. We can make and upload a distribution kit almost any day. We use a very large number of automated tests of various levels and types. If the revised version has passed all the tests, then it can be safely issued to users.

Unfortunately, all the beauty and reliability of the internal code sometimes falls apart due to the impacts of a hostile environment. As a result, the entire impression of the product spoils. It seems we are not to blame, but it is our product that does not work. I can cite just an infinite number of examples. The first thing that comes to mind is:

• A third-party tool spoils something in the Visual Studio environment. As a result, you have to create a crutch to work around a problem, or accept it and respond, "sorry, we can do nothing." One such example is the "Description of the Intel Parallel Studio Service Pack 1 Integration Error in Visual Studio 2005/2008 . "
• Visual Studio COM interfaces for project information may unexpectedly throw an exception. Apparently, the environment at this unfortunate moment is busy with something else. Challenges have to be wrapped in a loop for multiple repetitions. Dances with a tambourine, which do not always help.
• A developer has antivirus X and, according to corporate policy, he does not have the right to change his settings. This antivirus "holds" for some time some temporary files, and the analyzer cannot delete them. The result is that the analyzer "shits" in the project folder.
• Miscellaneous can be remembered. For example, you can see here , here and here again.

However, our readers are waiting for me to show the errors that PVS-Studio found in itself. I have been repeatedly asked to write an article about it. Unfortunately, I still can not save enough examples of errors. To be honest, I trivially forget to write them out when they are detected by incremental analysis. I saw, corrected and went further over the algorithm to think. The next day I remembered that it would be nice to write out such warnings. Yes, I do not remember where and what happened.


The article “Alternative to PVS-Studio for $ 250” presents a new product of the company - CppCat. At the very beginning it is stated that the project was created almost "from scratch".

Many products have a kernel around which everything is built, as, for example, for antivirus software - this is its antivirus engine. As a rule, this is one core for several products of the company. A whole system is built around this kernel from all sorts of logic, interface and various utilities for the user (for example, all the same report generation) - all this is a product that differs from other products of the company built on the same engine.

Would you like to know PVS-Studio and CppCat - are these products built on the same core? Should users expect the same quality of code analysis from a new product as from an old one in the segment to which it is oriented? If both products are built on the same core, then the user of a simplified and cheaper product gets some advantage in that he will receive the same defect fixes and innovations in the analyzer as the user of a complex product. Can this be said about CppCat?

Andrei: Yes, PVS-Studio and CppCat are built on the same core. But the interface is reworked very significantly.

In terms of general purpose analysis, they are very close. CppCat disables the most unsuccessful diagnostics, which gave a lot of noise. And not the fact that it is bad. It is better to correct 5 errors by viewing 10 messages than to inattentively scroll through the list of 100 warnings and correct only 2 errors.

Regarding the fact that CppCat will be partially delayed by PVS-Studio users. Yes, it will probably happen, but not massively. Many of our users need specific functionality. Someone buys PVS-Studio due to 64-bit diagnostics, someone needs a modified metric “cyclomatic complexity” (non-public functionality). Someone uses MSBuild and so on.

Speaking about testing large projects in PVS-Studio and CppCat, could you give some indicative figures, for example, products were tested on projects with a number of subprojects, say, just over 500-700, whose code in XXX MB can be collected on a machine with some specified configuration for such a time period?

Yes, it is quite possible that many projects are much smaller, but the developers of large software systems might well be interested in such figures.

Andrei: We did not deal with the special collection of such statistics. Perhaps in the future it is worth doing. Now I can give the following numbers.

If you take a test suite for working with Visual Studio 2010, which includes the following projects:

• CamStudio
• CrashRpt
• Dosbox
• ffdshow
• IPP Samples
• Logs
• Miranda
• MongoDB
• mpc-hc
• Shareaza_vc10
• STweet
• XUI-GUI Fraimework
• Zlib_IA64

All of them are checked on my working machine in 55 minutes . The source code of these projects is about 320 megabytes .
Configuration:

• Intel Core i7-4770 (3.40GHz, 4x256Kb + 8Mb, EM64T, GPU), number of cores - 4, number of threads - 8;
• 16 gigabytes of memory (2 strips: DDR3 8Gb Kingston PC10600, 1333 MHz);
• Everything related to the tests is on the SSD.

The pricing policy of CppCat involves the purchase of a license for some fixed fee, but for a year. For some products, the annual license also has a place to be, but every year the company may issue a new version of the product with the same license price. Whether there is or is thought to stimulate users with updates, new versions, etc .: all that the user would have more desire to purchase a product with an annual payment for its use instead of a product for which you can pay once and use as much as you want , for example, until the realization of the need to purchase a new version?

Andrei: We consider the sale of an “endless” license unacceptable. It seems to us that we are close in spirit to antivirus programs. The analyzer constantly learns to catch new errors, learns to maintain innovations in the C ++ language, supports new Visual Studio environments and new versions of Windows.

In a sense, we sell an “error prevention service”. This service requires constant investment and effort from our side. For this, honestly ask for the purchase of updates.

I foresee the objection: “I don’t need updates. I want to buy once and use as much as I want later. ” This is stupid. It is strange to use antivirus three years ago. Fresh antivirus will find a lot more viruses. So the new analyzer will find more errors than the old one. Anyone who asks for an "infinite analyzer" actually wanted to spit on the number of errors in his code. He does not need an analyzer and you can ignore it.

You offer to try CppCat for 7 days for free. It’s not always possible for a developer with his download to appreciate a product in 5 days, if he does it in the workplace in order to make a proposal to purchase the product to his superiors.

There is often a situation when the need for a product is determined by the management team without taking into account the opinions of the employees (direct users of the product), but there are also situations when it is the expressed needs of the employees that are the determining factor for purchasing a particular product.

Would you like to increase the period to a month and expand the target audience for advertising through direct user-programmers who would make their purchase offers to their management?

Andrew: Regarding the increase in time. Controversial moment. When a developer has 5 days, he remembers that he needs to try the tool. When he has a month, he will always be “once” or “lazy.” Short term stimulates to action.

And why don't you implement some online service similar to online compilers with the goal of simply seeing how the analyzer responds to some code fragment? Or also an online service, sending to which some test project or even a real one, the user could use the verification of his project without installing the software on his workstation. Here you can also organize some interesting model, of which there are already quite a few for, for example, web development: to give some restrictions on the project for free use and to offer to pay a fee for the absence of these restrictions. The fact that many people refuse to install software locally and go online is unlikely to me, because security policies of companies that protect their intellectual property, will not be their huge projects to fill in an online-resource. But it is quite possible that the audience of small projects would expand without paranoid protection of their mental work, and the resource could become a test and, as a result, an advertising platform for the company's main products. In addition, an interested developer could try products both at work and at home, since There is no license (even a trial license) to the identifier of its workstation.

Andrew: We do not like any ideas related to online. Here is the line of reasoning.

There is a person who develops applications in the Visual Studio environment. What could be simpler than running CppCat to analyze your project? A person either likes the result or not. If you like, he can buy a tool. Online service here is like a fifth leg dog.

Fantasies on the subject of “pouring a project somewhere and checking it” are very difficult to implement and the main thing is not clear why. The analyzer needs a project that can be compiled. Otherwise, he will not know what types what they mean, what preprocessor directives are set, how to open macros, and so on. This means that somewhere you will need to configure a complete environment. It is impossible to do this automatically. It is necessary to force a person to fill in the libraries used in the project, to create a certain analogue of the project in the web system, so that the analyzer knows about the preprocessor directives and so on. It turns out that a person needs to create and maintain a “parallel project” somewhere in the clouds. And we need to keep all this charm somewhere and also maintain it.

The only thing you can use online analyzer for is experimenting. But it is better not to do it. Programmers are very one-sided approach to such tests. I even wrote a note on this subject: “Myths about static analysis. The fifth myth - you can create a small program to evaluate the tool . " A person can easily make a negative conclusion based on one, two examples. So it's better that he checks his real project and finds errors in it.

CppCat is offered for use with some versions of MS Visual Studio, which determines its use platform - Windows and a whole class of applications - desktop software. Yes, Windows exists for both workstations and servers, but if you look at all sorts of publications about a major web resource or a new built cluster, you can basically see a lot of information about Linux on the server software segment. You may notice that the success of Windows in the server segment is similar to the success of Linux in the desktop software segment. Among the desktop applications for Windows, C ++ has more and more competitors; on the C ++ server software segment, it also does not lack competition. I would like to know your position regarding the release of products for Linux and market penetration of server software.

There was an interesting article about trying to sell your software to Google and checking the Chrome browser as an example of the utility of your products. Are there any new developments in this issue? Do you consider the possibility of cooperation with large companies in promoting your products to the server software market taking into account the release of versions under the * nix platform? There are also domestic players: Yandex, Mail.ru, etc.

Andrei: At first, I wanted to write something in the spirit of: “We do not have a Linux version and are not expected. The reason can be considered our incompetence, laziness and tyranny. Just leave us alone please. ” Then I decided that I’ll just ask you to remove Linux questions from the interview. But then a colleague asked: "Why am I saying that there is no Linux version, when here it is currently running in his virtual machine?" I realized that I still need to give a detailed answer to this topic.

We are a small and independent company. We do not have funding from other parent organizations or government grants. We can very easily cease to exist if we start doing stupid things. It is enough that expenses within 3 months exceed revenues and can be closed. One of such silly actions can be inflating the infrastructure in order to implement the “Linux version”.

Programmers stubbornly do not want to understand and realize that a compiled executable file is not the same as a software product. It sounds trite, but people really do not notice the difference in size with the elephant.

We have an assembly for the command-line version of the analyzer for openSUSE 13.1. It starts and even displays warnings. But we do not have a "Linux version". We do not have such a software product and are not expected in the foreseeable future.

Fortunately, I and Evgeniy EvgeniyRyzhkov combine not only programmers, but also managers. Otherwise we, as a company, would have disappeared long ago. We can foresee how technical enthusiasm can ruin everything.
Recall what makes up our software product:

1. Operating system support. Well, this is relatively simple. We took and compiled the program. But remember that for this we need at least virtual machines and the complexity of the build system and the preparation of releases.
2. Compiler support and system libraries. This is where the difficulties begin. Do not think that a C ++ program is just a text file that can be analyzed everywhere in the same way. Each compiler and system libraries are simply teeming with all sorts of non-standard extensions. Support for these features - invisible, but very large amount of work. The fact that the '__restrict' keyword is used 10,000 times less often than the 'for' operator does not mean that it can be not supported. Many programmers are unaware of such things as _Bool, __w64, __noop, __int3264, __if_exists, 123.0fi, __uuidof, __based, __LPREFIX, __builtin_offsetof, __volatile____, and so on and so forth.
3. Diagnostic rules. Each system has its own data model, its own libraries and other nuances. If you want to give a good diagnosis, you need to know the features of the system functions. For example, in PVS-Studio / CppCat there is a huge table that tells you how to use this or that function incorrectly and what you can expect from it. It says: suspiciously if the '_wcsicmp_l' function has the first and second actual argument. Now the file “functions_info.cpp” is more than 800 kilobytes and we still have to work and work on its completion. If we want to go to other systems, we need to take into account the corresponding system functions and strongly replenish the table. This is a terribly hard task.
4. Documentation. Documentation must be seriously reworked. It must be remembered that now the PDF file with the documentation for PVS-Studio contains 250 pages and is constantly growing.
5. Integration with development environments. It's one thing to type something into the console, and quite another to make the tool handy. In Windows, the priority is obvious - you need to integrate with Visual Studio. In the Linux world, everything is more diverse. Need to support some of the most common editors. Even if not in the form of plug-ins, but you still need to be able to give information in a suitable format for them so that you can navigate.
6. Distribution kit
7. Testing. Most damning point. We have many varieties of automated tests. The main system is a set of regression tests showing differences in diagnostics. In total, more than 100 open-source projects are used for this. Time to complete the tests takes many hours. If we want to master a new operating system, we must create a new, more complex, portable testing system. We must find at least 50 new open-source projects that can be built in this system. You also need to learn how to run unit tests in the new operating system, and so on.
8. Support. Support is becoming more complicated, as you will need to have on hand various virtual machines and people who are ready to understand the problems specific to this system.

And now the most important thing. Under the Unix / Linux version, everyone actually understands their own operating system. It is very important! From the fact that there is a package for system X, the user of system Y is neither warm nor cold. For him, we still do not have the "Linux version". This means that you need to make a version for at least five different most common systems (but still there will be dissatisfied).

Accordingly, everything described above should be multiplied by the number of supported systems. For example, multiply this by 5 and imagine the amount of work.

At the same time, infrastructure costs increase enormously. Looking for programmers. Yes, even just to support the zoo of virtual machines and regression tests, we need a special system administrator. It will be necessary to create a complex infrastructure where various virtual machines are started in turn for tests. In addition, since tests should take place at least overnight, it is likely that several computers or a powerful server will be needed at once.
New programmers, system administrators, computers, servers, virtual machines, an additional cable, so as not to burn out the electrical network, to rent additional room for the server.

Unfortunately, what I described did not understand 90% of programmers and laughed. He was going to pull some kind of wire ...
A simple programmer never ran to look for an electrician when trouble. He just has an extra day off. The programmer never went through the circles of bureaucratic hell, trying to stretch the wire from the electric booth. Wire, of course, I gave an example. There will be no wire, so something else.

We will not pull it in the near future.

Can you talk about interactions with domestic software companies? For example, with companies such as Kaspersky Lab, Acronis, and so on.

Andrei: So far, there has been no communication with these and similar companies. I do not even know what to remember. There was a passing communication with Yandex. I think you can tell, since no NDA was signed.

Yandex shortly before the IPO decided to strengthen itself a bit by acquiring some start-ups. At least we got the impression. They eagerly communicated with different teams. Talked to us. But it did not work out.

The question was asked: "How much money and to whom should your startup?" To which we honestly replied: “To no one and not at all. Startup brings us money and we are in the black. ” They were surprised and lost interest. It's clear. It is much easier and more profitable to buy out unprofitable startups and thereby get good staff in the staff.

PVS-Studio, . , Yandex Coverity.

, CppCat — PVS-Studio, , . Standart, Professional .. , ? . - ? , , - , , , .. CppCat?

: , PVS-Studio , . / . . CppCat . PVS-Studio , CppCat .

What can you say about the product Cppcheck? Are there any ideas, plans for the release of a similar commercial product for Linux with some chips?

Andrew: Cppcheck is a good tool for getting acquainted with the methodology of static analysis. You can even run on a non-
gathering project. That is just a set of some files. Of course, the quality of analysis suffers from this, but it is very easy to try.

I already answered about Linux. No, we do not plan.

In October 2013, the article “PVS-Studio. Counting prices in the opposite direction , in which the planned experiment was announced with the formation of prices on an individual basis.I would like to know about the results of this experiment. Is it completed or ongoing? Have interesting pricing options been found for you?

Andrew: The experiment continues. It’s too early to draw conclusions. There is more communication with potential users, but whether it will be possible to earn more money is not clear. We have a very long sales cycle. Often, PVS-Studio is sold for sale in half a year, or even a year after the first contact in the mail.

The article also describes the option when companies buy one license, and the whole team uses it, giving quite legitimate arguments. In my opinion, this situation should gradually disappear. It's still "some savings on matches," it's like buying one debugger for a small team of developers. Causes inconvenience.

, , , « »?

: CppCat. , . PVS-Studio . , .

CppCat . $250 . – .

, «» . , «» , , , . , , . , .

, , , . . , . , . , . . . , $250-$500 , . .

. (KAV) 3 . 2 1 . - . . : « 3 . , 2 1. ». : « , ».

. C++ .

C++ «» , C++ — « ++». C++ — , ; - …

, ? C++, Windows, — . ?

: . C++ , 20 . Chromium, , C++ .

Chromium . C/C++ . C/C++. -, FORTRAN COBOL. .

FORTRAN / COBOL . , Intel FORTRAN, Intel C++. COBOL .

. , . , . FORTRAN COBOL . .

- ? , ? , , ?

: . , :).

, . - . , . .

Summing up

What can be said at the end of our conversation? The company creating the products discussed in this article is a small business that provides very interesting software products. On the way of small business, one of the key problems is the mandatory need for strict accounting of funds. What sometimes makes it impossible to implement and test on consumers seemingly breakthrough ideas. This issue can of course be resolved, for example, by attracting "other people's money" (loans, investors, etc.). But in order to decide on such an attraction, it may be worth repeatedly thinking, weighing all the pros and cons.

— ! , — CppCat. . : « ?», « CppCat» , , .

Windows. Linux . . , . - . Linux . , , , . , 20% - 80% . , , , , 10/90. , 90% 10% , , Linux , . , . « » , — Windows . , , .
C++ , , , . , C++ ( CppCat Cppcheck ). , C++ , — , , , .. , .

CppCat PVS-Studio . . - , , , , , .

, !

Unfortunately, we no longer develop or support the CppCat project. You can read about the reasons here .