No Bugs, No Flaws or the reliability of systems as such

Today, looking through the publication of Egor Bugaenko When Do You Stop Testing! I was surprised to find that few people understand and think that the problem of testing and asserting that the system does not have flaws is not fundamentally solvable. By the way, this is also relevant in connection with the introduction of electronic elections, and the requirements from officials and journalists for evidence of the absence of security holes. For example, we (Estonia), this discussion is in full swing, and politicians want one hundred percent proof of the reliability of the electronic election system. Public politicians are usually unfamiliar with logic in its formal form. And Hume and Popper, at best, are reminiscent of some boring philosophical seminars that it is better to forget about. But when I hear that a tester or a senor begins to prove quite seriously that the code has been tested and there are no errors at all, my face acquires this expression:



There are several main points why the formulation of this type of questions should be excluded, especially from the lips of IT professionals.

The first moment is logical

The problem of induction, which Hume seriously considered, calls into question the very possibility of justifying a number of problems and questions. For example, the removal of some general conclusions from the presence of some private observations is impossible. Unlimited number of observations of something does not confirm the truth of the statement. Karl Popper, one of the most incredible geniuses in the history of mankind, drew attention to the problem of falsification, and became the author of a new paradigm in the philosophy of science. What is the essence of the problem of fraud, and how does this affect us? The most famous example is Black Swan. Black Swan was a symbol of the impossibility of anything in the 16th century in London. The Old World assumed that all swans are white, since all the historical records of the swans described their white feathers. And in this context, the black swans were impossible, or did not exist. However, in the 17th century, the representative of the Old World met the impossibility in Australia.
')
There is an asymmetry between confirmation and falsification. The problem of induction is that an individual statement, like this white swan, or by doing this and that, in such a context, did not happen, cannot be generalized to a universal statement. The singular statement of existence does not prove the general.

For example, if for the last 4.5 billion years the Sun has regularly (well — well, with some slowdown) rises above the horizon, this does not tell us that the Sun always rises above the horizon. Or a lover of paradoxes B. Russell gives an example with a turkey that is fed, cared for, and she thinks that it will always be like this until the very day, Thanksgiving Day.

Thus, in order to prove that all swans are white, we must check all the swans in the Universe, and Leibnitz whispers to me that these should be all possible worlds, which is naturally impossible. But in order to refute this statement, it is enough to come to Australia and meet one black swan. Thus, there is a fundamental asymmetry between confirmation and falsification.

Talking about the absence of bugs in the code, or holes in the system in this context is wrong. You can not even put the question in this plane, because it leads the discussion in a completely wrong direction, especially if it involves people uncorrupted by the matan.

The second point is computing.

Rice's theorem leaves an imprint of pessimism on our ability to create perfect systems. A consequence of the Rice theorem, which generally follows from the problem of stopping the Turing machine and computability, asserts that there is no universal algorithm for proving or stating that a program, system, or algorithm does not contain errors and does what is necessary. Automatic universal testing systems are generally not possible. In particular, yes, but no matter what we do, there will be bugs! Static typing, TDD and other techniques can only instill in us more confidence and relieve from some fears, no more.

The third moment is engineering

There are such situations that we may even know that we have some specific bug that destroys our system, because we observed this at some point. We know that there is a bug. But we cannot reproduce it. I mean problems in multi-threaded applications where non-determinism rules. There are no special problems to catch a bug if the server crashes every ten minutes. But what if we need the server to work for several months before the problem comes back. It is literally impossible to debug. Worse, as Paul Butcher mentioned, it is likely to write programs that contain multi-thread bugs that will never appear no matter how carefully or for how long we would test them. Just because stream access to memory can be reordered does not mean that it will be done that way. And we may well be completely in the dark about the existence of problems until we upgrade the JVM or switch to another hardware, where we suddenly encounter mysterious problems that no one understands.

All these three points make the very statement that there are no bugs, or about proving the safety of something meaningless. A very large percentage of IT professionals also do not understand this. A great example in the post Egor Bugaenko.

If I were offered a choice of two super abilities, the ability to divide by zero, or find all the bugs in the code, then I would think what to take, because these are things of the same order.