The Black Swan Theory and the Fundamental Vulnerability of Automated Systems
There is a current fundamental vulnerability in any software tool written in compiled languages.
The author of the theory is Nassim Nicholas Taleb, who described it in his book “The Black Swan. Under the sign of unpredictability. The theory considers hard-to-predict and rare events that have significant consequences. It is not possible to describe the processes of the real world from the point of view of mathematics alone, and to prove this we consider one simple example.
Live turkeys are bought at the poultry farm, and it falls into much better conditions.
She is intensively fattened, cared for, cared for and cherished. Every day her feeder is filled with food, her pen is warm and comfortable. Every day, a long time.
Possession of a turkey with the ability to predict, she would predict with almost 100 percent probability that the next day feed would be plenty in her warm corral. But one fine day in November, thanksgiving comes, and the same caring owner collapses her neck. Surprise. What happened was that the turkey could not know, and the whole picture changed fundamentally. What happened, and there is a Black Swan - a rare unpredictable event. Strike from an unexpected side. And such events occur in our lives.
')
If you have seen only white swans all your life, it does not mean that blacks do not exist.
Having examined this theory, one can understand the relevance of the potential fundamental vulnerability of any automated system. The problem is quite old, there is as much as C.
In 1975, Ken Thompson (the creator of the C programming language and the UNIX operating system) first posed the problem of a trusted compiler. Publicly published in 1984 in his Turing Lecture "Reflection on whether you can rely on trust."
In the lecture, Ken Thompson demonstrated the attack through the binary version of the C compiler without the need to modify its source codes. A tab was shown in the compiler, which adds virus code to programs compiled by this compiler. And also configured on the compiler itself.
Virus code can not be traced by traditional methods through the source code.
But compilers are also compiled by compilers. The trusted and verified source code of the compiler cannot be trusted until it has been compiled by the trusted compiler quietly. Collecting new versions of the compiler multiplies the viral code in new versions of the compiler. A striking example of the manifestation of the problem of "chicken and eggs."
For about 20 years, the problem was considered unsolvable.
In 2005, David Wheeler presented a solution to this problem, allowing him to accurately recognize the successful attack on the compiler.
The main complexity of the theory lies in the fact that you must have a trusted compiler.
In short, the technique is as follows:
1) the checked source code of the compiler S is taken (for example, gcc-4.7);
2) the compiler S is compiled by the checked C1 compiler and it turns out the C1 (S) compiler;
3) the compiler S is compiled by the trusted compiler C2 and the compiler C2 (S) is obtained;
4) on the compiler C1 (S) and C2 (S) the program P is compiled, we get the binary code of the program C1 (S (P)) and C2 (S (P));
5) compare C1 (S (P)) and C2 (S (P));
The resulting binary files must be identical, otherwise the compiler is discredited and has a virus code (software tab).
The distribution of binary versions of compilers is uncontrollable, and it is impossible to argue that at some stages the compilers with the described vulnerability did not lay out. At the moment, it is almost impossible to find a trusted compiler. This indicates the presence of a potential fundamental vulnerability in any automated system. Having verified source codes is not a panacea, since the presence of binary viruses nullifies all the benefits of easily auditable source code.
The example with a turkey at the beginning of the article is an attempt to prove the relevance of this vulnerability, which has been neglected for many years.
It is not known who and why could make a program bookmark in the compiler, and whether “Thanksgiving Day” will come for us all.
1) Nassim Nicholas Taleb, The Black Swan: The Impact of the Highly Improbable,
http://www.nytimes.com/2007/04/22/books/chapters/0422-1st-tale.html?_r=0
2) Thompson, Ken, Reflections on Trusting Trust,
https://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thompson.pdf
3) Wheeler, David A., Countering Trusting through Diverse Double-Compiling (DDC),
http://www.dwheeler.com/trusting-trust/wheelerd-trust.pdf
The Black Swan Theory
The author of the theory is Nassim Nicholas Taleb, who described it in his book “The Black Swan. Under the sign of unpredictability. The theory considers hard-to-predict and rare events that have significant consequences. It is not possible to describe the processes of the real world from the point of view of mathematics alone, and to prove this we consider one simple example.
Live turkeys are bought at the poultry farm, and it falls into much better conditions.
She is intensively fattened, cared for, cared for and cherished. Every day her feeder is filled with food, her pen is warm and comfortable. Every day, a long time.
Possession of a turkey with the ability to predict, she would predict with almost 100 percent probability that the next day feed would be plenty in her warm corral. But one fine day in November, thanksgiving comes, and the same caring owner collapses her neck. Surprise. What happened was that the turkey could not know, and the whole picture changed fundamentally. What happened, and there is a Black Swan - a rare unpredictable event. Strike from an unexpected side. And such events occur in our lives.
')
If you have seen only white swans all your life, it does not mean that blacks do not exist.
Having examined this theory, one can understand the relevance of the potential fundamental vulnerability of any automated system. The problem is quite old, there is as much as C.
Thinking about whether you can rely on trust
In 1975, Ken Thompson (the creator of the C programming language and the UNIX operating system) first posed the problem of a trusted compiler. Publicly published in 1984 in his Turing Lecture "Reflection on whether you can rely on trust."
In the lecture, Ken Thompson demonstrated the attack through the binary version of the C compiler without the need to modify its source codes. A tab was shown in the compiler, which adds virus code to programs compiled by this compiler. And also configured on the compiler itself.
Virus code can not be traced by traditional methods through the source code.
But compilers are also compiled by compilers. The trusted and verified source code of the compiler cannot be trusted until it has been compiled by the trusted compiler quietly. Collecting new versions of the compiler multiplies the viral code in new versions of the compiler. A striking example of the manifestation of the problem of "chicken and eggs."
For about 20 years, the problem was considered unsolvable.
Double split cross compilation method
In 2005, David Wheeler presented a solution to this problem, allowing him to accurately recognize the successful attack on the compiler.
The main complexity of the theory lies in the fact that you must have a trusted compiler.
In short, the technique is as follows:
1) the checked source code of the compiler S is taken (for example, gcc-4.7);
2) the compiler S is compiled by the checked C1 compiler and it turns out the C1 (S) compiler;
3) the compiler S is compiled by the trusted compiler C2 and the compiler C2 (S) is obtained;
4) on the compiler C1 (S) and C2 (S) the program P is compiled, we get the binary code of the program C1 (S (P)) and C2 (S (P));
5) compare C1 (S (P)) and C2 (S (P));
The resulting binary files must be identical, otherwise the compiler is discredited and has a virus code (software tab).
Conclusion
The distribution of binary versions of compilers is uncontrollable, and it is impossible to argue that at some stages the compilers with the described vulnerability did not lay out. At the moment, it is almost impossible to find a trusted compiler. This indicates the presence of a potential fundamental vulnerability in any automated system. Having verified source codes is not a panacea, since the presence of binary viruses nullifies all the benefits of easily auditable source code.
The example with a turkey at the beginning of the article is an attempt to prove the relevance of this vulnerability, which has been neglected for many years.
It is not known who and why could make a program bookmark in the compiler, and whether “Thanksgiving Day” will come for us all.
Literature
1) Nassim Nicholas Taleb, The Black Swan: The Impact of the Highly Improbable,
http://www.nytimes.com/2007/04/22/books/chapters/0422-1st-tale.html?_r=0
2) Thompson, Ken, Reflections on Trusting Trust,
https://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thompson.pdf
3) Wheeler, David A., Countering Trusting through Diverse Double-Compiling (DDC),
http://www.dwheeler.com/trusting-trust/wheelerd-trust.pdf
Source: https://habr.com/ru/post/199306/
All Articles