A short guide to the creation of oracles, the gods of the car and the errors of the second kind
Probably, in this text for many there will be no novelty. Perhaps others will say that this does not happen in real life. But, not the first of April, but everything written here is the absolute truth that happened to me or to the people around. Perhaps something from the above will make you rethink the phenomena around you.
If we approach these stories formally, then we can say that they are all generated by the fact that people do not take into account the error of the second kind. In Yudkovsky, with whom Habr's quarter is familiar, this error is usually called “ Confirmatory distortion ”.
')
What it is? In a nutshell - “a person is looking for confirmation of his model, and not its refutation”. The only chance to explain it better is examples, examples, examples and experience. Only in this way can one develop the feeling that “something is wrong here.”
It seems to me that this short story will allow you to look at errors of the second kind from a very different side. From the way they have already entered our lives, they affect almost every decision. And help us make the gods of the surrounding technology. In machine learning, I bump into it every day.
I'll start with the stories of my utravuka.
Once, in the days of the Union, a meeting of the Council of several design bureaus was held. One of the doctors of science, the head of a large department in one of the design bureaus, presented a new method for finding flying objects with a radar. At the same time, the accuracy of finding was almost perfect, exceeded the existing algorithms by orders of magnitude.
The basis was some kind of clever fractal mathematics. No one on the board could understand her from the report / related papers. And the person who presented the algorithm cited more and more arguments that all radars should be transferred to this algorithm. And his reputation was well-established. Everyone felt the catch. But to understand where the error in mathematics - no one could.
And only one post-graduate student on the council, who got there accidentally, replacing his boss, after an hour of discussion, decided to ask: “And what is the probability of a false detection if there are no objects”? He did not understand mathematics, but he clearly felt that he could not jump to an order of magnitude.
Everyone was silent. Everyone was sure that such a trivial thing was obviously verified. But the doctor who defended the mathematics turned pale, leaned over and only squeezed out of himself "We forgot to look ...".
People, even if they are intelligent and titled, are primarily looking for confirmation of their theory. And the more they are sure of it, the more evil a joke it can play with them.
If the article continues like this, then there will be no pictures! But I know that articles on Habré open only for the sake of cool pictures!
And there are a lot of them on this topic. They confirm that the feature of trusting the simplest solution lies in us at the level of nature. All of you have seen them many times. And, probably, it is not necessary to explain:
Classic yellow (blue?) Dress.
Necker cube , how is it without him
Rotation right / left
The peculiarity of these illusions - they have a double interpretation. The brain focuses only on one point of view, which solves the problem. And misses other points of view. And just tell me that you can keep two performances in your head at the same time !!!
It is like a religion that can explain the nature of the phenomena. Like horoscopes that are about anything, but a person is looking for confirmation. It is often impossible to determine what the smart system counted for you and how to compare it with reality. Do not believe?
Well, how about this project ? Using Neurosky to assess the talents of a child? Anyone more or less versed in technology is clear that this is layout. Noisy testimony Neurosky barely allow to distinguish playing the guitar from reading a book.
And this is not to mention the fact that everything is 100 times more difficult for children. Children are restless, which generates a high level of noise. And, of course, the developers do not provide any evidence of the algorithm and statistical significance. It would work - it would be a breakthrough. But this is only a way of distributing.
After my article about NeuroSky, five people wrote to me with similar ideas in the mail / lichku. Someone was just crazy, and someone quite cynically hinted that "well, you make the system, because she still shows something, she cannot give complete nonsense after training." And the site that I brought a little higher - cited as an example.
And why are you not horoscopes?
What is it here? Well, there is one office that deals with layout. Well, Scientologists are there . After all, it is not massive!
But, it seems to me that not a couple. Such things fill daily life. And technology generates them every day. Below in the text there will be some more illustrative examples about which you all know very well (willyou have time to remember before you finish reading? ). And in the comments you can bring your own.
The key to me in this case is that it shows that many people want to believe in technology / methodology. And one thing when people who do not understand it believe. And the second thing is the self-deception of the creators. The second, unfortunately, is progressing.
It seems to me that the stories should not be repeated, that each of them should show human nature from some other side. So go to the other side.
Car plate numbers!
What ?? What are the errors of the second kind!? What is magic?
And they appear here very cool. Here is what you say, what is the number of the car:
Is “M” or “H” here? And the region is “71” or “21”?
And, nevertheless, on the basis of single images of such a plan, people begin to evaluate the quality of the system. Why do you think? Usually because a person saw the number of such a car. And it is difficult for him to admit that in such a photo he may have a different interpretation.
There is a collapse of consciousness in favor of the nearest solution which is known. I gave the most illustrative example, but if it is suddenly interesting, here I am analyzing a few more.
And to convince him that the number could be different is often almost impossible. It is easier to say "this is included in the percentage of algorithm errors."
This is a very frequent trap of consciousness when working with images and recognition systems. A person expects a ready-made solution. And all that came to another solution is wrong. The path is not interesting. Only the result is interesting. A couple of times a month I have to explain that a trained algorithm is not required to work with frames with noise.
I am afraid to think how some firms offer such a plan. Even on very good pictures it is difficult to find glasses / gloves and track their owner. Even if you are not an algorithm, but a living person.
At the same time, engineers at technical enterprises are not the people who set themselves a philosophical task “is it possible to understand from this video whether a person has safety glasses or not.”
And still many people consider that unreadable numbers are also well recognized. And the resolution can be increased hundreds of times. On Habré was a wonderful article on this topic. And sometimes they are very offended that this is not so. The magic of the car does not work;)
I brought three different options. And everything can be answered with a standard one: “oh well, it's just incompetent people.” Or: "people made a mistake, with whom it does not happen."
But, such errors are much more common than you think. As soon as we deal with ambiguous decisions, with situations where 3-4 outcomes are possible, it is impossible to assess the level of one’s competence. Even if you are a very cool engineer.
I'll tell you an epic story about how experts can make mistakes (or not make mistakes), finding God for themselves from a car.
Once they wrote to me asking for a consultation on the recognition of some medical tests. On the phone, I didn’t really understand the manager’s explanation of what to recognize. Especially according to him, the case was about some kind of “absolutely new” method. So we agreed to meet with the doctors who developed it (Skype is too difficult for doctors).
The next day, two hours later, my brain was boiling. I did not understand with whom I spoke for two hours. With absolute rogues, with future Nobel laureates, or with people who are crazy.
A group of several doctors invented a universal test that allowed to identify dozens, if not hundreds of diseases. By urine, by blood, by tears and by saliva.
The method was the same: “knead the liquid with the reagent (the same), see how it dries and cracks”. The diagnosis was made according to what the cracks would look like, what color the whole thing would look like, what shape the crystals grow on the dried drop, how it looks in polarized light.
And I must say that the first part, which was urine-based, was fairly well substantiated.
When in the following days I tried to dig up information on the topic and interviewed the doctors, they confirmed that much is very distorted standard tests that take place in reality. Although somewhere about a third of the book with a doubt shook his head.
But the second part, by blood, was the product of madness. What is there Theranos ! Just it was necessary not to allocate some American 4.5 billion there, but to these comrades.
Researchers told me how, by drop of blood, they can deliver cancer at the earliest stages. Just with a microscope, a couple of drops of reagents from any laboratory (if memory serves, it was albumin). And told how to successfully apply these methodologies in clinical practice for the treatment of patients of the medical center (sic!). Here is a small list of what they detected:
(Clickable)
In my somewhat wounded look and the question about the evidence base, I was told about hundreds of articles in literature, about two departments in Russia working on this methodology and developing it. About a dozen candidates and doctors of science defended on the topic. And at parting they presented two books on the topic, where the preface was written by some academician:
Is this all true? I do not know. I realized that I lacked an understanding of how chemistry and biology work. No understanding of how to find this information. And on the formal grounds. Well yes. There are a bunch of doctors and related articles. None of them was published in a peer-reviewed international journal with a high citation rate. There are academics who talk about the success of the methodology. But they, sorry, academicians of the Russian Academy of Medical Sciences. There is not a single criticizing article, only articles reporting on progress (Google year in 2016, since then something new could come out).
No, well, I, of course, found the e-mail of the chairman of the commission on pseudoscience, and some of his deputies. And I wrote about everything about it ... I received no answer :)
And by personal communication. I understand that these people believe in it. Damn, Russian medicine is not the place where you can do something on this level of deception. You need to really believe in your business.
But I understand that with the bases on which they measure, with the parameters that are there - at least half of the story is questionable. But something really can work.
That work thank God I did not go. Perhaps due to the fact that I said that in order to automatically recognize something, you must first assemble a large base and confirm with independent markup that it exists. And maybe the truth is due to bureaucratic formalities.
Who is right here? Doctors who have come up with a theory and believe in it? Or am I a man from the outside, whom she seems insane? I dont know. If doctors make mistakes, then this is a classic mistake “lack of checking counter-arguments”. Or maybe me. And then it's almost the same :)
But since then I have two epic books and a phantosmogoric story.
You know. Doctors are generally fertile topic. When you try to recognize something with neural networks, you constantly come across it. I worked with a large number of radiographic examinations: fluorograms, mammograms, talked and looked at CT, saw different teeth.
And almost everywhere (in different measure) there is one and the same problem: different doctors read pictures in different ways. You ask two doctors to note the pathology in the pictures - and they have a 20-30 percent intersection area. And each of them is confident in his decision, shows clear boundaries.
Justifies your answer. And again, the problem is that a person builds a good model in his head, projects it onto the image, and then looks for confirmation. But very often this is the wrong answer. Or unreliable.
Unfortunately, the past centuries, medicine was no longer built on the “proof of each step by statistics”, but on the “logical explanation of each step”. It sits very deep in psychology. Sometimes it seems to me that a homeopath and a poor therapist work on the same principle. There is a set of basic settings and faith in it. Without analysis where it went. Without knowing what you can trust. Most likely the set of therapist’s settings is closer to reality. But globally neither one nor the other can prove the schemes that are used. Maybe even both appointed Validol or send to drink motherwort.
Once I was asked to consult a surgeon ... The head of a department, a prominent specialist with 30 years of practice. Uncle told me how with 100% accuracy on the questionnaire of the level “date of birth, date of first menstruation, number of children ...” from 30 questions can be predicted when there will be breast cancer. Well, that, of course, is ridiculous. But he offers this questionnaire to fill in all his patients, making conclusions based on it.
Want more? .. Want something everyone knows about? Well, take the great and terrible car that everyone fears. Polygraph . What, do you still believe that it works? Link leads to Wikipedia. Go and see the criticism. Why it doesn’t work and what the reliability is. And still many use them. What do not you god of the car?
I regularly write to different companies that offer to make a lie recognition by face. Very offended when I ask them to say what size they have a base. They say that they have a cool model developed by the best physiologists.
I especially remember one company with a luxurious office in the center of Moscow. Absolutely inadequate NDA and insane pathos about the superiority of their technology ... Almost five years have passed since then. The company has even appeared on Habré’s blog. Only here the products to the recognition of lies had nothing to do, and they solve much more mundane tasks (as I understand it, they used the same things, which is good).
It's funny that later it turned out that not only I sent them, but also other companies involved in the development of machine vision. Perhaps it allowed them to rethink what they were doing - and start doing business ^ _ ^
Finally, I can not help but give a link to the announcement of the portal Superjob which was a few days ago. I would like to offer them another 100% proven method to learn more about the candidate.
Perhaps it is time to come to some conclusions? The conclusions that I made for myself - any work to begin and do only after verification of the input data and methodology.
Do not trust anyone until he shows verification of his method by open statistical research / access to the algorithm.
I am sure that these conclusions are not perfect. They are not suitable for medicine. Most likely there is no verified aspirin test anywhere. For thousands of years, people know that willow bark extract helps to bring down the temperature. Why investigate it?
And to summarize the findings and understand the universal strategy of trust vs. check - I can not.
But probably if you start wondering “can we trust one method or another”, the article will fulfill its goal.
PSDisclaimer
I deliberately tried not to mention the names of people / companies where there is at least some chance that people are honestly mistaken. Or where I may be mistaken. Probably, by what is in the text, many can be calculated or googled. I would ask not to post it in the comments. All who need to - do it yourself / be able to ask in a personal.
In many situations, I may not know the deep details of the decisions, so I do not consider it necessary to stigmatize some people. But it seems to me important to show on these examples that our interaction with the modern world generates a lot of errors of consciousness in people.
If we approach these stories formally, then we can say that they are all generated by the fact that people do not take into account the error of the second kind. In Yudkovsky, with whom Habr's quarter is familiar, this error is usually called “ Confirmatory distortion ”.
')
What it is? In a nutshell - “a person is looking for confirmation of his model, and not its refutation”. The only chance to explain it better is examples, examples, examples and experience. Only in this way can one develop the feeling that “something is wrong here.”
It seems to me that this short story will allow you to look at errors of the second kind from a very different side. From the way they have already entered our lives, they affect almost every decision. And help us make the gods of the surrounding technology. In machine learning, I bump into it every day.
Introduction
I'll start with the stories of my utravuka.
Once, in the days of the Union, a meeting of the Council of several design bureaus was held. One of the doctors of science, the head of a large department in one of the design bureaus, presented a new method for finding flying objects with a radar. At the same time, the accuracy of finding was almost perfect, exceeded the existing algorithms by orders of magnitude.
The basis was some kind of clever fractal mathematics. No one on the board could understand her from the report / related papers. And the person who presented the algorithm cited more and more arguments that all radars should be transferred to this algorithm. And his reputation was well-established. Everyone felt the catch. But to understand where the error in mathematics - no one could.
And only one post-graduate student on the council, who got there accidentally, replacing his boss, after an hour of discussion, decided to ask: “And what is the probability of a false detection if there are no objects”? He did not understand mathematics, but he clearly felt that he could not jump to an order of magnitude.
Everyone was silent. Everyone was sure that such a trivial thing was obviously verified. But the doctor who defended the mathematics turned pale, leaned over and only squeezed out of himself "We forgot to look ...".
People, even if they are intelligent and titled, are primarily looking for confirmation of their theory. And the more they are sure of it, the more evil a joke it can play with them.
Feel for yourself
If the article continues like this, then there will be no pictures! But I know that articles on Habré open only for the sake of cool pictures!
And there are a lot of them on this topic. They confirm that the feature of trusting the simplest solution lies in us at the level of nature. All of you have seen them many times. And, probably, it is not necessary to explain:
Classic yellow (blue?) Dress.
Necker cube , how is it without him
Rotation right / left
The peculiarity of these illusions - they have a double interpretation. The brain focuses only on one point of view, which solves the problem. And misses other points of view. And just tell me that you can keep two performances in your head at the same time !!!
It is like a religion that can explain the nature of the phenomena. Like horoscopes that are about anything, but a person is looking for confirmation. It is often impossible to determine what the smart system counted for you and how to compare it with reality. Do not believe?
Example number two, about rogues
Well, how about this project ? Using Neurosky to assess the talents of a child? Anyone more or less versed in technology is clear that this is layout. Noisy testimony Neurosky barely allow to distinguish playing the guitar from reading a book.
And this is not to mention the fact that everything is 100 times more difficult for children. Children are restless, which generates a high level of noise. And, of course, the developers do not provide any evidence of the algorithm and statistical significance. It would work - it would be a breakthrough. But this is only a way of distributing.
After my article about NeuroSky, five people wrote to me with similar ideas in the mail / lichku. Someone was just crazy, and someone quite cynically hinted that "well, you make the system, because she still shows something, she cannot give complete nonsense after training." And the site that I brought a little higher - cited as an example.
And why are you not horoscopes?
What is it here? Well, there is one office that deals with layout. Well, Scientologists are there . After all, it is not massive!
But, it seems to me that not a couple. Such things fill daily life. And technology generates them every day. Below in the text there will be some more illustrative examples about which you all know very well (will
The key to me in this case is that it shows that many people want to believe in technology / methodology. And one thing when people who do not understand it believe. And the second thing is the self-deception of the creators. The second, unfortunately, is progressing.
Example number three, per-week
It seems to me that the stories should not be repeated, that each of them should show human nature from some other side. So go to the other side.
Car plate numbers!
What ?? What are the errors of the second kind!? What is magic?
And they appear here very cool. Here is what you say, what is the number of the car:
Is “M” or “H” here? And the region is “71” or “21”?
And, nevertheless, on the basis of single images of such a plan, people begin to evaluate the quality of the system. Why do you think? Usually because a person saw the number of such a car. And it is difficult for him to admit that in such a photo he may have a different interpretation.
There is a collapse of consciousness in favor of the nearest solution which is known. I gave the most illustrative example, but if it is suddenly interesting, here I am analyzing a few more.
And to convince him that the number could be different is often almost impossible. It is easier to say "this is included in the percentage of algorithm errors."
This is a very frequent trap of consciousness when working with images and recognition systems. A person expects a ready-made solution. And all that came to another solution is wrong. The path is not interesting. Only the result is interesting. A couple of times a month I have to explain that a trained algorithm is not required to work with frames with noise.
I am afraid to think how some firms offer such a plan. Even on very good pictures it is difficult to find glasses / gloves and track their owner. Even if you are not an algorithm, but a living person.
At the same time, engineers at technical enterprises are not the people who set themselves a philosophical task “is it possible to understand from this video whether a person has safety glasses or not.”
And still many people consider that unreadable numbers are also well recognized. And the resolution can be increased hundreds of times. On Habré was a wonderful article on this topic. And sometimes they are very offended that this is not so. The magic of the car does not work;)
Example number four, or how to get to the epicenter
I brought three different options. And everything can be answered with a standard one: “oh well, it's just incompetent people.” Or: "people made a mistake, with whom it does not happen."
But, such errors are much more common than you think. As soon as we deal with ambiguous decisions, with situations where 3-4 outcomes are possible, it is impossible to assess the level of one’s competence. Even if you are a very cool engineer.
I'll tell you an epic story about how experts can make mistakes (or not make mistakes), finding God for themselves from a car.
Once they wrote to me asking for a consultation on the recognition of some medical tests. On the phone, I didn’t really understand the manager’s explanation of what to recognize. Especially according to him, the case was about some kind of “absolutely new” method. So we agreed to meet with the doctors who developed it (Skype is too difficult for doctors).
The next day, two hours later, my brain was boiling. I did not understand with whom I spoke for two hours. With absolute rogues, with future Nobel laureates, or with people who are crazy.
A group of several doctors invented a universal test that allowed to identify dozens, if not hundreds of diseases. By urine, by blood, by tears and by saliva.
The method was the same: “knead the liquid with the reagent (the same), see how it dries and cracks”. The diagnosis was made according to what the cracks would look like, what color the whole thing would look like, what shape the crystals grow on the dried drop, how it looks in polarized light.
And I must say that the first part, which was urine-based, was fairly well substantiated.
When in the following days I tried to dig up information on the topic and interviewed the doctors, they confirmed that much is very distorted standard tests that take place in reality. Although somewhere about a third of the book with a doubt shook his head.
But the second part, by blood, was the product of madness. What is there Theranos ! Just it was necessary not to allocate some American 4.5 billion there, but to these comrades.
Researchers told me how, by drop of blood, they can deliver cancer at the earliest stages. Just with a microscope, a couple of drops of reagents from any laboratory (if memory serves, it was albumin). And told how to successfully apply these methodologies in clinical practice for the treatment of patients of the medical center (sic!). Here is a small list of what they detected:
(Clickable)
In my somewhat wounded look and the question about the evidence base, I was told about hundreds of articles in literature, about two departments in Russia working on this methodology and developing it. About a dozen candidates and doctors of science defended on the topic. And at parting they presented two books on the topic, where the preface was written by some academician:
Is this all true? I do not know. I realized that I lacked an understanding of how chemistry and biology work. No understanding of how to find this information. And on the formal grounds. Well yes. There are a bunch of doctors and related articles. None of them was published in a peer-reviewed international journal with a high citation rate. There are academics who talk about the success of the methodology. But they, sorry, academicians of the Russian Academy of Medical Sciences. There is not a single criticizing article, only articles reporting on progress (Google year in 2016, since then something new could come out).
No, well, I, of course, found the e-mail of the chairman of the commission on pseudoscience, and some of his deputies. And I wrote about everything about it ... I received no answer :)
And by personal communication. I understand that these people believe in it. Damn, Russian medicine is not the place where you can do something on this level of deception. You need to really believe in your business.
But I understand that with the bases on which they measure, with the parameters that are there - at least half of the story is questionable. But something really can work.
That work thank God I did not go. Perhaps due to the fact that I said that in order to automatically recognize something, you must first assemble a large base and confirm with independent markup that it exists. And maybe the truth is due to bureaucratic formalities.
Who is right here? Doctors who have come up with a theory and believe in it? Or am I a man from the outside, whom she seems insane? I dont know. If doctors make mistakes, then this is a classic mistake “lack of checking counter-arguments”. Or maybe me. And then it's almost the same :)
But since then I have two epic books and a phantosmogoric story.
And a couple more examples about doctors ...
You know. Doctors are generally fertile topic. When you try to recognize something with neural networks, you constantly come across it. I worked with a large number of radiographic examinations: fluorograms, mammograms, talked and looked at CT, saw different teeth.
And almost everywhere (in different measure) there is one and the same problem: different doctors read pictures in different ways. You ask two doctors to note the pathology in the pictures - and they have a 20-30 percent intersection area. And each of them is confident in his decision, shows clear boundaries.
Justifies your answer. And again, the problem is that a person builds a good model in his head, projects it onto the image, and then looks for confirmation. But very often this is the wrong answer. Or unreliable.
Unfortunately, the past centuries, medicine was no longer built on the “proof of each step by statistics”, but on the “logical explanation of each step”. It sits very deep in psychology. Sometimes it seems to me that a homeopath and a poor therapist work on the same principle. There is a set of basic settings and faith in it. Without analysis where it went. Without knowing what you can trust. Most likely the set of therapist’s settings is closer to reality. But globally neither one nor the other can prove the schemes that are used. Maybe even both appointed Validol or send to drink motherwort.
Once I was asked to consult a surgeon ... The head of a department, a prominent specialist with 30 years of practice. Uncle told me how with 100% accuracy on the questionnaire of the level “date of birth, date of first menstruation, number of children ...” from 30 questions can be predicted when there will be breast cancer. Well, that, of course, is ridiculous. But he offers this questionnaire to fill in all his patients, making conclusions based on it.
Examples of what we fool around every day
Want more? .. Want something everyone knows about? Well, take the great and terrible car that everyone fears. Polygraph . What, do you still believe that it works? Link leads to Wikipedia. Go and see the criticism. Why it doesn’t work and what the reliability is. And still many use them. What do not you god of the car?
I regularly write to different companies that offer to make a lie recognition by face. Very offended when I ask them to say what size they have a base. They say that they have a cool model developed by the best physiologists.
I especially remember one company with a luxurious office in the center of Moscow. Absolutely inadequate NDA and insane pathos about the superiority of their technology ... Almost five years have passed since then. The company has even appeared on Habré’s blog. Only here the products to the recognition of lies had nothing to do, and they solve much more mundane tasks (as I understand it, they used the same things, which is good).
It's funny that later it turned out that not only I sent them, but also other companies involved in the development of machine vision. Perhaps it allowed them to rethink what they were doing - and start doing business ^ _ ^
Finally, I can not help but give a link to the announcement of the portal Superjob which was a few days ago. I would like to offer them another 100% proven method to learn more about the candidate.
Perhaps it is time to come to some conclusions? The conclusions that I made for myself - any work to begin and do only after verification of the input data and methodology.
Do not trust anyone until he shows verification of his method by open statistical research / access to the algorithm.
I am sure that these conclusions are not perfect. They are not suitable for medicine. Most likely there is no verified aspirin test anywhere. For thousands of years, people know that willow bark extract helps to bring down the temperature. Why investigate it?
And to summarize the findings and understand the universal strategy of trust vs. check - I can not.
But probably if you start wondering “can we trust one method or another”, the article will fulfill its goal.
PS
I deliberately tried not to mention the names of people / companies where there is at least some chance that people are honestly mistaken. Or where I may be mistaken. Probably, by what is in the text, many can be calculated or googled. I would ask not to post it in the comments. All who need to - do it yourself / be able to ask in a personal.
In many situations, I may not know the deep details of the decisions, so I do not consider it necessary to stigmatize some people. But it seems to me important to show on these examples that our interaction with the modern world generates a lot of errors of consciousness in people.
Source: https://habr.com/ru/post/446038/
All Articles