Add some statistics to Tesla's autopilot safety statement.
Translator's note. A couple of days ago, an article appeared on the GT, where the owner of the Tesla Model S claims that the autopilot of his electric car saved a pedestrian from death . It says that the author of the letter wrote this letter, since “Now there are a lot of negative reviews about Autopilot and about the death of the driver a few weeks ago. I think it is important for you to learn about this story. ”It looks like a completely normal reaction to criticism, if not for a few moments. This letter is, in fact, a response to the criticism of the figures given in the article below, which was published on the Fortune website on July 5, 2016.
"Figures never lie, but liars always figure."
“Tell me which side you support, and I will provide statistics in support of your point of view”
2016 Tesla Model S 70D with Autopilot (photo: Sam Abuelsamid)
')
Variants of the above quotes have existed for many years and their origin is unknown. But there is a great deal of truth in both idioms. Whether the number of unemployed, economic growth, voting results or safety statements are discussed - before accepting the answer received, it is necessary to determine which question was asked. Different questions can give completely different answers.
This also applies to security applications for vehicle automation, including a recent blog post on Tesla, which said Joshua Brown was killed in a collision with a trailer in early May, who was driving his Tesla Model S with autopilot activated. Tesla's statement said :
As usual, we have to ask: which question was answered? Based on Tesla's statement, it can be assumed that since the autopilot appeared in October 2015, all S and X cars in the world have traveled 130 million miles. If we accept this interpretation, we must also assume that there were no other deaths in the world that we do not know about at the moment. Considering that Tesla collects a huge amount of telemetry from its cars, we believe that this is the case - at the moment, 1 death case per 130 million miles.
The Autopilot options screen shows that all functions are in beta, although most users may not understand what this means (photo: Sam Abuelsamid)
And what about 1 death per 94 million miles in the USA? The best source of such information is the Fatality Analysis Reporting System (FARS) of the US Department of Transportation, which provides crash data by state and local organization for the entire US. This system is a delightful source of information on accidents and trends. Since the data for 2015 were only previously announced, the latest complete data presented is for 2014, so we will use it. In 2014, US residents traveled 3.026 trillion miles ( approx. Translator 3.026x10 ^ 12) ) along the roads and 32,675 people died on the roads during this time. This means about 1 death per 92 million miles, a little worse than Tesla’s statement, and 2015 will be slightly worse in terms of this indicator, since more motorcyclists and pedestrians died.
An animation of what is happening around in the toolbar showing moving vehicles around, but without specifying pedestrians, cyclists or road signs (photo: Sam Abuelsamid)
The latter is very important, as the FARS system includes data of all the deaths on the roads, both in cars and trucks, as well as those that occurred in the event of a car crash with other road users (motorcycles, bicycles or pedestrians). As far as is known today, none of the cars equipped with the Autopilot has yet hit or killed a pedestrian or cyclist. Because the Tesla company essentially compares two different data sets. In 2014, 4,586 motorcyclists and 5,813 pedestrians / cyclists died in a traffic accident in the United States.
This leaves 22,276 road users in cars (drivers and passengers) who died. This number should probably be compared with the figure of 1 death per 130 million miles. In this case, there is one death per 135.8 million miles when driving a man. And according to this statistics, live drivers drive better than computers. However, even this, most likely, is also not the most correct comparison. Each death is a tragic loss; today's data of 1 death for more than 90 million miles is actually a very small amount. The “only one death” data set is insufficient for statistically correct judgment.
These 130 million miles sound like a huge number apart from reality and 87 times more than Google's test fleet hit Silicon Valley and Austin, Texas. However, this is only 0.0043% of the total distance that vehicles traveled in 2014. Before the advent of a large amount of data (I don’t want more people to die, although it is inevitable on a certain total number of trips) it is too early for any judgments about whether there is any progress in security matters.
We know exactly the following: Tesla did not report if they had any data on how often drivers behaved not like Brown and paid attention to the fact that the Autopilot did not work correctly and did not see what it should have been, and took control . If most drivers behaved like Joshua Brown, considering the car to be completely autonomous, the number of victims could be higher now. Or not. There is simply not enough data for this output.
I believe that automatic driving technology will improve over time and reduce the number of accidents, injuries, deaths and associated costs. I’m just not convinced that we have such data today to confirm or deny this information.
I am also not convinced that people should be used as experimental mice, as I am not sure that they can be provided with adequate information and provide real informed consent from them. Even if these participants were properly informed, millions of other people on the roads did not agree to be part of the experiment, and their lives are as valuable to society as the lives of Tesla drivers. The fact that Mr. Brown’s life was the only one lost in that incident was just a coincidence.
The author is a senior analyst for the Transportation Efficiencies team at Navigant Research.
"Figures never lie, but liars always figure."
“Tell me which side you support, and I will provide statistics in support of your point of view”
2016 Tesla Model S 70D with Autopilot (photo: Sam Abuelsamid)
')
Variants of the above quotes have existed for many years and their origin is unknown. But there is a great deal of truth in both idioms. Whether the number of unemployed, economic growth, voting results or safety statements are discussed - before accepting the answer received, it is necessary to determine which question was asked. Different questions can give completely different answers.
This also applies to security applications for vehicle automation, including a recent blog post on Tesla, which said Joshua Brown was killed in a collision with a trailer in early May, who was driving his Tesla Model S with autopilot activated. Tesla's statement said :
This is the first known fatal event for more than 130 million miles in which the Autopilot was activated. Among all vehicles in the USA, deaths occur once every 94 million miles
As usual, we have to ask: which question was answered? Based on Tesla's statement, it can be assumed that since the autopilot appeared in October 2015, all S and X cars in the world have traveled 130 million miles. If we accept this interpretation, we must also assume that there were no other deaths in the world that we do not know about at the moment. Considering that Tesla collects a huge amount of telemetry from its cars, we believe that this is the case - at the moment, 1 death case per 130 million miles.
The Autopilot options screen shows that all functions are in beta, although most users may not understand what this means (photo: Sam Abuelsamid)
And what about 1 death per 94 million miles in the USA? The best source of such information is the Fatality Analysis Reporting System (FARS) of the US Department of Transportation, which provides crash data by state and local organization for the entire US. This system is a delightful source of information on accidents and trends. Since the data for 2015 were only previously announced, the latest complete data presented is for 2014, so we will use it. In 2014, US residents traveled 3.026 trillion miles ( approx. Translator 3.026x10 ^ 12) ) along the roads and 32,675 people died on the roads during this time. This means about 1 death per 92 million miles, a little worse than Tesla’s statement, and 2015 will be slightly worse in terms of this indicator, since more motorcyclists and pedestrians died.
An animation of what is happening around in the toolbar showing moving vehicles around, but without specifying pedestrians, cyclists or road signs (photo: Sam Abuelsamid)
The latter is very important, as the FARS system includes data of all the deaths on the roads, both in cars and trucks, as well as those that occurred in the event of a car crash with other road users (motorcycles, bicycles or pedestrians). As far as is known today, none of the cars equipped with the Autopilot has yet hit or killed a pedestrian or cyclist. Because the Tesla company essentially compares two different data sets. In 2014, 4,586 motorcyclists and 5,813 pedestrians / cyclists died in a traffic accident in the United States.
This leaves 22,276 road users in cars (drivers and passengers) who died. This number should probably be compared with the figure of 1 death per 130 million miles. In this case, there is one death per 135.8 million miles when driving a man. And according to this statistics, live drivers drive better than computers. However, even this, most likely, is also not the most correct comparison. Each death is a tragic loss; today's data of 1 death for more than 90 million miles is actually a very small amount. The “only one death” data set is insufficient for statistically correct judgment.
These 130 million miles sound like a huge number apart from reality and 87 times more than Google's test fleet hit Silicon Valley and Austin, Texas. However, this is only 0.0043% of the total distance that vehicles traveled in 2014. Before the advent of a large amount of data (I don’t want more people to die, although it is inevitable on a certain total number of trips) it is too early for any judgments about whether there is any progress in security matters.
We know exactly the following: Tesla did not report if they had any data on how often drivers behaved not like Brown and paid attention to the fact that the Autopilot did not work correctly and did not see what it should have been, and took control . If most drivers behaved like Joshua Brown, considering the car to be completely autonomous, the number of victims could be higher now. Or not. There is simply not enough data for this output.
I believe that automatic driving technology will improve over time and reduce the number of accidents, injuries, deaths and associated costs. I’m just not convinced that we have such data today to confirm or deny this information.
I am also not convinced that people should be used as experimental mice, as I am not sure that they can be provided with adequate information and provide real informed consent from them. Even if these participants were properly informed, millions of other people on the roads did not agree to be part of the experiment, and their lives are as valuable to society as the lives of Tesla drivers. The fact that Mr. Brown’s life was the only one lost in that incident was just a coincidence.
The author is a senior analyst for the Transportation Efficiencies team at Navigant Research.
Source: https://habr.com/ru/post/396395/
All Articles