5 autopilot sensors and their (so far) unsolvable difficulties

The first part of the review of autopilot problems has gone well and the time to move on. Life does not stand still and "Tesla" is still fighting (Tesla Model X, too, but so far without the help of the autopilot). What is wrong with ADAS system sensors? And why in the coming years nothing will change dramatically?



Many, surprisingly, closer to their physical capabilities. We can speak with confidence about this, because the systems listed below are used not only on automobiles and have large sales markets outside the auto world. This is a lot of money, billions in R & D budgets and quite slow progress.

')

Now more.



Types of sensors - like fingers on a hand, five:

• cameras,
• radar,
• lidars
• ultrasound,
• infrared cameras.

The classification is chosen for convenience, rather than adhering to technological truth. In general, you can divide in different ways:

• passive (cameras) and active (radar lidar),
• in the visible range (camera) and not (all others, including IR),
• by frequency (from ultrasound to lidar).

All of these sensors are very different in nature and give different information to the electronic brain.



What is important to know about the market as a whole?







Estimates vary, but the order is the same. The balance is skewed in favor of cameras and radars, which for two occupy 85 percent of the market in money. Lidars remain a niche solution (only if there are no breakthroughs), and ultrasonic sensors cannot go beyond their niche, and again, breakthroughs are not visible. Night vision systems are lost even against the background of lidars.



Now more.

CAMERAS

It works like this: we got a picture of the world in the visible range, processed (individual frames and a sequence), recognized.



Pros:

1. Only cameras can recognize road markings and marks.
2. The cameras have a good resolution in the angle, that is, the camera “understands” objects that move across the movement of the car (a pedestrian crossing the road).
3. According to the camera image, direct classification of objects is possible (“dog” - “child” - “adult” - “bicycle” - “car” - “truck”).
4. And, of course, cameras are affordable - from a few hundred dollars to the buyer.

Minuses:

1. The cameras have poor resolution in range, they hardly understand the distance to the obstacle or the speed of approach (stereo cameras partially solve the problem).
2. There is no direct measurement of the speed of objects, it can be understood by a series of frames, and still the measurement is fuzzy.
3. Strongly influenced by weather and road conditions - the notorious "white van against the bright sky" in a recent accident.
4. Processing data from the cameras takes time: the delay is greater, the more complex the camera system.

The market is trampled by elephants from Intel and NVIDIA to Mobileye and Magna International. There are a lot of startups, the entry threshold is low. The development of technology rests on the speed and quality of data processing, that is, processors and algorithms. It is impossible to drastically increase the processing speed, and the breakthrough algorithms have not yet broken through anything.

Conclusion:

Without cameras on the car still can not do.

RADARS

It works like this: they radiated at ultra-high frequency, listened to what came back, immediately recognized the distance to the obstacles, their speed and angles on them.



Pros:

1. The weather conditions affect the radar operation much less, the color of objects and the illumination do not affect at all.
2. Good resolution in range and speed, direct measurement of range and speed is available with good accuracy.
3. Instant (1 / 20–1 / 50 seconds) data processing, which is especially important for high speeds.
4. The ability to easily accumulate and analyze information over time due to a small amount of information. For the same reason, suitable for V2I / V2V

Minuses:

1. Not cheaper, but often more expensive than cameras. More expensive 1000 dollars for a distant radar.
2. Bad angle resolution (radar is difficult to understand the movement across).
3. There is no direct way to classify an obstacle or assess its dimensions.
4. The strong dependence of the distance from the viewing angle. If we shine far and narrow beam, we do not see interference on the road and on the sides. We shine widely - we get a short range.

There are elephants on the market, only with other names - Bosch Global, Delphi, Smartmicro. There are almost no startups, the entry threshold is very high due to the high cost of equipment for the development of microwave. The processing methods have changed little over the past 50 years, investments in the development and organization of the production of components (as the antenna-feeder path) are measured in hundreds of millions of euros. Affordable and high-quality transceivers, without which no mass decision can be made at all, produce five companies for the whole world. Radars are gradually improving, but development takes decades.

Conclusion:

The strengths of the radar are the camera's weaknesses and vice versa. Without radar, too, can not do.



I did not invent it, but Mr. Mask:

“... That said, I don't think you need LIDAR. Forward if you’re driving in the rain or snow or dust. I think it is a solo. “I’m not a big fan of LIDAR,”

LIDARS

Laser rangefinders: shone with a laser, looked at what was back, gave an array of points with distances to them.



Pros:

1. Vundervaflya, sees everything around in detail and with high accuracy. Except for them, no one knows how and never will.

Minuses:

1. Scanning lidars see the same stretch of road only two or three times a second.
2. High cost, no - LIDAR COST.
3. More lidars are fragile.
4. In short, everything else - cons

There are cool players on the market - Velodyne LiDAR, Quanergy, Sick. All solutions are relatively young, startups are a few, but relatively large. Indeed, there are no major Bosch players at the automotive level of tens of billions in the market, and there will not be any more. With the decline in the cost of the lidar and the abandonment of moving parts, the possibilities and necessity of the lidar also catastrophically fall. Everyone is waiting for a decision for 250 bucks, but there is something for him.

Simple and understandable: the lidar car market is a hundred times smaller than the radar or camera market, and for those full and other applications

Conclusion:

Until the revolution happened, the lidars will remain a niche solution. Good for prototyping, no way for stock car. Yes, and for prototyping it is bad: the logic ground under the lidars does not adapt well to other sensors.







ULTRASOUND

Well-known parktronics: they emitted a high-frequency signal, waited until they returned, counted time.



Pros:

1. Very cheap, hundred-other dollars for a multi-sensor system
2. Sensors are easy to combine.
3. The technology is developed and understood, you can experiment even at home on arduino.

Minuses:

1. The range is limited to units of meters.
2. It is not possible to understand the angle to the object or directly measure the speed.
3. Low data reliability.

Our market is dominated by Chinese noname, the glade has long been trampled down, there are few people who want to invest in new projects. There are few startups, but there is for example Toposens , who does a cool thing on ultrasound, but something suddenly starts to do radar, too.

Conclusion:

As soon as radar and lidar will learn to work well near range, ultrasound only for arduino will remain. Joke:).







INFRARED CAMERAS

How it works: we looked in the infrared range, analyzed images on the matrix.



Pros:

1. Hot men and forest animals are clearly visible.

Minuses:

1. Expensive, a couple thousand dollars just for the front camera
2. Narrow scope of application
3. Restrictions on temperature use.

Conclusion:

An even more niche story than lidars. Radars with lidars will throw them out of the car market very soon, if not yet.



And now - sweet. Summing up:

1. No sensor or sensor system can become universal. If we talk about the role of the only one in low-end solutions, these are cameras. Secondarily - the radar.
2. Any ADAS system and, moreover, autopilot will use a system of several sensors. This is called sensor fusion and in the first place will combine cameras and radar. I suspect they will finish.
3. The more heterogeneous information you want to collect, the more expensive it is (half the trouble) and the more time it takes. The longer we process, the slower we must go to make it safe. Hence the turtle ride of most auto pilots and the love of good weather.

And that is not all. Now let's talk about the problems of the real world .