Unmanned cars. Intel Expert Answers

In this post, we publish the answers of Intel expert Ivan Kuzmin to questions from readers on the topic “Artificial Intelligence and Autonomous Vehicles”. There were a lot of questions, as usual, and we left Ivan to choose the ones he can and wants to answer. For various kinds of clarifications, we invite comments, but do not guarantee quick answers.
At the very end, according to tradition, we announce the name of the user who receives the prize for the best question.

We are grateful to Ivan Kuzmin for his dedicated work and time spent!

Lingvo questions

Is AI used in reality in unmanned vehicles that are actually tested today or are these all marketing bikes and in fact the autopilot is based on clear mathematical algorithms and logic?

Answer: The control of an unmanned vehicle uses various mathematical algorithms. All algorithms will be very difficult to count, because the management task itself is complex and must both take into account various parameters and calculate data from different sources. The algorithms of artificial intelligence or machine learning include both classical algorithms (trees, logistic or linear regression, support vector machine), and deep learning algorithms (convolutional and recurrent neural networks). The combination of various algorithms allows you to effectively reliably solve the control problem.
')

How is this software certified for compliance with safety standards adopted for automotive software and is it produced at all? Is the autopilot considered the same critical vehicle control unit as ESP or airbags, where there is also hardware and software?

Answer: Of course, the autopilot is as critical a block as airbags or other devices. There is an ISO 26262 standard or its equivalent GOST R ISO 26262 in Russia, which imposes a requirement on the entire process of developing systems related to safety. The developer of such systems at the end of development should show that he followed this standard.

Does the autopilot work in hard real-time or is it all on Linux / Windows?

Answer: In general, we should be able to keep within less than 140 milliseconds from receiving signals or images on sensors or cameras and sending a signal to a machine’s control system. This task can be solved in hard realtime mode, but also a number of auto companies and partners are considering the option of developing the direction of the Linux version, which could meet all the requirements of the industry.

Does Intel plan to bring its work to OpenSource and attract more developers, or will it keep everything closed and sell to interested customers?

Answer: the company always supports and supports open source ideas and actively participates in the development of a large number of OpenSource projects. At the same time in the development of unmanned vehicles, an important issue is compliance with the standard, which may limit the participation of a large number of developers. In any case, we will weigh all the pros and cons for each individual product and make decisions individually. There is no general strategy for this yet.

Is it used in the development of modeling tools and automatic code generation (for example, Matlab / Simulink) or is everything written with C / VHDL pens? If so, what is the percentage of coverage of the total amount of software?

Answer: The automotive standard requires the developer to have a complete understanding of the code that is written. This limits the use of code generators. At the same time, models built in tools such as Matlab are fully used to test the written algorithm.

What hardware is used for “rapid prototyping” of the developed algorithms in a real car - processors, FPGAs, ready boards, sensors / lidars / radars / parking sensors, actuators?

Answer: if we are talking about rapid prototyping, then all the same it happens more likely not in the machine, but in the data center. Here, Intel has many solutions both in hardware and in software, including our Intel Distribution for Python package, which significantly speeds up Python as the most convenient language for rapid prototyping. When we talk about a car, here is C and C ++, and the platform itself is usually set in advance, considering the cost, power consumption and many other factors.

Are there already industry standards for connecting the specified peripherals to the computing platform?

Answer: Yes, there are already such standards, and they are actively used in existing machines and ADAS solutions.

Kababok questions

Does Intel refer to Tier-1-Supplier or Tier-2-Supplier?

Answer: Intel and Mobileye work together with more than 50 different partners, both directly and through suppliers. This list is constantly updated and the latest information about partners can always be found on our website .

Are these solutions from Intel compatible with AUTOSAR architecture (architecture levels)?

Answer : Intel is among the partners of AUTOSAR and actively cooperates with this organization.

The official releases reasonably state that Intel autonomous driving solutions are optimized for Intel processors. Is it possible to use / porting solutions from Intel to processors of other manufacturers: for example, Texas Instrument, Infineon, nVidia, Altera, etc.?

Answer: in optimizing mathematical libraries, we not only rely on the compiler, but also actively use instructions of the processor on which the code is executed. Theoretically, you can try to mute the instructions from IA to other architectures, but here it is important to remember that the standard imposes a requirement on the entire development process and you will need to transfer not only the code, but also the tests, and it is also possible to revise the guarantees that are given. So, I think it will be extremely difficult to port the code.

Have the solutions mentioned come from Intel to Automotive Certification? If so, what certificates?

Answer: development is carried out in accordance with ISO 26262 and fully complies with it.

Whether the use of solutions from Intel is based on only a single ECU in the architecture of the electronic components of the car, or is it possible to supply solutions in the form of separate libraries that are then connected via dedicated interfaces to the software functions of other ECU other auto suppliers (Suppliers / Zulieferer).

Answer: Intel has a large number of components (FPGA, CPU, 5G, Mobileye, and so on) that can be used both separately and as a full-fledged platform provided by the company.

Does Intel interact with its mentioned solutions directly with OEM?

Answer: Of course, we work with a large number of partners. All information about those with whom we work can be found on our website .

Does Intel use in its own development processes anything from a chain of SiL / HiL / XiL stands?

The answer is: yes, in some cases

In the automotive industry at the level of software / hardware solutions for development / commissioning / testing / acceptance and other parts of R & D processes, de facto leaders have long been the leaders of Vector Informatik and ETAS. Are the solutions from Intel optimized for working with products from these firms?

Answer: we work and interact with many leaders in this industry.

Does Intel provide ready-made simulation blocks in Matlab / Simulink?

Answer: at the moment such blocks are not provided. Information on what is included in the Intel GO SDK can be found here .

What processors, communication interfaces, operating system are used on the control unit in the Audi A8?

Answer: this car uses FPGA solutions and VxWorks operating system.

Is it possible to get a private solution from Intel in the form of an evaluation kit or a complete test suite from Intel GO Automotive SDK?

Answer: You can request access to this link .

Does Intel work on autonomous / robotic control solutions in the aviation industry?

Answer: Intel WindRiver is developing an operating system that is already used in various aircraft, including the Boeing 787 Dreamliner or Airbus A400M. Some details can be found here .

What exactly are the tools used in the Python language solutions from Intel? What are C ++ tools for? What are the differences in the use of different languages within the same product?

Answer: Python language is used at the stage of rapid prototyping of solutions in the data center. Directly in the car is proposed to use languages C or C ++

Is the ASIL D certification only enough for the AURIX processor - if other components of this level fail to achieve it?

Answer: according to the requirement of the standard, we must reach the level of ASIL D for the whole system; however, you can apply the decomposition of the overall system into submodules, some of which may have a level lower than ASIL D.

Questions Nick_mentat

Are the possibilities of developing cars without machine vision systems on non-networks considered? If not, what iron will be used, and will the necessary computing power not be too expensive cars?

Answer: There are various approaches to the creation of unmanned vehicles and there are 2 opposite camps. Some argue that you need to build autopilot, based solely on the cameras and algorithms of view. Others believe that radar and lidar can be used, and information about signs can be obtained from maps or from cameras of low resolution. There are also many who are in the middle and actively use various means for orientation on the roads. We propose to connect a processor to each camera that will cheat an image. Its power may be limited, since in the process of calculating we do not need to recognize all objects, but it is enough to classify them and carry them to some class road, obstacle, sign, pedestrian, and so on. We also care about the final cost, together with the car manufacturers, as the final solution must be available.

Is it possible to organize a sufficiently reliable communication system so that the miners' clusters of some so far non-existent cryptocurrency, all over the world, are engaged in calculations for cars currently traveling in real time? If so, what kind of communication channel will it be, and what equipment will be needed to support it?

Answer: We plan to use a 5G modem in the platform, which will allow you to download high-resolution maps in real time, interact with other machines, and also enable passengers to connect to high-speed Internet. The planned channel will be wide enough, but not sure that the power of the machine, which could be used for some third-party calculations, will remain untapped.

Noonv questions

Please tell me, what architecture of building a control system for an unmanned vehicle do you consider the most promising? Fully based neural networks or modular - where solutions based on "AI" work together with classical algorithms?

Answer: I would say modular. The car uses many algorithms that are used all together to achieve the final result. Of course, for pattern recognition, neural networks today have the best accuracy, but at the same time they require a lot of data for training, are more costly in terms of computation, and are not always stable. On the other hand, for the problem of calculating the trajectory and its smoothing, it is possible to completely manage with classical algorithms.

What was the most unexpected problem you encountered while working on unmanned systems?

Answer: probably, as such, there is no one problem; there is a general scope and complexity of the task, as well as the responsibility that we take upon ourselves. We are talking about a rather complex platform, in which there are many different components that interact with each other. Understand and analyze these dependencies, make sure that they work as they should, and at some point to say, yes it is ready, you can release. For me, when I say that it is ready, I think not only about how many times and which tests we drove and how formally we fulfilled all the requirements, but also whether I am ready to put my family and children in an unmanned vehicle and remove my hands from steering, entrusting the management of what we did with the team.

Question rost99

He is interested in how it is planned to collect / collect data for training, and how much data for this can be considered minimally sufficient.

Answer: Each car that drives along the road during the day collects about 4 TB of data. We plan to release on the roads a few dozen cars that during the year will collect these data, which later will be used for training. So we are talking about the fact that there will be a lot of data, and it will probably be that sufficient level to put the car on the road with confidence. Is this enough - I think not, and additional training will continue to occur as the machines become larger, the algorithms will be improved, the rules will change, and so on.

Question asympro

How do you test your products, sensors and decision making?

Answer: there are various testing methods. Some of them are imposed by the requirement of the standard itself: tests built on the basis of the requirement of functionality, testing the operability of interfaces, checking for resource consumption, checking for working with faulty data, and so on. In addition to all this, we accumulate a large amount of data taken from sensors and cameras while the vehicle is moving with the driver, the data are digitized, and the algorithm is tested under conditions similar to real ones. In addition, work is also underway on the use of virtual reality tools.

Sith questions

What prevents (and what does not prevent) the most affordable unmanned family car on public roads today?
As options:
Lack of inexpensive / necessary radar on the market.
Lack of inexpensive / necessary calculators.
Imperfection / lack of reliable for all situations algorithms for the calculator.
Not enough good road conditions (bad markings, sudden road obstacles, etc.).
Not always clear road conditions sometimes even for a live driver.
Unpredictable actions of other road users.
Large energy loss to the computer (especially important for EV).
Insufficient accuracy and accessibility (tunnels) of data from navigation systems (GPS)
Strange appearance (protruding radar).
Other inconveniences (the trunk is partially occupied by the computer).
Laws.

Answer: There are many factors here and in fact all of them have been listed. First of all, it is, of course, the ability to efficiently and accurately process data and calculate the trajectory of movement, while remaining within the framework of the consumption of electricity, which is permissible for a car (in the long run, electric). There are many different partial solutions that are actively used (row adjustment, adaptive cruise control, and so on). Creating the same full-fledged unmanned vehicles requires a qualitative transition in terms of the types and number of algorithms used in the final solution, multiple duplication of their work by other algorithms, and so on. For example, we say that neural networks have an accuracy of 90+ percent for pattern recognition. This is a very good result, but in order to entrust the management, we cannot rely only on such an algorithm that is known in advance that it is mistaken.

Are radars really necessary if a person quite successfully copes with driving a car in general in any situation, even closing one eye and directing it with four buttons on the keyboard watching the process on a flat screen with a very low frame rate and black and white picture 640x480?

Answer: the question is really very interesting. As I said, answering another question, there are different trends. Some argue that a single high-precision camera is enough and they are researching algorithms that can restore the image depth. Others, on the contrary, argue that radars and lidars may well cope on their own and the cameras are completely unnecessary. I tend to think that at the very beginning of the path of unmanned vehicles that will drive in the next few years, we will see solutions that combine both of these components, yet the risk of error is too great and excessive duplication will not interfere. At the same time, I am confident that work on optimizing the solution and reducing the total number of elements will continue actively. Where science will lead us, let's see in 30 years.

Areso Questions

Is it possible to produce such cars on roads with unpredictable traffic? For example, some parts of the Russian roads, or in large parts of Southeast Asia?

Answer: movement in dense traffic conditions will need to be tested separately. Work in this direction, including those already underway.

How will such a car react to a motorcyclist moving between the rows?

Answer: as a motorcyclist who travels between rows :) the car is not programmed for the specific behavior of other road users. It has its own strategy and adapts it depending on the behavior of other road users.

Will the car violate the rules (traffic, parking), if asked about it by the owner?

Answer: so far it is not assumed that the driver can tell how to drive the car. Choose one of the proposed route options yes, but do not speak at what speed to drive or when you need to turn around. This is the meaning of unmanned vehicles, that such situations are becoming less possible.

questions vassabi