Logic of auto-robot: from engine vision to transmission control

A.Zhukovsky, S. Usilin, V.Postnikov

Today we want to talk about the new project, which started a little more than a year ago at the department of "Cognitive technologies" of MIPT.

It consists in creating a machine vision system, a robot - car ( Fig. 1 ), which should process the video stream in real time, recognize the surrounding scene, detect objects and form a control action aimed at solving the set task.

Fig. one

At the same time, we did not attempt to completely recreate the real conditions of the road scene, excluding all the charms of small-sized modeling.

To begin with, on simple examples, we wanted to work out the main architectural components of the system (the base for receiving video stream and distributed processing on combinations of minicomputers and video cameras, as a prototype System – on – a – Chip (SoC)), potentially suitable for solving more complex problems.

We taught the robot to move along the corridor and detect simple objects, for example, an orange road cone. The task was that he could drive up to the object and stop. And then they decided to play with the ball. In the current version, if the ball is in the field of view of the camera, the robot detects the ball, accelerates and pushes it with a bumper. If the ball leaves the field of view of the camera, the robot starts searching for it.
')

The video was shot during a report at the autumn conference of young scientists at MIPT,
right in the hallway of the main building

Now we teach the robot to pass the "snake" for a while. This exercise allows you to assess the quality of the control system and its progress from version to version. And also to compare with the quality of manual control with human participation.

Initially, our robot contained only a control computer, a camera and, in fact, a chassis. This model is a sports SUV Traxxas Slash 2wd made in the ratio of 1:10. ( Fig. 2 , Fig. 3 )

Fig. 2 Traxxas Slash 2wd

The chassis controller is made on the basis of the Arduino nano, but in fact only the ATMega32 microcontroller is used from it.

A little later, we added front sonar to the circuit to control the distance to the obstacles - in other words, so that the robot would not bumper into corners and walls.

Fig. 3 Traxxas Slash 2wd

If in the first version, the robot broadcast video via HTTP back and control signals were generated on the desktop, in the current version 2.0 (shown in the video) the whole cycle is closed on board, with the main video processing burden placed on the Odroid U2 minicomputer. ( Fig. 4 -1 )

In addition to the computer, the equipment version 2.0 includes:

• robot control device (Fig. 4 -2);
• Logitech HD Pro C920 / Genius WideCam 1050 camcorder (almost arbitrary webcam can be used) (Figure 4-3);
• ASUS USB-N10 Wi-Fi adapter (Fig. 4-4);
• USB hub (Figure 4-5)
• sonar LV-MAXSONAR-EZ2 (Fig. 4-6)

Fig. four

The functions of the robot control device include:

1. implementation of commands of the controlling computer :
   
   • generation of control PWM signals,
   • external load control (7 channels);
   
   
2. signal processing from sensors :
   
   • Sonars (8 channels),
   • Hall Sensor,
   • battery voltage sensor (ADC);
   
   
3. robot protection :
   
   • emergency stop for frontal sonar,
   • stop at loss of control signal.
   
   

Fig. 5 Scheme of the equipment of the robot

Now we are collecting the 3rd version. In it, the video capture system will already include two professional IDS video cameras, the video signal from which (including detection of objects) will be processed on separate minicomputers, in turn, connected to the central minicomputer, which produces the final scene recognition and control actions.

It is also planned to put several sonars around the perimeter of the robot in order to more fully represent the environment for solving parking problems.

Further, quite a few improvements are planned, for example, we want to hang on the robot light equipment, like real cars, for which some of the parts are printed on a 3D printer. It is necessary to simulate the movement behind the car in front with a certain distance (a case of traffic in a dense stream or traffic jam).

The first lyrical digression.

By the way, if you do something like this, we immediately warn against using Chinese analogs - the first version of the motor controller was made on it, which resulted in a few weeks of searching for the cause of the strange motor behavior - it turned out that the microcontroller from the “analog” closed some inputs at the exits. Perhaps, we were so lucky, but with the original Arduino there were no such problems.

The second lyrical digression.

Before creating the chassis controller, it turned out that it is not known how and what to control. The lack of official documentation on the control signals of the chassis components made it possible to recall the labs on physics and dig deeper with the oscilloscope. As a result, it turned out that pulse-width modulation is used there. In general, nothing complicated.

The third lyrical digression.

Somewhere between the discovery of the problem and the gathering of a new controller, it was decided to abandon the circuit board on which the first controller was assembled, and therefore the “iron-printer” method of PCB layout was taken from the naphthalene. The result was very neat and cool.

The robot behavior algorithm for chasing a ball is shown schematically in the figure below. It seems that there is nothing incomprehensible in it. Is that worth saying a few words about the reversal algorithm. This is a test of the situation when the ball goes out of sight of the robot. In fact, he has only 4 possibilities for a turn: forward right, forward left and back right and back left. The robot remembers where it saw the ball for the last time and turns in this direction, by complete analogy with a football player, from whose sight the ball disappeared. He seeks to turn in the direction where the ball flew.



For the reversal, we apply the "asterisk" algorithm: we are going, for example, first to the right and forward, then to the left and back, we get such arcs, convex toward the common center point. This maneuver is reminiscent of a turn in a limited space, known to many by the traffic police exam ( Fig. 6 ).

Fig. 6

If there is a deadlock due to the fact that the robot gets stuck, for example, having caught on the leg of the chair, the control program identifies this situation due to the discrepancy between the engine speed and the angle of rotation of the wheels. In this case, the robot is trying to pass back, make a maneuver on the "asterisk" and continue to move to the goal.

The fourth lyrical digression.

When we were preparing for the conference of young scientists of the Moscow Institute of Physics and Technology, we increased the parameter of acceleration to increase the audience perception. As a result, the robot began to run into obstacles more often, since the detection circuit of the “stuck” case stopped working adequately - the wheels began to slip. In addition, at higher speeds, the robot began to miss the ball more often (this can be seen in the first video). Therefore, we had to solve the problem of optimal balancing between the processing speeds of the video stream, the movement of the robot and decision making. In terms of complexity, it resembled the problem of getting used to an old Soviet-made car with a manual gearbox, such as the Moskvich or Zhiguli, each of which had its own grip and ignition adjusted. Whoever faced this, he understands that it takes some time to adjust to catch the balance between the clutch and the gas pedal, so that the car accelerates smoothly.

Fig. 7 Robot Behavior Behavior Algorithm

In the third version of the robot (it is almost complete), we switched to using a “professional” video camera and lens.

Fig. eight

In parallel, we are conducting experiments on the installation of cameras on slats, which are mounted on the rails of a full-size vehicle ( Fig. 8 ). This will allow to reproduce the real geometry and conditions of the road scene.

We are going to tell about the new version of the robot in the next article.