How we did a robot soccer player
On November 25, 2012, the largest robot competition in the Baltic States - “Robotex” - was held in Tallinn. We decided to build a robot in the professional football category. Of course, this will not be Cristiano Ronaldo, but an interesting challenge. I will describe the details of creating and programming the robot. His name is Palmer.
Football itself takes place on the ground in green, on which there are 11 orange golf balls. There are gates, 15 cm tall and about 37 cm wide, yellow on one side and blue on the other. The robot must search for balls on the field, capture them, select the desired goal and score. There are two robots on the field. The winner is the one who scores more balls. Technical requirements for the robot: a cylinder with a height of 35 cm and a diameter of 35 cm. Everything is simple.
After discussing the details and a couple of crates of beer came to this configuration. Onboard there will be a mini-format motherboard with an Atom processor. Pros - does not require cooling and separate power, compactness. Cons, respectively, in low productivity. The image is captured by a Sony Playstation 3 camera, the perfect camera for the price and quality. The frame rate is critical, since the maximum possible speed of the robot depends on it. Suppose, with a frequency of 30 frames per second and a robot speed of 3 meters per second, it will travel 10 cm between two frames, which is clearly not enough for an accurate ball grab. PS3 gives up to 125 frames per second. The camera driver fits well with the Linux kernel. Ubuntu operating system with USB download. For image recognition using your favorite OpenCV.
In addition to the camera, IR sensors and lighthouses are used to explore the surrounding world. Infrared sensors allow you to avoid collisions with another robot or gate. Power supply - lithium battery, with a decrease in power from 14.8V to 12V for the motherboard. Power supply for motors LiPo batteries (2x 4S 14.8V 2200mAh + 2x 2S 7.4V ~ 2000mAh). Separate power for the motors is used because when starting the motors there is a strong voltage drop in the circuit, which in turn can reset the microcontroller. Yes, and a separate power allows you to remove the high-frequency noise from the motors, which also knocks the microcontroller. There are two types of capacitors in the circuit, some smooth the voltage drops, others, faster, smooth the high-frequency noise. Broadband noise occurs in the process of sparking the brushes of the motor, which, passing through the circuit, knocks microcontrollers. To solve this problem, H-Bridge with optical breakers is used. H-Bridge is a bridge relay whose circuit allows you to change the polarity of the current on the motors without the risk of a short circuit. Signals suitable to the bridge pass through optical breakers. Optical interrupters break the network, and the noise from the motors does not fall into the controller circuit. They consist of an LED and a photocell in one package, i.e. the circuit is open, but the signals are on. Such bridges are used to control anything. We make boards for the bridge ourselves.
')
For hitting the ball, a solenoid is used, which has its own block of capacitors connected in parallel to create a large impulse. To power the solenoid is a separate battery.
Next in the kit are electrolytic capacitors to stabilize the voltage, and ceramic capacitors and ferrite rings to combat the ubiquitous noise, because ALL is noisy. And there remain such little things as the frame, wheels, motors, bolts and other cogs. But the beginning.
And in a few days.
Mechanics and electronics were collected in non-stop mode, sometimes working at night. The code started writing in parallel, using C ++ and as the Qt development environment. A robot is a classic abstract machine gun that has a set of states (a ball is found, a movement, a search for a gate, or a target is found, a rocket is aimed), and it does not matter whether a military robot, a toy or an industrial robot. The problem is that there should be as few states as possible, and as many as necessary. If the algorithm is made too confusing, then at some point you will not understand why the robot reacts this way. We have 8 states: Start, Ball Search, Gate Search, Aiming, Obstacle Avoidance, Solenoid Capacitor Charge, Impact, End. Transitions between them form a graph, which is an algorithm.
Most of the time is spent on writing and fine-tuning the code, testing, calibration. What really is the most important thing.
The recognition of the robot allocated to individual classes, is constructed as follows. Each frame is processed, where colors are first converted from RGB to HSL, which is more close to our perception, and with which it is more convenient to work. For convenience of conversion, the built-in OpenCV function is not used, since conversion is calculations and such expensive time is spent on them, but a pre-calculated lookup table, where the RGB value is the offset in the table, and then the color is sampled at. We win in speed, we lose in memory, everything is like in theory. Next, objects (balls, gates) are searched for sequentially and written into the list. Objects are searched for by a common algorithm, each pixel runs through the loop, and if it falls within the range of the object's color, a bit is put on the map. The result is a bitmap card with selected objects. The bitmap card is sent to the input of the OpenCV function, which returns the list of contours. Next, we find the area of the contours and if the area of the contours is more than 40-50% of the area of the circle (rectangle) or in the 4 / Pi range, then we consider this object recognized and add it to the list. Balls are searched in a certain range (small noise, there is no big noise, there is no sense to look for balls on the ceiling either). Since the camera gives out up to 125 frames per second, you cannot use some of the built-in functions of OpenCV, they are long, but the library is convenient and you can run through the pointers over the whole matrix. It remains to make a check that the ball has not rolled out over the edge of the field (black is limited), since the robot should not go beyond the black line. The check is quite simple, draw a line from the center of the bottom of the image to the ball and see if it crosses the black area. If it intersects, the contour from the list is thrown out.
Next - the most interesting. A robot has a certain number of states in which it moves depending on the conditions. It's simple. The algorithm is written at this stage. The most difficult thing is calibration. How to build a function of speed, direction from the obtained objects and their coordinates?
We run the entire issued list and find the largest ball by area. This is the closest. Now, in order to obtain a function of speed and direction from the coordinates of an object, it is required to think and smoke a matan. The higher the coordinates of the ball in the image and the smaller it is in area, the farther it is. The more shifted along the X axis and further / closer, the sharper, or vice versa, obtuse angle. To get the coefficients, the robot is placed on the platform at one end in the center, then the process of crawling with a tape measure, measuring points with a certain step (33 cm) and taking readings of object values in the recognition system (contour area) begins. Putting values and trying to find a function that would be the connection of some values with others. Strong accuracy is not needed, because the robot does not build a path once and for all, but corrects itself 17-18 times a second, so much our small system betrays. Those. it is necessary to find a function whose limit tends to the desired results with small changes in the arguments. Next, the angle and direction are fed to the wheels, from where the speeds are calculated by the simplest vector equations. Figure will explain.
Further, when we are able to move to objects, the algorithm of the game of football is not so difficult to write. But already close to the competition. My right.
Result in action:
On Saturday evening, we already moved to the sports hall, where competitions are held. At night it was great, your social circle, various clubs and laboratories from different countries. A lot of people. No one sleeps. All communicate with each other and are interested in everything that is possible. I myself slept 2.5 hours. And all this time was tuning. As a person participating for the first time, I did not know about the subtleties and problems that occur.
The most important thing is that it is impossible to calculate everything, and the emphasis is on testing. He who tests and drives longer is the main one. Suppose it’s better not to make turns with all robots, and the robot must rotate around the ball, because when the robot rotates, the ball rotates the robot's impulse; Here problems come out. The first is that you don’t know what kind of rotation speed the robot had before. That is, if the robot travels at a leading angle directly to the ball, it will strike immediately and by, since it did not have a rotation. Or vice versa - he will drive up to the ball from the turn to the other side, start turning and hit, but this will be by, since due to inertia its speed of turn will not yet be calculated. Those. too many choices to calculate. Turning around the axis of the ball does not add a rotational momentum to it, and you can shoot at once.
The second is a problem with recognition. When the robot is spinning, at long distances the balls and the gate again have a greater linear speed, and the frame is blurred for recognition. The contour search function works on gradients, but blurred gates have no clear gradients, and therefore instead of a single gate contour, dozens of contourists are obtained at the exit (this eats resources) or nothing happens (the ball is smeared into a little orange fog). I had 17 frames per second at the input, respectively, distant objects during rotation were not available to me. Plus the delay in processing. Those. when the robot realizes that there were gates in the distance, they are no longer there (although this was what I used to my advantage when I gave the rotation from my gates, and the delay was such that my robot turned almost to the enemy gates). An ultrabook on the robot's head would solve the problems.
One of the solutions that our other team used is to cut the picture, i.e. when we see the ball, we start to cut off the excess, center it, and cut the sides and throw it out, FPS jumps to 60. The same logic when recognizing gates, especially when searching, the lower part is not needed, we cut it.
I had another camera and there would be problems with cuts. A good solution with two cameras. One wide-angle, looks directions and gives an overview. The second gives distance and sees far.
To solve the problem of turning the robot, I used a stop, if after a couple of seconds during the turn it does not find anything. The stop gives you the opportunity to look further, but at an angle of 60 degrees, which gave our camera. Next, search again.
My second omission is that I did not know that robots would start from the right corner. The robot must remember in which side of the enemy's gate and where it should turn, otherwise the wrong turn takes a lot of time. I determined the sides of the gate by color, but until the gate came into view, the directional variable (plus or minus one sets the direction of the turn) defaults to -1 counterclockwise, and it was necessary to clockwise. And because of this, in the losing match, our first two goals scored on the opposite side and lost time. A total of 6-4, and in the second draw 5-5, and our loss. 5 place from 18 teams. A still high center of gravity prevented, not to set a great speed - it starts to dance. I called it a printer.
In general, there are a lot of nuances. What is remembered is an interesting get-together. And the rhythm. Those. speed and constant refinement of cars. After every match. With the champion, we played 4-3! (tactics of protection, closed the gate and beat from afar). The first one-on-one match we had at competitions! Hence the inability to test the strategy of behavior. During the week we were chased by breakdowns. Plus, we lost the camera (axis), I tried to solve the problems of mechanics with code, which can never be done, but there was no time, and I decided everything just before the last match. In total, we played 4 matches.
In the first defeated robot cube.
In the second they won the Skype team, who had a very interesting robot. Insanely fast, small, with 8 cameras and FPGA! That is, they did something that shouldn’t work at all. He quickly flew out, but was charming. Here he is.
There were very interesting decisions. The robot that won the first place (it was being finalized for 3 years) has an overview of 360 degrees. The camera looks up at the cone-shaped mirror. It turns out the picture in polar coordinates.
And about everything to calculate, there is such a story in the annals of "Robotex", on which everyone is constantly laughing.
Some 5 years ago two masters came to the competitions and said that they had all calculated and built the robot. They were met with a burst of laughter. And the subsequent defeat.
Thanks for attention.
Football itself takes place on the ground in green, on which there are 11 orange golf balls. There are gates, 15 cm tall and about 37 cm wide, yellow on one side and blue on the other. The robot must search for balls on the field, capture them, select the desired goal and score. There are two robots on the field. The winner is the one who scores more balls. Technical requirements for the robot: a cylinder with a height of 35 cm and a diameter of 35 cm. Everything is simple.
After discussing the details and a couple of crates of beer came to this configuration. Onboard there will be a mini-format motherboard with an Atom processor. Pros - does not require cooling and separate power, compactness. Cons, respectively, in low productivity. The image is captured by a Sony Playstation 3 camera, the perfect camera for the price and quality. The frame rate is critical, since the maximum possible speed of the robot depends on it. Suppose, with a frequency of 30 frames per second and a robot speed of 3 meters per second, it will travel 10 cm between two frames, which is clearly not enough for an accurate ball grab. PS3 gives up to 125 frames per second. The camera driver fits well with the Linux kernel. Ubuntu operating system with USB download. For image recognition using your favorite OpenCV.
In addition to the camera, IR sensors and lighthouses are used to explore the surrounding world. Infrared sensors allow you to avoid collisions with another robot or gate. Power supply - lithium battery, with a decrease in power from 14.8V to 12V for the motherboard. Power supply for motors LiPo batteries (2x 4S 14.8V 2200mAh + 2x 2S 7.4V ~ 2000mAh). Separate power for the motors is used because when starting the motors there is a strong voltage drop in the circuit, which in turn can reset the microcontroller. Yes, and a separate power allows you to remove the high-frequency noise from the motors, which also knocks the microcontroller. There are two types of capacitors in the circuit, some smooth the voltage drops, others, faster, smooth the high-frequency noise. Broadband noise occurs in the process of sparking the brushes of the motor, which, passing through the circuit, knocks microcontrollers. To solve this problem, H-Bridge with optical breakers is used. H-Bridge is a bridge relay whose circuit allows you to change the polarity of the current on the motors without the risk of a short circuit. Signals suitable to the bridge pass through optical breakers. Optical interrupters break the network, and the noise from the motors does not fall into the controller circuit. They consist of an LED and a photocell in one package, i.e. the circuit is open, but the signals are on. Such bridges are used to control anything. We make boards for the bridge ourselves.
')
For hitting the ball, a solenoid is used, which has its own block of capacitors connected in parallel to create a large impulse. To power the solenoid is a separate battery.
Next in the kit are electrolytic capacitors to stabilize the voltage, and ceramic capacitors and ferrite rings to combat the ubiquitous noise, because ALL is noisy. And there remain such little things as the frame, wheels, motors, bolts and other cogs. But the beginning.
And in a few days.
Mechanics and electronics were collected in non-stop mode, sometimes working at night. The code started writing in parallel, using C ++ and as the Qt development environment. A robot is a classic abstract machine gun that has a set of states (a ball is found, a movement, a search for a gate, or a target is found, a rocket is aimed), and it does not matter whether a military robot, a toy or an industrial robot. The problem is that there should be as few states as possible, and as many as necessary. If the algorithm is made too confusing, then at some point you will not understand why the robot reacts this way. We have 8 states: Start, Ball Search, Gate Search, Aiming, Obstacle Avoidance, Solenoid Capacitor Charge, Impact, End. Transitions between them form a graph, which is an algorithm.
Most of the time is spent on writing and fine-tuning the code, testing, calibration. What really is the most important thing.
The recognition of the robot allocated to individual classes, is constructed as follows. Each frame is processed, where colors are first converted from RGB to HSL, which is more close to our perception, and with which it is more convenient to work. For convenience of conversion, the built-in OpenCV function is not used, since conversion is calculations and such expensive time is spent on them, but a pre-calculated lookup table, where the RGB value is the offset in the table, and then the color is sampled at. We win in speed, we lose in memory, everything is like in theory. Next, objects (balls, gates) are searched for sequentially and written into the list. Objects are searched for by a common algorithm, each pixel runs through the loop, and if it falls within the range of the object's color, a bit is put on the map. The result is a bitmap card with selected objects. The bitmap card is sent to the input of the OpenCV function, which returns the list of contours. Next, we find the area of the contours and if the area of the contours is more than 40-50% of the area of the circle (rectangle) or in the 4 / Pi range, then we consider this object recognized and add it to the list. Balls are searched in a certain range (small noise, there is no big noise, there is no sense to look for balls on the ceiling either). Since the camera gives out up to 125 frames per second, you cannot use some of the built-in functions of OpenCV, they are long, but the library is convenient and you can run through the pointers over the whole matrix. It remains to make a check that the ball has not rolled out over the edge of the field (black is limited), since the robot should not go beyond the black line. The check is quite simple, draw a line from the center of the bottom of the image to the ball and see if it crosses the black area. If it intersects, the contour from the list is thrown out.
Next - the most interesting. A robot has a certain number of states in which it moves depending on the conditions. It's simple. The algorithm is written at this stage. The most difficult thing is calibration. How to build a function of speed, direction from the obtained objects and their coordinates?
We run the entire issued list and find the largest ball by area. This is the closest. Now, in order to obtain a function of speed and direction from the coordinates of an object, it is required to think and smoke a matan. The higher the coordinates of the ball in the image and the smaller it is in area, the farther it is. The more shifted along the X axis and further / closer, the sharper, or vice versa, obtuse angle. To get the coefficients, the robot is placed on the platform at one end in the center, then the process of crawling with a tape measure, measuring points with a certain step (33 cm) and taking readings of object values in the recognition system (contour area) begins. Putting values and trying to find a function that would be the connection of some values with others. Strong accuracy is not needed, because the robot does not build a path once and for all, but corrects itself 17-18 times a second, so much our small system betrays. Those. it is necessary to find a function whose limit tends to the desired results with small changes in the arguments. Next, the angle and direction are fed to the wheels, from where the speeds are calculated by the simplest vector equations. Figure will explain.
Further, when we are able to move to objects, the algorithm of the game of football is not so difficult to write. But already close to the competition. My right.
Result in action:
On Saturday evening, we already moved to the sports hall, where competitions are held. At night it was great, your social circle, various clubs and laboratories from different countries. A lot of people. No one sleeps. All communicate with each other and are interested in everything that is possible. I myself slept 2.5 hours. And all this time was tuning. As a person participating for the first time, I did not know about the subtleties and problems that occur.
The most important thing is that it is impossible to calculate everything, and the emphasis is on testing. He who tests and drives longer is the main one. Suppose it’s better not to make turns with all robots, and the robot must rotate around the ball, because when the robot rotates, the ball rotates the robot's impulse; Here problems come out. The first is that you don’t know what kind of rotation speed the robot had before. That is, if the robot travels at a leading angle directly to the ball, it will strike immediately and by, since it did not have a rotation. Or vice versa - he will drive up to the ball from the turn to the other side, start turning and hit, but this will be by, since due to inertia its speed of turn will not yet be calculated. Those. too many choices to calculate. Turning around the axis of the ball does not add a rotational momentum to it, and you can shoot at once.
The second is a problem with recognition. When the robot is spinning, at long distances the balls and the gate again have a greater linear speed, and the frame is blurred for recognition. The contour search function works on gradients, but blurred gates have no clear gradients, and therefore instead of a single gate contour, dozens of contourists are obtained at the exit (this eats resources) or nothing happens (the ball is smeared into a little orange fog). I had 17 frames per second at the input, respectively, distant objects during rotation were not available to me. Plus the delay in processing. Those. when the robot realizes that there were gates in the distance, they are no longer there (although this was what I used to my advantage when I gave the rotation from my gates, and the delay was such that my robot turned almost to the enemy gates). An ultrabook on the robot's head would solve the problems.
One of the solutions that our other team used is to cut the picture, i.e. when we see the ball, we start to cut off the excess, center it, and cut the sides and throw it out, FPS jumps to 60. The same logic when recognizing gates, especially when searching, the lower part is not needed, we cut it.
I had another camera and there would be problems with cuts. A good solution with two cameras. One wide-angle, looks directions and gives an overview. The second gives distance and sees far.
To solve the problem of turning the robot, I used a stop, if after a couple of seconds during the turn it does not find anything. The stop gives you the opportunity to look further, but at an angle of 60 degrees, which gave our camera. Next, search again.
My second omission is that I did not know that robots would start from the right corner. The robot must remember in which side of the enemy's gate and where it should turn, otherwise the wrong turn takes a lot of time. I determined the sides of the gate by color, but until the gate came into view, the directional variable (plus or minus one sets the direction of the turn) defaults to -1 counterclockwise, and it was necessary to clockwise. And because of this, in the losing match, our first two goals scored on the opposite side and lost time. A total of 6-4, and in the second draw 5-5, and our loss. 5 place from 18 teams. A still high center of gravity prevented, not to set a great speed - it starts to dance. I called it a printer.
In general, there are a lot of nuances. What is remembered is an interesting get-together. And the rhythm. Those. speed and constant refinement of cars. After every match. With the champion, we played 4-3! (tactics of protection, closed the gate and beat from afar). The first one-on-one match we had at competitions! Hence the inability to test the strategy of behavior. During the week we were chased by breakdowns. Plus, we lost the camera (axis), I tried to solve the problems of mechanics with code, which can never be done, but there was no time, and I decided everything just before the last match. In total, we played 4 matches.
In the first defeated robot cube.
In the second they won the Skype team, who had a very interesting robot. Insanely fast, small, with 8 cameras and FPGA! That is, they did something that shouldn’t work at all. He quickly flew out, but was charming. Here he is.
There were very interesting decisions. The robot that won the first place (it was being finalized for 3 years) has an overview of 360 degrees. The camera looks up at the cone-shaped mirror. It turns out the picture in polar coordinates.
And about everything to calculate, there is such a story in the annals of "Robotex", on which everyone is constantly laughing.
Some 5 years ago two masters came to the competitions and said that they had all calculated and built the robot. They were met with a burst of laughter. And the subsequent defeat.
Thanks for attention.
Source: https://habr.com/ru/post/249229/
All Articles