How I screwed the speedometer to the tanks

Once I saw a video a long time ago, where a person was playing in a kind of car simulator and he had 2 large dial gauges (voltmeters, if I'm not mistaken) on his desk, which served as a speedometer and tachometer. A few years later I decided to repeat something similar.

I only played World of Tanks (and then I abandoned it a long time before I reached the 10th level), so I decided to do it with tanks. At once I will tell, all development, except for debugging was made on linux, the code executed on the PC is written on a python. Yes, it is slow and I am ashamed to show the code, so we can do without it.

Speedometer

As a speedometer, I also decided to use something with an arrow, this role fell out of the old, cheap, Chinese multimeter with an arrow indication.
')
It was decided that the ATMEGA16 would be the speedometer, since there was a debug board with all the necessary strapping on board. PC communication -> MK is organized via the serial port.



The firmware is primitive, the algorithm is something like this:

1. In an endless loop, listen to USART.
2. When receiving speed data, they were checked for validity and with the help of PWM the necessary voltage level was set.

This was the simplest part, then it was necessary to obtain the value of speed. Immediately, there were 2 ideas: Get data from the game process (wool memory in the hope of finding the desired address) or scan the value directly from the interface itself and recognize it. The last for me was a completely dark forest and sounded interesting.

Getting the cherished speed

First we get a screenshot of the game, I naively thought that there would be no problems with it, but as it turned out, capturing the screen in DirectX mode is not the easiest thing, but it can be solved of course. Instead of the game interface, Malevich’s rectangles were obtained; useless images, without thinking twice, I switched to windowed mode and maximized the game window to full screen, the difference in informativeness with fullscreen mode is minimal, but the screen began to be captured remarkably.

I was not capturing the entire screen, but a small area of ​​200x200 approximately pixels, I already worked with it.



Then I opened such a wonderful thing as OpenCV for myself, using it to discolor the picture and invert the colors (initially the background is dark and the numbers are light).

Our speed represents a maximum of two-digit number, and in a monospaced font, which also pleased, because the digits occupied a strictly defined place and did not shift as the speed value changed. It only remained to select these 2 positions and recognize them.

Recognition

I reduced the recognition of the value to the comparison with the standard. The comparison took place according to the difference of the image hashes: the smaller the difference, the higher the probability that this is the exact number. I made a lot of standards for all numbers (for example, 50 pieces of the number 3 from the first and second category), captured our positions, numbered, then manually sorted into folders according to the denomination. And then he compared the hash in the inside of one nominal (ie, units with units, etc.) to find out how much the same values ​​differ from each other. Then he mixed similar patterns (for example, to 3 - 8) and again considered the difference. This information was key to the accuracy of the definition. But as it turned out, there were borderline values, i.e. sometimes all the same, 3 and 8 and 9 were confused, though not often, but not pleasantly. The reason for this was the translucent background and the very small size of the numbers themselves.

I had to climb into the client and change the standard damage panel to the custom one, where I increased the font 2 times, changed the background color, and also turned off the transparency completely. I ran tests with a new panel and, as expected, recognition accuracy increased by an order of magnitude. Also refused to invert the color.



I gathered all the code together, I wrapped it up in a loop and went, in video, in principle, what happened. The WoT client itself after the last update began to slow down strongly, while at the time of lags the speedometer (in the interface) sometimes shows completely inadequate values. The work is not perfect, but I am satisfied with the result.



To capture used Python Imaging Library (PIL) .
To work with the image: OpenCV .

I apologize for the uneven handwriting, this is my first article on Habré.