What for?

I was thinking about laser vision correction for a long time, and then, finally, I decided on a procedure. After a brief study of the market (I live in St. Petersburg), it turned out that prices around the city are about the same everywhere, and there is also no sense in medical tourism (in Moscow, it is not much cheaper). However, it turned out that the operation can be significantly saved, because one of the clinics provides an extensive system of discounts on its services.



Discounts for veterans and pensioners, of course, did not interest me. But I decided to use the unusual action “play a flash game on our website and convert points scored into a discount”. Sub-description of the process.



In general, the idea at first was astonished by its absurdity - it seems that computer games are harmful to eyesight, and then the action is similar to "scoop out 10,000 buckets of ice-cold water from the basement and get a discount on rheumatism treatment." The game itself, I must say, was also struck by its stubbornness - it is obvious that the authors wanted to make the game without violence, so the legend says: “with the help of LASIK technology, help restore the sight of the moles”. And, judging by the animation, the treatment of myopia is performed by instantaneous evaporation of the patient.

')

Well, okay, this is the lyrics. In fact, I immediately tried to knock out a discount, however, my entire gaming experience did not help me get even 17% from the first time. Having played several times, I still scored the required 17,000 points, but it was clear that even 20,000 were an unattainable bar, not to mention the cherished 25,000. Damned moles climb from all cracks, but quickly hide back. In this case, for the "healing" of the mole is given 100-200 points, so you can not miss them. I do not know whether it is a man.



The decision came to mind immediately - you need to write a bot that will play for me! The process of writing a bot on C # rolled up.

How?

Concept

Two forces started fighting in me, about the battles of which many posts have already been written. On the one hand, I wanted to write a beautiful and well-designed application. On the other hand, the thought “the code should work, it should not do anything” pulsed in my head. In general, for the experiment, I decided to fully adhere to the second concept. Well, let's go.

Eyes

First, take OpenCV to capture frames from the screen and recognize objects ... STOP. I don't need some kind of super app, I just need to get this discount. Should I tinker with OpenCV? Maybe it is easier to start a stream that will take a screenshot of the screen in an infinite loop and view it? For example:

Bitmap bmpScreen = new Bitmap(Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height); Graphics g = Graphics.FromImage(bmpScreen); while (true) { g.CopyFromScreen(Screen.AllScreens[0].Bounds.X, Screen.AllScreens[0].Bounds.Y, 0, 0, bmpScreen.Size, CopyPixelOperation.SourceCopy); } 

Arms

And how to "treat" moles? Obviously, at startup you need to find the browser window and send it a WM_CLICK message with the necessary parameters. However, you can make everything easier - physically move the cursor to the desired location on the screen and emulate keystrokes.



Import the appropriate WinAPI functions.

 [DllImport("user32.dll")] static extern bool GetCursorPos(ref Point lpPoint); [DllImport("user32.dll")] static extern bool SetCursorPos(int X, int Y); [DllImport("user32.dll")] static extern void mouse_event(uint dwFlags, uint dx, uint dy, uint dwData, IntPtr dwExtraInfo); 

And write the function to click

  static public void MouseClick() { Point ptCoords = new Point(); GetCursorPos(ref ptCoords); uint x = (uint)ptCoords.X; uint y = (uint)ptCoords.Y; System.IntPtr ptr = new IntPtr(); mouse_event(MOUSEEVENTF_LEFTDOWN, x, y, 0, ptr); mouse_event(MOUSEEVENTF_LEFTUP, x, y, 0, ptr); } 

Now that all the service functions are there, it remains to write the logic.

Brain

Having passed the first round several times manually, I made several observations:

1. Moles appear in the same places.
2. Additional targets (robot and UFO) scare away moles, so they should be “treated” first. Moreover, they appear in strictly defined places.
3. 500 points are given for the Copper rocket, and it hangs in one place.

In general, since our goals appear in the same places, the first thing that comes to mind is to make a reference screenshot of the screen, and then just check if the color of the specified pixels has changed.



How to store the coordinates of the goals? Generally it is not so simple. You need to consider the screen resolution, page scroll level, and so on. Those. Not only can the game window have a different size, it can also be in different places on the screen. Fortunately, I did not write a universal game for the game, I just needed to score 25,000 points. Therefore, I decided to open the game in a browser deployed to the full screen, scroll to the very top of the page and record the coordinates of the targets as the physical coordinates of the pixels on the screen.



And how to record the coordinates? The most convenient way for the user is to make a hot key, by pressing which the coordinates of the cursor are saved, for example, to a file. Then, when the mole appears, you need to hover the cursor on it and press the hotkey. Frankly, at first I did. Later it turned out that it was much easier to make screenshots of moles, measure the position of each mole in the graphic editor, and these coordinates are simply hard-coded. It turned out something like this

 List<Point> m_lpTargets = new List<Point>(); m_lpTargets.Add(new Point(557, 623)); //  m_lpTargets.Add(new Point(261, 654)); //  m_lpTargets.Add(new Point(352, 486)); //   m_lpTargets.Add(new Point(450, 500)); // m_lpTargets.Add(new Point(592, 698)); //  m_lpTargets.Add(new Point(756, 631)); //  m_lpTargets.Add(new Point(373, 514)); // m_lpTargets.Add(new Point(481, 440)); //   

Add an Aim button to the form that will make a reference screenshot.

  private void btnAim_Click(object sender, EventArgs e) { m_bmpReference = new Bitmap(Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height); Graphics g = Graphics.FromImage(m_bmpReference); g.CopyFromScreen(Screen.AllScreens[0].Bounds.X, Screen.AllScreens[0].Bounds.Y, 0, 0, m_bmpReference.Size, CopyPixelOperation.SourceCopy); } 

Now we add our bot code to the main loop and run it!

  foreach (Point pt in m_lptTargets) { if (m_bmpReference.GetPixel(pt.X, pt.Y) != bmpScreen.GetPixel(pt.X, pt.Y)) { MouseMoveTo(pt, 10, 200); MouseClick(); } } 

What happened?

First result

Well, it’s not that it doesn’t work at all ... First, in a hurry, as always, I forgot that when a program intercepts user input (especially in such a barbaric way), it becomes problematic to close it. After the launch of the bot, the mouse cannot be brought out of the playing field - it will always jump at the specified points, even when the task manager is started. I admit that I came out of this situation with the help of a reboot ... Actually, to be honest, I think that this is how robots will ever take over the world.

Main switch

It is probably worth making some wise algorithm that will determine that the game is over, and also release the mouse when, for example, the game window is minimized. But, fortunately, I just need to find a way to stop the bot according to my desire, and this will help me to know the Windows hotkeys. I simply add a form resizing handler in which I will stop the flow with the firing code.

  private void Form1_Resize(object sender, EventArgs e) { if (WindowState == FormWindowState.Minimized) { m_objAimingThread.Abort(); } } 

Now, to stop the bot, you just need to click Win + D.

First problems

But this is not even the most important thing. The trouble is that, although our bot successfully heals the moles, something is wrong: when it hits a mole, glasses are knocked out of it, but the mole itself does not disappear, or a white silhouette remains after it. It is clear that in this case, our bot continues to peel at this point to infinity (the color has changed).



Frankly, I fought over this problem for quite a long time. As a result, I came to the conclusion that the game somehow implemented crookedly drawing and accounting of moles. Those. if I hit the mole earlier than it had completely emerged from the hole, then I earn points for it, but at the same time it continues to crawl out. And when the mole is marked as “healed”, then re-get points for it does not work. It can be seen that several variables are responsible for the mole state, in which consistency is violated. It is important that this problem arose only with moles, which crawl out of four holes — there were no difficulties with jumping across the field and flying into space.



I reasoned so - once in the manual mode, such a problem does not arise, then you just need to pause, and maybe even emulate mouse movement to the goal.



After launching a modified program that brings the pointer to a mole in 250 milliseconds, the problem with non-disappearing moles is gone. But another one came out immediately - the bouncing moles at the bottom of the screen move too fast so that they can manage to transfer the sight to it in 250ms. After all, the moles from the holes appear at fixed points, and they sit there for some time, and the jumping moles jump over our “target points” very quickly.



Finally, on closer examination, it turned out that the moles jumping into the atmosphere actually have different X-coordinates, so simply listing their positions would be too tedious. It seems that we need to change the concept ...

New concept

So, moles appear in different places, so it is impossible to track them along the coordinates. Do you have to do pattern recognition? I would not want to. I will try before this another "stupid" implementation. What is common to all moles? Suits and boots are not visible to everyone. But all the moles visible head in the helmet. In addition, the helmet has a very unusual color ... What if you scan the entire image for the presence of this bluish tint? Let's try!

 Color clHelmet = Color.FromArgb(102, 142, 193); for (int j = 400; j < 880; j += 1) for (int i = 200; i < 850; i += 1) { if (bmpScreen.GetPixel(i, j) == clHelmet) { Point ptTarget = new Point(i, j); MouseMoveTo(ptTarget); MouseClick(); } } } 

In the screenshot, I looked at the coordinates of the working area of ​​the flash drive and the color of the helmet. Now on my Core i5, I simply loop through all the pixels using the super-braking method GetPixel, and shoot when the color matches the reference one. With this approach, the execution time of one cycle is about 200 milliseconds, which seems to be a valid value. Run!

And new problems

Everything turned out to be not so simple - there is more than one dot of reference color on the helmet, so the bot, when a mole appears, starts to peel into white light like a pretty penny, ignoring other patients. The solution was found quite simple - we will save the coordinates of the last shot, and when processing the current frame, ignore the pixels in the vicinity of these coordinates.



The second problem of this approach is that the moles move, therefore, often during the frame processing time, the mole manages to escape from the healing beam of photons. In addition, the color chosen by us is on the very edge of the helmet, so even with a slight delay we lose the patient.



To begin, reduce the frame processing time. In general, for this you need to copy the image to the area of ​​direct memory access, and work with the bikes directly ... But we just change the step of scanning cycles to two, thereby increasing the scanning speed of the image four times. Stronger increment is no longer painless - some pixel zones will be skipped.



The second step to the guaranteed cure of moles - when it detects a point of the reference color, shoot not only at it, but immediately turn into the detected area. Here is the code, enter and run.

  Point ptLastHit = new Point(0,0); for (int j = 400; j < 880; j += 2) for (int i = 200; i < 850; i += 2) { if (bmpScreen.GetPixel(i, j) == clHelmet) { Point ptTarget = new Point(i, j); if (Distance(ptLastHit,ptTarget) > 70) { ptLastHit = ptTarget; MouseMoveTo(ptTarget); MouseClick(); ptTarget.Offset(20, 20); MouseMoveTo(ptTarget); MouseClick(); ptTarget.Offset(-40, 0); MouseMoveTo(ptTarget); MouseClick(); } } } 

Patient sorting

Now everything is fine - the moles are successfully healed, but the problem with the moles from the holes has returned. Remember, they did not disappear if you shoot at them without delay? I returned the delay to 250ms, but with such a delay it becomes impossible to get into the running moles ... The solution was found quite quickly - we put the coordinates of the holes in the code, and we will give a delay to the shooting only if the target appears in these areas.



Even to increase the effectiveness of shooting at jumping moles, it is worth stopping frame processing after hitting the first target found. Let me explain - even if there are two moles on the screen, then while we are shooting at one, the second one already has time to shift a little, while in the old frame we still have his previous image. At this moment we are faced with one of the global problems of all C-like languages ​​- the impossibility of interrupting two nested loops with the help of the break command. In such cases, I commit a terrible karmic crime using the goto operator.



Finally, let's not forget about flying saucers, robots and rocket, which must be tracked strictly in certain places. As a result, the cycle began to look like this.

  g.CopyFromScreen(Screen.AllScreens[0].Bounds.X, Screen.AllScreens[0].Bounds.Y, 0, 0, bmpScreen.Size, CopyPixelOperation.SourceCopy); foreach (Point pt in m_lptTargets) { if (m_bmpReference.GetPixel(pt.X, pt.Y) != bmpScreen.GetPixel(pt.X, pt.Y)) { Point ptMouse = new Point(); GetCursorPos(ref ptMouse); if (ptMouse != pt) { MouseMoveTo(pt); MouseClick(); } } } // Rectangle hole1 = new Rectangle(539, 612, 60, 50); Rectangle hole2 = new Rectangle(577, 690, 60, 50); Rectangle hole3 = new Rectangle(738, 621, 60, 50); Rectangle hole4 = new Rectangle(243, 641, 60, 50); // Rectangle hole5 = new Rectangle(379, 415, 45, 40); int iDelay = 5; Color clHelmet = Color.FromArgb(102, 142, 193); DateTime tmNow = DateTime.Now; Point ptLastHit = new Point(0,0); for (int j = 400; j < 880; j += 2) for (int i = 200; i < 850; i += 2) { if (bmpScreen.GetPixel(i, j) == clHelmet) { Point ptTarget = new Point(i, j); if (hole1.Contains(ptTarget) || hole2.Contains(ptTarget) || hole3.Contains(ptTarget) || hole4.Contains(ptTarget) || hole5.Contains(ptTarget)) { iDelay = 200; } if (Distance(ptLastHit,ptTarget) > 70) { ptLastHit = ptTarget; Thread.Sleep(iDelay); MouseMoveTo(ptTarget); MouseClick(); ptTarget.Offset(20, 20); MouseMoveTo(ptTarget); MouseClick(); ptTarget.Offset(-40, 0); MouseMoveTo(ptTarget); MouseClick(); goto next; } } } next: } 

What is the result?

In general, this program has already successfully gained 21-22 thousand points. To get 25, it was necessary to change some magic values ​​such as delays and shot coordinates for the “queue”. At a certain point, the stars were successful, and I passed the cherished mark of 25,000, for this, a couple of dozen launches were required.



It took two nights for all the work from the moment of creating the project to the moment of printing the discount coupon. Now I understand that the work could be reduced if the algorithm had been thought out in advance. However, I saved a lot of time due to the fact that I constantly tugged at myself when it occurred to me to create a product that would pass the game for the person.



I originally wrote bad code in terms of architecture and functionality, trying to solve a specific task in the shortest possible time. This program works only on my monitor configuration and requires quite a lot of computing resources, but it solves its problem, and that is enough. Actually, this post was conceived as an illustration of the effectiveness of this approach, since In the past, I often missed many opportunities precisely because of "perfectionism" and the desire to write the perfect code, instead of just doing the thing that solves the problem.



It is not at all clear to me what the creators of the game hoped for, because it seems to me that it’s unrealistic to go through the game manually. Probably, the concept just assumed that only the Red-eyed, who can write a bot, deserve a 25% discount.



UPD. The comments set out more ways to get discounts. To my shame, I did not even think about the simplest of them ... It turns out that the post as an illustration of what not to do is difficult, when it can be done simply, was a success thanks to the commentators. Indeed, the bot could not write ... I hope this will serve as a lesson not only for me, but also for the rest of the unfortunate, suffering from computer disease, identified and classified fifty years ago.

As for Mr. Frenkel, who started all this activity, he began to suffer from a computer disease - everyone who worked with computers today knows about it. This is a very serious disease and it is impossible to work with it. The trouble with computers is that you play with them. They are so beautiful, so many opportunities - if an even number, you do it, if it is odd, you do it, and very soon you can do more and more sophisticated things on a single machine, if you are smart enough.



After some time, the whole system collapsed. Frenkel did not pay any attention to her, he no longer supervised anyone. The system acted very, very slowly, and at that time he was sitting in a room, wondering how to make one of the tabs automatically type the arctangent x. Then the tabulator was turned on, typed columns, then - bang, bang, bang - calculated the arctangent automatically by integration and compiled the entire table in one operation.



Absolutely useless exercise. After all, we already had arctangent tables. But if you have ever worked with computers, you understand what kind of a disease this is - admiration for being able to see how much can be done. Frenkel picked up this disease for the first time, poor guy; the poor guy who invented the whole thing.