McPaintio - a program that converts an image into a set of mouse commands that draw this image

Hi, Habrahabr!
This time it will be a purely entertaining experiment. The article claims solely on Friday reading and there is nothing phenomenal in it. It tells about the history of the creation and development of the McPaintio application, which can draw images in any * context of any ** drawing program. The article will be of interest to people interested in bots programming and graphic animation. Ave, welcome!

Instead of entry.
Once upon a time, in one social network I came across an interesting application. It was a collaborative (collaborative multiplayer) online drawing tool. Something in the spirit of flockdraw.com and webcanvas.com , but at any given time there were many online users ... a lot ... much more than now. Just now such games are no longer relevant; Rarely one or two people will drop in, something will be pinched, and, without trying even minutes, they will come out. Boredom. Such entertainment in our days has died as a class, and after all, before the people hawala such fun with a bang! Ah, these golden times of one megabyte flash toys and gif animations.
So, what am I talking about ?! I'm talking about this game here. Now, of course, there is practically no one there. And in 2007, each room was literally kishila people. Well, man for a hundred was definitely online. Day and night. And this entertainment really entertained people, and from small to large. Cool it was thought to draw something really beautiful. Everyone saw the very process of your drawing, they could express delight, they could draw something, and they could ruin it. Of the tools - it is ridiculous to remember - then there was only a pencil (pencil) and a hand (pan) for scrolling through the endless canvas. Even the line could not draw a line, all - by hand. No gum for you, no color or thickness settings. There was no chat, so we talked like this: we wrote the text directly on the drawing canvas, it turned out to be terribly long and very clumsy, because the mouse is far from an ideal tool for drawing by hand. Other players answered below under your inscription, even lower - the following, and the canvas turned into a solid wall of text. Fortunately, the wall is endless. It was fun.

Especially cool were those who had a graphic tablet at home. They wrote by hand beautifully, quickly, cleanly, and their drawings were much better. No one could beat them. Personally, I did not have a tablet, so your humble servant for the first three weeks was amused by the mouse. But then I got tired of such a course of affairs. I wanted to be not a boyar nobleman, but a mistress of the sea cool artist and cool to draw very complex drawings.

At first, I began to train on cats: I took the AutoIT meta-script language under Windows and wrote a couple of simple programs in it. The first drew just a circle, and the second - a straight line. Well, as I drew ... I programmatically put the mouse cursor in a certain place, programmatically clamped the left mouse button and programmatically moved the cursor along a straight or curved line. The application in which such actions were carried out, "thought" that this user moves the mouse and presses the buttons. In fact, everything was painted by the AutoIT robot (something like a cnc machine ). The browser interpreted the movements and clicks in the context of the game as a drawing process, and the game translated the drawing to the server. Other users do not see that it is not me who draws, but the robot. Here is such a trick =) In fact - a bot!
')
Thus, I learned to draw perfect circles and straight lines. For schoolchildren (and the vast majority of the players were just schoolchildren) sitting in the game, I was, frankly speaking, a kind of deity. I was a deity even for people with a tablet, because even on a tablet, using a stylus, you cannot draw a perfect circle. And in my hands now were alien drawing technologies! For each perfect circle drawn by me, a whole crowd of drawers pounced, and turned it into a painted smesharika, now into a clock, then into a face, and into any other drawing, based on the circle. The rest were divided into two camps: those who were jealous, and those who admired. Those envious of anger wrote “ cheater ,” “schmuck,” and so on, but the admirers crossed out the envy and expressed delight in writing: “wow cool,” “how do you do it?”, “Is this a tablet?”, “Which program? "and so on ... flooded!

Literally within a week, I achieved such popularity that none of my strokes in the game went unnoticed. But entertainment with circles and lines bore me pretty quickly, since the perfectionist soul always wants something more and trubut of creating the perfect tool. I wanted this so that any drawing, any file, any graphics could be transferred to the canvas. Then I sat down to create what is now called McPaintio.

McPaintio is a program that converts an image into a set of mouse commands that draw this very image. A kind of meta-draftsman. And since any image can be analytically redrawn, it is possible to draw anything in any outline of any application! Imagine that you take a drawing with an intricate ornament and immediately draw the most complex spline that describes this ornament, right in the 3DMax window! Isn't it really cool ?!

I really zamorochilsya and equipped McPaintio cool chips. For example, if the drawing to be drawn is larger than the drawing canvas provided by the application and the application supports the pan operation, McPaintio can draw the image block by block by switching the pan and pen tools at the right time and drawing the correct fragments of a large image in the right ground canvas! Further. In McPaintio there are already three different drawing algorithms designed to perform various kinds of image tracing and transfer it to a different canvas. These algorithms to varying degrees satisfy the classic trade-off between drawing speed and drawing accuracy. I also equipped McPaintio with the ability to draw images in grayscale with several different algorithms. For example, if you want to draw an image in shades of gray and the canvas provides a tool for selecting colors, you can customize McPaintio so that he opens the palette in the application, selects the desired color, closes the palette and continues the drawing process in the desired color. McPaintio also knows how to write custom text. McPaintio has in its arsenal a flexible setting of control buttons and timings, allowing you to customize the drawing process for the most demanding applications.

In general, I worked for the glory. True, the application turned out with a very crumpled interface. This is the result of his improper planning: over time, more and more new functions were added in a random order, all of them reflected one way or another in the interface, it turned out a lot of small, and redoing laziness.

Example 1
Let's start with the simplest. Download the application (check its size) and open a simple small black and white bmp file. It is necessary to translate what is drawn in black from a file into another outline. For now, for the experimental canvas, we’ll choose simple Microsoft Paint . We will draw the first of the presented algorithms - “to the nearest point”. This algorithm moves the pen from the current pixel towards the nearest non-shaded pixel, painting all intermediate points along the way. If there are several such nearest empty pixels, then a random one is chosen. Drawing starts in a random pixel that is not yet shaded. At certain points in time, there is a jump to a new random pixel that has not yet been shaded (this specially programmed artifact serves the purpose of adjusting to the interface of the first program of collective drawing, which limited the length of the maximum line that the user can draw). We configure the start button of the broadcast to F10 (by default), and the stops to F12. First we try to draw right in the McPaintio itself (for clarity). Put the mouse in the upper left corner of the picture and press F10. See how the mouse moves. In the title, the program shows the number of parts (line breaks, lifts of the left mouse button) that it produced. At this moment, the program completely took control of the mouse, and even if I try to move it, nothing will work out for me, since in the next time slot the mouse will be programmatically moved to the desired program point. Violet we see already painted pixels of the image. The filling process takes place, which we can stop at any time by pressing F12. You may notice that the distance between the closest filled pixels is not one. One gets the impression that the program is missing some pixels. In fact, this is true. And this is not a mistake, but another specially programmed artifact, which serves the purpose of adjusting to the interface of a collaborative drawing program, in which the pen radius was 2 pixels. And if this is the case, then, having painted a pixel (x, y), it makes no sense to paint over pixels (x + 1, y) and (x, y + 1); You can simply skip them, since they are already painted over on the canvas. To ignore this artifact and force the processing of absolutely all pixels, use the “exactly” checkbox.

When the exponential run shows an acceptable result, without waiting for the end of the drawing, stop the program and go to Paint. Once again: it is necessary to translate from the file what is drawn in black (and not white). Black is pixels with an intensity of 0. White is pixels with an intensity of 255. 50% gray are pixels with an intensity of 127. McPaintio allows you to select those pixel intensities that should be considered portable (drawn, processed). And not just one specific intensity (say, 0), but even a range. This was done in order to correctly render even those raw images that contain color fluctuations in the vicinity of zero and the black color in them does not always have an intensity of 0, but is scattered from 0 to, say, 8. Then, specifying the range [0 .. 8] the program will correctly draw "black". To indicate the range, use the “layering” fields - specify black as [0..8] and draw our file with a brush in Paint.


Example 2
Here I show more clearly what it means to choose different ranges for drawing. Consider a simple gradient: black on the left, white on the right. If you select the range [0..8], then, as I said and showed, the left area of ​​the gradient will be drawn. If we choose, say, [128..132], then pixels will be drawn from about the middle. And finally, if you select [242..255], then the rightmost sections of the gradient will be drawn. Select the range [0..32] and turn on drawing in Paint. Notice how the widest part of the bar is drawn. Pay attention also to the fact that the right edge of the strip is not even, but torn. This is due to the fact that the gradient was prepared in Photoshop using dithering. See how the pen tightly wraps around the already drawn areas.


Example 3
In this example, I demonstrate the exact drawing mode that I talked about earlier. Notice that the image is now processed pixel-by-pixel, without taking into account the shading of its neighbors.


Example 4
Now open the new image, which is the contour of the previous one. I want to demonstrate drawing in “fast” mode, in which there are no additional delays in the drawing code. They are needed so that “slow” programs like Photoshop without errors hawali the drawing script. It is noticed that some drawing programs discard some of the window messages sent to them. So, open the file, put a tick "quickly" and draw for greater clarity in the McPaintio. Pay attention to the drawing speed. This is the maximum speed at which McPaintio draws in this mode.


Example 5
It is time to demonstrate the new drawing algorithm - “the dynamics of lines”. This algorithm moves the pen from the current pixel to the side of the most distant pixel that has not yet been shaded, painting all the intermediate ones along the way. The most distant pixel that is not yet shaded is not chosen arbitrarily, but with some limitation: from it to the current pixel there should be a straight line that passes a) only along the processed pixels b) the number of already filled pixels in the line should not exceed 30% of the total number of pixels lines. Such a technique most closely resembles the natural drawing by man by hand, and was invented and introduced instead of the old algorithm “to the nearest point”, which too clearly exposed in the program for collaborative drawing the fact of drawing not by man, but by robot. First, we draw for greater clarity in the McPaintio, and then brush in Paint.


Example 6
I will now demonstrate this algorithm on the example of a gradient. I will draw twice, in two different ranges. And if so, then, in order to get the final picture without breaks, it is necessary both in the first and second drawing cases to place the mouse cursor exactly in a certain position. I will call this position an anchor point. I will note the so-called anchor point in Paint with a red cross. First I open the gradient and select the range [0..8]. This will draw the leftmost fragment of the gradient. I remove the checkbox "quickly" and set the mouse approximately at the anchor point. See how the process of drawing, or rather, painting a large rectangle. Very similar to the movement of the human hand. After finishing the drawing, I select the range that follows exactly the current one - [9..12]. For clarity, I choose a different color in Paint. Again, I put the pointer at the anchor point and draw. Please note that the new lines are located strictly to the right of the previous figure, and right next to it. This is the essence of the choice of the drawn range in this mode.


Example 7
I will now demonstrate two parameters such as “mouse pause” and “line time”. McPaintio allows you to set the time that the drawing program needs to perceive the line and draw it. This parameter is called “mouse pause” and is expressed in milliseconds. He actually says McPaintio: “after holding the next line, release the left mouse button and wait n milliseconds”, where n is the parameter to be set. The business is that programs such as, for example, Paint do not miss, do not drop, and do not discard the window messages that were sent to them. Paint is an ultra-simple and super-fast drawing program, so you can put at least 0 such a pause in it. But there are load and braked programs, for example, Photoshop, which draw lines very slowly (especially with gaussian soft brushes) that lead history that can be dropped by window messages sent to them. For such programs, the “pause of the mouse lift” must be set to even 100 milliseconds so that they can figure out what to do and do it.

The “time line” parameter serves similar purposes. He speaks for himself: this is the time for which one straight line is drawn, in milliseconds. For example, for Paint it is possible to set this parameter to 10 or even 5, whereas for Photoshop it should be 50 or even 100. Otherwise, he simply does not have time to draw this line, as window messages about drawing the next line will arrive and in As a result, Photoshop will break the current line, dropping part of the window messages.

In the video, I first set these parameters to 0 and 1, respectively. Pay attention to how quickly the image is drawn. Then, for the demonstration, I put these parameters at 200 200. Now drawing is going on, oh, very slowly. Finally, at the end I put the average values ​​for most programs - 25 25. Now everything is in order.


Example 8
Now let's play with the text, because McPaintio can write it too! This function was introduced to realize the possibility of a non-stressing letter for the very first program of collaborative drawing. In McPaintio there is a special “text” field where you can enter text. To begin, select the font size 40 and press the "write" button. The program has generated an image of the written text, and now it will draw it on a common basis. Turn on the exact mode and see what happens. It turned out very average. Now, change the font size to 80 and write the same with a bold brush. So much better!


Example 9
Now I will demonstrate one of the most interesting drawing modes - drawing in blocks. Looking ahead, I will say that this mode was introduced to draw huge images on an endless canvas through a very small “hole” (window) for drawing. For the same first collaborative drawing program. For experimental experiments now take Photoshop. So, there is a small program window (actually no more than 512 x 512 pixels) through which we can have access to a large canvas (actually 2000 x 2000 pixels). It is necessary to draw an image, much larger than the viewing window. We open in McPaintio a huge image to match the canvas. Create a new 2000 x 2000 pixel document in Photoshop. We see the entire canvas only in 30% of the form, in 100% of the form we see only a small part of it. Next, you need to show McPaintio those points on the screen that contain the pan tool and pan tool enable buttons. Once having specified them, now McPaintio will be able to draw images block by block, scrolling the canvas in the desired position. For learning, first tick the “blocks” and indicate the size of one block, like, say, 256 x 256 pixels.That is, McPaintio will draw an image of 1025 x 1024 pixels through a viewing window of only 256 x 256 pixels. Now you need to become the mouse cursor on the pan photoshop button and McPaintio will remember it (the checkbox near the sound will become checked). Similarly, you need to do for the brush button. Let's set the correct timings for Photoshop 20 and 20. Let's start drawing and admire block drawing. First, the upper left corner of the image is drawn. After the rendering is finished, McPaintio goes into panning mode and drags the canvas so that the new piece of data will be drawn right next to the old one. And so on for all blocks. At the end of the video you can admire the finished image.


Example 10
It's online time! Now we will do the same, only in the very collaborative draftsman. Let's set the block size this time to 512 x 480 and re-train McPaintio with the pan and pen buttons a little. We admire the result. Please note that in any room no soul! This testifies to the fact that after 7 years, people found more interesting games for themselves than collaborative drawing.


11.
! , -, . . « ». — , « », . McPaintio , — , « » . « » — « ». That's all.

, . , McPaintio . , . , [0..0], (), , [1..1], , , 255?!?! , ! -, . , {[0..0], [1..1], [2..2],…, [255..255]} , , , : {[0..8], [9..16], [17..24],…, [248..255]}. 16. 32. . , , . , 8 . , , -: McPaintio (!), (!!), (!!!), (!!!!) .

, , «+0» . . : , , , [0..32]. +0 +0 +32 +32. , +32 . [0..32], [32..64] 255. , , . 10 10 ( , ) «»! . McPaintio 4 : « » , , , « » . 4 McPaintio . , ( — ) , 2 : « , » « , ». McPaintio «o», «w», «b» «x» , , , . , , .


Example 12
, Photoshop'. . McPaintio . 200 10 .


13.
It's online time again, FlockDraw! «AM» . . .


14.
.


Afterword
, . Very interesting.
Thanks for attention!