Read more about Sikuli in test automation

Introduction

Sikuli is an API that allows writing automation scripts on Jython based on the visual component of any program / site, etc. Especially nice for automating Flash.
About Sikuli few articles are written and most of them are overview. Even less Russian-language help, and even fewer code examples. And the absence of the latter is perhaps the most tragic for a software tester who is faced with the need to automate any flash. It was precisely this that prompted me to write a more capacious article on Sikuli and describe in some detail some features of use.

Installation

To install and properly operate Sikuli, you will need:
1. Java of the sixth version and necessarily 32 bit.
2. Distribution
3. Service Pack.

Due to the fact that Sikuli comes only in 32-bit configuration, in order to work correctly on 64-bit systems, we must install Java 6u38 32x (with the 7th version does not work).
It is also necessary to put the update package. Installs easily. The contents of the Sikuli-IDE folder are simply copied from the archive (point 3) into the folder with the installed program C: \ Program Files (x86) \ Sikuli X, by the way, do not change the default installation folder.
If suddenly something is wrong, here is a reference to read.
')

Functionality and features.

Screenshot
It is used to highlight a specific area of ​​the screen and then create a pattern.
When you click on the pattern, the "Template Setup" window appears.
Tabbed everything is primitive and clear. The only thing I will note is that on the “target offset” tab, the goal is where to move the cursor and make a click / enter. And the place of the click can lie far beyond the pattern.

Now about the features.
*****. png is the name of the desired picture that should be in the folder of a previously saved project.
similar = similarity, where 1 = 100% (it is better not to use because it does not always find it), for example, 0.85 (through a dot) = 85%, which in my opinion is the most optimal value.
targetOffset (x, y)) and here it is important that the coordinates "x" and "y" is the offset of the target from the center of the pattern.

Area selection / wait () / Click ()


Great function and most importantly, extremely useful.
Many say that Sikuli has a bad understanding of work and is still mistaken great. I completely disagree with this, and I believe that a competent approach significantly increases the performance of the program compared to similar IDEs.

This function sets the active screen area.

in which the search for the desired pattern will be performed.
You can also like this:

Region(x,y,w,h).find(Pattern("*****.png".similar(0.85)targetOffset(x,y))) 

It is extremely useful if we need the script to react to changes promptly, not to slow down the system.
As an example:
from all of our large screen, we select only a small area where every time a cherished button appears.
Next, we describe what we need to look for in this area and then what to do with it.

In order to optimize the process, we need to run this script and make it click for us 1 time. After we go to the tab "Message" where we see the coordinates of the click.

Then do a cycle and click on the coordinates (this is provided that the button will always appear in the same place).
Similarity in the search do about 70 percent, will find faster, the expectation of 999 ...

As a result, we have almost instantaneous triggering.


Find ()

The find () function searches for a pattern. But why? It's simple. Once found, and then he will know where the found object is always. How to put into practice:
We have a lot of things either ...
And the program will have to press this “something” long and hard ...
In order for the program to know where to click, you need to set targetOffset ()

Repeat for each goal, assign variables, optimize if the field with the set is in the same place.
Further we can use the found zones as we want. And the click will occur without delay in searching, etc.


The rest of the functionality will not be considered it is primitive and simple.

Examples

And a video about how it works (shot in debug mode).

Utilities Used

Alt + Shift + c stops the script execution and returns the IDE.

Set the cursor movement time:

 Settings.MoveMouseDelay = 0.02 

Assign an event to key combinations:

 def off (event): exit() Env.addHotkey(Key.SPACE, KeyModifier.ALT+KeyModifier.CTRL, off) 

Calling the dialog box (integer):

 n = int(input("    ?")) 

Inaction in 1 sec:

 sleep(1) 

If after inserting the copied text you have replaced the code of a pattern or region with a picture, simply press 1 or 2 times the key combination Ctrl + z.


This is not all, and there will be more. But at this stage I think that I expressed the main idea.
I would be happy to feedback.
If you have questions about Sikuli, then write.

Next time I’ll probably tell you how to teach Sikuli to recognize the text and write the error log to a separate file.

Thanks for attention.