A special case of using Sikuli to solve a problem on Facebook
Some time ago, Facebook began to use an NFS-based solution to store user photos. It helps to cope with a pressure of about 220 million new files occupying 25TB every week and at peak hours handing out more than 550 thousand images per second [1] . The maximum number of files uploaded to one album has been increased from 60 to 200 to the delight of users.
The presence of privacy settings and the fact that the free space on Picasa was safely eaten by the masterpieces of past years, prompted me and some acquaintances to try to use Facebook as a photo storage.
Practice, however, revealed some inconvenience to the service, the most annoying of which was the impossibility of removing the album after loading a lot of photos into it. This article is dedicated to everyone who has been affected by this problem and who is curious about another example of the use of the remarkable cross-platform visual automation tool Sikuli.
')
Replication scenario was as follows:
The Bug Report was sent over a month ago, and at the time of writing this article has not lost its relevance, judging by the experience of other users.
The only found method for deleting photos from an album in the state described above was to use the “Delete This Photo” link in the viewer. Manually delete 200 photos in this way is not a fun lesson.
Good articles by zodiak [2] and KL7 [3] inspired the use of Sikuli.
Briefly, Sikuli can be described as a visual technology for automating graphical user interfaces using images (screenshots). The current implementation of Sikuli includes Sikuli Script — a visual script API for Jython and Sikuli IDE — an integrated development environment for writing scripts using screenshots. With the help of Sikuli Script, you can automate everything that you see on the screen without using internal APIs. You can programmatically control web pages, applications running on Windows / Linux / Mac OS X, and even iPhone applications running on an emulator.
Unexpectedly, a simple script to automate actions was “assembled” and debugged in Sikuli GUI within 15 minutes, most of which went to get acquainted with the documentation and search for optimal visual patterns uniquely characterizing the elements of Facebook web pages.
The screenshot is given due to the peculiarity of the Sikuli syntax, which includes the graphics in the source code.
Pay attention to the 11th line in which there is the upper part of the picture frame and the empty space on the left, which is present only on the album photos on the left. It should be noted that the search and recognition of patterns on the screen Sikuli performs starting from the top left corner, so in our case the first top picture on the left is selected in the album.
The click function on the found image click () performs a click in the geometric center of the specified area. Increasing the area of ​​the desired pattern in the script can reduce the likelihood of erroneous recognition on similar interface elements.
The documentation also suggests that you can use the MinTargetSize parameter to fine-tune the speed of the script:
English captions are inserted for the sake of people who do not own the great and powerful.
Download the script here .
NB! Most likely, you will have to change the browser name, keyboard shortcuts and wait time between commands for your environment.
Interestingly, from GUI, scripts to disk are saved in folders or zip archives (when exporting to an executable format) with nested * .png images, a python source and an HTML page with embedded CSS for highlighting the code, when opened, the browser displays the nested graphics.
Sikuli made it possible to solve the problem, perhaps not in the most elegant, but in an effective enough way, while saving time and inspiring to write this article.
You can learn more, download and install Sikuli for Linux, Windows and Mac OS X on the official website of the project http://sikuli.org .
The presence of privacy settings and the fact that the free space on Picasa was safely eaten by the masterpieces of past years, prompted me and some acquaintances to try to use Facebook as a photo storage.
Practice, however, revealed some inconvenience to the service, the most annoying of which was the impossibility of removing the album after loading a lot of photos into it. This article is dedicated to everyone who has been affected by this problem and who is curious about another example of the use of the remarkable cross-platform visual automation tool Sikuli.
')
The essence of the problem
Replication scenario was as follows:
- Using the drag-and-drop upload form we upload photos to the new album in an amount exceeding 200.
- After uploading two hundred photos, the process is terminated by an error message.
- Attempting to delete an album through “Edit Album” - “Delete Album” fails.
- Deleting a photo through “Edit Album” - “Edit Photos” - “Delete” - “Save changes” also does not work.
The Bug Report was sent over a month ago, and at the time of writing this article has not lost its relevance, judging by the experience of other users.
Solution search and implementation
The only found method for deleting photos from an album in the state described above was to use the “Delete This Photo” link in the viewer. Manually delete 200 photos in this way is not a fun lesson.
Good articles by zodiak [2] and KL7 [3] inspired the use of Sikuli.
Briefly, Sikuli can be described as a visual technology for automating graphical user interfaces using images (screenshots). The current implementation of Sikuli includes Sikuli Script — a visual script API for Jython and Sikuli IDE — an integrated development environment for writing scripts using screenshots. With the help of Sikuli Script, you can automate everything that you see on the screen without using internal APIs. You can programmatically control web pages, applications running on Windows / Linux / Mac OS X, and even iPhone applications running on an emulator.
Unexpectedly, a simple script to automate actions was “assembled” and debugged in Sikuli GUI within 15 minutes, most of which went to get acquainted with the documentation and search for optimal visual patterns uniquely characterizing the elements of Facebook web pages.
The screenshot is given due to the peculiarity of the Sikuli syntax, which includes the graphics in the source code.
Pay attention to the 11th line in which there is the upper part of the picture frame and the empty space on the left, which is present only on the album photos on the left. It should be noted that the search and recognition of patterns on the screen Sikuli performs starting from the top left corner, so in our case the first top picture on the left is selected in the album.
The click function on the found image click () performs a click in the geometric center of the specified area. Increasing the area of ​​the desired pattern in the script can reduce the likelihood of erroneous recognition on similar interface elements.
The documentation also suggests that you can use the MinTargetSize parameter to fine-tune the speed of the script:
from org.sikuli.script.natives import Vision Vision.setParameter("MinTargetSize", 6) # , , 6 - . Vision.setParameter("MinTargetSize", 18) # , 18 - . Demonstration of work
English captions are inserted for the sake of people who do not own the great and powerful.
Download the script here .
NB! Most likely, you will have to change the browser name, keyboard shortcuts and wait time between commands for your environment.
Interestingly, from GUI, scripts to disk are saved in folders or zip archives (when exporting to an executable format) with nested * .png images, a python source and an HTML page with embedded CSS for highlighting the code, when opened, the browser displays the nested graphics.
Conclusion
Sikuli made it possible to solve the problem, perhaps not in the most elegant, but in an effective enough way, while saving time and inspiring to write this article.
You can learn more, download and install Sikuli for Linux, Windows and Mac OS X on the official website of the project http://sikuli.org .
Source: https://habr.com/ru/post/130094/
All Articles