Script for checking the availability of free dates at the embassy

Preface:

In Belarus, there is an acute problem with obtaining visas to the Eurozone (i.e. Schengen). All because of the fact that the Polish Embassy provides the so-called multi-visas for shopping (ie multiple). Registration is made on the website of the embassy online. But the whole problem is that there is no free dates. The only option is to check the page around the clock, and if a date appears, quickly “catch” it and complete the registration. Since there is no free time for round-the-clock checks, it was decided to automate this process.
At once I will make a reservation that there are various scripts that catch free dates and for which people receive money. My script does not claim their place for speed, quality, etc. This script was made only for myself, I did not pursue any commercial or other benefits.

Task setting and input data:

To begin with, it was necessary to study how the registration process goes.
Link to the website of the embassy: by.e-konsulat.gov.pl
On the main page we see two selects, with a choice of country and city. Having chosen the necessary parameters, we redirect to by.e-konsulat.gov.pl/Informacyjne/Placowka.aspx?IDPlacowki=94 .
Then choose from the menu "Schengen visa - Register the form" and go to by.e-konsulat.gov.pl/Uslugi/RejestracjaTerminu.aspx?IDUSLUGI=8&IDPlacowki=94 - this url and took as an entry point, because there is no point in automating the previous pages (of course, before that I checked the ability to log in with this URL with clean cookies)
Next we see the captcha. By entering it, we are given the result - Lack of free dates.
Based on this data, we can make a sketch of our future script plan:

Tool selection

After I decided on what needs to be done - the question of a suitable tool became. Just want to make a reservation, I am not a programmer, I am a tester. But some knowledge of languages ​​is present.
At the very beginning, I wanted to automate this process at TestComplete. After automation, I ran into some problems, most of which was the speed of script processing, and plus I used the old version of testkompleat 7.5, which works maximum with Mozzila 3.5 browser. You understand that in such an old browser the display of elements is lame, and the layout is going in some places. Therefore, I scored this tool and looked at Selenium WebDriver.
The script was written in Python. The choice fell on him for the sole reason, I was a little familiar with this scripting language, and there was neither time nor desire to go into Java, for example, and to study it.
')

Work with captcha

In fact, it is not difficult to automate these actions, but the hated captcha spoils everything. The whole problem consisted in the fact that the captcha changed with a frequency of once or twice a month and therefore there was no point in thinking through the technology of deciphering the captcha (creating templates, masks, etc.). For this reason, I decided to use antigate.
By registering there and throwing 3 dollars, I received resources for 3000 captchas.
But now it was necessary to think over the algorithm for processing this captcha, sending it to antigate and getting the value of captcha. It looked like this:

To work with antigate, I used the API of this service. I had to deploy a PHP server on the local machine, not bothering with the choice fell on Denwer. I created a local website test1.ru and put a php page there to work with the service API.
Listing this page

<?php function recognize( $filename, $apikey, $is_verbose = true, $sendhost = "antigate.com", $rtimeout = 10, $is_phrase = 0, $is_regsense = 1, $is_numeric = 0, $min_len = 4, $max_len = 4, $is_russian = 1) { if (!file_exists($filename)) { if ($is_verbose) echo "<b>file $filename not found</b>"; return false; } $fp=fopen($filename,"r"); if ($fp!=false) { $body=""; while (!feof($fp)) $body.=fgets($fp,1024); fclose($fp); $ext=strtolower(substr($filename,strpos($filename,".")+1)); } else { if ($is_verbose) echo "<b>could not read file $filename<b>"; return false; } if ($ext=="jpg") $conttype="image/pjpeg"; if ($ext=="gif") $conttype="image/gif"; if ($ext=="png") $conttype="image/png"; $boundary="---------FGf4Fh3fdjGQ148fdh"; $content="--$boundary\r\n"; $content.="Content-Disposition: form-data; name=\"method\"\r\n"; $content.="\r\n"; $content.="post\r\n"; $content.="--$boundary\r\n"; $content.="Content-Disposition: form-data; name=\"key\"\r\n"; $content.="\r\n"; $content.="$apikey\r\n"; $content.="--$boundary\r\n"; $content.="Content-Disposition: form-data; name=\"phrase\"\r\n"; $content.="\r\n"; $content.="$is_phrase\r\n"; $content.="--$boundary\r\n"; $content.="Content-Disposition: form-data; name=\"regsense\"\r\n"; $content.="\r\n"; $content.="$is_regsense\r\n"; $content.="--$boundary\r\n"; $content.="Content-Disposition: form-data; name=\"numeric\"\r\n"; $content.="\r\n"; $content.="$is_numeric\r\n"; $content.="--$boundary\r\n"; $content.="Content-Disposition: form-data; name=\"min_len\"\r\n"; $content.="\r\n"; $content.="$min_len\r\n"; $content.="--$boundary\r\n"; $content.="Content-Disposition: form-data; name=\"max_len\"\r\n"; $content.="\r\n"; $content.="$max_len\r\n"; $content.="--$boundary\r\n"; $content.="Content-Disposition: form-data; name=\"is_russian\"\r\n"; $content.="\r\n"; $content.="$is_russian\r\n"; $content.="--$boundary\r\n"; $content.="Content-Disposition: form-data; name=\"file\"; filename=\"capcha.$ext\"\r\n"; $content.="Content-Type: $conttype\r\n"; $content.="\r\n"; $content.=$body."\r\n"; $content.="--$boundary--"; $poststr="POST http://$sendhost/in.php HTTP/1.0\r\n"; $poststr.="Content-Type: multipart/form-data; boundary=$boundary\r\n"; $poststr.="Host: $sendhost\r\n"; $poststr.="Content-Length: ".strlen($content)."\r\n\r\n"; $poststr.=$content; $fp=fsockopen($sendhost,80,$errno,$errstr,30); if ($fp!=false) { fputs($fp,$poststr); $resp=""; while (!feof($fp)) $resp.=fgets($fp,1024); fclose($fp); $result=substr($resp,strpos($resp,"\r\n\r\n")+4); } else { if ($is_verbose) echo "<b>could not connect to anti-captcha</b>"; if ($is_verbose) echo "<b>socket error: $errno ( $errstr )</b>"; return false; } if (strpos($result, "ERROR")!==false or strpos($result, "<HTML>")!==false) { if ($is_verbose) echo "<b>server returned error: $result</b>"; return false; } else { $ex = explode("|", $result); $captcha_id = $ex[1]; if ($is_verbose) echo "<b>$captcha_id</b>"; } } $text=recognize("captcha.png","       ",true,"antigate.com"); ?> 

I did not begin to thoroughly understand what was happening, but the only thing I set was the following settings:

  $is_phrase = 0, //     $is_regsense = 1, //   ? $is_numeric = 0, //  ? $min_len = 4, //  $max_len = 4, //  $is_russian = 1 //    

As a result, we need to place the image captcha.png in the directory where index.php is located and go to url test1.ru
As a result, the captcha will fly to the service, when it is solved, we will receive its id, framed in the tag b, or some error will appear that is displayed.
It will remain the case for small, only to take the value of the captcha from the page by its id.

Script creation

Since All preliminary preparations have been made, we can proceed directly to writing the script.
We will work with two open windows Firefox. Since in one window we will have a check of dates, and in the second all the work is relative to the captcha. To display captcha in a new window, we will simply find the element on the page by id and read the current captcha URL. When accessing this url, we get only the image of a captcha, without any extra elements.


Now listing the script, with comments:

 from selenium import webdriver from selenium.webdriver.support.ui import WebDriverWait import time driver = webdriver.Firefox() #   () add_driver = webdriver.Firefox() #       driver.get("https://by.e-konsulat.gov.pl/Uslugi/RejestracjaTerminu.aspx?IDUSLUGI=8&IDPlacowki=94") #    captcha_url = driver.find_element_by_id('c_uslugi_rejestracjaterminu_ctl00_cp_botdetectcaptcha_CaptchaImage').get_attribute('src') #     id   ,      add_driver.get(captcha_url) #      add_driver.set_window_size(50,200) #   ,     ,     add_driver.get_screenshot_as_file('captcha.png') #  ,                test1.ru, ..     ,     add_driver.get(http://test1.ru) #    ,     captcha_id = add_driver.find_element_by_xpatch('//b') # ,     b,  ,    id  count = false while (count == false) add_driver.get('http://antigate.com/res.php?key=    &action=get&id=" + captcha_id) captcha_complete = add_driver.find_element_by_xpatch('//pre').text #    (      pre) if (captcha_complete.find('ERROR') >= 0) #,    time.sleep(5) # 5  else count = true #    #        captcha_complete,      driver.find_element_by_id('ctl00_cp_BotDetectCaptchaCodeTextBox').send_keys(captcha_complete) #       driver.find_element_by_id('ctl00_cp_btnDalej').click() #       result = driver.find_element_by_id('ctl00_cp_lblBrakTerminow').text if (result.find('') >= 0) print(' ') else print(' ') 

Future improvements

The basis is ready, our script goes to the page, receives the captcha, recognizes it through the recognition service, enters the captcha, clicks on and checks for the date. For myself, I did the following - I drove all this action into a while loop (true) and checked the site until I got the date (I also added sending a letter to the soap, in case of a positive result). Of course, you can make a lot of modifications to the script, for example:
1) check for errors and take various actions based on the errors
2) check the wrong captcha and send the report to the antigate (complain about a bad employee)
3) add autoregistrar, and not just a checker date
etc.

Afterword

Once again I want to make a reservation that this script is rather weak, but the result from it was. It is also worth noting that the embassy is far from fools and often change the captcha, so it will be necessary to rewrite the script for the new conditions.