Bypassing ReCaptcha in Selenium tests

ReCaptcha (a popularly popular “captcha” ) is one of the most painful things that the testing automation engineer may encounter on its way. Thousands of various videos, recorded by people from sunny India, are walking around the web regarding how to dance with a tambourine is possible to deceive this beast. Indeed, it is quite difficult to try to interact with the help of programmed scripts with a piece, the main purpose of which is to make sure that “you are not a robot”.

Very important disclaimer: it is impossible to deceive captcha.

If you have already encountered this problem, and are reading this article, trying to google a panacea recipe, then know that it does not exist. Moreover, innovative ideas about simulating realistic user behavior with the help of WebDriver, by randomly mouse-overing elements, clicking on inputs, and carefully placed Thread.sleep (), most likely have already appeared in your head. It is absolutely certain that this approach will not work, do not waste your time.
')


So, there is no way out?

Not everyone is so pessimistic. Sometimes it is enough to try to give yourself the most accurate answer to the question “What is the challenge before me?” And look at the situation more broadly. In most cases, you will understand that your goal is not to deceive the captcha, but to bypass it in order to test the functionality hidden behind it. Using the example of my case, I will share with you the solutions I have found for solving the task.

Context: we integrated a part of our product into a third-party service, and wanted to monitor whether everything was in order on their side, since they do not cover the third-party parts of their platform. To gain access to our functionality, it was first necessary to log in. It was then that I met the captcha face to face. Further I cite all the ways I have tried to circumvent this problem.

---

Non-working

Log in via Google or Facebook

In addition to classic authentication, there were canonical “Login with Google / Facebook”. Of course, their captcha were also there, so this option did not help solve the problem.

Simulate user behavior

Yes, I tried it too. It was fun, but too naive.

---

Workers

Chrome / Firefox Profiles

Let's talk about the first "live" version. The drivers for these browsers (chromedriver / geckodriver) have the ability to boot under the pre-prepared User Profile. It stores all saved passwords, cookies, sessions, and even browser history and bookmarks. Those. in this way, we simply skipped the login step that was absolutely unimportant for our task, and thus got directly to the page with the test object. It is implemented as follows:

1. Create a “clean” browser profile
2. Manually enter the captcha and login to the desired resource.
3. Copy the necessary profile to our project (HOWTO for Firefox and Chrome )

After that, we need to tell the driver that he should boot from the specified profile:

Firefox:

//   FirefoxProfile profile = new FirefoxProfile(new File("////")); //      FirefoxOptions options = new FirefoxOptions(); options.setProfile(profile); //   WebDriver driver = new FirefoxDriver(options); 

Chrome:

 //      ChromeOptions options = new ChromeOptions(); options.addArgument("--user-data-dir=/////"); options.addArgument("--profile-directory=____"); //   WebDriver driver = new ChromeDriver(options); 

This approach showed itself well when testing on a local machine with a browser installed and the usual gecko / cromedrivers, but there were problems when running on Jenkins. We are raising the Selenium hub and nodes inside the Kubernetes cluster, so we faced troubles in the form of too long mount directories inside the container (a clean profile weighs about 25 MB on average, which is quite a lot), as well as some problems with CRUD browser rights, which Could not make changes to the profile in runtime, and fell from the “unknown error: failed to write to prefs file” event. In addition, updating a profile after reaching Expiration Dates with cookies and sessions is rather inconvenient, and I didn’t want to keep a huge folder with profile internals in the project, so the next option was finally final.

Cookies

“The casket just opened” —that was how it was possible to characterize the situation after we simply added the manually received cookies to the driver. The action algorithm is as simple as possible and does not depend on the selected browser:

1. Login by hand
2. Through Network we look at Request Headers -> Cookies which our browser sends

We add them to our tests as follows:

 //  cookie private static final Cookie COOKIE = new Cookie("", "", "", "", new Date("")); //   WebDriver driver = new ChromeDriver(options); //  cookie   driver.manage().addCookie(COOKIE); 

The obvious disadvantage of this approach is the need to manually change cookies after their validity period has expired. But, since this period is 3 months on the platform under test, we chose this solution.

---

And if I do not need to login?

But what about the situation when it is not about authorization and sessions, but about performing a one-time action (eg placing an order from the basket, registering a new user, etc.)? Here the situation is even worse. Two options that I could discover are:

1. Agree with your developers to provide you with a certain workaround. Google provides such an opportunity , but remember that you are deliberately making a small hole in security.
2. Use third-party paid services that take a captcha screenshot from your side, try to decode it, and send you a decoded value. I myself have not tried this method and can not fully recommend it.

---

Let's sum up

As you can see, there are no hopeless situations. However, it would be foolish to deny that absolutely all of the above options have their own, rather significant, disadvantages, so the choice is yours.

Thanks for attention.

PS If you know any other solutions that work in real life - please describe them in the comments, it will be very interesting to read.