Selenium: Pumping Up Muscles

Quite a long time has passed since our last article on efficient Selenium infrastructure. If you are at the very beginning of the difficult path of Selenium - I advise you to read our articles about scalable Selenium ( Part I , Part II ), Selenoid is a universal tool for test automation in browsers ( one , two ), Selenium under Windows ( link ). If you like motivating stories more, check out the video of my report about scalable Selenium at SeleniumConf Berlin 2017 .

Since the publication of the last article in our community, many interesting things have happened. Today I want to talk about the most important features added to our tools in recent months.

Ggr

Our Selenium balancer got some cool improvements.

1. First, an API was added to define a host with browsers by session ID. This can be done with this HTTP request:

$ curl -s http://test:test-password@my-ggr-host.example.com:4444/host/8e82f31f408b4a906f715228b5176efb0528ce32-66db-417f-8fa7-b12d85ce1ab5 {"Name":"my-hub-1.example.com","Port":4444,"Count":5,"Username":"","Password":""} 

1. Also added support for guest quotas . Browsers from the guest quota can be obtained without a username and password. It turns out to be convenient to maintain a small sandbox where new users can try to run their tests without changing their code. You can enable guest quota with command line arguments Ggr:

 $ ./ggr -guests-allowed -guests-quota test < ...> 

When adding such flags, any browsers from the test.xml file will be accessible without specifying a password.

1. The third cool thing is the ability to proxy requests to external services like SauceLabs , BrowserStack or TestingBot . Some testing tasks, such as testing in mobile browsers or testing mobile applications on devices, are complex in terms of maintaining the test infrastructure. You can decide to support the main part of the browser infrastructure yourself and use paid services only for such complex cases. To redirect requests to an external system, simply enter the username and password in the XML file:

 <qa:browsers xmlns:qa="urn:config.gridrouter.qatools.ru"> <browser name="some-mobile-browser" defaultVersion="45.0"> <version number="45.0"> <region name="1"> <host name="ondemand.saucelabs.com" port="4444" count="1" username="test-user" password="my-password"/> </region> </version> </browser> </qa:browsers> 

1. All our tools, including Ggr, are now compiled with Golang version 1.9. This version has much more efficient garbage collection algorithms, which gives a significant decrease in response time. Ggr now really flies.
2. The most important change is the addition of support for VNC data proxying . If the hosts from the Ggr quotas have a VNC server running on some port (to look at the browser screen at runtime), then you can configure the Ggr to be able to access the VNC data by session ID. Without any changes in quotas, you can now use these links:

 ws://test:test-password@my-ggr-host.example.com:4444/vnc/8e82f31f408b4a906f715228b5176efb0528ce32-66db-417f-8fa7-b12d85ce1ab5 

Note the ws:// protocol, meaning WebSocket . Using a VNC client that supports data transfer via web sockets (for example, noVNC ), you can see the browser screen of any running session. By default, Ggr expects the VNC server to be running on the standard port 5900, but this can be reconfigured.

Selenoid

The most numerous changes were made in our, as they say, "flagship" open-source product - Selenoid . Selenoid is a complete replacement of the Selenium hub that launches browsers in Docker containers:

More Customization Options

Now you have much more possibilities for setting up the browser environment. For example, in the configuration file you can set any environment variables, the /etc/hosts file entries and the shmSize value for the operating system inside the container where the browser will be launched:

 { "firefox": { "default": "46.0", "versions": { "46.0": { "image": "selenoid/firefox:46.0", "port": "4444", "env" : ["TZ=Europe/Moscow"], "hosts" : ["example.com:192.168.0.1"], "shmSize" : 268435456, } } } } 

This allows, for example, to override the time zone for each browser version or add hosts from the internal network without changing the DNS settings. In addition to these settings, you can change some parameters separately for each running session with the help of capabilities.

New Capabilities

1. If you run several tests in parallel, in order to distinguish them in the Selenoid UI, you can specify the capability of the name with an arbitrary string inside:

 name: "MyCoolTestName" 

Here is how it looks in the UI:

2. If the application under test is also launched in the Docker container - you can automatically link (link) the container with the browser to the container of the application, specifying its name:

 applicationContainers: my-application-container 

3. Sometimes it is required to redefine the contents of /etc/hosts for only one test. This can be done like this:

 hostsEntries: "some-host:192.168.0.1" 

4. Finally, to override the time zone for a single session, specify:

 timeZone: "Europe/Moscow" 

Logging Improvements

We added two important logging improvements:

1. If browsers run in containers — any logs sent to a centralized log repository (such as Amazon CloudWatch or Google Cloud logging) can now be labeled with a random label. The label value is specified using the capability name , which I wrote about in the previous section.

2. If you run Selenoid without Docker, you can finally turn on the web driver logs in the Selenoid logs. In order for this to work, you need to add the -capture-driver-logs flag when starting Selenoid:

 ./selenoid -conf ~/.aerokube/selenoid/browsers.json -capture-driver-logs 

Images with Browsers

We rebuilt all images with browsers , adding support for all UTF-8 locales and additional fonts to correctly display such characters as:

• Chinese characters
• Japanese characters
• Korean characters
• Hindi (Devanagari)
• Thai alphabet
  All these changes allow the use of new images for testing sites translated into many languages.

Selenoid UI

We completely redesigned the Selenoid UI - a graphical web interface for Selenoid.

The most frequently used features such as browser usage statistics and buttons for viewing sessions are now located on the main page. The less commonly used capabilities selection screen has been carried to a separate tab.

The running browser screen and session logs are now displayed on the same screen next to each other.

Configuration manager

Configuration Manager , a small application that greatly simplifies the installation of our tools, has become even more convenient. The ability to customize Selenoid to work with Microsoft Edge and Safari is now supported. Fresh CM versions work without problems on Windows 10. You can also redefine the port on which Selenoid and Selenoid UI listens, for example, in order to simultaneously launch Selenium server and Selenoid.

We also concluded that CM was pleasing to the eye by coloring the logs in different colors.

Video Recording

The cherry on the cake is the newly added ability to record video browser sessions. In addition to the ability to look at the browser screen in real time in the Selenoid UI, it is now possible to record video from the browser screen and save it in a H264 file. The video looks like this:

To record a video, just add one capability to the tests:

 enableVideo: true 

By default, all recorded videos are called <sessionID>.mp4 , where <sessionID> is a unique identifier of a browser session that can be easily pulled out of the test logs. If you want to use your name, add the videoName capability:

 videoName: "my-cool-video.mp4" 

Selenoid automatically provides access to a directory with video over HTTP. To open a file in a browser, use the URL:

 http://my-selenoid-host.example.com:4444/video/my-cool-video.mp4 

To view the entire list of files - delete the file name:

 http://my-selenoid-host.example.com:4444/video/ 

I hope you now have much more motivation to set up a truly efficient Selenium infrastructure in your team. Believe me, running tests in browsers can be painless! If you have any questions - do not hesitate to write to us in the mail , in the Telegram-support channel or send questions to the StackOverflow tag . If you already use Kubernetes in your processes - you may also be interested in our new product - Moon , which was specifically designed to deploy an effective Selenium cluster in Kubernetes and supports all the best that Selenoid has.

Until new meetings.