When Apple with the release of Xcode 8 abandoned the UI Automator, we, like many, were left with nothing. Appium, which was used here, lost its relevance, we began to look for alternatives and found the tool WebDriverAgent from Facebook. Under the cut there is a text transcript of the report on what problems we encountered, how we solved them, and how this affected our testing infrastructure for iOS applications.

Avito is several web interfaces, APIs and applications for iOS and for Android. All this needs to be tested, so we have our own test framework. It looks like this:

And consists of two main parts.

The first is, of course, the tests themselves. They are a set of high-level steps: log in, open a page, log out, and so on. They work with Stages, where work with all specific elements is described: press the button, fill in the input field, and so on. And (where could they be without them?) Under this all hides page objects, in which the elements themselves, their locators, descriptions are described.

The second part of the framework is a large set of libraries that allow you to create an ad, find a ready one, delete it, do something with the user, something else, apply some services, and so on. In short, everything you need to get a prestate test.

All tests "communicate" with the tested applications, except for the API, via the WebDriver protocol. For iOS, we used Appium. Everything was cool: we had a test coverage, everything worked. And then Apple announced the release of Xcode 8. Its key feature is that they completely abandoned UI Automation, which was in Apple tools. And the whole scheme just stopped working.

We had to make some decision. Apple offers to write tests on Swift instead. But we do not want: we have a large set of libraries, 200 thousand lines of code, 400 tests for different applications, and so on. I do not want to lose all this. As a replacement, we found a tool from Facebook - WebDriverAgent . The system of its work is similar to Appium, it also raises the web server, however, immediately on the device or on the simulator, and transmits calls from test scripts via XCUITest to the application under test.

WDA Pros

What can WebDriverAgent? Supports Json Wire protocol. This means that we save our tests and the entire test framework. Under the hood of his XCUITests. This is cool because it is a technology supported by Apple, and there is a chance that it will last for another two or three years. Written by WebDriverAgent on ObjC, thanks to this we don’t need to rewrite everything with each new Xcode release.
Also from the pros: WDA supports various locator strategies, you can search by item type, by name, XPath - all as we love. Of the additional advantages - allows you to work with the system outside the application. We can go to the settings in Safari, drive DeepLink, open the application right away, where we need to, and so on. In the settings, disable or allow geolocation and so on. WDA also supports Touch ID technology, which is a separate plus of this tool.

Cons WDA

There are downsides. The first is the Inspector. In any system of functional testing, it occupies a very important role, this is how the auto-tester sees the application, the page code on the screen, and so on. With WebDriverAgent this is pretty bad. More specifically, I will tell about it below.
And the key problem is that it is slow in our infrastructure. And so much so that they can not use. But this is all we learned not immediately. At first everything was cool, but about half of our auto testers did not work.

Hacking

The first edit we made to WebDriverAgent was this.

The guys from Facebook didn’t bother; after all, everyone knows that MacOS is a case-insensitive file system. But in our department, we write code that runs mainly on Linux servers, so we immediately advise everyone to rearrange MacOS, and those who did so could not compile the WDA: they simply mixed up the letter.

The next funny edit we made looked like this.

This is a method of erasing text from a field. Editing has sped up some of our tests by about 30 seconds. Why? The main problem is that the length of the array being iterated was calculated directly in the body of the loop. Usually in compiled languages ​​you don’t need to think about it: there is an optimizer there, it does everything for you. But in functional testing systems this does not work. Because we have an element on the screen that lies somewhere in the WDA cache, and which needs to be retrieved from there, found on the page, take its attributes, find the value among them, and erase one character. Then go again, get the item from the cache again, find it on the page again, again calculate the length of the value field and erase one character again. We have a description field, in my opinion, 1000 characters. Minus 30 seconds.
But this is not the worst. The worst thing looked like this:

Here is the Refine page in the Avito iOS app. It is implemented through XPath. Two elements are sought: the minimum price and the maximum. This video is accelerated 6.5 times. The real travel time is a minute forty. Of these, 20 seconds, I drive my hands across the screen, typing text, and so on. For 40 seconds, each of the two search queries is performed. At the same time, extra 16 MB of RAM is “consumed”. We thought and realized that it was impossible to live with it: now it takes 40 seconds, and if half of the test passes, more memory is accumulated, requests will start to run even longer. And either WebDriverAgent will fall when it uses too much memory, or our HTTP requests will fall off by timeout.

We sat down, looked at what we could do about it, and found a solution: we invented our system of allocators. They called it XUI: eXtended UI Interator. It just well reflected our attitude towards this at that time. Under its hood - binding on XCUI locators. Here is the second demo:

Exactly the same elements are searched for, but now through our new locators. The video is doubled, its real time is 20 seconds, and these are the same 20 seconds that I drive with my hands on the screen, because each request is executed in less than one second. And only 1.5 MB of memory is used.

How did we do it? If someone wrote tests on XCUI natively, then he knows that everything there begins with the fact that we have an application object, from which we then look for just elements, for example, text, a button with an inscription on, and so on. If something more complicated - you can find an element by index, and so on.

XCUI selectors:

• let app = XCUIApplication ()
• app.staticTexts ["Volley"]
• app.buttons ["On"]
• app.windows.element (boundBy: 0)

In real life, however, it looks more like this:

• app.children (matching: .window) .element (boundBy: 0) .children (matching: .other) .element.children (matching: .other) .element.children (matching: .other) .element.children (matching : .other) .element.children (matching: .other) .element.children (matching: .other) .element.children (matching: .other) .element.children (matching: .other) .element (boundBy: 0 ) .children (matching: .other) .element (boundBy: 1) .children (matching: .other) .element.children (matching: .other) .element (boundBy: 0) .children (matching: .other). element.children (matching: .button) .element.tap ()

But the essence remains the same: there is a parent, he has either a direct descendant of some type, or, if deep into a tree, then indirect.
Therefore, we need to know the type of the element and have some sort of direct and indirect descendant. We climbed into WebDriverAgent, started coding. Took the item type. If type is not important to us, simply *. The dot indicates a direct or indirect descendant, children are looking for or descendants.

Everything is cool, but it can all be invested, so we need some kind of separator. We chose pipe (“|”), just to avoid confusing with XPath.

And everything that was higher is now executed in a loop.

With this sorted out. Next you need a choice on the index of the element. If an index arrived, if we detected that we need it, we simply select from the collection of found elements. Here is the keyword last, to take immediately the last.

The most important thing is that we still need a choice on difficult conditions, because XPath has XPath access and a bunch of other functions. And here we were greatly rescued by NSPredicate , a class that comes in the Apple Foundation Framework and serves to filter and select items from the collection.

In fact, he can do a lot. Here is the link , you can read.

Who remembers, in Appium at the time of UI Automation there were BEGINSWITH, MATCH and so on, it is right there. The syntax is somewhere between RegExr and the WHERE clause in SQL.
We put it together and we got such locators. In parentheses - NS predicates, in square - indices, pipes and points.

XUI locators

• xui = StaticText [1] | TextView [0]
• xui = Button (label == 'Stop')
• xui = NavigationBar [last]
• xui = Table [0] | Cell [3] | .StaticText (id = Address)
• xui = Table [0] | Cell [3] | .StaticText (id = Time)

They figured it out: it worked at an acceptable speed, they started to deal with the next problem. Inspector. The WebDriverAgent inspector is there, there is even a cool instruction on how to start it, you can execute a command, two, three, and ... It didn't start, in short:

Even if wound up, there is also nothing useful. So we had to write our own. It is also simple, but quite functional.

There is a tree of elements, green is marked with accessibilityID, if marked. You can select an item and in the upper right corner you can see the information on it, in the left - in the screenshot to see exactly where it is located on the screen. The key thing we needed was a search string to test whether we compiled locators correctly or not, whether an element is searched for on the page or not.

Results

What is the result? There was a webdriver with its pluses and minuses. We worked on them a bit -
It became much better.
In our scheme, Appium has changed to WebDriverAgent:

We left the tests as is:

Stage left as is. In page objects we changed the Appium locators to ours. They have become even more readable.

It took about 2-3 weeks and saved us 200,000 lines in libraries and about 400 tests.
On this we did not stop, of course.
Appium besides the fact that he just allowed to test, he solved some other problems. Now we had to deal with them ourselves. For example: we want to parallel tests, we need to run several WebDriverAgent instances, run tests, specify an API in each, and so on.

But people invented the grid! However, here is the problem: when you connect more than three nodes, it blows down the tower, it starts to eat memory, flow, stupid. Some people are perverted like this. Do not try to understand anything here:

There is a grid, followed by 4 more, followed by 4 more, and so on. I considered: taking into account the peculiarities of our project, we would have to install one hundred such grids, allocate a whole server for this. We did not bother with this. We wrote our own. It is simple, written in Go, uses approximately 10 MB of RAM and serves 300 nodes. Registers the nodes, proxies all calls to these nodes and selects the appropriate session with regard to the requirements for the request. And in the end you need to release it. Either when the session was closed, or on time-out, if the test fell and could not report that he had finished. All this is compatible with the Selenium grid so that you can work. Now we have this scheme, it is working this time:

But our problems do not end there, because we are testing the application, we need to put it on the phone, do something with it, so we wrote a thing called grid-wda-agent :

It does several simple things: it is registered in the grid, because WebDriverAgent itself does not know how. Proxies all calls to WDA, and at the start of the session, selects or launches the desired simulator that we requested in Capabilities, deletes the old version of the application, installs a new one, and restarts WDA, if necessary, because it still sometimes eats memory, it is better to restart it sometimes . And in addition, he records the video of the test and sends it later to the storage via the S3 protocol .
We put all this on the Github organization qa-dev , you can go, read, send pull requests, issues.

findings

What to do now, if in 2018 we want to test iOS applications?
Two ways. If we can Swift / ObjC - we can write native tests, XCUI Tests, wait until it is compiled to test one test. Or we can take WebDriverAgent. There is more choice: there is either Appium, the developers of which, six months later, after the release of XCode 8, they nevertheless filed their implementation on top of the WDA Agent, having forked it and adding something. There is an original version from Facebook. And ours.

What does Facebook have? He is an official, everything is quietly contributing to him.
There is an Appium. First, I quote Dan, who Appium is developing.

"De-facto standard for automating mobile applications".

Secondly, Appium has support for the open-source community. And the guys recently wrote down their inspector as an application for MacOS.

There is our option. We have fast locators, our inspector, grid, grid-agent. The last three, in principle, work with any WebDriverAgent, you can take them separately and try to use them.

What conclusions can we draw?
First, choose what suits you. Second - do not be afraid to make your tools. In 2015, my life looked like this:

Then in 2016, all these events started to happen, we started writing our own “crutch” version of WDA. It became so:

Now we have a Grid, Grid-agent ... Life is getting better!

When you make your bikes, do what you need for the project. And more importantly - do not do what is not needed. Because when the whole story happened, we thought: “cool, there is an XPath, we will now take it and make it normal, not like them, and it will work”. But XPath has such a powerful syntax and so many functions that we probably would have written so far if we went this way. We needed fast locators, not XPath.
Perhaps that's all. Ask your questions.