We test user scripts with Hermione. Yandex lecture
Hermione is an open-source automation tool for integration testing. At the FrontTalks conference, the developer of the search interface service Nikolay Ilchenko tavriaforever explained why our interfaces still sometimes break, what are the advantages of Hermione, and showed how to start using it.
- I arrived with a report about Hermione. But this is not about Harry Potter, and this is not a fairy tale. I look like a pretty young guy, sometimes they don’t sell juice in a store, they require a passport. I am 28 years old. I found the Internet still like this, my dad had strange boxes at work near computers that did this. (The sound of the modem.) I think some of you caught it. Sites were very simple. Text and a set of hyperlinks to navigate through the pages. Over time, the Internet developed, pictures were added, Lebedev’s studios appeared: one, second, eighth ...
The most difficult thing for developers was to correctly cut the pictures to pretty buttons. But complex interfaces were still far away. Everything was complicated on computers. ')
We listened to music in Winamp. I recently learned that it exists, it is developed. We wrote special letters. We went to the “My Pictures” folder to find our photos there. There was nothing difficult on the Internet. But everything developed. New highways of communication were being laid. In my youth I managed to build one kilometer of the Internet, I know how to do it. Everyone has 3G, 4G, you watch movies, send letters, do everything on the Internet. When was the last time you installed the program? Not very frequent case.
We, the makers, had to make complex interfaces. We now have a lot of buttons, custom scripts. Not just pictures. Browsers have already learned to do everything for us, they round everything up for us. But our interface is very complicated.
In one of these services I work. This is Yandex.Search. Millions of people ask questions, find answers. Use Yandex, Google or other services. The service is very complicated, but only I know about it. It seems to you that there is one page, what is there such. The department employs approximately 120 people. 120 people! When I tell my relatives and friends about this, they say, what are you doing there? There is one page! Who you are? And this is only a frontend.
120 people - the army of managers, every day, walk, talk, slaughter negotiations and think about how to make the search better. They come up with incredible stories. For example, I jumped out of the train, I need to find the next bus - how to do it in two clicks? They constantly improve the search. Maybe you do not see it, but since 2014, 10 thousand experiments have been conducted in the search. Changed the color of the links, the size of the text, added new pictures, buttons. And about a quarter of them has come to you. We sometimes experiment on you.
In order to bring this all to life, we have 40 interface developers in five cities and several countries. Now the infrastructure is not an easy story, you need to collect these projects, a lot of files, optimize everything. Therefore, we have a super-team that makes us concentrate on the product, and how the project is going to deploy, which tools to write tests - all this magic, what happens in the console, is a separate command. And we, 120 people, make this one page every day. QR code is about our work. Perhaps when you read, you want to work with us. 120 people will not engage in any strange activity. From today I officially declare: search is not 10 links. It is much more difficult today.
The search must comply with the laws of the country in which it operates. The search should adapt to the user in order to provide relevant links that you most often use, so register and log in with Yandex. And it turns out that there is a huge settings page in the search; there are more than 20 checkboxes there. And probably someone uses them. This is generally a separate project - search settings.
There are special answers in the search. I often listen to the reports of my colleagues, and they say that they would never understand what “witches” is. These are special answers. If you ask what time it is, you will see a special answer above the links. If you ask for the weather forecast, you will see the weather brinker and put a raincoat on just so you don't get wet.
If you came to your mother, said a bunch of different words, she boiled over, went to the Internet to read who such a geek is. This we know, and she does not know. I entered Yandex in search and received a special response. There are a lot of such witches. And it also happens that after work you want to think about nothing, you want a cool series. We have a special serial wizard. And this is only a small part of the answers in the search for Yandex.
Some statistics, just numbers. 35,000 lines of css, not dead code. We constantly refactor. The company has a lot of money, so the most important thing for the company is quality. We have long been a startup.
The search has one page. Market has more than one, there are filters, lists, product cards, iPhones, that's all. How many lines of templates are there in the Market? Are there any options? A couple of thousand? One hundred thousand? In search of 160,000 lines of patterns. In the Market, about four times less. And these are live templates. 160,000 lines of templates are written for one page. We, of course, not React, sorry, but still. It is a fact.
And the last figure. You have seen a promotional ad, that search is number one, we are fighting Google ... 50 million people use Yandex search every month. Normal audience, the site is popular.
But there is a small but. With the help of these logos you all know, there are also big projects on the Internet that are used by a large number of people. I think some of you work with a no less code base with more complex interfaces, where there are more user scripts, drop-down menus, editors ... I was not surprised by this search.
We and you have a common situation. Our users should look like this: a smiling young guy with a MacBook without a logo. Or this girl who loves this site so much is ready to kiss him. And it would be better to kiss her boyfriend. But it should not be so. A person should not come to the page and suffer. We are cool developers. What are we doing? We write tests.
There is a super application, React, server rendering, everything is cool. Developers write tests for each component so that it is rendered, some HTML starts up, everything seems to be cool. If something fell, reporters what you want to eat. Even over the past few years, we have learned how to run it in different operating systems on different devices. Thank you Selenium, we learned it, everything is cool. And now it has become fashionable to use regression testing with screenshots, when you make a template layout, change it, a new template is made and the layout is compared. If wrong, something moved out, then we will see.
We do not write the code as much as the tests. Everywhere these articles about tests, tests ... But in real life, everything happens a little differently. Despite the fact that our code, from our point of view, works correctly when a real person arrives, sometimes it happens like this. Cool, the window worked in the factory, I collected it.
Another life example. March 8, 2017, I have to go somewhere. I met a girl, I really liked her. I wanted to congratulate her on March 8, but I could not physically. I went to one of the sites of ordering flowers, chose flowers, clicked on "pay", and hloby - in my browser just a white page. Okay, I'll try again. Once again I clicked on “buy” - the story repeated. History repeated for the third time.
At this point, the developer was born in me. I took and opened developer tools. And I saw that in the center of the iframe page, my form that I want to fill out. I want to pay money, but I can not. And on the iframe tag is written inline style height 0 px. As in the movie "The Diamond Hand", I changed the movement of the hand by 300 pixels and saw the shape. She is real. I filled all these fields, clicked to pay. Probably, they have beautiful validation there, as Pavel said. Everything is cool, clicked to pay. And the girl received flowers. I am a developer, and she got flowers, even sits in this room. What if it was my neighbor? He would not have a girlfriend. And if I was not a developer? I would not have a girlfriend. By doing interfaces, you and I can really influence people's lives. And when they do not work, we really create difficult situations. We want to make people happy, like in those pictures.
In the search there is a certain number of different features. These pieces are used by many, and any change in these features seriously changes the metrics. People start to click less, less money becomes more serious.
Example. The page should open, click on the thumbnail of the image, the preview should appear. All buttons should work, everything should close. How should this work from a tester's point of view?
Open the seals, click on the sorcerer, open this, that ... And we have a competition, we are releasing every day. Every day there are so many commits in the repository, sometimes I don’t have time: in the morning the folder was - did git pull - there was no folder. And I was hoping for her.
In search of 550 test cases mandatory, which in any way should work. Imagine what we need a team of testers to check it? We will spend hours testing it with them. For us, this is not an option; we cannot test it like this. Naturally, I did not come up with integration testing, we had tests, they were written in Java. But we are frontenders, with Java does not really fit. It was a separate team in a separate room, they wrote tests, what we did. All the time was rassinhron. We have torn off some piece - they did not have time to boil this test - the tester has everything red already. “Kolya, you broke this thing!” But it didn’t break, this thing was just torn off. It was constantly.
Someone from our guide — maybe there were more than one — decided that we should write JavaScript tests and put them in a folder with all the code. These infrastructural cats have taken this task, have thought about it for a long time, went to the Internet, searched for what it is. We were looking for writing our own tool. As usual, your cycling is better than a stranger.
This project was called Hermione. Any coincidence is random. When choosing a name, they simply voted for Hermione more, I don’t know why. Everybody likes Hermione. This is a node.js application, open source, right now you are opening laptops, downloading and starting to use.
What is under the "cloak"? In order not to call Hermione in the wrong words, this is a tool representing a wrapper over such famous things, mocha. If you wrote tests, you wrote using mocha. Describe it, no need to invent anything and learn.
Convenient client to Selenium to write "open browser, click, flip." And Selenium itself to run these tests in all browsers. When I was chased, they told me, why are you clever with your Selenium? Do you think every person has come across? Explain what it is. I think everyone knows, but what is it? This is a project that includes a number of tools sharpened to ensure that these tests automatically run on MacOS, Windows XP, and run different browsers there. It all happens automatically, and it is called Selenium.
You can endlessly tell what a delicious salad, but until you try it, it is not tasty. Therefore, right now, without screencast, I will write code in real time. I will show you how to write tests simply and very quickly.
I created a folder called ft. Open it in the editor. I use Visual Studio Code, they did not pay me for it, but I just love the quick stuff. There are some files here. There is a package.json, it is written that there are three dependencies, Hermione herself. I need an HTTP server, because there is no beta. You will probably have your own beta, you do not need it. And I put the local Selenium standalone, because I don’t want to pay the money. But if necessary, then I will tell you how to pay money. I have two teams. The second will run tests. The first will run two modules, an HTTP server and Selenium standalone. Let's create index.html. With a flick of the wrist, I created a half-project. I will show you how we write code in Yandex. Time. Half an interview has already passed.
CSS will write. Everything is already beautiful and designers are not needed, everything is ready. There is also JS, no React and jquery are needed. There is a div, inside the button and a div with text. I suppose to press this button, the text will be hidden and opened. Then there are some styles, but this is not interesting, the main thing is to look beautiful. A little bit js. We are here listening to a click on the buttons, a callback is triggered if a click happens, where we use the classList API to toggle the class to the house node. There is a special class that adds a display block.
Cool, we write code in Yandex, right? Come.
Here is our supersite already. Everything works, the button opens, opens. The project can already pass. Everything is super.
Now I want to write a test. I set everything depends, I will not show how to write.
I show that the project is open source. I am writing hermione test. The developers say that you need to put a package, add a configuration file, here is an example of a test on github, everything is for real. And a lot of documentation, if you're bored, you read.
Use quick start. They say you need to create a configuration file. I am creating a file at the root. Another life hack. I wrote a config. This is the minimum set to run tests. You specify a gridUrl. I told you what Selenium is. I will raise it locally, it will launch Google Chrome. The infrastructure guys told me that there are several projects, for example, SauceLab. You go there, choose Selenium, indicate which device I want - PC, Win 10 and some browser, Firefox. And he will show you in different languages ​​how to insert it. On node.js says that you need to fill in three fields. Specify the browser Firefox, the platform such and such, the version is. I do not need it, I have only one Chrome field. You just copy from there to here, paste in the address. All, Selenium set up, the tests are accelerated everywhere. Not very difficult.
Then there is the field sets, here you can point dot, for example, in the “Phones” folder there are tests for phones, in the folder for desktops - for desktops, and so on, you can infinitely distribute among folders, platforms. I wrote platformName for example, said that look for tests in the tests folder. It is necessary to create such a folder. I create a file, it can be called whatever you like, as long as there is an extension hermione.js.
Another life hack. Super. Mocha is well known. They wrote what kind of block it was to later understand what was happening with the tests. I am writing a super code. return this.browser. From now on, you can use the API provided by webdriver.io. I go to the site, show that I do not invent it. In the left menu, all these commands: click, double click, everything that a user can actually do on the site, you can play with webdriver.io. It's very simple, the methods are clear, even without reading the documentation.
I have already learned three commands by heart. I am writing a URL. Opened the browser. Need to open some address. I will locally raise index.html. I am writing localhost: 8080. It is necessary to click on the button and check that the text appeared. Therefore, I use the click method and specify the button selector, ft_button. When clicked, you need to check that it appeared on the page. Therefore, in webdriver there is a pack of helpers, which can as you asserter immediately check some condition.
I use waitForVisible. We point it to a selector of some block, in this case, ft_text. If the block becomes the display block, visible, opacity = 1, then we assume that this condition is true, then the test will be executed.
Everything, it remains to start the server in package.json, run Hermione itself. Super, everything started, Selenium started on 4444. Our server started, everything works. We will send Hermione here to test our feature. In another tab, write npm test.
Everyone, look at Google Chrome. As you can see, the test passed. You can see which test in which browser was launched. If there are many browsers - in each browser will show all the necessary statistics.
Imagine that your colleague decided to do everything as usual. You tell him - make a girl, and he says - you need to zarefachit. He rewrote everything, he decided that this code is dead. Removed everything. When you click on the button, the text does not appear. We want our friend not to commit such a thing and break us. Therefore, he most likely has some kind of continuous integration configured or runs them locally, and everything should fall. All fell test. It can be seen that such an element did not appear after 1000 ms. These 1000 ms can be configured, for example, you have long Ajax, you can specify at least 20 seconds. Thus, in a matter of minutes, we wrote a test. It's complicated? I think no.
Let's return to the presentation.
As usual, open source is very developed, for every sneeze there are 20 more similar ones. When the other guys who developed Hermione talked a lot about her at the beginning, they constantly asked the question: there is a Nightwatch, wdio, a lot of plugins, you install, you customize ... There is a Testcafe new trendy, there is Magellan with such documentation that Hermione's documentation is just an alphabet for children. What for?
When they started doing two years ago, half of these services did not exist. There was a nightwatch. When the pull request was sent there, they tried to communicate with Nightwatch, the maintainer did not respond very quickly. And we decided to write our project. We have a serious service, Yandex.Search, Pictures, Videos, News, millions of people. We can not use tools that no one maintenit.
Now, most likely, the situation has changed, all the projects really pulled up. Now there are a lot of users, Nightwatch has a lot of stars on github.