Five ideas "for service", or Impressions of the Moscow "Heisenbag"

How to make sure that the conference for which you take a ticket, will not be "lazhi"? In my opinion, the best way is to become a speaker at one of the conferences of the same organizer and test the selection and preparation of reports on yourself. In the spring, I was a speaker at a conference held by the JUG.ru Group, and everything that happened before this was reminiscent of defending a degree. Several times my report was tapped by the program committee, after each of the runs I received a long list of questions and comments. One member of the committee became an “opponent” and, in endless discussions in Slack, required additional experiments in order to get unconditional evidence of the effectiveness of what I was going to talk about in the report. When the news came that the report was approved - it was not yet the final: I was invited, along with other speakers, to take a two-day course of public speaking. Having passed through this test, I know: the guys from the JUG.ru Group will not allow a drop of “Laji” at their conferences. And I trust them: this year I attended most of their conferences.

Moscow “Heisenbag” for me is the last conference of the outgoing year, closing the busy season. Judging by the statistics voiced by program director Andrei Dmitriev during the opening speech, most of the people at Heisenbag are professional testers. Feelings from communication are mostly people working in large companies with dedicated QA teams.

I am not a typical visitor to the Heisenbag. In our company there are only about 25 people, and many combine roles. I have to be a technical leader, a project manager and just a “problem solver”, and only a couple of people are involved in testing automation. The survival of our company literally depends on the effectiveness of the team. Increasing efficiency is possible only through the acquisition of new knowledge and broadening of horizons, for which I and two other colleagues went to Heisenbag.
')

Day 1

Conference opening

Good mood was set from the very opening of the conference. During the opening, program director Andrei Dmitriev gave the public a task: in thirty seconds, get to know one of the neighbors sitting around. The hall was immediately filled with a loud noise of voices. It was a very useful task! After all, the conference is a tool that you need to skillfully use so that it gives you maximum benefit. Although it seems that the main component of the conference is reports, in fact, you can get only about 40% of the total benefit from sitting on reports, and you need to take the remaining 60% by talking to experts on the sidelines, discussion areas and at a conference party. If you are an introvert IT specialist, then in order to start a conversation with a new person, you have to go overboard yourself. But it is difficult only at the very beginning - very soon you will find out that everyone here, like you, came for something new, and it’s really easy to communicate at the conference.

When the noise subsided, Andrew gave another task: in the next two days, remember at least five key ideas for yourself, which, having come out of the conference, we could put into practice. I took note of this, and reports began in three halls.

one

In the first timeslot, I went to the report of Dmitry Buzdin "How to build your framework for autotests?" . Based on the ISTQB Generic Test Automation Architecture theoretical scheme, Dmitry talked about how to combine open-source solutions so that they completely cover all the tasks of the testing process. For me personally, the report did not contain any ready-made recipes, but I found out about the ISTQB model, and my knowledge of what details any test framework should consist of was seriously ordered.



At the same time, Sergei Egorov gave a report on TestContainers in a parallel room. I didn’t go to him for a very simple reason: for the first time Sergey’s report on TestContainers I heard in spring at JEEConf, in the beginning of autumn we successfully implemented this technology, and in October at Joker we managed to talk in detail and successfully with Sergey features that we met. The idea of ​​TestContainers is simple and powerful: using the existing Docker infrastructure and downloaded images, in your test you can get, for example, a JDBC connection to a real Oracle database simply by instantiating a class, and no more difficult than any HashMap. Browsers, databases and much more is available out of the box by simple class instantiation! What used to require complex setup of the test environment and the configuration of the tests themselves, now boils down to three simple requirements: the machine must have JDK, Maven (Gradle) and Docker. The creators of TestContainers are great, and I wish them good luck.

2

Then I went to Simon Stewart's Scaling Selenium talk. Simon talked about common sore problems that arise as UI tests grow and become more complex: “fragile” locators, expectations, scaling with Selenium, including in the cloud. Although I did not feel deeply immersed in the topic, I suddenly realized that Simon had voiced the first idea that I definitely wanted to apply in my office: “check for preconditions”. Before you run the system tests, check to see if all the services on which the system depends are running. Do the database, authentication server and other components that the application depends on work? After all, if at least one of them "lies", there is no sense in testing, but it is worth stopping and explicitly reporting a problem. Otherwise, a large number of irrelevant errors fall into the logs, which undermine the credibility of the test results as such.

At the same time, my colleague Nikolai visited Alexey Vinogradov and Andrey Solntsev's “Selenide Puzzlers” . Quiz presentations are a great idea to show off with quickness of mind and earn a T-shirt. Returning from there, Kolya said that he was dragging his hand with all his might, but he was never given a word. Knowing the delicate Kohl and the excitement, which is usually created on “puzzlers”, I can assume that it was worth pulling the hand even harder :-) Selenide is a great framework that solves most of the problems of “bare” Selenium. We experimented quite successfully with him, but now we use another. Although after Heisenbag we thought about the correctness of his choice.

3

Following we thought whether it was worth going to the report of Vladimir Sitnikov “Testing the performance of web applications on the browser side” or on the round table “What should the tester know in 2018?” We decided that we don’t rest on the performance on the browser side and on the "round table".



There was no table there, but there were five speakers sitting in a row, the presenter and the questions asked from the audience and on the telegram channel. The discussion was mainly conducted not around technologies, but around the career path of the tester and the organizational structure of the company - things that are not very relevant for such a micro-company with a flat structure like ours. Although there were five speakers, Nikolai Alimenkov spoke more and more actively than others, and some would be good if several replicas could be inserted in an hour. I think Nikolay is one of the best experts and trainers in the IT field, and just a cool witty guy. But at this conference he already had his two reports, and therefore, it may be, sometimes, worth giving more time for other guys to speak.

I understood such a thing: it makes sense to go to the “round table” only if you have a specific question that you would like to ask the group of experts. If you go to "listen", then the thread of conversation can go not where you expect. It is more convenient for the speakers to ask their own specific question in the discussion area or at a party, which makes the very idea of ​​a “round table” rather doubtful.

But not to say that we had a bad time in this time slot. Once near my colleague, I showed him a few falling UI tests, because of which Jenkins spammed me in the mail, and I was nervous before the upcoming show next week to the customer. Together, we managed to quickly get to the bottom of the reasons for the drop in these tests, and I left the room satisfied. Sometimes the conference may need to work. And the report of Vladimir Sitnikov, we will definitely look in the video!

Dinner

What I will say can be treated with irony, but I consider lunch to be the most important indicator of the quality of any conference. You spend the whole day, you are away on a business trip, or you just come home in the middle of the night, constantly focused on gaining knowledge - it’s energy-consuming. Meals at the conference should be such that thoughts about food and thirst are not distracting, and given the fact that you may have specific requirements for food. JUG.ru Group in this regard, well done, the quality of food they have is good, although there are some punctures: telegrams and kamenty on Habré boiled over for a long time about quinoa on Joker 2017 :-) This time, being at dinner, I found that although all the people were clearly full of food, the tables were still full of snacks and hot dishes. This instilled in me a pleasant confidence, and I did not deny myself in one more portion of the fish under vegetables and baccalaree with cheese , after which I was ready to continue :-)

four

In the previous St. Petersburg “Heisenbag”, Andrei Satarin talked about special tools for analyzing code execution in C ++. We write in Java, so I thought that his current report “How to check the system without starting it” will not be relevant for us. But still, I went up to Andrew himself and asked him. It turned out that the report was not limited to C ++ at all, I stayed and did not regret. Hence the conclusion: never hesitate to clarify in advance what the report will be about, the speaker or the program committee member. You will be answered with readiness, and this will increase the efficiency of your pastime at the conference.

The idea that Andrew expounded is both cheap and efficient: the configuration files of distributed systems can and must be verified automatically. The simplest thing you can check is that these files generally exist and are valid JSON / YAML / XML, but more complex checks are possible: for example, that the distribution of services among nodes / racks / data centers meets the requirements for system resiliency. Or that the hosts and ports are connected to a single network, otherwise it may turn out, for example, that due to incorrect configuration of the ports in your cluster there are fewer connected instances than you thought.

For me, the picture, however, did not develop in one place: Andrei talked about the case when the configuration files are in the version control system in finished form, but, for example, in our configuration files are created dynamically and decomposed into virtuals using Ansible, and on virtuals they are no longer available. I went to the discussion area and a small “brainstorming” led to a solution: it is enough to run the ansible-playbook run to put the copies of the configs into a separate folder and immediately start checking the resulting configs. And although the risks that Andrei was talking about for now are not essential for my project, I know that over time they will become significant, and then these checks can and should be implemented. Thus, the second idea “for armament” was obtained.

five

The author of the next report, “The Abuse and Misuse of Test Automation,” Alan Page , participated in the Microsoft Windows releases in the years when I wrote “hello world” at the institute in Borland C ++. That alone was worth it. The report referred to the well-known concept of the “pyramid of testing”, which often tends to degenerate into such a ridiculous “ice cream cone”:



“A test pyramid means that you must write as few UI tests as few as possible,” said Alan. UI tests 1) are difficult to write, 2) they require huge computational resources and are still slow, 3) they are unreliable (flaky). The most reliable and cheap tests are Unit tests, and they should be the most. And in case of detection of a bug, a test for it should be written on the lowest tier of the test pyramid - ideally, if it is Unit-test.

It would seem - this is the basics! But it is strange that I did not formulate for myself this before in such a crystal clear way. Thus, the updated idea of ​​the essence of the test pyramid was for me another idea “to adopt” from the conference.

6

The first day was closed by Nikolai Alimenkov’s report “Household classification of testers from a developer’s point of view,” where he wittily described the archetypical cases of testers' ineffective and destructive behavior in the company: “tester-Nazi”, “tester-slug”, “tester-dandelion” ... Indeed much more depends on efficiently working people in an organization than on technology, and such a report will probably open someone’s eyes to what is going on in their team and let the managers diagnose cases ciency behavior of employees.

A party

Conference parties are held mainly so that you can meet and talk with specialist colleagues without being constrained by the time frame between the presentations. Therefore, they need drinks, food and fairly quiet comfortable places where you can chat. At huge conferences like Joker, noisy crowds are formed, a crush arises around popular speakers and it can be difficult to communicate. In this regard, I prefer less numerous conferences, and the Heisenbag on this score turned out to be good. We had a great talk with Vladimir Sitnikov on topics related to SQL, and then I moved on to the group around Nikolai Alimenkov: as it turned out, there was a discussion about his closing report, people were discussing organizational problems in their companies. Everyone agreed that "fixing" problems with people is a much more difficult task than the most intricate bugs. I also managed to talk with Nikolai and learn something about how best to carry out the transformation of processes in our company, to set a goal, to build a delivery conveyor. The conversation went far away from the topic of testing in the topic of process management, and this is wonderful: you go to conferences to communicate with experts on any topics that concern you, not necessarily limited to the scope of the conference names.

Day 2

one

Came the second day. I came to the first slot nepodvatsya and with his head, already swelling from the information received on the first day. I also had to code and pull-request something. Therefore, although I was in the room at the report of Artem Eroshenko “Simplicity, trust, control - three whales of web testing automation” , half of my focus was directed to my IDE. PageElements are cool and correct. Retrofit turns the HTTP API into a Java interface. Note to the hostess: the Yandex htmlelements in his improved incarnation should be taken from Artyom's gitkhabnik eroshenkoam.

Sorry, Artem, I was not the best listener of your report. Perhaps this report should somehow be reviewed in the video. There is nothing to worry about missing one of the slots at the conference: it’s impossible to grab everything. Information, like food, requires digestion.

2

In his report “Developer + Tester = Quality ++” Nikolai Alimenkov continued yesterday’s topic of the struggle for improving processes in the organization. Not for the first time, Nikolay is an ardent opponent of the existence of a “testing department”, in which, like through a brick wall, the developers “shower” the product in order to get it back from the tester after a while. Nikolay talked about the benefits of Agile Feature Teams with the joint work of developers and testers, and the "key effectiveness ingredients" that testers and developers can contribute during the execution of the main practices of this process, and as a result, the transition from the ideology of Quality Assurance to the ideology of Quality Assistance .

3

We are long-time users of Jenkins, so I necessarily went to the report by Oleg Nenashev “We ​​are building our own test framework, with Jenkins Pipeline and libraries” . However, as I see from the reports at the meetings, we are lagging behind in the level of using Jenkins: in Jenkins we propose to use Pipeline, but we still sit on Freestyle Jobs / Maven Build Jobs. Oleg spoke about the need to transfer all jobs to Pipeline in the conclusions of his report in plain text.

But those examples that are shown on Oleg Jenkins reports are, in my opinion, quite complex, replete with code on Groovy, which we do not know, references to the API, which we do not know, and generally give the impression of hard-to-debug hacks. Although, judging by the depth of questions from the public, there are a lot of people around who use this "to the fullest." In our simple case, the threshold of entry somewhat scares. But we need not so much from Jenkins: that he collect Maven-projects, publish artifacts and at the same time be completely configured through the code on version control. Where to go - we need to learn all this, we must deal with Oleg's examples on the githaba.

And we also heard some people, coming out of the report, say among themselves: “Everything is clear, we must switch to TeamCity”. Between the CI systems there is now a serious competition, and Jenkins has yet to fight for itself ...

four

Pavel Senin in his report Selenoid - hundreds of parallel UI tests easily and quickly demonstrated how using a Selenoid system you can replace an unreliable and poorly scalable Selenium hub with a beautiful solution that runs browsers in containers and issues sessions to tests, with a lack of resources it builds session requests in the queue, and, most importantly, it does not require that the Selenium tests you have yourself have been rewritten.

Because of the nature of the design work, I know well that the browser is a beast ready to devour all resources, and therefore it is better to keep it “in a cage”, completely isolating it from other processes, I really liked the idea with Selenoid. This is definitely another takeaway from the conference . We will try to deploy Selenoid-server as a common resource for UI-tests in our company.

In the course of the report, Pavel unwittingly uncovered organizational problems common to many in his team that did not escape from those who wrote to Twitter. The solution of organizational problems is what happened to me on this Heisenbag in the red line.

five

The report of Andrei Solntsev under the short title “Flaky tests” , in my opinion, turned out to be the best at the conference. Going to him, I thought: “Well, yes, there is such a thing as tests, which, for some unknown reason, sometimes do not work - what can I say?” And really, what can you say? Andrew did not give ready-made simple recipes for deliverance of such situations, although he outlined the main directions of the fight against them.

But he told a number of truly detective stories from his own practice, sometimes lasting months and years. For example, the “Curse of the green button”, which for a split second closes the pop-up progress bar, because of which the button was not pressed.

Or the best: “Why does Chrome hang?” Is a mystery that Andrew could not solve for two whole years, having tried everything that was possible: logs, timeouts, debugging ... Once, when he unsuccessfully tested the most insane hypotheses, the daughter called him play cubes. He reluctantly distracted from work, gazing regretfully at the monitor. The test was left unattended for about twenty minutes. Suddenly the following appeared on the screen, and the clue came:



Andrei finished the same thing as Nikolai said: developers must participate in the work of testers, so that through the “interpenetration” teams will mutually enrich themselves.

One of the reasons for the flakiness of our own tests on one of the projects is that the data in our test database is “polluted” according to the results of previous tests. In the discussion area, I wanted to ask Andrei, with the help of what mechanisms it is better to quickly pick up a fresh copy of the database with data before each test. Nearby were the guys experienced in this area, a discussion ensued, and in just a couple of minutes we developed a solution that is suitable for us and optimal in our case. So we are implementing in the near future, this is certainly another idea “for armament” . And it’s cool when exactly your engineering task at the conference is helped by the best experts in the field :-).

6

The conference ended with the report of Alan Page 'Truths about technical testing' . Alan talked about what awaits testers in a transforming world, where the separation between the specialties "developer" and "tester" is erased.

For example, the current stereotype that the “tester” - this is certainly the “automator of UI tests” - must break, especially since the principle of the test pyramid forces us to write as few UI tests as possible. Instead, the tester should be integrated into the development process as much as possible. It seemed that Alan directly continues the thoughts of Nikolai and Andrei, who spoke in this hall in front of him, and so it was: for example, Nikolay had a slide about “throwing the product through the wall” between the development teams and testers, and Alan ridiculously portrayed this throwing pantomime .

Alan said that when he was a developer, he experienced the greatest satisfaction not when he created new functionality, but when he caught a bug, and I completely agree with him - I have the same thing. After his report and the official closing of the conference, Alan went into the empty corridor to the most recent discussion area of ​​the conference, and we talked a little on these topics with him: he was a very pleasant person to talk to.

Conclusion

In the final word, Andrei Dmitriev asked the public - “how many take-aways do you take away from the conference”? Honestly, by this time I did not count. But on the other hand, I knew exactly which one was taking the most prestigious take-away chief, I tweeted about him: