Test automation: who should do this, who needs it and how this area is changing

In IT, everything happens rapidly, and six months or a year is enough time for dramatic changes. This applies to automated testing. To find out how this segment and the attitude of the testers themselves to their profession, let's talk with two experienced experts in this field - Igor Hrol and Ilari Heinrik Egerter.




Igor Khrol is a test automation specialist, has been working in this field for about 10 years in various roles: as an engineer, architect, manager, trainer, consultant. He has extensive experience using most popular tools (Selenium, HP QTP, TestComplete, JMeter). It programs mainly in Ruby and Scala, but in other languages ​​it was also necessary to write (Python, Java, C #, JavaScript, VBScript).
Now working in Toptal.
')
- Good afternoon, Igor. Many readers are already familiar with you and saw a speech last year at the Heisenbag conference . If it's not a secret, please tell us, what are you doing now? What is the work going on?

Igor Hrol: Good day! Since the past of Heisenbag, my occupation has not changed fundamentally - I am still working in the analytics department of Toptal. True, since that position has changed, I was QA Automation Engineer, and now Team Lead. Now I can influence the quality of the final result on all fronts: both from the point of view of testing, and from the side of development, and of organizational improvements, too, cannot do without me.

We are working to give the business relevant information. Actually, I will talk about this in my report in St. Petersburg: how to check reports, correctness of incoming data, which application architecture is suitable for solving such problems. And, of course, for the most part it's all about test automation and code writing. We do not like to repeat the same thing many times and we mainly test with autotests.

From fashion trends Toptal does not lag behind, of course. We introduce the technology of machine learning and other fashionable data science. It looks like a report on this will be a good topic for the next Heisenbag.

- Is a modern auto tester a jack of all trades or a narrow specialist?

Igor Khrol: I would like to believe that the master of all trades. If this were not so, then I would hardly have been doing this for 10 years - it would have become boring. I do not believe in narrow specialization in IT in general. In my picture of the world, a good engineer should know related areas. The developer should be able to test and write autotests. Tester - write code and understand management issues. I saw a lot of nonsense and lost time precisely because people sawed their own narrow task and for various reasons did not solve it together with other specialists.

I will give examples. The automator can come up with various sophisticated xpath-locators instead of spending two minutes and adding an ID on the page. You can ask the developer, or even better - just add it yourself. Well, or the automator automates the test case written by the tester. He does the work one-to-one, as it is written: he presses the buttons, checks the result that is displayed to the user. It is possible to spend half a day on writing such a test, although in a large number of cases these checks are reduced to several simple unit tests that are written within 10 minutes.

Many companies continue to develop products with narrow specialists on the conveyor principle. Of course, the result will be, but the same could be achieved with less cost. Well, from the side of the engineer it is more interesting for me to work when I am not limited to the list of job descriptions, but I can influence the result from different sides.

- Often we can see tips on how to quickly become a guru in a certain area, for example, learn a language in two months. Do you think testing automation accepts such a rush or specialists in this field are mostly former developers?

Igor Khrol: According to my observations, experts in test automation are mostly former testers, not developers. There are among them developers, but not the former, but the current ones. These are people who, in the course of development, write a large number of autotests and in one way or another face the problems of this field. What is interesting, more often they, rather than automators, who grew out of testers, move the expertise in this area forward.

Regarding the study of languages ​​in two months. During this period, you can become a junior specialist, if there is a tendency and desire. You will not become a guru, but you will be able to find work. And then - years of real experience, until enlightenment comes.

- There is an opinion that unit testing is the responsibility of code developers. Do you agree with him? When there is a huge project, where interconnection is often present, for example, classes of a program, does the participation of test automation teams in such projects and such tests be required, or does everything depend on the specific situation?

Igor Khrol: The unit tests interact directly with the code and very often, in order for it to be tested, it needs to be changed. I have heard stories of individual teams writing unit tests after the code, but I have little idea how this can work normally. Plus, I do not understand why, in principle, to allocate the writing of tests to individuals. With us (in Toptal), the author of the change himself adds the necessary tests (not only modular ones), and nothing bad happens. When you do this, you better understand how exactly the class should be used, the API, or how to use what is written.

As for the huge projects and automation teams for testing: I led the automation of projects according to the service principle, when a separate team receives a request for automation in the form of test cases, and together they write autotests. These were really big projects for several dozens of automations. Honestly, I didn’t really like the result. Autotest is an integral part of the product. And when they are dealt with by individuals who are not tightly involved in either the development process or the testing process, these tests become a separate thing in themselves, which is necessary only for the automation specialists. I call it "writing tests in the bedside table." From the point of view of formal indicators, everything looks decent - work is boiling, coverage is growing, and in fact these autotests are used very little.

I prefer the Spotify approach when there are separate autonomous teams consisting of specialists of different profiles, and at the same time there are “guilds” uniting the same specialists and solving common issues. Working within teams, you better understand what you need to solve specific problems, and quickly solve the issues that stand in the way of automation.

- I would like to ask you a question related to the use of Selenium WebDriver. You said that you actively follow this direction since version 0.8. How do you feel about the fact that the W3C WebDriver standard received the status of a W3C Candidate Recommendation? Do you think this will lead to active refinement of Selenium?

Igor Khrol: WebDriver itself has not been changing much for a long time, at least externally. Including thanks to this, he became the standard. What changes is the quality of implementation and support on different browsers. Browser developers know better how to do it right, and can change, including the browser itself. And the W3C "forces" them to engage in supporting WebDriver by the browser itself.

Now superstructures are actively developing over Selenium, like Selenide. The situation resembles javascript and jQuery. Pure Selenium / WebDriver is good to know, but it is more convenient to use higher-level libraries, which solve typical problems specific to interaction with the user interface.

- Since participating in last year’s conference, Heisenbag, has your approach to testing or the set of programs you use changed?

Igor Khrol: Half a year is not such a long time in order to radically change the approach. Although, of course, we are reviewing our testing, and it is evolving. Recently, we have begun to actively use model based testing to test our business processes. In a nutshell, we randomly walk through our workflow, looking at the problems and contradictions that arise.

If we talk about my department of analytics, then data profiling appeared in testing, where we explore various invariants on our input and output data. I will talk in detail about him in my report.

- Thank you so much. We look forward to seeing you at the conference.

Igor Hrol: Thank you!

Ilari Henrik Egerter is a software engineering and software testing specialist who has been working in this field for more than 10 years, starting with software and ending with e-commerce on eBay. In 2013, he co-founded the International Software Testing Society (ISST), which advocates the return of common sense to testing, and is now the president of the organization. In 2015, he was elected to the Board of the Software Testing Association (AST), where he serves as vice president of marketing.
Currently managing director of House of Test GmbH.

- Ilari, good afternoon. Please tell us about yourself and your work.

Ilari Henrik Egerter: I came to testing rather by chance. Fourteen years ago, I studied linguistics and sociology at a university in Zurich, and there was a job at a medical equipment company related to installing a PC. They also needed to be tested, which is why I thoroughly hooked on this type of activity. I really liked the combination of social elements and technology in this work. After a couple of years, having already become the line manager of the whole team, I also decided to study software development. Then I changed jobs and moved to eBay, where I headed the testing team for eBay sites in Europe.

Two years ago, I became independent with House of Test, and we began to develop my team. We currently have seven full-time QA testers who work as outsourcing contractors in a wide range of industries. Besides managing the company, I am the vice president of marketing at the Software Testing Association . For every tester who is proud of his craft, I advise you to join us. In my personal life, I love to read, spend time with my family and enjoy brewing.



- Last year you gave a talk on "No Such Thing as Manual Testing and Other Confusions" . Your audience consisted of people from different areas: testers, managers, developers. Why do you think people from different directions are so actively interested in testing issues? Is this due to the fact that companies are gradually becoming dependent on test automation?

Ilari Henrik Egerter: I think that when you are a tester, it’s quite natural to be interested in testing issues. In addition, various non-testing positions are certainly involved in the process, and I welcome their interest in this area. The use of tools in automation is certainly becoming more advanced. I would like to note that testing automation cannot be an end in itself. The goal is to find adequate solutions to problems. Some of them can be solved using automation, while others require more suitable solutions.

- You say that not many good testers met on your way. Do you think so due to the fact that this direction (testing automation) is quite young, or is this position connected with a lack of understanding of the meaning of the requirements for the profession of a candidate for this role? Maybe there are other reasons?

Ilari Henrik Egerter: I think that most people are not very suitable for the work they do, and therefore the result is not always good. Expertise by definition is rare.

You mention automation: I would like to point out that you cannot automate testing. You cannot automate software development. You can automate tasks in testing and development.

My understanding of testing is human-centered, since testing satisfies the need for information about the state of the product and identifies threats to the timely delivery of the product. Much of this can be done only by sensible people. You cannot automate the question "Is automation appropriate here?".

Software is created by people to meet human needs. So when I say that most testers are not very good, I, among other things, point out the idea that automation is a kind of improved testing, a kind of add-in. I want to convey the idea that I am not an opponent of automation. I am her supporter in those places where it is really necessary, and to understand this, to come to an understanding is a rather non-trivial task.



- In your materials there are many parallels with life and attention is paid to such things as communication, human curiosity, the desire for knowledge. Tell me, do you really think that automation really depends on non-technical things?

Ilari Henrik Egerter: I never said that you do not need technical skills for automation. Automation testing is a software development project and therefore, of course, it needs engineering knowledge. The reason I pay a lot of attention to communication, critical thinking, and epistemology is that they are often ignored, although they are important elements that should be — let's call it — a complete tester set.

- Soon you will make a presentation on "Think Bigger - World to Class in Testing" . What do you want to achieve from the audience, showing examples and steps to achieve goals in test automation? What kind of feedback do you want to get? Is it important for you to know that your information helps to put data in the heads of the audience and direct their thoughts in the right direction?

Ilari Henrik Egerter: Again, I have a holistic view of testing, and automation is part of it. I want to make people start thinking about what it means to be a great tester. I want people to come to their own conclusions, and they do not have to be the same as mine. I do not have the truth, but I have an informed opinion. I want people to challenge me too. If I start active conversations with people, and if I have the impression that people began to think, it would be that I would like to get an answer from them.

- Thank you very much for your answers. We look forward to your presentation.

Ilari Henrik Egerter: See you at Heisenbag in St. Petersburg.

You can learn more information and knowledge on test automation, tools, environments for testing and other areas in this difficult craft at the upcoming Heisenbag 2017 Piter conference on 4 June. The event will bring together automatists, developers, managers, and just testing fans. Waiting for you!

You might be interested in testing automation materials.
Video recordings from Heisenbag 2016 Moscow , as well as interviews with Nikita Makarov on Testing: a simple IT track or a serious undertaking?