Introduction

As you probably already know, on December 8-9, the next Heisenbug will take place in Moscow , so we decided to acquaint Habr with the program of the upcoming event.

But not so fast! As an introduction - a little life story. At the end of the entry there will be one brain-breaking fact about this habrapost. Immediately after him - a detailed presentation of the program .

Why do you, dear chabrozhitel, generally worth listening to some marketer who writes this article? What can he understand in our hard work? Behind this info, I had to climb into the darkest depths of LinkedIn: Once upon a time, in 2010, I got a job in one small cozy company in the Novosibirsk Academgorodok on my first Java programmer. To study the product a little bit, the management assigned a responsible mission: to manually run through the interface of our web application, click on the buttons and pull errors out of there. Pretty soon I was assigned to invent test plans, and then they gave me an incredibly responsible task: to write a completely new framework for automatic testing.

How cruelly they made a mistake in my mental abilities! It turned out that testing is not a fucking muhra. The next few months had to go to blood and sweat, trying to take several directions by storm: simultaneously writing a remote test system to Java (opening parallel connections via SSH - first using a crutch as a Python call from Java and pyexpect to enter a password - it was impossible to authenticate on the keys, do not ask why). The servers didn’t have all the necessary software, and the admins didn’t want to install the necessary software, so I had to write a framework for building from the sources of the small GNU distribution directly in the home directory of the current user. The most monstrous are tests for performance, I did not understand them then, and I don’t understand them now (by the way, should there be a marketer as a particularly clever creature? :)

Then it was necessary to collect test results and visualize them in order to give them to the customer in PDF. As I remember, I chose the Concordion framework for writing and subsequent visualization - this is such a BDD framework for Java. Why not cucumber? He was not there yet. But in Concordion everything was so beautiful and elegant - the tests were implemented right in the middle of HTML. There were many tests, many had their own specifics, seasoned with OSGi-madness on the part of the backend. The developers hated me.

Years passed, and we have a bunch of new tools. In 2012, Cucumber came out for Java and a whole bunch of different BDDs, then the whole world was captured by Seleniums and Dockers, and their influence only grew over time. Unfortunately, by that time I had already quit testing and was simply silently envious of all these nishtyaks. Still, no longer do you need to script the constantly falling off SSH yourself, because Ansible can smoothly be used.

Similarly, conferences for testers changed: people increasingly introduced new tools and could already say something intelligible. If at the very beginning people were just talking about trying to introduce something new, over time, more and more reports began to focus on real-world methods of use, problems and solutions, details of the internal design of tools, and so on.

That is, the conferences themselves have qualitatively changed. Now they are about implementation and scaling. Every second word is about scaling. This may seem annoying, but, it seems to me, this is good - previously, one could only dream of scaling.

In parallel with the general evolution, the conferences of the JUG.ru Group also developed. The quality of reports, organization and video broadcasts, discussion areas.

And here I want to talk about the program, because both the eyes are a mirror of the soul and the program is a reflection of the surrounding reality. There, below, I will carefully write out all the main reports, but in the introduction I will focus on trends.

• First , it is necessary to note the strengthening of the direction of mobile development. What do Tinkoff, Yandex and Badoo have in common? They all came to Heisenbug 2017 Moscow. And they all make reports about mobile phones.

  
  

• Secondly , the eternal themes have not disappeared anywhere. Jenkins, flaky tests, Selenium - no escape from them. I would say some kind of Selenium domination is going on, it is present in every third report either explicitly or by hints. And still - Docker and scaling. What does the Docker do? Docker - always with it!

  
  
• Thirdly , there are quite non-standard topics. Vyacheslav Alenkov will tell about testing at Rosatom , more precisely, about the digital launch of nuclear power plants (nuclear power plants). How do you like this quality standard? Roma Poborchy with a report on A / B testing will provide an opportunity to feel like a little data scientists. And we also have an excellent report by Alexander Shukov from World of Tanks , suddenly, about World of Tanks. It is already intuitively clear that in such a popular game everything should be at a very good level, you can learn from their example. These people are not talking about the introduction of some kind of hipster technology, but about the successful solution of problems in large projects, and these solutions are tested by time and reputation.

I ended the bullet points in the list, so a couple more interesting in one paragraph! At conferences JUG.ru Group traditionally held reports with all sorts of puzzlers - that’s how they got to us . The report with puzzlers will be at the very beginning, in the morning, and who are used to waking up not because of coffee, but with the help of interesting tasks - you are welcome.

Of course, the reports are read by time-tested speakers like Nikolai Alimenkov and Artem Yeroshenko . Artem has an architectural report about tests, there are not so many such reports.

And at this conference there will be real stars, celebrities: Simon Stewart (the person who is the living incarnation of Selenium on Earth) and Alan Page (formerly director of testing at Microsoft, acting director of quality at Unity).

And of course, in the end there will be a noble afterparty! With fluids for ingestion, music, warm tube conversations - everything is as it should be. Just come, it will be cool.

An interesting point: before, they cursed us terribly, that we immediately take speakers after the conference’s closing and take them away somewhere. As a result, there was almost no speaker left at the party. In fact, this makes sense: someone needs to recover from a tense report, someone has an airplane in an hour. But in your outrage, dear reader, there is a large share of sense, and in general - the customer is always right. Therefore, this time we will try for the speakers to come to the party and use liquids with us. I can not give guarantees, but we will try.

Little game about this post.

In general, if a person once learned how to write code (for example, test code), it deteriorates from the realization of this strength. It becomes a cunning lazy creature who wants to automate everything that he sees. If there is a test - it must be automated! If there is a post - his uh ... also need to automate!

In the course of writing this post, I had to shovel the entire program of the conference - in some places to reformulate the descriptions of the reports, translate English texts into Russian, turn the resulting data into a habrapost, and so on. To do this, you need to obtain data on reports in a structured form. Such data are on our website in the “program” section, and I could easily take them from the database. Had taken care of this in the evening! The fact is that I wrote this post deep in the night, when sins scrape the soul most strongly, and the lines fall on the paper most smoothly. At this time, admins are sleeping, and if I woke up the admin with a request to give a dump of the base, the conversation would have turned out, frankly, intense.

So I picked up the web driver and wrote a simple table scraper from our site! The principle of its operation is simple:

• starts with SpringBoot (project generated using Spring Initializr - it's very important to be lazy),
• he downloads the page (using jbrowserdriver ),
• then all the pages of all reports on the links,
• neatly parsit all of this (in java objects of Talk type, the “Report” object with fields like “speaker”, “company in which it works”, etc.),
• and all the photos of the speakers are saved to the hard disk,
• applies patches with translations from Russian to English (simple ArrayList<Talk> in Java),
• transforms this into Markdown (using the Velocity template engine - yes, I'm a fan of antiquity),
• and the output generates a text markdown file for further processing.

A couple of notes. At first, I tried to use Unirest + jSoup , but it turned out to be some kind of thrash. Whether we have a highly intelligent JavaScript sits on the site, or in jSoup bugs, but at four in the morning did not want to search for them. But JBrowserDriver turned out to be a fairly stable thing and normally returns the DOM, received immediately after all the transformations performed by the browser after loading pages. Pictures are saved to the hard disk, and not directly to HabraStorage, because Habr does not give an API token, and hacking their API across their wills is ugly. The launch was tested against Java 9 with the --add-opens java.base/java.lang=ALL-UNNAMED and Java 8 --add-opens java.base/java.lang=ALL-UNNAMED .

The results of this senseless and merciless work can be viewed on my GitHub .

The main factor was that I, without thinking, began to write this without tests. As a result, the initial assessment of the project on writing a parser + obtaining the source code of the article quickly turned from 20 minutes to 2 hours.

It's a shame, very embarrassing. This is Oleg, and he is crooked. Don't be like Oleg, always write unit tests ! Especially when developing a compiler similarity (even as simple as a web page parser). On the other hand, should a marketing employee be a smart creature? In short, I can be forgiven. And you - no, that's it!

On this life-affirming note, we turn to the most important thing - to the program!

Program

The first day. December 8th.

Testing the browser performance of web applications (javascript, rendering, that's all)

Vladimir Sitnikov / Netcracker

Often, the word "performance testing" means only testing the server side. Much less often test directly the browser. In the simple case, we connect Yandex.Metrica and / or Google Analytics, and go ahead. But there is a nuance: in a corporate environment, sending data to NM / GA may not be available, and simply connecting NM / GA is not enough to collect the necessary amount of information about the application's performance. In the report we will look at how to measure the duration of operations in the browser. Find out why the times obtained from Selenium show the weather, find out what the benefits of Selenium tests can be when measuring performance. Let's look at boomerang.js and find out what moments to pay attention to when integrating such libraries into a project. The report does not address the issues of optimizing the browser / server.

Vladimir Sitnikov

For ten years, it has been working on the performance and scalability of NetCracker OS - the software used by telecom operators to automate network management processes and network equipment. He is interested in Java and Oracle Database performance issues. His responsibilities include planning load measurements, analysis and explanation of the results. Vladimir is a committer in Apache JMeter.

Selenide puzzlers

Alexey Vinogradov / Radio QA; Andrey Solntsev / Codeborne

Puzzlers is a quiz report in which all viewers will become direct participants of the show. The presenters came up with half a dozen interesting problems for you, almost all of which are taken from the actual design practice of the authors. For each problem you will be shown a list of options and give convincing arguments for each! You just need to choose the right one. Sounds easy? And we are sure that the tasks will be more complicated than it seems at first glance. Want to bet? :) The theme of this Puzzler is a Selenide UI testing framework. Questions are available for beginners with superficial knowledge of the framework, but even some professionals will break their teeth about some of them. We expect that the report's visitors have already dealt with Selenide at work or have been actively exercising at home, you do not want to just guess? After each assignment, the presenters will of course disclose and explain the correct answer, as well as demonstrate the practical application of the knowledge gained from the decision.

Alexey Vinogradov

He has been working in IT projects in Germany for more than 15 years. Advises on testing and automation. Junior developer framework Selenide. Founder and one of the leading podcast Radio QA.

Andrey Solntsev

Andrei is a developer at Codeborne Estonian company. Author of the Selenide framework, organizer of the Devclub in Tallinn, a frequent speaker at conferences. An ardent supporter of extreme programming, automated tests, pair programming, and clean code.

Enterprise Automation with Selenium and Selenium

Michael Palotas / Element34 Solutions GmbH

As Selenium becomes the W3C standard, more and more organizations are beginning to use it as the main test automation tool. Most teams focus on writing tests and dealing with problems with the help of Selenium itself. However, in our experience, Selenium is the smallest problem when creating a test solution from scratch that is suitable for use in enterprise projects. This report will look at many real-life examples of how test automation with the help of Selenium eventually turns into a project to create a large software product. It is necessary from the very beginning to understand that this is precisely such a software project, and conduct it accordingly. The report will show the main problems that prevent teams from building expandable and reliable solutions based on Selenium tools. We will also see how you can successfully apply the principles of lean in creating a similar solution based on Selenium.

Michael palotas

Founder and CEO of Element34 Solutions. One of the developers of Selenium Grid. Former Director of Quality Engineering at eBay. For more than 10 years, Michael has been developing an testing approach at eBay International, being in the position of head of Quality Engineering. He was the manager responsible for transforming eBay International from the abyss of Waterfall into a very flexible organization using new ways and approaches, especially in the field of engineering practices. Before joining eBay, Michael held leading positions at companies such as Ericsson, Nortel Networks and Intel.

Development and Testing with Google

Vsevolod Brekelov / Grid Dynamics

A report for testers and developers who are interested to know: what rakes can arise when working with Google Cloud Standard Environment, how to avoid them (test), what Google tools you can take and use in your projects. You will also learn a little more about GAE, Memcache, Task Queues, Objectify, Protobuf, Bazel.

Vsevolod Brekelov

Over 5 years in software testing / test automation. The last year is working Full Stack Developer / Tech Lead. Experience in building test automation from scratch for mobile, desktop, web projects (mainly for financial companies). He likes to participate in hackathons and work with smart colleagues. He conducted many interviews (more than 200, and has already stopped counting) for test automation engineers, developers, and analysts. The last few years working in the company Grid Dynamics. Now living in California, working on a contract with Google.

Mobile testing automation with native tools

Ekaterina Bateeva / Tinkoff Bank

The popularity and convenience of mobile applications is growing, and now in many areas, users are much more likely to use mobile devices than personal computers. Therefore, automated testing does not stand still, and there are already many solutions that are suitable for most tasks. To test the Tinkoff mobile bank, we were looking for a reliable, configurable tool that would also allow us to effectively solve our problems. After trying Experitest's SeeTest Automation solution, we decided to turn to the automation tools provided by Android and iOS developers themselves - UIAutomator, Espresso and XCUITest. In this report, we want to tell about our experience in applying these frameworks, what we found the pros and cons, what difficulties we faced and how we solved them. The report will be of interest both to beginners in the automation of mobile testing, and to already experienced specialists. It will allow them to expand their horizons, to assess the possibility of using one or another framework in their project or to make a choice in favor of other decisions.

Ekaterina Bateeva

Senior Automated Testing Engineer at Tinkoff Bank. Works in testing since 2012, engaged in the automation of instant messengers, web services, banking systems. He has significant experience in functional and integration testing. The last few years engaged in testing mobile applications.

Report from Sergey Egorov

Sergey is an active member of the open source community, a member of the Apache Software Foundation and a contributor in various projects (Apache Groovy, TestContainers, Spring Boot, JBoss Modules, Zipkin and more). He is also one of the founders of the Russian DevOps podcast “Two Devs One Ops”, where he shares his experience in DevOps, cloud solutions and modern infrastructure solutions like Docker (user since 2014). Sergey will give a talk at Heisenbug 2017 Moscow, but the topic is still doped.

Pandora's White Box

Nikita Makarov / Classmates

In the testing industry, a lot is said about the “black box”, but occasionally “white” is mentioned. This is due, among other things, to the fact that testing the “white box” has always been considered the prerogative of programmers. In our report we will answer these and other questions with examples and a clear demonstration.

Nikita Makarov

He worked in outsourcing and grocery companies. He was engaged in the automation of embedded operating systems based on Linux, integrated VPN solutions for business, software and hardware systems. Since January 2012, he has been the head of the test automation group in the Odnoklassniki project.

Measure energy consumption

Alexey Lavrenyuk / Yandex; Timur Torubarov / Yandex

Previously, we were able to test only the performance of server applications, now we are mastering mobile applications - this is a new trend. In particular, we test the power consumption of phones. We will tell you how you learned how to collect the energy consumption metric in the hardware way. We collected a small Arduino-based circuit that measures current, and wrote a library to work with it. We posted the library in open source (github.com/yandex-load/volta). We will tell you how to prepare phones, assemble boxes for measurements and how to use the library.

Alexey Lavrenyuk

Developer in Yandex in the service of load testing. Makes tools and services for performance testing. Participates in the development of open source tools for load testing, such as Yandex.Tank (github.com/yandex/yandex-tank) and Pandora (github.com/yandex/pandora), as well as the service for load testing - Overload (overload. yandex.net).

Timur Torubarov

Engaged in productivity in IT for 8 years. 4 years in telecom (MTS and related web / wap / sms-services), 4 years in Yandex. A brief history of life: "... and unexpectedly for us gave a much more load than we wanted ...".

How to check the system without running it

Andrey Satarin / Yandex

The systems we develop become more complex every day. And it seems that there is no salvation from the omnipresent complexity that penetrates everything. One aspect of this complexity is configuration. On the one hand, the configuration greatly affects the stability and availability of the system, on the other - very little attention is paid to checking its correctness. The report will tell you how we test the configuration and how it was useful in our project. This report will be useful to anyone who wants to learn a simple way to increase the stability and availability of the system in production.

Andrey Satarin

Engaged in testing distributed systems in Yandex. In his career, he managed to work in completely different projects: he tested the game at Mail.ru, the cloud detection system at Kaspersky Lab, the system for calculating currency prices at Deutsche Bank. Interested in testing backend and distributed systems.

How to perform many UI tests in parallel using Selenium Grid?

Mikhail Podtserkovsky / Avito

How to perform many UI tests in parallel using Selenium Grid? No, Selenium Grid is not able to perform a large number of tests in parallel. Want to register a really large number of nodes? Well, try it. Want speed? It will not be - the more nodes registered on the grid, the slower each test is performed. Want fault tolerance in case the Grid stops responding? Also not - you can not run a few replicas and put a balancer in front of them. Do you want to update the Grid without downtime and so that tests that are currently running do not fall? No - this is not about Selenium Grid. Do you want not to keep thousands of Seleniums of different configurations in memory, but to pick them up on demand? Will not work. Want to know how to solve all these problems? Then we invite you to listen to the report.

Mikhail Podtserkovsky

Mikhail works at Avito, where he develops testing tools.

Automate the impossible: blending the best of Android drivers

Rajdeep Varma / Badoo

Now we are on the verge of amazing changes - more and more mobile developers are looking at tools for automatic testing with open source. It may be Appium or Calabash, or something else: they are all good, but it is also true that everyone has important limitations. How do we get the most out of every tool? How to identify their best side? Despite the fact that the selected tool can work quite well at the very beginning, when you have just started using it, everything can change dramatically if your business requirements change. Fortunately, the open source world is generous enough to give you the opportunity to go and overcome their limitations on their own. This report is about how the speaker dealt with this problem. Despite the fact that he had been working on Calabash-Android for several years, new business requirements forced us to add additional features to the driver. This is a report on a specific problem, how it was solved, and what opportunities we generally have in solving such problems.

Rajdeep Varma

Senior Test Automation engineer in Badoo is currently working. There is a real passion for test automation, as they say - automates everything that moves. Engaged in the automation of many mobile applications in many subject areas for more than seven years. Rajdeep has written several open source utilities, including " parallel_calabash " and " nakal " - they are designed as RubyGems and are intended for testing mobile applications.

Tester Tools

Julia Atlygina / ALM Works

The purpose of the report: to show the tools that can be used on the same day, which will speed up the work of manual testers, as well as analysts, developers and others. Many people think that manual tests can only be accelerated by automation - but this is not quite so :) In this report, I would like to show simple tools that would allow you to spend less time and at the same time help increase the test coverage. Target audience: manual testers, as well as all those who at least sometimes need to analyze and do something with their hands.

Julia Atlygina

In testing over 9 years, the last of which spent in ALM Works, developing plugins for Atlassian JIRA and Confluence. The role of the tester combines with the roles of the Product Owner and SAFe consultant. If you have questions about JIRA, feel free to ask!

The (Ab) use automation test

Alan Page / Unity

If you are a tester writing code, then most likely your work includes test automation — in particular, writing tests that automate the user's workflow. Being a veteran of automated testing (for twenty years now, and this figure continues to grow!), Alan Paige has seen a lot of teams whose attempts to do automation at least once and for all turned out to be completely unsuccessful. Nevertheless, he managed to take part in the few teams that did it all, and on this report he will share his wisdom and the patterns that really work (and also share a bunch of patterns that are worthless). Alan will show successful automation strategies, how to deal with flaky tests, guide the dangers of UI automation and give a number of other tips, based on many years of experience in test automation in a variety of large products. You can be an advanced tester, or on the contrary, just start a career - in this report you will find tips for everyone, and such tips that can be put into practice almost instantly.

Alan Page

Alan Page has been working as a software tester for about 25 years. He was the main author of How We Test Software at Microsoft, and was involved in the creation of "Beautiful Testing Software Test Automation" . In addition, he writes articles on various engineering topics in his blog, his posts can be found everywhere on the Internet. His latest "book" is a collection of essays on automatic testing under the common name "The A Word". Alan joined Microsoft and became part of the Windows 95 team, and has since worked on multiple releases of Windows, on earlier versions of Internet Explorer and Office Lync. Including, Alan worked for two years at Microsoft as director of testing. In January 2017, Alan left Microsoft for the position of director of quality at Unity.

Badoo geolocation testing: bumps, stones, crutches and selfie stick

Alexander Host / Badoo; Nikolay Kozlov / Badoo

Working with locations is quite nontrivial, and in the process there are a lot of things that are difficult to predict in advance. We will try to highlight the problems and nuances that we have encountered, give advice, talk about the tools used. There is still not much information on this issue - we will be happy to share it with everyone.

Alexander Host

In testing for 7 years, the same amount has been testing mobile applications: slightly_smiling_face: Responsible for all manual mobile testing at Badoo. He loves all sorts of gadgets (one of the reasons why he went into mobile testing), testing using the free search method, detective work on finding complex problems.

Nikolay Kozlov

5+ years of Android experience.

Testers' home classification from a developer’s point of view

Nikolai Alimenkov / XP Injection

Testers often talk about confrontation and conflicts with developers. But there are teams where everyone lives in peace and harmony. Apparently, something is wrong here? I would like to talk about how testers see the developers themselves. The report will be a fun classification. In addition to the famous monkey tester, the fly tester, the Nazi tester, the panda tester and many other heroes will be presented. You can once again think about how you are seen from the side and, perhaps, make a difference for the better. The report will also be useful to project managers and team leaders. You will be able to quickly recognize those or other patterns of behavior testers and take measures to improve the level of teamwork. Come, it will be interesting!

Nikolay Alimenkov

Practicing java techlide and Delivery Manager. Java, Agile- . Java 12 , . . XP Injection. Selenium Camp, JEEConf, XP Days Ukraine IT Brunch. « ».

Second day. 9 .

, c Jenkins Pipeline

/CloudBees

Pipeline Jenkins, . . Java- , Pipeline- QA . Docker, Maven, JUnit, FindBugs, overity, . , , .

CloudBees, core team Jenkins. Since 2008 he has been engaged in automation, infrastructure and framework building for large software and hardware projects using Jenkins and dozens of other tools. , Jenkins, .

/

, . , , , , , . , , , , ? , . , , , , data scientist' . - .

2004 Sun Microsystems, JDK. Intel, , Java. , . 2015 , .

Flaky tests

/Codeborne

Flaky tests — . , — . . . . flaky- . , , . , — , .

— Codeborne. Selenide, Devclub, . , , .

staging differential testing

/Deutsche Bank

— , , . — . , ? (Prod Parallel) — , , , , . , , . QA, . , , . . , QA-.

Deutsche Bank, server-side- . — . Lomonosov. Intel , Intel C/C++ open source Clang-. , , .

Scaling Selenium

Simon Stewart /The Selenium Project

, Selenium. ? ? . — . ? ? , , , . , . ! . , .

Simon Stewart

— WebDriver Selenium. , Facebook, Buck, . , Facebook, Google — ThoughtWorks. He saw really a lot of code. , -- .

-

/

, -. <-> <-> . , , . , - Java.

8 -. During this time, he worked in different teams and in different roles: testing automator, testing tool development team manager, testing automation team leader. (Selenium, HtmlElements, Allure, Jenkins). Java, Groovy.

, «, - »

/

— , — . « № 392904» «, - », , . , , , . . , , . .

5 . : , , , .

«»

/

, . , . . , . , . , . , - , . «». « 2 »? : , V- . , . , , . , , «» .

-.

Selenoid — UI-

/EPAM Systems

. , Selenoid . Selenium Server Selenoid.

Test Automation Engineer EPAM Systems. 10 . , , , .. .

Loading time testing and results visualisation of web games

Andrejs Kalnacs /Evolution Gaming

— , user experience, - . -, , , (JS/HTML5, Flash) (Android iOS, ). : -, , Selenium WebDriver, Selenium Grid, Appium, Zaproxy, Linux, Jenkins Docker — , , . , , , InfluxDB Grafana.

Andrejs Kalnacs

Andrejs , - . , — , , , , . «Lead Software Developer in Test» Evolution Gaming, - — , — , . , , — , , , .

BDD

/EPAM Systems

Continuous Delivery. , , , ? , , UI? , - BDD- , , , .

Lead Software Test Automation Engineer EPAM Systems. BDD. , e-commerce-. BDD- DevOps- -.

World of Tanks:

/Wargaming.net

GameDev World of Tanks. , , ( -), «bot-net» – «World of Tanks» Python, . , . QA GameDev, , WebDriver, JUnit, PyTest .

2011 Wargaming.net World Of Tanks QA-, « » . 2 .

: Truths about technical testing

Alan Page /Unity

( ) « ». , — . , , , , - . , , , . , .

Alan Page

25 . «How We Test Software at Microsoft» «Beautiful Testing and Experiences of Test Automation: Case Studies of Software Test Automation» . , , . «» «The A Word». Microsoft Windows 95, Windows, Internet Explorer Office Lync. , Microsoft . 2017 Microsoft Unity.

Conclusion

, « ». , , — , . — « , », : -. 8-9 !