Practice writing tests. Yandex lecture

Happy holidays, friends! If you don't mind learning something new during the holidays, give a lecture by Kirill Borisov, developer of Yandex authorization systems. Kirill explains how to build the process of testing Android applications, introduces modern tools and the specifics of their use.


“Before we go ahead, let's do a little opinion poll.” Who of you knows what tests are? Who writes the tests? And who knows why he writes tests? About the same people.

Those who are familiar with hearsay with tests and did not put their hands on them, I want to present an example of the simplest test.

')
As you can see, do not worry. This is the simplest test that checks that the laws of mathematics have not changed yet and 2 + 2 is still equal to 4. That's all. You have a full test before your eyes.

In fact, a test is just a function in some programming language. In our case, this will most likely be Java, although it may be Kotlin, etc.

The test is run by a certain software package, a test framework that takes care of all the hard work of finding, running, processing results, etc. The most frequent is the jUnit package, which will be discussed further in our lecture, but nothing stops you from using some other packages or writing your own.

Tests are grouped into classes. Decent function must live in the classroom. Then these classes are broken down into different categories of tests, which we will discuss later.

Tests are grace and benefit. First, they will free up your testers from the routine tasks of verifying already fixed bugs, the so-called regression, and, in turn, will help them reproduce complex cases that require a large set of actions and are amenable to automation.

But you say: wait, what testers? I am a simple independent developer, I am writing an application, I post one, I earn money alone. I do not want to upset you, but you are the tester, simply because you will still have to check how the application works, stick with the main usage scenarios, etc., anyway. And this poor tester will be helped by auto tests.

Secondly, they will increase you, the developer, confidence in your code. Knowing that a soulless computer written in your code checks that this all works as expected will help you not to worry about it and freely develop the code. As soon as something changes in an unpredictable way, a red flag will immediately jump up and you will understand what needs to be fixed.

In fact, this leads to the fact that you improve the quality of the code. As you cover your code with tests, as you rework it so that you can test it, you will suddenly notice that the code is becoming more and more pleasing to the eye, more and more easy to read and understand. You can even involve other programmers who will also read this code and understand that this is a wonderful piece of programming art.

But most importantly, they help you maintain compatibility. As your application grows and blooms, you will somehow hit other parts of the code that other people can rely on. Imagine that you are not alone in the team, you have the same fellow developers, and they write their modules, focusing on how your modules work. If you are covered with tests, if you have ensured that it is still working as you planned it, as soon as you break something, you will know about it right away. Thus, you will help them not to think that suddenly he broke something today, I will go recheck. In fact, you place all your concerns on a soulless computer, leaving yourself just for creativity.

Unfortunately, like everything else in this world, nothing comes for nothing. Everything has its price. I think it is obvious that you will have even more time to develop. From this you can not escape. Tests you have to write, you have to think about them, have to test them, and all this takes precious time. But as well as investing in something good, it will pay off sooner or later.

Secondly, you have to improve skills. We all have room to grow, including in technical skills. As you write tests, you will have to learn new tools and techniques. But all this is good for you.

And the worst thing that can scare not only you, but also your manager, may need refactoring. A terrible word that strikes terror in any person planning to release software. Unfortunately, tests are not a magic wand. In order for them to work, you will have to rework your software code, for example, to use some kind of software interfaces that were previously unavailable, or make it more modular to make it easier to test. Unfortunately, all this all requires time, money and effort - the most valuable in our industry.

And in the end, in sum, all this complicates the release of the code. When you could take and simply compile the code, send it to the Play Store and go eat pizza, this time you will not succeed. Most likely, you will have processes such as running the code that checks your code, viewing the test report, sending it to the Continuous Integration server, etc. Unfortunately, this is the price you have to pay, but like all the previous ones, it in the future will pay off.

The most intricate question associated with the tests, which I hear most often when I try to push through this idea: how can I convince my manager? Because these people do not understand why we need tests. They think that again these programmers have come up with something, they want some tests, we need to do features, release.

Unfortunately, there are few arguments for this, but there is a verified list that will always help you.

Firstly, they are very happy when there are less bugs in your final product. As you cover your code with tests, the number of errors, including stupid ones that slip completely by chance, will decrease. After all, you will detect them and correct them in a timely manner. This is what leads to the fact that you are accelerating the search for the causes of errors, and as soon as you are rescued by managers with shouts: “Chef, everything is lost”, you immediately look at the list of passed and filled up tests, you understand where and what broke correct and save the day.

All this leads to the fact that money and time are saved. It pleases almost everyone. As you spend less effort finding errors in your code, less effort reworking this code and fixing it, you can spend more time on really interesting things: developing new features, bringing more money in less time. In theory. I do not know how it works specifically in your case, but it should work something like this.

And most importantly, think about it, you come to a new company and ask if you have any tests? They say: "Yes, we have a 100% coverage, everything is checked." And you think that I have come to the right company. Agree, very cool. And when young proud developers come to you, you will say - and we have tests here, 97% coverage, we all test here. And they will look and understand that yes, this is a cool company, a cool development team, I want to stay with them.

We proceed to a specific theory. Let's look at the already mentioned tests in the section.


Before you is the skeleton of almost any type test in a vacuum. It consists of several blocks, which are necessarily marked in yellow, and gray - for those who need it.

The most important is the name of the test. I met a lot of people who think, why give the tests some meaningful names? Test1, Test2, Test3 are already good, they differ and well.

In fact, the name of the test as the name of the book. This is something that should fit as much as possible in a very short period of time. This is what you see in your report, what will glow in your code editor. By the name of the test you should already get a rough idea of ​​what it checks, what is happening in it. Therefore, it is worth making an effort and thinking about how to put in the same sentence of three or four words the meaning of what you are testing.

Next, their required blocks are committing the action. To check something, we need to do something. In this block, there is some kind of impact on your system. For example, you pull the function, start the service, open the window. Having obtained the result of this action, you proceed to the main, sweetest part of any test - checking the results. This is where the heart of the dough is. This is where you check that the world has changed in the way you expected it to. What opened the window, and not deleted all the files from the device. That you started the video, but did not erase the memory, etc.

And what do the gray blocks noted here, the preparation of the environment and the release of resources, carry? In fact, these are the boring duplicate parts that will sooner or later begin to appear in your code.

If all your tests are connected with, for example, files, if you create the same file in each test, open the same files, then close them, delete, why should you carry it all from the test to the test? You can simply use any of the tools of your test framework, and put them into a separate small function that will be called before and after your test.


In fact, this may not always come in handy. Your test may well go without these optional blocks. But if something happens, know that it does not matter, you just take them to a separate function, and everything works.



Thus your test remains solely from the required parts, simple and elegant.

We understood what a test is and how to write it. And what will we test?

There are several theories about how to test using autotests. Almost everyone agrees that there are a number of such typical cases. First, the first is the happy path, the typical path for your code. This is what you know that I will click on the button and a window will appear. You first check it, that you really pressed the button, and a window appeared. Or entered your name, and it was highlighted in a special way. Everything, you know how it should work, you expect it to work that way, but just in case, write a test for it. Because if suddenly it breaks, it will be sad.

Then you check all possible marginal cases. For example, if a person enters his name in Japanese hieroglyphs, or if he suddenly enters Emoji in the age box. What will I do in this case? Will my code handle this?

You write for each such case a separate test that checks that your application will act in a certain way, for example, throws out a window with an error, or simply ends and refuses to start more - everything is up to you.

Then you go to the most banal. What will happen if I start stuffing null anywhere in my code, wherever I think about it. Aha, and I have a function - I will send null. I have an optional argument - I will send null. and so on. Your code, in a good way, must be resistant to the fact that one of your arguments is suddenly emptied. And at the very least, I think it’s worth touching on the script when nothing works. Your application, designed to send your photos to Instagram every five seconds, suddenly realizes that it does not have a network. And there is no camera. What to do? In a good way, it is necessary that your application in some meaningful way makes it clear to the user that, excuse me, I will not work. This is what you should test, that in the case when everything went wrong, your application still works in some expected way. Nothing so upsets the user as a NullPointerException window with an error from Android or something like that, it's terrible to think.

When to test all this? How do we start development or when have we finished? On this account there is no consensus, but there is a set of established concepts. First of all, there is no point in writing tests when the code of your application changes literally every hour. If you and your friends are captive to the muse, and literally every hour the concept changes, you change the entire structure of the program, the architecture floats, if you still write tests for this, you will have more time to constantly rewrite these tests after your creative thought.

Well, your code is already more or less settled, but your designer also fell into the clutches of this terrible woman (muse) and starts pulling your UI every minute - oh my goodness, the new design, we support the new concept.

Writing tests that interact with UI is also not a good idea at this point, because you will also have to rewrite them from scratch.

As soon as your application code has stabilized, the UI no longer skips across the screen, is it worth writing tests further? Yes, if only because sooner or later so-called regressions will be found in your application, there are bugs in common people. This is something that means a malfunction in your application. For example, your name is displayed from right to left, because we accidentally thought that we were in a country with Arabian cuneiform, etc. This is a regression, this is a bug, but you need to write a test to check that in the future under these conditions your application will still work in the expected way.

Here are three cases where exactly you should write code. When it does not swim, when it does not swim UI, and when you already have a bug. From this you will begin your journey.


Tests are divided into several categories. Such a pyramid of tests shows an approximate order of the plan in which to start writing tests in your application. The basis of this pyramid are the so-called unit tests. This is the lowest level salt of the earth. These are low-level tests when you are testing individual units in isolation from each other.

But what is a unit?

Scientists are still arguing on this issue, they write scientific work. Everyone decides for himself what is a unit in his application. Most often, a class is selected as a unit, and a function of this class, methods of this class, and various conditions for its interaction are tested. We assume that a class is some kind of self-contained entity, for example, a class that calculates the length of a string or an encryption class, etc. It is usually not related to any other classes in a relatively explicit sense, and can be tested.

You will have the majority of these tests. These are the tests you will run most often. If you have the habit of running every five minutes - this is normal. We do it, and everything goes well.

Unit tests are designed to control you in the first place in the process of writing your code, precisely because they are the lowest-level and should pass as quickly as possible.

As you run them, for example, you pressed Ctrl + S, and right there you got rid of the tests, and you immediately noticed that something was broken. Agree, it is better to detect a mistake before it has yet had time to penetrate somewhere else.


Consider an example of such a unit test. Consider our favorite class of static utilities. There is a class that contains exactly one function that checks our hypothetical application, whether a user has entered a strong password, whether hackers crack it or not. This simplest function contains three basic conditions, three basic invariants that our strong password must not contain less than seven characters, must contain at least one capital Latin letter, and at least one digit. Otherwise, it is chickens to laugh.

If all these three conditions pass, we return that all is well, register the user, we move on.

How are we going to test it? Our unit, we choose this function isStrongPassword, and we will test each of these three cases separately.


We begin with the first condition that strings with a length of more than 6 characters must be passed to our functions in order for them to be considered successful. In our first test case, we check that if we pass lines that have less than seven characters, our function will return false. The assertFalse function is responsible for this, which will throw up its hands in a panic and stop the whole testing process if, instead of false, it suddenly returns true as an argument.

In the same vein, we check our main cases, and check one counterexample, that if we still pass our function to a length of more than 6 characters, it also returns true. Such a test case in a vacuum. We checked some conditions that caused the fall of our function. We checked that if we pass the expected parameters to it, it responds in the expected way. And in the same vein, we test everything else.



We have a separate test case for the condition of checking that there is at least one digit in our password. We have a separate test case to check that there is at least one letter in our password. And you ask, where is the fourth test case? We had four ways out of the function there. In fact, we have already checked in the previous three test cases that if we pass a password that meets all of our conditions, then we will return true anyway.



Let's look at the main star of these tests, at functions beginning with the word assert. They belong to a class of functions called asserts. These functions are just auxiliary tools, represented by the jUnit test framework, which simply help you to express your intentions. , assertEquals, , , . — , , . assert = null, not null . .

. , . , , .

, , , .

AssertJ.


, , . — That(count).isGreaterThan(original). , assert true a < b. , .

- , AssertJ . , , count. - . , , , 1, 3, 4. . AssetJ : assertThat(counters).extracting(“count”) .contains (1, 3, 4) .doesNotContain(2). , , , , , . , .

AssertJ , , .


Hamcrest . — , . AssertJ, , . , , - , .

, - , . counters , . , , .

.

-, - .

-, , , , . , ? , , - , . .

, , , , -, . , , .

, . , -. -, , , .

— , , . , , , , . .

, , . , - , , , . . . , -, .

— UI-. , , - , , . , . , .

, , , , , . Play Store, , « , , ». , , , , , , . Sadness What to do?

UI-, . , Instagram, . , .

, , , . , , . . , , , , , . -, , , .

, , . , .

, .

, , , , ? , , ? . . , .

, .

-, . , . -, , , — .

. , . , « » « », , , , , .

, . . , , , , , , . .

. . , , , , .

, — , , — , — . ? , . . Does not work. Sadness , .

. , .

, , , , .


-, . , . , , , — assertThatNoFileDownloaded.

? , , , ? , ? , — - . , , , , , .

, . «, , 2017 , , , , ». , ? ?


.

, . , . , , . , , , , , . .


, , . , -? , . , . , , .

, , . : for, - . . , , . , , for , ?

, , — . , , .

, , , , , .


for Hamcrest, : , . , , , false. , — . , , , — .

— «» .


, . , .

, ?

, . , , - . , 10. , , — . . , . , , , .


« », , . . . , , .



, , 1 3, , 10, .

. Android.

Not so simple. Android — , Java. — , , .

. .

— , . , Java-, Android, , , . Windows, Linux, Android. , IDE, .

- Android. , , . , API Android. , , . , , . , Android? .

. , , , , , . , , , APK, , , , , .

, test suite . , . , Continuous Integration, - , , . .

. - , , Windows Linux, ? , , .

- . . , , 100 , . , , , - . , .

, .

, . , . Python, , , , , , . ? I do not know.

- , , , , . , , .

Android, , PackageManager, , , .

— . , , , ? , , , . . , . , , , , .

, .

Mockito, . «» , , . , . , , , . . , , , — 42. — null. etc.

. .

, . .


, . . Restaurant, — getFreeSeats getOccupiedSeats. .

, , , issued preferences, - , , Airbnb . . , , . . , . , , Restaurant, -, , , .

, , getFreeSeats — 42. getOccupiedSeats — 56. . - «», -, , , , — 42 56. , , , , .



verify , . , , , , - .



YandesPackageCounter, , . — , “yandex”. , , . , . PackageManager. , , PackageManager .



. PackageManager.

. , - . ApplicationInfo, , - , , PackageManager, , , .



, , getInstalledApplications GET_META_DATA.

, . YandexPackageCounter, , , , , , . .

, . . .

Android , SDK , , , . , , , , .

- Robolectric. API Android. , , SDK Android, , Android, .

, , , , Robolectric, , , , .

Robolectric PackageManager , , - , .

. .

, . , SDK . , Robolectric, 100 . , , , , .

. , , .



.



, RobolectricTestRunner. , , , . RuntimeEnvironment.application.getPackageManager, ShadowPackageManager, ShadowPM. PackageManager, , , , application info . .



, , , . PackageManager Environment, counter, . . , , , .

Android UI . ?

-, , UI, . , , , , , , . , , , , , , . , , . . — , , , , ! , , ?

, UI . , - - . , , , .

Espresso. , . Android Testing Support Library, API . , . , . UI , , WebView. , . . — , .

.


, , isStrongPassword, true, false, . . , , , UI. , «» . , .


? -, . Espresso , . , , .



. AndroidJUnit4, CheckerUITest. , . , string.xml, , .

— Rule mActivityRule. Rule — JUnit, , . , , . ActivityRule Activity, , , , . ? .



— badPasswordDisplayed. . -, , , R.id.password. , . , . , , , , . , .

. « ». id.check_strength, — click. . , R.id.result, , , , . Hamcrest. , matches(withText(WEAK_STATUS)). , .

, , . « - , null, -…»

, Espresso , , , . .

Espresso, UI , , , Espresso Test Recorder.

, Android Studio . , , , , -, . Run Run Debug, .


, , , , assert. -?


:





, — , , , , . .

, - , , , . . .

. - , , .

? , Android. ! UI , -. , ? , , , , . . .

, , , Robolectric, . ServiceTestRule. , Android Testing Support Library, , , . , , — , , - -, , , .

?


, , Android Testing Support Library . , . , - -. .

Rule, ServiceTestRule, , , , .

Intent, . - , .



getService, , , , , , , integer.

, . , .

Android. , . , ? , ? .

, -, Continuous Integration. , , . .

- , , , Jenkins, TeamCity — , , , - pull request github -, , , , , .

, , , , , . , .

Continuous Integration — , . , . , , , , , , , , . , Continuous Integration , , , , — -.

. Continuous Integration , , . , , , , , .


, , , — Test Driven Development. , , . , , . . , , , . , , . . , , ? ? . , , , isStrongPassword, - - , . , , , ? , , . , , , . . Fine.

. . , , . , … . But in fact, there is a solution. - , . , , .

, , , … -.

, , , , .

, , . . , . : , , , , , … .

. , , , , , , . : ? , , .

, , ? Why do they need us? , , , . , . , , , , , , - , . , , . , «» OAuth , « », . ? . . , , , . , , , . , , .

, , , , , , . , , , . .

, , , , — 14 , … , . But how? , ? 4 . , , .

, — . , , . , , , , , , , . .

, , -. , .

, , . In theory. , .

, , , — , , , , . . , , .

, , , - . , , . , , . , . . . , .