Acquaintance with testing in Python. Part 1

All good!

From our desk to yours ... That is, from our Python Developer course, despite the rapidly approaching New Year, we prepared for you an interesting translation about various testing methods in Python.

This guide is for those who have already written a cool Python application, but have not yet written for
them tests.
')
Testing in Python is an extensive topic with a lot of subtleties, but it is not necessary to complicate things. In a few simple steps you can create simple tests for the application, gradually increasing the complexity based on them.

In this tutorial, you will learn how to create a basic test, run it, and find all the bugs before users do it! You will learn about the tools available to write and run tests, check the performance of the application, and even look at security issues.



Testing Code

You can test the code in different ways. In this tutorial, you will learn from simplest to advanced methods.

Automated vs. Manual Testing

Good news! Most likely you have already done the test, but have not yet realized this. Remember how you first launched the app and used it? Did you test and experiment with the features? This process is called exploratory testing, and is a form of manual testing.

Research testing - testing that is conducted without a plan. During research testing, you explore the application.

To create a complete list of manual tests, it is enough to make a list of all the functions of the application, the various types of input that it accepts, and the expected results. Now, every time you change something in the code, you need to re-check each of the elements of this list.

Sounds bleak, right?

Therefore, we need automatic tests. Automatic testing - the execution of a test plan (parts of an application that require testing, the order in which they are tested and expected results) using a script, and not by human hands. Python already has a set of tools and libraries to help you create automated tests for your application. Consider these tools and libraries in our tutorial.

Modular Tests VS. Integration Tests

The world of testing is full of terms, and now, knowing the difference between manual and automated testing, we will go down a level deeper.

Think about how you can test the headlights of a car? You turn on the headlights (let's call it the test step), get out of the car yourself, or ask a friend to check that the headlights are lit (and this is a test proposition). Testing multiple components is called integration testing.

Think of all the things that need to work correctly for a simple task to produce the correct result. These components are similar to parts of your application: all those classes, functions, modules that you wrote.

The main difficulty of integration testing occurs when the integration test does not give the correct result. It is difficult to assess the problem without being able to isolate the broken part of the system. If the lights are not lit, the bulbs may be broken. Or maybe the battery is dead? Or maybe the problem is in the generator? Or even a failure in the computer machine?

Modern cars themselves will notify you of the failure of light bulbs. This is determined by the unit test.

The unit test (unit test) is a small test that checks the correctness of a separate component. A unit test helps isolate a breakdown and fix it faster.

We talked about two types of tests:

1. Integration test, verifying the components of the system and their interaction with each other;
2. A unit test that checks a particular component of an application.
3. You can create both tests in Python. To write a test for the built-in function sum (), you need to compare the output of sum () with known values.

For example, this is how you can verify that the sum of the numbers (1, 2, 3) is 6:

>>> assert sum([1, 2, 3]) == 6, "Should be 6" 

Values ​​are correct, so nothing will be displayed in the REPL. If the result of sum() incorrect, an AssertionError will be AssertionError with the message “Should be 6” (“Should be 6”). Check the assertion statement again, but now with incorrect values, to get AssertionError :

 >>> assert sum([1, 1, 1]) == 6, "Should be 6" Traceback (most recent call last): File "<stdin>", line 1, in <module> AssertionError: Should be 6 

In the REPL, you will see AssertionError , since the value of sum() not equal to 6.

Instead of REPL, put this in a new Python file called test_sum.py and execute it again:

 def test_sum(): assert sum([1, 2, 3]) == 6, "Should be 6" if __name__ == "__main__": test_sum() print("Everything passed") 

Now you have a written test case (test case), statement and entry point (command line). Now this can be done on the command line:

 $ python test_sum.py Everything passed 

You see a successful result, “Everything passed”.

sum() in Python accepts any iterated input as the first argument. You checked the list. Let's try to test the tuple. Create a new file called test_sum_2.py with the following code:

 def test_sum(): assert sum([1, 2, 3]) == 6, "Should be 6" def test_sum_tuple(): assert sum((1, 2, 2)) == 6, "Should be 6" if __name__ == "__main__": test_sum() test_sum_tuple() print("Everything passed") 

Having test_sum_2.py , the script will test_sum_2.py error, since s um() (1, 2, 2) must be equal to 5, not 6. As a result, the script produces an error message, a line of code and a traceback:

 $ python test_sum_2.py Traceback (most recent call last): File "test_sum_2.py", line 9, in <module> test_sum_tuple() File "test_sum_2.py", line 5, in test_sum_tuple assert sum((1, 2, 2)) == 6, "Should be 6" AssertionError: Should be 6 

You can see how an error in the code causes an error in the console with information about where it occurred and what the expected result was.

Such tests are suitable for simple verification, but what if there are more errors than in one? Test runners come to the rescue. The test runner is a special application designed for conducting tests, checking output data and providing tools for debugging and diagnosing tests and applications.

Selection of Test Artist

For Python there are a lot of executors of tests. For example, unittest is built into the standard Python library. In this tutorial, we will use test cases and unittest test executers. The principles of unittest work are easily adapted for other frameworks. We list the most popular test performers:

• unittest;
• nose or nose2;
• pytest.

It is important to choose a test performer that meets your requirements and experience.

unittest

unittest is built into the standard Python library since version 2.1. You will surely come across it in commercial Python applications and open source projects.
In unittest there is a test framework and a test performer. When writing and executing tests you need to follow some important requirements.

unittest requires:

• Put tests in classes as methods;
• Use special approval methods. Class TestCase instead of the usual assert expression.

To turn a previously written example into a unittest test case, you need to:

1. Import unittest from standard library;
2. Create a class called TestSum that will inherit the class TestCase ;
3. Convert test functions to methods, adding self as the first argument;
4. Modify assertions by adding the use of the self.assertEqual() method in the TestCase class;
5. Change the entry point in the command line to call unittest.main() .

Following these steps, create a new test_sum_unittest.py file with this code:

 import unittest class TestSum(unittest.TestCase): def test_sum(self): self.assertEqual(sum([1, 2, 3]), 6, "Should be 6") def test_sum_tuple(self): self.assertEqual(sum((1, 2, 2)), 6, "Should be 6") if __name__ == '__main__': unittest.main() 

Doing this on the command line will result in one successful completion (indicated by.) And one unsuccessful (indicated by F):

 $ python test_sum_unittest.py .F ====================================================================== FAIL: test_sum_tuple (__main__.TestSum) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_sum_unittest.py", line 9, in test_sum_tuple self.assertEqual(sum((1, 2, 2)), 6, "Should be 6") AssertionError: Should be 6 ---------------------------------------------------------------------- Ran 2 tests in 0.001s FAILED (failures=1) 

Thus, you have completed two tests with the help of the test performer unittest.

Note: If you are writing test cases for Python 2 and 3, be careful. In versions of Python 2.7 and below, unittest is called unittest 2. When you import from unittest, you will get different versions with different functions in Python 2 and Python 3.

To learn more about unittest, read the unittest documentation .

nose

Over time, after writing hundreds or even thousands of tests for an application, it becomes increasingly difficult to understand and use unittest output data.

Nose is compatible with all tests written with the unittest framework, and can be replaced by a test performer. The development of nose, as an open source application, began to slow down, and nose2 was created. If you start from scratch, it is recommended to use nose2.

To get started with nose2, you need to install it from PyPl and run it on the command line. nose2 will try to find all test creaks with test*.py in the name and all test cases inherited from unittest.TestCase in your current directory:

 $ pip install nose2 $ python -m nose2 .F ====================================================================== FAIL: test_sum_tuple (__main__.TestSum) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_sum_unittest.py", line 9, in test_sum_tuple self.assertEqual(sum((1, 2, 2)), 6, "Should be 6") AssertionError: Should be 6 ---------------------------------------------------------------------- Ran 2 tests in 0.001s FAILED (failures=1) 

This is how the test created in test_sum_unittest.py from the nose2 test performer. nose2 provides many command line flags for filtering executable tests. To learn more, we advise you to familiarize yourself with the documentation of Nose 2 .

pytest

pytest supports unittest test cases. But the real advantage of pytest is its test cases. Pytest test cases - a series of functions in a Python file with test_ at the beginning of the name.

It has other useful features:

• Support for assert built-in expressions instead of using special self.assert * () methods;
• Support filtering test cases;
• The ability to restart from the last failed test;
• An ecosystem of hundreds of plug-ins that extend functionality.

An example TestSum test case for pytest will look like this:

 def test_sum(): assert sum([1, 2, 3]) == 6, "Should be 6" def test_sum_tuple(): assert sum((1, 2, 2)) == 6, "Should be 6" 

You got rid of TestCase, using classes and command line entry points.
More information can be found on the Pytest Documentation Site .

Writing the First Test

Let's unite everything that we have already learned, and instead of the built-in sum() function, we will test a simple implementation with the same requirements.

Create a new folder for the project, inside which create a new folder called my_sum. Inside my_sum, create an empty file called _init_.py . The presence of this file means that the my_sum folder can be imported as a module from the parent directory.

The folder structure will look like this:

project/
│
└── my_sum/
└── __init__.py

Open my_sum/__init__.py and create a new function called sum() , which takes as input iterable (list, tuple, set) and adds the values.

 def sum(arg): total = 0 for val in arg: total += val return total 

In this example, a variable called total , all values ​​in arg are arg and added to total . Then, at the end of the iteration, the result is returned.

Where to Write a Test

You can start writing a test by creating a test.py file, which will contain your first test case. For testing, the file should have the ability to import your application, so put test.py in the folder above the package. The directory tree will look like this:

project/
│
├── my_sum/
│ └── __init__.py
|
└── test.py

You will notice that as you add new tests, your file becomes more cumbersome and difficult to maintain, so we advise you to create the tests/ folder and divide the tests into several files. Make sure that the names of all files begin with test_ , so that test performers understand that the Python files contain the tests that need to be run. On large projects, tests are divided into several directories depending on their purpose or use.

Note: And what is your application is a single script?
You can import any script attributes: classes, functions, or variables using the built-in function __import__() . Instead of from my_sum import sum write the following:

 target = __import__("my_sum.py") sum = target.sum 

When using __import__() you do not have to turn the project folder into a package, and you can specify the file name. This is useful if the file name conflicts with the names of the standard package libraries. For example, if math.py conflicts with the math module.

How to Structure a Simple Test

Before writing tests, you need to solve a few questions:

1. What do you want to test?
2. Are you writing a unit test or integration test?

Now you are testing sum() . For it, you can check different behaviors, for example:

• Is it possible to sum up the list of integers?
• Is it possible to sum up a tuple or a set?
• Is it possible to sum up the list of floating point numbers?
• What happens if you give a bad value to the input: a single integer or a string?
• What happens if one of the values ​​is negative?

The easiest way to test a list of integers. Create a test.py file with the following code:

 import unittest from my_sum import sum class TestSum(unittest.TestCase): def test_list_int(self): """ Test that it can sum a list of integers """ data = [1, 2, 3] result = sum(data) self.assertEqual(result, 6) if __name__ == '__main__': unittest.main() 

The code in this example is:

• Imports sum() from the my_sum() package you created;
• Defines a new test case class called TestSum, which unittest.TestCase ;
• Defines a test method .test_list_int() for testing an integer list. The .test_list_int() method will do the following

:

1. Declare a data variable with a list of values (1, 2, 3) ;
2. my_sum.sum(data) value my_sum.sum(data) variable result ;
3. Determines that the result value is 6 using the .assertEqual() method on the unittest.TestCase class.

• Specifies the entry point of the command line that launches the unit test executable .main() .

If you don’t know what self is, or as defined by .assertEqual() , then you can refresh your knowledge of object-oriented programming with Python 3 Object-Oriented Programming .

How to Write Statements

The final step in writing a test is to check that the output matches the known values. This is called assertion. There are several general guidelines for writing statements:

• Check that the tests are repeatable and run them several times to make sure that they give the same results each time;
• Check and confirm the results that relate to your input data - check that the result is really the sum of the values ​​in the example sum() .

In unittest, there are many methods for validating the values, types, and existence of variables. Here are some of the most commonly used methods:

Method	Equivalent
.assertEqual (a, b)	a == b
.assertTrue (x)	bool (x) is true
.assertFalse (x)	bool (x) is False
.assertIs (a, b)	a is b
.assertIsNone (x)	x is None
.assertIn (a, b)	a in b
.assertIsInstance (a, b)	isinstance (a, b)

.assertIs() , .assertIsNone() , .assertIn() , and .assertIsInstance() have opposite methods called .assertIsNot() and so on.

Side effects

Writing tests is more difficult than just looking at the return value of a function. Often, code execution changes other parts of the environment: class attributes, filesystem files, values ​​in the database. This is an important part of testing, which is called side effects. Decide whether you are testing a side effect before including it in your list of statements.

If you find that there are a lot of side effects in the block of code you want to test, then you are violating the principle of sole responsibility . Violating the principle of sole responsibility means that a piece of code does too many things and requires refactoring. Adherence to the sole responsibility principle is a great way to design code for which it’s easy to write simple repeatable unit tests and, ultimately, create reliable applications.

Launch of the First Test

You have created the first test and now you need to try it out. It is clear that it will be passed, but before creating more complex tests, you need to make sure that even such tests are performed successfully.

Running Test Performers

The test runner is a Python application that executes test code, validates assertions, and displays test results in the console. At the end of test.py, add this small piece of code:

 if __name__ == '__main__': unittest.main() 

This is the command line entry point. If you run this script by running python test.py on the command line, it will call unittest.main() . This starts the test unittest.TestCase , detecting all the classes in this file inherited from unittest.TestCase .

This is one of the many ways to run the unittest executor. If you have a single test file called test.py , calling python test.py is a great way to get started.

Another way is to use the unittest command line. Let's try:

 $ python -m unittest test 

This will execute the same test module (called test ) via the command line. You can add additional parameters to change the output. One of them is -v for verbose. Let's try the following:

 $ python -m unittest -v test test_list_int (test.TestSum) ... ok ---------------------------------------------------------------------- Ran 1 tests in 0.000s 

We ran one test from test.py and output the results to the console. Verbose mode lists the names of the tests performed and the results of each one.

Instead of providing the name of the module containing the tests, you can request auto-detection using the following:

 $ python -m unittest discover 

This command will search for files in the current directory with test*.py in the name to test them.

If there are several test files and the pattern of the test*.py name is observed, you can pass the directory name with the -s flag and the folder name.

 $ python -m unittest discover -s tests 

unittest will run all the tests in a single test plan and display the results.
Finally, if your source code is not in the root directory, but in a subdirectory, for example, in a folder called src /, you can use the -t flag to tell unittest where to run the tests in order to import modules correctly:

 $ python -m unittest discover -s tests -t src 

unittest will find all the test*.py files in the src/ directory inside the tests , and then execute them.

Understanding the Results of Testing

It was a very simple example where everything went well, so let's try to understand the output of the failed test.

sum() must accept other lists of a numeric type, such as a fraction, for input.

test.py Fraction fractions .

 from fractions import Fraction 

, . , , ¼, ¼ ⅖ 1:

 import unittest from my_sum import sum class TestSum(unittest.TestCase): def test_list_int(self): """ Test that it can sum a list of integers """ data = [1, 2, 3] result = sum(data) self.assertEqual(result, 6) def test_list_fraction(self): """ Test that it can sum a list of fractions """ data = [Fraction(1, 4), Fraction(1, 4), Fraction(2, 5)] result = sum(data) self.assertEqual(result, 1) if __name__ == '__main__': unittest.main() 

python -m unittest test, :

 $ python -m unittest test F. ====================================================================== FAIL: test_list_fraction (test.TestSum) ---------------------------------------------------------------------- Traceback (most recent call last): File "test.py", line 21, in test_list_fraction self.assertEqual(result, 1) AssertionError: Fraction(9, 10) != 1 ---------------------------------------------------------------------- Ran 2 tests in 0.001s FAILED (failures=1) 

:

• : (F), (.);
• FAIL :

1. ( test_list_fraction );
2. ( test ) - ( TestSum );
3. ;
4. (1) (Fraction(9, 10))

, -v python -m unittest .

PyCharm

PyCharm IDE, unittest pytest, :

1. Project tool, tests.
2. unittest. , 'Unittests in my Tests…'.

unittest PyCharm:



PyCharm .

Visual Studio Code

Microsoft Visual Studio Code IDE, unittest, nose pytest Python.

, , Command Palette Ctrl+Shift+P “Python test”. :



Debug All Unit Tests, VSCode . (unittest) (.).

, , :



, , .

THE END

, Django Flask.

, , .