Why developers don't like Unit Tests
Maybe they just do not know how to "cook" them?
On duty, I participate in the development of applications for microcontrollers. But it so happened that I was more involved in various kinds of testing (both my own and someone else's code) than, actually, in development. Far from the first attempt, I managed to master TDD. Now the volumes of the test and “combat” code more or less dwindled :)
I hope that after reading this article the question “Why not the first time?” Will be removed.
In my professional activity, I often hear statements of the following nature:
Even supporters of flexible development methodologies do not always understand the value of this type of testing. Actually, article Agile from the point of view of the programmer served as the trigger to this publication.
Let's imagine that in the process of developing a certain system there was a need to implement a linked list. For simplicity, I will limit myself to the functions of push and pop (FIFO) and an integer as payload.
Without additional requirements for this list, we can expect that the experienced developer Maxim will first study the examples that are on the Internet and take one of them as a basis.
')
As a result, we have the following implementation:
Well, the implementation is. Integrated the code into the system, "clicked" everything works.
Then someone remembers that linked lists are a very important matter, address arithmetic is there ... memory leaks ... And you should write unit tests, at least for this module - well, that would have been easy to save.
And I’m almost 100% sure that this will be handled by another developer, Andrei. Andrey is a novice developer and he just needs to gain experience. And since the development of the system is not yet finished, the guys with experience still have something to do.
Andrew: "And how to test it?"
Maxim: “Well, look at the code, figure out how it is implemented, and cover all the code branches with tests so that you don’t miss anything”
Andrei: “I want to start testing with the list_pop () function. It allocates memory for a new item and adds it to the list. But in the same place static and I cannot reach the list from the test code. "
Maxim: “Oh, well, let me make a crutch especially for your tests. In the production build it will not work, but it will help you. Externally in the test and all. "
It is natural to expect the following test implementation:
I think the further expansion of the code coverage with new tests is obvious to the reader. The result is achieved - the module is tested by unit-tests, the coverage is 100%. You can sleep well.
Of course, the story described above may have another development. I'm just trying to say that unit tests are different.
In this case, the tests have the following disadvantages:
Unfortunately not. Or not always.
I am not an ardent supporter of the basic principle of TDD, forcing me to first write a test for non-existent code, and then write code in order for this test to pass. Sometimes, I write a small piece of code before tests to it.
The main thing is different. It is very important, in my opinion, to consider each module as an independent system:
Someone will probably notice “this is BDD”. And most likely will be right. But, it does not matter what comes first in your development: tests, or behavior, or the code itself, which is already written very, very much. It is important how you write unit tests.
For example, the first test for the list implemented above may be:
Second test:
The first test, we checked that the API module in principle workable. We also made sure that what we keep on the list can later be retrieved.
By the second test, we checked that the elements are retrieved from the list in the order in which they were placed there.
And it was precisely this kind of functionality that interested us initially when designing the entire software complex, but certainly not the way it was implemented.
With this approach, the above-described shortcomings of the tests are eliminated:
In addition to the above advantages, unit tests have another, in my opinion, very important advantage - they improve the quality of the code.
Whether we like it or not, the code under test (one that can be physically tested) is more flexible, more portable, more scalable. Maybe something else (I'm afraid to praise).
Unfortunately, the list implemented above has not yet been tested for memory leaks. But this moment was far from the last in the list of fears that made the team think about unit tests for a linked list.
In order to verify the absence of leaks, we must control the allocation / release of memory. And making a mock-and on the functions of the standard library is not the easiest task.
There is a way out - add an abstraction layer between the module and the standard library with the following interface:
Then, the implementation of the list will look like:
Already implemented tests will not change in any way, with the exception of adding mocks:
And finally, new memory management tests:
As a result, on the one hand, we checked the correctness of working with memory, on the other - we implemented an additional layer containing wrappers for malloc () and free () functions. And if in the future the memory allocation mechanism is changed (a static array of elements of a fixed size, the memory_pool of some RTOS) - our code is ready for these changes, and the list itself and tests for its functionality will not be affected in any way.
Yes, ... only two conclusions
1. Unit tests are good, the main thing is to write them correctly.
2. but in order for this to be possible, you should think about testing when developing code.
All coincidences with real people are random.
As a basis for the implementation of the text used material www.learn-c.org
All tests are written using Unity / CMock / Ceedling tools.
Intro
On duty, I participate in the development of applications for microcontrollers. But it so happened that I was more involved in various kinds of testing (both my own and someone else's code) than, actually, in development. Far from the first attempt, I managed to master TDD. Now the volumes of the test and “combat” code more or less dwindled :)
I hope that after reading this article the question “Why not the first time?” Will be removed.
Data
In my professional activity, I often hear statements of the following nature:
- “Why are we going to spend time on unit tests, we don’t have time to complete the project on time?”
- “Why do tests dictate how to write code?”
- “Let's just write the code, testers will find all the defects. Then fix it. "
- “Here, colleagues have implemented a new feature, it is necessary to cover it with unit tests”
Even supporters of flexible development methodologies do not always understand the value of this type of testing. Actually, article Agile from the point of view of the programmer served as the trigger to this publication.
As it usually happens
Let's imagine that in the process of developing a certain system there was a need to implement a linked list. For simplicity, I will limit myself to the functions of push and pop (FIFO) and an integer as payload.
Without additional requirements for this list, we can expect that the experienced developer Maxim will first study the examples that are on the Internet and take one of them as a basis.
')
As a result, we have the following implementation:
my_list.h file
#ifndef MY_LIST_H #define MY_LIST_H #ifndef NULL /* just for this example */ #define NULL 0 #endif void list_push( int val ); int list_pop( void ); #endif my_list.c file
#include "my_list.h" #include <stdlib.h> typedef struct node { int val; struct node * next; } node_t; static node_t * list_head; void list_push( int val ) { node_t * current = list_head; if(list_head == NULL) { list_head = malloc(sizeof(node_t)); list_head->val = val; list_head->next = NULL; } else { while (current->next != NULL) { current = current->next; } current->next = malloc(sizeof(node_t)); current->next->val = val; current->next->next = NULL; } } int list_pop( void ) { int retval = -1; node_t * next_node = NULL; if (list_head == NULL) { return -1; } next_node = list_head->next; retval = list_head->val; free(list_head); list_head = next_node; return retval; } Well, the implementation is. Integrated the code into the system, "clicked" everything works.
Then someone remembers that linked lists are a very important matter, address arithmetic is there ... memory leaks ... And you should write unit tests, at least for this module - well, that would have been easy to save.
And I’m almost 100% sure that this will be handled by another developer, Andrei. Andrey is a novice developer and he just needs to gain experience. And since the development of the system is not yet finished, the guys with experience still have something to do.
Andrew: "And how to test it?"
Maxim: “Well, look at the code, figure out how it is implemented, and cover all the code branches with tests so that you don’t miss anything”
Andrei: “I want to start testing with the list_pop () function. It allocates memory for a new item and adds it to the list. But in the same place static and I cannot reach the list from the test code. "
static node_t * list_head; void list_push( int val ) { node_t * current = list_head; if(list_head == NULL) { list_head = malloc(sizeof(node_t)); list_head->val = val; list_head->next = NULL; } ... Maxim: “Oh, well, let me make a crutch especially for your tests. In the production build it will not work, but it will help you. Externally in the test and all. "
#ifdef UNIT_TEST node_t * list_head; #else static node_t * list_head; #endif It is natural to expect the following test implementation:
file test_my_list.c
#include "unity.h" #include "my_list.h" void setUp(void) { } void tearDown(void) { } typedef struct node { int val; struct node * next; } node_t; extern node_t * list_head; void test_1( void ) { list_push( 1 ); TEST_ASSERT_NOT_NULL( list_head ); /* Check that memory is allocated */ TEST_ASSERT_EQUAL_INT( 1, list_head->val ); /* Check that value is set*/ TEST_ASSERT_NULL( list_head->next ); /* Check that the next pointer has appropriate value */ } I think the further expansion of the code coverage with new tests is obvious to the reader. The result is achieved - the module is tested by unit-tests, the coverage is 100%. You can sleep well.
What's wrong with that?
Of course, the story described above may have another development. I'm just trying to say that unit tests are different.
In this case, the tests have the following disadvantages:
- Tests are testing the code (as strange as it sounds)
- Tests force the developer to do crutches
- Tests require titanic efforts to support them, even in the case of refactoring, not to mention significant changes.
- "Failed" tests do not mean at all that some kind of functionality does not work
And if you first write tests, and then the code. Will this help?
Unfortunately not. Or not always.
I am not an ardent supporter of the basic principle of TDD, forcing me to first write a test for non-existent code, and then write code in order for this test to pass. Sometimes, I write a small piece of code before tests to it.
The main thing is different. It is very important, in my opinion, to consider each module as an independent system:
- Try to formulate the requirements for this system, which it must meet
- It is the compliance with these requirements to try to test unit-tests.
- Try not to delve into the features of the implementation of this system and use only its external API for testing.
Someone will probably notice “this is BDD”. And most likely will be right. But, it does not matter what comes first in your development: tests, or behavior, or the code itself, which is already written very, very much. It is important how you write unit tests.
For example, the first test for the list implemented above may be:
/* * Given the list is empty * When I push 1 to the list * Then the pop function shall return 1 */ void test_simple( void ) { list_push( 1 ); TEST_ASSERT_EQUAL_INT( 1, list_pop() ); } Second test:
/* * Given the list is empty * When I push 1 to the list * And I push 2 to the list * Then the first call of the pop function shall return 1 * And the second call of the pop function shall return 2 */ void test_order( void ) { list_push( 1 ); list_push( 2 ); TEST_ASSERT_EQUAL_INT( 1, list_pop() ); TEST_ASSERT_EQUAL_INT( 2, list_pop() ); } The first test, we checked that the API module in principle workable. We also made sure that what we keep on the list can later be retrieved.
By the second test, we checked that the elements are retrieved from the list in the order in which they were placed there.
And it was precisely this kind of functionality that interested us initially when designing the entire software complex, but certainly not the way it was implemented.
Benefits
With this approach, the above-described shortcomings of the tests are eliminated:
Tests test codeTests test the behavior of a module without knowing anything about its implementation (black-box)
Tests force the developer to do crutcheswhen testing through an API, this is extremely rare
Tests require titanic efforts to support them, even in the case of refactoring, not to mention significant changes.in our example, the implementation can be changed completely (an array instead of a linked list, bidirectional list instead of unidirectional, etc.), which should not affect its behavior
"Failed" tests do not mean at all that some kind of functionality does not worksince code refactoring (if successful) does not affect the test results in any way, there is only one reason for the “failures” of the tests - something really does not work
Extra buns
In addition to the above advantages, unit tests have another, in my opinion, very important advantage - they improve the quality of the code.
Whether we like it or not, the code under test (one that can be physically tested) is more flexible, more portable, more scalable. Maybe something else (I'm afraid to praise).
Unfortunately, the list implemented above has not yet been tested for memory leaks. But this moment was far from the last in the list of fears that made the team think about unit tests for a linked list.
In order to verify the absence of leaks, we must control the allocation / release of memory. And making a mock-and on the functions of the standard library is not the easiest task.
There is a way out - add an abstraction layer between the module and the standard library with the following interface:
my_list_mem.h file
#ifndef MY_LIST_MEM #define MY_LIST_MEM void * list_alloc_item( int size ); void list_free_item( void * item ); #endif Then, the implementation of the list will look like:
my_list.c
#include "my_list.h" #include "my_list_mem.h" typedef struct node { int val; struct node * next; } node_t; static node_t * list_head; void list_push( int val ) { node_t * current = list_head; if(list_head == NULL) { // list_head = malloc(sizeof(node_t)); list_head = (node_t*)list_alloc_item( sizeof(node_t) ); list_head->val = val; list_head->next = NULL; } else { while (current->next != NULL) { current = current->next; } // current->next = malloc(sizeof(node_t)); current->next = (node_t*)list_alloc_item( sizeof(node_t) ); current->next->val = val; current->next->next = NULL; } } int list_pop( void ) { int retval = -1; node_t * next_node = NULL; if (list_head == NULL) { return -1; } next_node = list_head->next; retval = list_head->val; // free(list_head); list_free_item( list_head ); list_head = next_node; return retval; } Already implemented tests will not change in any way, with the exception of adding mocks:
file test_my_list.c
#include "unity.h" #include "my_list.h" #include "mock_my_list_mem.h" #include <stdlib.h> static void * list_alloc_item_mock( int size, int numCalls ) { return malloc( size ); } static void list_free_item_mock( void * item, int numCalls ) { free( item ); } void setUp(void) { list_alloc_item_StubWithCallback( list_alloc_item_mock ); list_free_item_StubWithCallback( list_free_item_mock ); } void tearDown(void) { } /* * Given the list is empty * When I push 1 to the list * Then the pop function shall reutrn 1 */ void test_nominal( void ) { list_push( 1 ); TEST_ASSERT_EQUAL_INT( 1, list_pop() ); } /* * Given the list is empty * When I push 1 to the list * And I push 2 to the list * Then the first call of the pop function shall return 1 * And the second call of the pop function shall return 2 */ void test_order( void ) { list_push( 1 ); list_push( 2 ); TEST_ASSERT_EQUAL_INT( 1, list_pop() ); TEST_ASSERT_EQUAL_INT( 2, list_pop() ); } And finally, new memory management tests:
file test_my_list_mem_leak.c
#include "unity.h" #include "my_list.h" #include "mock_my_list_mem.h" #include <stdlib.h> static int mallocCounter; static int freeCounter; static void * list_alloc_item_mock( int size, int numCalls ) { mallocCounter++; return malloc( size ); } static void list_free_item_mock( void * item, int numCalls ) { freeCounter++; free( item ); } void setUp(void) { list_alloc_item_StubWithCallback( list_alloc_item_mock ); list_free_item_StubWithCallback( list_free_item_mock ); mallocCounter = 0; freeCounter = 0; } void tearDown(void) { } /* * Given the list is empty * When I push an item to the list * Then one part of mmory shall be allocated * And no part of memory shall be released */ void test_push( void ) { list_push( 1 ); TEST_ASSERT_EQUAL_INT( 1, mallocCounter ); TEST_ASSERT_EQUAL_INT( 0, freeCounter ); } /* * Given the list is empty * When get the item from the list pushed before * Then one part of mmory shall be released * And no part of memory shall be allocated */ void test_pop( void ) { list_pop(); TEST_ASSERT_EQUAL_INT( 0, mallocCounter ); TEST_ASSERT_EQUAL_INT( 1, freeCounter ); } As a result, on the one hand, we checked the correctness of working with memory, on the other - we implemented an additional layer containing wrappers for malloc () and free () functions. And if in the future the memory allocation mechanism is changed (a static array of elements of a fixed size, the memory_pool of some RTOS) - our code is ready for these changes, and the list itself and tests for its functionality will not be affected in any way.
Conclusions
Yes, ... only two conclusions
1. Unit tests are good, the main thing is to write them correctly.
2. but in order for this to be possible, you should think about testing when developing code.
PS
All coincidences with real people are random.
As a basis for the implementation of the text used material www.learn-c.org
All tests are written using Unity / CMock / Ceedling tools.
Source: https://habr.com/ru/post/250053/
All Articles