TDD wrong?

DISCLAIMER: you got caught on clickbait. Obviously, TDD can not be called erroneous, but ... There is always some kind of but .

Content

• Introduction
• Tests are the problem!
  
  
  • A bit of history
  • Small quiz
• TDD Review
  
  
  • Tests are formal requirements.
  • TDD encourages good architecture
  • TDD saves time
  • Tests are the best documentation.
  • Few conclusions
• What is the solution?
  
  
  • A few examples
  • Little about plans

Introduction

The first six years of my career, I freelance and participated in the initial stages of life of small startups. There were no tests in these projects ... In fact, not a single one.

Under these conditions, you must implement features for yesterday . Since market demands are constantly changing, tests become obsolete even before you finish them. And even these tests can be created only if you are sure of what exactly you want to create, and this is not always the case. Being engaged in R & D you may well not know what the end result should be. And even achieving some success, you cannot be sure that tomorrow the market (and with it the requirements) will not change. In general, there are business reasons for saving time in testing.

Granted, our industry is not just startups.

About two years ago, I got involved in a fairly large outsourcing company that serves customers of all sizes.

During conversations in the kitchen / smoking room, I found that almost everyone agrees that unit testing and TDD is a kind of best practice . But in all the projects of this company in which I participated, there were no tests. And no, I did not make that decision. Of course, we have projects with excellent test coverage, but they are also quite bureaucratic.

So what's the problem?

Why does everyone agree that TDD is good, but nobody wants to use it?

Maybe TDD is wrong? - Not!

Perhaps there is no business benefit? - And again, no!

Maybe the developers are just lazy? - Yes! But this is not the reason.

The problem is in the tests themselves!

I understand that it sounds strange, but I will try to prove it.

Tests are the problem!

Based on this study, the lowest overall satisfaction in the entire ecosystem belongs . So it was in 2016 and 2017. I have not found earlier studies, but this is not very important.

A bit of history

In 2008 , one of the first JS frameworks for testing ( QUnit ) was released.

In 2010 , Jasmine appeared.

In 2011 - Mocha .

The first Jest release I found was in 2014 .

For comparison.

In 2010, released angular.js .

Ember appeared in 2011 .

React - 2013 .

And so on…

At the time of this writing, no JS framework has been created ...

Anyway, by me.

For the same period of time, we saw the rise and fall of the grunt , then the gulp , after which we realized the power of the npm scripts and a stable release of the webpack was released.

Everything has changed in the last 10 years. Everything except testing.

Small quiz

Let's test your knowledge. What are these libraries / frameworks?

one:

 var hiddenBox = $("#banner-message"); $("#button-container button").on("click", function(event) { hiddenBox.show(); }); 

2:

 @Component({ selector: 'app-heroes', templateUrl: './heroes.component.html', styleUrls: ['./heroes.component.css'] }) export class HeroesComponent{ hero: Hero = { id: 1, name: 'Windstorm' }; constructor() { } } 

3:

 function Avatar(props) { return ( <img className="Avatar" src={props.user.avatarUrl} alt={props.user.name} /> ); } 

Answers:

1. Jquery
2. Angular2 +
3. React

Good. I am sure that all your answers were correct. But what about these frameworks for testing?

one:

 var assert = require('assert'); describe('Array', function() { describe('#indexOf()', function() { it('should return -1 when the value is not present', function() { assert.equal([1,2,3].indexOf(4), -1); }); }); }); 

2:

 const sum = require('./sum'); test('adds 1 + 2 to equal 3', () => { expect(sum(1, 2)).toBe(3); }); 

3:

 test('timing test', function (t) { t.plan(2); t.equal(typeof Date.now, 'function'); var start = Date.now(); setTimeout(function () { t.equal(Date.now() - start, 100); }, 100); }); 

four:

 let When2IsAddedTo2Expect4 = Assert.AreEqual(4, 2+2) 

Answers:

1. Mocha
2. Jest
3. Tape
4. Test for F #

You could guess some of them, but, in general, they are all very similar. Note that even with a change in language, there is little change.

We have at least 8 years of unit testing experience in the JavaScript world.

But we just adapted the already existing at that time. Unit testing, as we know it, appeared much earlier. If we take the Test Anything Protocol (1987) release as a starting point, then we use current approaches longer than I live.

TDD is not much younger, if not older . All this leads us to the fact that we can already objectively evaluate all the pros and cons.

TDD Review

Let's remember what TDD is.

Test-driven development ( TDD ) is a software development technique that is based on repeating very short development cycles: first a test is written covering the desired change, then code is written that allows the test to pass, and finally refactoring is done new code to the relevant standards. (c) Wikipedia

But what does this give us?

Tests are formal requirements.

This is only partially true.

TDD as a practice was "reinvented" by Kent Beck in 1999, while Agile Manifesto was adopted only 2 years later (in 2001). I have to emphasize this, so that you would understand that TDD was born in the Golden Age of the cascade model and this fact determines the most favorable conditions and processes for which it was designed. Obviously, TDD will work best in these conditions.

So, if you work in a project, where:

1. The requirements are clear;
2. You fully understand them;
3. They are stable and will not change frequently.

You can create tests as formalization requirements.

But in order to use existing tests in the same way, the following points must also be fulfilled:

1. There are no bugs in the tests;
2. They are relevant;
3. And they cover almost all usage scenarios (not to be confused with code coverage).

So "Tests are formalized requirements" is true only when these requirements exist before the development itself, as in the "Waterfall model" or NASA projects, where the "clients" are scientists and engineers.

Under certain conditions, this will work with "Agile" processes. Especially if something like BDD will be used, but that's another story.

TDD encourages good architecture

And again this is only partially true.

TDD encourages modularity , which is necessary, but not enough for a good architecture.

The quality of the architecture depends on the developers. Experienced developers are able to create excellent code, despite the use or non-use of unit testing.

On the other hand, weak developers will create low-quality code covered with low-quality tests, because creating good tests is a kind of art, like programming itself.

Of course, tests are like sex: "better is worse than none at all." But...

This test will not take you on the path to good system design:

 import { inject, TestBed } from '@angular/core/testing'; import { UploaderService } from './uploader.service'; describe('UploaderService', () => { beforeEach(() => { TestBed.configureTestingModule({ providers: [UploaderService], }); }); it('should be created', inject([UploaderService], (service: UploaderService) => { expect(service).toBeTruthy(); })); }); 

Because he does not test anything.

Note that we used 15 lines of code to test nothing.

But this test will not make the design of your system better:

 var IotSimulation = artifacts.require("./IotSimulation.sol"); var SmartAsset = artifacts.require("./SmartAsset.sol"); var BuySmartAsset = artifacts.require("./BuySmartAsset.sol"); var BigInt = require('big-integer'); contract('BuySmartAsset', function (accounts) { it("Should sell asset", async () => { var deliveryCity = "Lublin"; var extra = 1000; // var gasPrice = 100000000000; const smartAsset = await SmartAsset.deployed(); const iotSimulation = await IotSimulation.deployed(); const buySmartAsset = await BuySmartAsset.deployed() const result = await smartAsset.createAsset(Date.now(), 200, "docUrl", 1, "email@email1.com", "Audi A8", "VIN02", "black", "2500", "car"); const smartAssetGeneratedId = result.logs[0].args.id.c[0]; await iotSimulation.generateIotOutput(smartAssetGeneratedId, 0); await iotSimulation.generateIotAvailability(smartAssetGeneratedId, true); await smartAsset.calculateAssetPrice(smartAssetGeneratedId); const assetObjPrice = await smartAsset.getSmartAssetPrice(smartAssetGeneratedId); assert.isAbove(parseInt(assetObjPrice), 0, 'price should be bigger than 0'); await smartAsset.makeOnSale(smartAssetGeneratedId); var assetObj = await smartAsset.getAssetById.call(smartAssetGeneratedId); assert.equal(assetObj[9], 3, 'state should be OnSale = position 3 in State enum list'); await smartAsset.makeOffSale(smartAssetGeneratedId); assetObj = await smartAsset.getAssetById.call(smartAssetGeneratedId); assert.equal(assetObj[9], 2, 'state should be PriceCalculated = position 2 in State enum list'); await smartAsset.makeOnSale(smartAssetGeneratedId); const calculatedTotalPrice = await buySmartAsset.getTotalPrice.call(smartAssetGeneratedId, '112', '223'); await buySmartAsset.buyAsset(smartAssetGeneratedId, '112', '223', { from: accounts[1], value: BigInt(calculatedTotalPrice.toString()).add(BigInt(extra)) }); assetObj = await smartAsset.getAssetById.call(smartAssetGeneratedId); assert.equal(assetObj[9], 0, 'state should be ManualDataAreEntered = position 0 in State enum list'); assert.equal(assetObj[10], accounts[1]); const balanceBeforeWithdrawal = await web3.eth.getBalance(accounts[1]); const gas = await buySmartAsset.withdrawPayments.estimateGas({ from: accounts[1] }); await buySmartAsset.withdrawPayments({ from: accounts[1], gasPrice: gasPrice }); const balanceAfterWithdrawal = await web3.eth.getBalance(accounts[1]); var totalGas = gas * gasPrice; assert.isOk((BigInt(balanceAfterWithdrawal.toString()).add(BigInt(totalGas))).eq(BigInt(balanceBeforeWithdrawal.toString()).add(BigInt(extra)))); }) }) 

The biggest problem of this test is the initial code base, but even in this case it could be significantly improved, even without refactoring a project that is already running.

In general, the impact of TDD on the resulting architecture is approximately at the same level as the influence of the selected library / framework, if not less (for example, Nest , RxJs and MobX , in my personal opinion, have a significantly stronger effect).

But neither TDD nor frameworks will save from bad code and unsuccessful architectural solutions.

There is no silver bullet .

TDD saves time

And it depends on many factors ...

Let's assume that:

1. Everyone in the project has a good command of the selected test tool, TDD methodology, and best unit testing practices;
2. And everyone understands all of the above in the same way;
3. And the requirements are transparent and stable;
4. In addition, the development team understands them in the same way as "Product Owner";
5. And management is ready to solve all organizational problems caused by TDD (for example, the longer process of introducing new developers into the team).

Even in this case, you need to invest the time and effort first, which will lengthen the initial development phase and only after some time you will get the benefit by reducing the time required for correcting errors and product support.

Of course, the second may be greater than the initial investment and in this case, the benefit from TDD is obvious.

Also, in some cases, you can save time on the introduction of new functionality, because the tests will immediately detect unintended changes.

But in the real world, which is very dynamic, the requirements may change and what was the correct behavior before will become incorrect. In this case, you need to rewrite the tests in connection with the new realities. And, obviously, make new efforts that will not pay off immediately.

You can even get into this type of loop:

Well, this cycle is contrary to the principles of TDD. But the next one is gone:

Try to find significant differences in them.

Tests are the best documentation.

Not. They are good at it, but definitely not the best.

Let's take a look at the angular documentation :

Or react :

What do you think they have in common? - They are both built on code examples . And even more. All these examples can be easily run (angular uses StackBlitz , and react - CodePen ), so you can see what it gives out and what happens if you change something.

Of course, there is also plain text there, but these are like comments in the code — you only need them if you do not understand something from the code itself.

Executable code examples are the best documentation!

Tests are close to this, but not enough.

 describe('ReactTypeScriptClass', function() { beforeEach(function() { container = document.createElement('div'); attachedListener = null; renderedName = null; }); it('preserves the name of the class for use in error messages', function() { expect(Empty.name).toBe('Empty'); }); it('throws if no render function is defined', function() { expect(() => expect(() => ReactDOM.render(React.createElement(Empty), container) ).toThrow() ).toWarnDev([ // A failed component renders twice in DEV 'Warning: Empty(...): No `render` method found on the returned ' + 'component instance: you may have forgotten to define `render`.', 'Warning: Empty(...): No `render` method found on the returned ' + 'component instance: you may have forgotten to define `render`.', ]); }); 

This is a small piece of the real test in react . We can extract code samples from it:

 container = document.createElement('div'); Empty.name; 

 container = document.createElement('div'); ReactDOM.render(React.createElement(Empty), container); 

Everything else is manually written infrastructure code.

Let's be honest, the sample test above is much less readable than the real documentation. And the problem is not in this particular test - I'm sure that the guys from facebook know how to write good code and good tests :) All this garbage from testing tools and assertion libraries like it , describe , test , to.be.true just to.be.true up your tests.

By the way, there is a library called tape with a minimal API, because any test can be rewritten using only equal / deepEqual , and thinking in these terms is generally good practice for unit testing. But even tests for tape still very far from just executable code examples .

But it is worth noting that the tests are still quite suitable for use as documentation. They are really less likely to be outdated, and our consciousness just throws out too much when we read them. If we try to visualize what the test turns into in our head, it will look something like this:

As you can see, this is already much closer to the actual dock than the original test.

Few conclusions

1. Tests are formal requirements if they are stable;
2. TDD encourages good architecture if developers are skilled enough;
3. TDD saves time if you invest it first;
4. Tests are the best documentation if there are no other executable code examples.

So TDD is still wrong? - No, TDD is not wrong.

It indicates the correct direction and raises important questions. We just have to rethink and change the way it is applied.

What is the solution?

Do not take TDD as a silver bullet .

Do not even perceive it as an Agile process, for example.

Instead, focus on his real strengths :

1. Preventing unintended changes, in other words, fixing existing behavior as a kind of 'baseline' (the English term 'baseline' will still come in handy);
2. Using examples from documentation as tests.

Think of unit testing as a developer tool . Like a linter or compiler , for example.

You will not ask Product Owner for permission to use the linter - you will simply use it.

Someday it will become a reality for unit testing. When the necessary efforts for TDD will be at the level of using a tickpicker or bandler . But up to this point, simply minimize your costs by creating tests as similar as possible to the executable examples and use them as the current baseline of the state of your project.

I understand that it will be difficult, especially given the fact that most popular tools are designed for other purposes.

True, I created one such, taking into account all the above problems. It is called

The basic concept is very simple. Write code:

 export function sampleFn(a: any, b: any) { return a + b + b + a; } 

And just use it in your test:

 import { sampleFn } from './index'; export = { values: [ sampleFn(1, 1), sampleFn(1000000, 1000000), sampleFn('abc', 'cba'), sampleFn(1, 'abc'), sampleFn('abc', 1), new Promise(resolve => resolve(sampleFn('async value', 1))), ], }; 

NOTE: The test is, of course, very synthetic - just for demonstration.

Then run the baset test command and get a temporary baseline:

 { "values": [ 4, 4000000, "abccbacbaabc", "1abcabc1", "abc11abc", "async value11async value" ] } 

If the values ​​are correct, run baset accept and commit the created baseline to your repository.

All subsequent test runs will compare the existing baseline with the values ​​exported from your tests. If they are different, the test fails , otherwise - passed .

If requirements have changed, just change the code, run the tests and accept the new baseline .

This tool still protects you from unintended changes, but it requires minimal effort. All you need is to simply write an executable code example , which, moreover, is the basis of good documentation.

A few examples

Use with react. Here is the test:

 import * as React from 'react'; import { jsxFn } from './index'; export const value = ( <div> {jsxFn('s', 's')} {jsxFn('abc', 'cba')} {jsxFn('s', 'abc')} {jsxFn('abc', 's')} </div> ); 

will create such .md file as baseline :

---

exports.value:

 <div data-reactroot=""> <div class="cssCalss"> ss </div> <div class="cssCalss"> abccba </div> <div class="cssCalss"> sabc </div> <div class="cssCalss"> abcs </div> </div> 

---

Or with pixi.js :

 import 'pixi.js'; interface IResourceDictionary { [index: string]: PIXI.loaders.Resource; } const ASSETS = './assets/assets.json'; const RADAR_GREEN = 'Light_green'; const getSprite = async () => { await new Promise(resolve => PIXI.loader .add(ASSETS) .load(resolve)); return new PIXI.Sprite(PIXI.utils.TextureCache[RADAR_GREEN]); }; export const sprite = getSprite(); 

This test will create a baseline like this:

---

exports.sprite:

---

Little about plans

I must say that this tool is still at a very early stage of development and there are still a lot of innovations ahead, for example:

1. Watch / Workflow mode
2. TAP compatibility
3. Git acceptance strategy
4. VS Code extension
5. ... and at least 24 others.

Only about 40% of the planned was implemented. But all the basic functionality is already working, so you can try to play with it. Maybe you even like it, who knows?