Testing: from rags to riches

Prologue

In this post I will share personal impressions of the formation and maturation of software testing processes in Russian IT. By the will of fate, this happened, one might say, in front of me and in the process of mastering this profession on the way from a junior tester in a small company to a quality director in a solid company with multimillion-dollar turnover.

It is recommended to abstract a little from the current perception of testing by the reader in his current status, in order to understand how it happened in the past, so as not to dismiss the article from the summer with exclamations “yes, this is already clear”, “these are well-known facts, why write about it” and etc. I am not trying to tell you about how to walk, but just want to talk about how I once learned to walk myself and how we all learned how.

Lived once

About 8 years ago, when I first started testing software, it seemed to me that this was a ticked job, saying someone out there somewhere said that testing software before distributing to users is cool, fashionable and generally good, because and you need it. The industry itself, by the way, had a similar opinion about this process. Of course, there were software giants and representative offices of Western companies, but in particular the culture of quality assurance and testing was imposed on the outside and was perceived as this rather than the fruit of the industry’s development in the Russian Federation.

I started with gamedev, which somehow was just like the representation of Western companies in terms of attitude to testing. Quality control was an integral part, but more often not because of an understanding of why and why, but because “there is testing in the West, which means we will have it,” especially since the company I worked at at that time was one of the largest distributors of foreign games. At that time, everything was already “serious” - a team of 10 specialists, a couple of leading specialists, a department head, i.e. its own hierarchy, orders and some beginnings of system processes in the spirit of “build - testing - bugs - fixes - release”. Certainly, everyone “understood” why we worked there, “we look for mistakes, so that users would not find them,” at least then the general line of the governing party was reduced and they certainly were right in something.
')
After a couple of years, before leaving for web development, I saw the beginnings of systematization in the company — the first checklists, test cases, and even some sketches of reporting on test results. But all of them have not yet developed into a single puzzle "We are testing because ..."

Years went by

After moving to a large media company that was developing a number of very popular web projects, I noticed not only changes in myself and the perception of testing, but also in the processes surrounding this testing. The development concerned both development and project management, no, it was still too early to talk about Scrum and other Agile, but even then the first answers to “Why do you need testing?”, “How can you implement it?” And “How did you can you improve? ”at the design level.

These were modest undertakings at the level of “we test, so that our products are of better quality”, “testing helps make our products better” and similar conclusions that already resembled child’s meaningful phrases more than babbling at the beginning of the journey.

It was then, 5-6 years ago, that test plans, checklists and test cases began to appear in mass order, because there was an understanding that with them you can speed up the testing process, make it predictable, standardized and manageable. Finally, it became possible to measure the coverage not with ephemeral man-hours, but with quite specific figures, reflecting, albeit approximate, but real data on test volumes and the percentage of code coverage of tests.

Testers-professionals, not monkey-pushers, became more and more, some even stumbled into the community, recruited members of thematic sites and forums, while at the same time a small group of enthusiasts organized the first and now fairly well-known conference dedicated to testing and quality assurance.

At the same time, they talked about testing automation at the level of functional tests for the first time, I personally did research and development on TestComplete and Selenium (which was then not a formidable tool in the hands of hundreds and thousands of automation engineers around the world, but only got the version of the first public beta, which was used desperate enthusiasts from the world of web development). These were the first attempts to bring the testing process to a new level, when the process became an indispensable and integral piece of development, without which not a single release would pass. It is understood that testing can help protect the development from unnecessary hassle and editing risks on live (or by uploading patches if you take software), and also helps perfectionist developers to do their job better in principle.

Somewhere at the same time I discovered freelancing for myself, but these were rather rare forays than full-fledged projects, distributed teams, etc. Rather, it was a kind of analogy in imitation of big companies, as in the years before, big ones imitated the West. Occasionally, at vip-customers, it was a matter of image, web studios wrote a beautiful line about the testing that was being done to the business proposal, which increased the cost of work, but added solidity.

Yesterday

Small companies have begun to struggle for quality with the hands of testers, but not in an attempt to get rid of suffering, suffering and development risks, but with the aim of profit. To be more precise, for the sake of saving, an understanding finally began to appear that testers are able to save companies a lot of money.

As it turned out (oh, a miracle, isn't it?), Bugs incur direct losses to the company, be it the vulnerability through which ingenious hackers pulled the client base of a large company’s accounts, the crookedly designed layout of an online store that scared away the target audience or even minor bugs that in consequence of their uncorrectedness they put resources for long hours, thereby causing indirect image and direct financial losses.

At that time, 3-4 years ago, as many might have thought, I reached a solid career point, replacing a couple of employers, I returned to that very large media company where I had previously worked as a tester, but now to the position of PMA of a popular and large project. I went to the role, not in the desire to develop to the field of project management, as such, but to learn to understand the PM as such and look at the process from the height of the bird, as they say, flight (I will not go into details, this is a topic for a separate post in the spirit of “Why I left project management and“ downgrade ”to a tester”).

On the one hand, I saw how sometimes the fruits of testers are insignificant and why when a tester screams at the PM "I have a critical bug in Opera, it is impossible to release" he skeptically covering his face with his hand, says "It is necessary!", Because Today, we are still negotiating with partners, discussing advertising with clients, 20,000 content units for moderation, setting 3 tasks and working out 6 features for the next week.

Lyrical digression:

“When I came to manage the project, I saw 500 tickets in the ticket system assigned to myself, including the bugs that I myself once set, while still being a tester 2-3 years ago. The first thing I did to these bugs ... lowered the priority. After all, if the project has lived with these bugs for 2-3 years, then they are really non-critical. ”

On the other hand, there were reverse cases where insufficient testing depth or lack of automation led to the site losing huge amounts of money or falling by half a day, which was unacceptably natural for a project with an audience of 100k hosts.

There were, for example, such cases

"From 00:00 on March 1, the site went down to 12 o'clock in the morning (before the fix was made by the developer) because it was possible to indicate in the user's personal card the fake date of birth, February 30-31, which was incorrectly validated by the output handler in the block "Today is a birthday for:", which clearly cut the list of births on March 1 and displayed a limited number of usernames on the main page, but did not cut fakes moved on that date, the software conflict decided to easily fall down and not rise until the arrival of the developer. "

“Unexpectedly, it turned out that our distributed server system for data storage failed and the registration of accounts with the letter“ P ”stopped. All anything, but on the day we had 4500 registrations. Those. According to averaged data, every day 200 people faced an inexplicable problem when registering, assuming that half left (100 people), the company lost about $ 30 every day, as according to statistics, every third bought during registration a “premium account for $ 1” . The losses are miserable, but due to the incomplete coverage of autotests on this site, this bug was discovered only ... six months later, i.e. on a similar triviality, the company has lost the order of pure $ 5000 "

Automation was introduced to the fullest, sometimes very thoughtlessly, simply because it was “fashionable” and it seemed to be how profitable it was. Again, it touches on approaches in past development milestones when implementing testing as such.

Freelance orders have become much more, now the companies understood that taking the tester team as a team also saves on image and financial losses (one thing to fix the bug half an hour after uploading the code to the test server, another thing is to fix it on the client’s in fact has already been handed over).

It was the era of great savings on the backs of testers, the mass demand for testers generated no less a supply, because the threshold for entering the profession was still low. There were companies and trainers teaching the basics of the profession, people began to use testing as the door to the development world more and more often, and companies used testers as a relatively cheap way to optimize the development process of their products as a whole.

Today

Two years ago, I went from project management to quality management, where becoming a quality director, I fundamentally began to work on optimizing not only testing processes (breaking it up into phases, creating instructions, templates, metrics, etc.), but also developing and even analysts in general. Helping thereby make the cost of mistakes even less, preempting their very appearance at the level of thought, and not written code. If before mistakes were corrected de facto on the battlegrounds, now quality assurance processes allow analysts to “get into their heads” and correct them in the problem statement before the programmers write the first lines of code. And developers should be encouraged to write competent unit tests so that the code going out of their hands should be checked not only by testers, but also by unit tests, passed the code review, as well as cross-testing by the developers themselves.

The community has grown, filled up with rows of recruits, outsourcing testing projects have appeared like mushrooms after the rain, training centers, tester conferences have already become regular and twice a year gather some hundreds of enthusiastic listeners, and at the general conferences the block devoted to quality is more and more widely represented.

Quality assurance and quality control processes in many companies now occupy a very prominent place, as they not only insure from an economic point of view, they allow to avoid the stresses of poor-quality development, but also help to create competitive products on the market. After all, it is no secret to anyone that the quality of products today is one of the defining characteristics when choosing any product.

And testers in many companies are the very driving force that exalts the quality on a pedestal of deserved honor, moving all processes forward, making not only products, but also this world more qualitative.

What awaits us tomorrow?

I am not a soothsayer, but only made up a small historical slice of testing development along the path of my own growth in the profession. Therefore, let me say the words from the famous movie:

“The future is not predetermined and there is no other destiny except that which we create.”