Testing: a simple IT track or a serious undertaking?
Sometimes, even in authoritative sources, a condescending attitude to testing software products and, accordingly, to people involved in this direction slips. There, they say, the requirements for workers are lower, and the cadres themselves are so-so, and you don’t make much money. We actually talked about how testing from the inside looks like from Nikita Makarov, who is engaged in manual and automated testing at the same time in Odnoklassniki.
- Please tell us a little about yourself and your work. How are you related to testing?
Nikita Makarov: At the moment I am in charge of testing at Odnoklassniki. I have come to the current position in several steps. Initially, I was an engineer, but at some point, at the initiative of the technical director, the introduction of test automation began and I was asked to take up this direction.
')
The first four years I have been working exclusively on testing automation. When we won the biggest problems in this area, I was given the management of delivery. Those. I have become not only the head of test automation, but also a delivery manager who determines who, in what sequence and by what rules, will break production.
A year later (from April 2016), I began to lead all testing at Odnoklassniki, retaining the previous positions as well. As a result, the scope of my duties is quite wide - most often I can call myself a psychoanalyst in IT.
- The market is pushing to release products faster. Does it often happen at the expense of testing? Is this justified?
Nikita Makarov: Indeed, the speed of release of releases is very often the determining factor. This is due to the fact that it is very difficult to find a fresh idea now and throw it onto the market. It is doubly difficult to do it alone, so that no one will know about it and try to copy it (copying ideas, using a slightly more fragmented niche audience can also be successful; a vivid example is Uber, Yandex.Taxi, GetTaxi and a bunch of similar services around the world). To enter the market faster, it is often necessary to shorten the testing phase.
In varying degrees in mass products, users are ultimate testers. About what you have done wrong, that the product works poorly, you can learn from feedback from users through the market, feedback forms, etc.
Is this justified? In terms of speed, yes. From the point of view of the audience of products - not always, not every product can be sold that way.
I focus on what we are talking about products for the mass user market - not life critical, not mission critical - about things like Instagram, Facebook, Uber, etc. In such products, in case of failure, someone may not lose very much money. In the areas of life critical solutions, testing, as well as the development itself, is very rigidly defined by the standards, so there it is not always possible to cut off something. And if it turns out, the cost of the error here can be very large.
One example is the case of the Knight Capital Group : within 45 minutes, the guys lost about half a billion dollars. A completely recent case is the DoDo Pizza, when inaccurately depot the service rolled back transactions for several million rubles.
How justified is this risk? Every business decides in its own way.
- That is, the amount of testing determines the testing department in a dialogue with the business? Who, in this case, should understand the interlocutor's language, putting forward arguments clear to him?
Nikita Makarov: The initiative is here on the testing side. The testing department, which does not understand the language of business, is obviously dead. The testing department in itself does not earn money (unless we are talking about selling this department to outsourcing). Accordingly, it should contribute to the reduction of losses or greater earnings. Therefore, the correct selection of arguments for business is one of the main qualities of a good tester, especially in product projects.
- How do you feel about the idea that the way in IT is easier to navigate through testing? They say that requirements are lower there, market demand is higher. And after a couple of years, you can do programming when you have mastered all the processes, entrenched in the profession?
Nikita Makarov: This is a stereotype. On the one hand he is faithful, on the other - not.
If you want "right off the bat", working as a phone seller, to get into IT, do it quickly and with some more or less intelligible money, now you most likely will not succeed. But through testing you can really get into the industry, because Demand for testers is high, and requirements for juniors are indeed lower. This confirms this stereotype.
And after the person got “inside”, the daily, hard work begins, and here the stereotypes fail - you either like to do it or not. And it happens in different ways - someone stays in testing for a long time, someone quickly realizes that this is not his, someone understands that IT is not for him at all, “here you have to work with your head ©”.
In the market, by the way, there are many case studies of joint projects of companies with higher education institutions. Within the framework of such projects there is a niche training for their personnel needs. Through them, in principle, it is also easy to enter IT, especially for a student. But we must understand that most often you sharpen to the needs of a particular company.
- It turns out that the next steps in the industry are questionable?
Nikita Makarov: If you continue to develop, the next step within the company can be quite good and definite. I have a lot of examples of how we took interns and they shot very cool. They are immersed in the culture of the company, they understand what is required of them, they themselves begin to develop. This is cool. But if you study in the educational project of one company and go to the market, this is not always fair. The needs of the market are much broader than the needs of any one company.
The fact that there is a big drift from testers to programmers and related professions is indeed true. This is evident even in the age statistics in IT: there are quite a lot of software engineers aged 35–40 years, which is not the case with testers. A lot of testers go either to programming or to project management (recently, more and more project management is going away).
- Is there an influx of personnel from related industries, such as technical support?
Nikita Makarov: There is an influx, and technical support is one of the places where people can really be taken. Technical support is located at the end of the product supply chain and its employees know the product based on communication with customers.
True, a good technical support engineer on a fairly complex product, which has been doing this for years, will not always go to testing for at least money. A good support engineer gets not much less, and maybe even more.
- Is it true that the cadres themselves in testing for “quality” are much inferior to developers, analysts, etc.? If so, why do you think this happens?
Nikita Makarov: It was observed earlier, in the mid-90s - early 00s, when we had an era of mass production of fairly standard enterprise-software. Then there really was a fashion to hire people in the hundreds and thousands to test standard things using well-defined scripted documents, and the tester seemed like a “monkey with a test plan.”
But now it is not so - to scale in public is much more difficult than on hardware or technology, not to mention the fact that it is more expensive.
Everything is changing very quickly. Those who think that they will work as testers now or 10 years from now, as they did in 1995 - 96 or at the beginning of zero, are mistaken. Now, at the input level, there is indeed a low frame quality. But if the tester has been working in the industry for 3-4 years (especially if he has worked in more than one place), his baggage will quickly begin to grow with a variety of knowledge and skills.
- Automated testing helps to cope with a personnel problem (for example, a shortage of personnel)? Or rather, generates new ones?
Nikita Makarov: The shortage of employees means that there is a problem at the organizational level (no one thought about hiring additional employees). And then automated testing is likely to impose more problems on top of what you already have.
For automated testing should be treated with caution. Probably, automated testing can really solve the problem. But it certainly can ruin a lot of things, primarily due to the fact that some scenarios will not be considered in principle. In addition, support for automated testing is 80% of its total cost, and people are needed there. And the cost of engineers for automated testing in the market is comparable to the cost of middle-developers. And very often it is cheaper just not to do it.
If there is a desire to engage in automated testing, it should not be due to personnel problems. You need to know the exact answer to the “why” question, what needs to be obtained, and somehow analyze in the process the achievement of goals.
Successful projects for the introduction of automated testing generally begin with the development department. Developers themselves roll up their sleeves and grow additional competence. Knowing their product and their code, they build quite specialized things around test automation around it.
- How do you improve your skills and qualifications of subordinates?
Nikita Makarov: I'll start with a story about myself. To somehow improve the qualifications, it must not be lost. I have long been engaged in automated testing and application development around it. Now, when I do only 90% of my time with managerial tasks, I still try to devote time to reviewing the code. On the one hand, not to forget how it all looks, but on the other - to write something yourself. In other words, I try to be a “playing coach” in order not to lose my hold and understand the context: what the automated testing team lives with, what problems it works with, everything is all right there.
And I improve my qualifications with a very large number of reading and viewing professional videos. Every day, when I can, I try to set aside at least an hour or two (usually on the way to work and back) to watch a video, study professional literature — articles, books, blogs.
- And as for subordinates?
Nikita Makarov: Everything is somewhat more complicated with subordinates. They must first be brought to the conclusion that the qualification does not correspond (or will cease to correspond in the near future) to the tasks that they are trying to solve. And then work out a constructive solution.
In Odnoklassniki, we periodically send employees to internal and external courses. It is very good when they themselves come and ask for it (this means that the employee has already shot the thought in his head that he does not know something, and he even looked for where he could pull up knowledge on this topic).
In general, I still work with a small number of people where you can communicate with each individually, find out the desires and needs, relate them to the goals of the company.
- What do you think, is it possible in such conditions to build training on a university-wide basis?
Nikita Makarov: I think it will not work. The processes developed in educational institutions are generated from fundamental areas that have not changed for at least dozens of years, and sometimes for centuries. And training in our industry should be exclusively applied in nature, since the industry is changing very fast.
When I first got to work on a project as a tester, no one has ever heard of session based test management or context driven testing, because they have not been invented yet. And that was 10 years ago. At this time, the first companies in Russia were just starting to work on agile methodology, and at the word scrum, people did not have any sighs or hatred, because no one knew what it was.
A lot has changed in 10 years. You can say everything.
- A modern tester should be able to program, often, no worse than a regular developer. That is, it is no longer a tester, but a developer who writes code for testing. So who is a modern tester?
Nikita Makarov: A modern tester is a person who helps a project not to make mistakes. He does not indicate the project to its mistakes, but helps them not to do in the process. He actively participates in the whole operational activity of the team, communicates perfectly with all the participants, regardless of their roles.
For example, a good tester can shake out the necessary information from the administrator, put it into some speculative (albeit not always true, but fairly orderly) form in his head and take it to the developer. Thus, the tester can provoke a dialogue between the developer and the administrator, or between any other roles within the project.
A modern tester understands very well the architecture of the project at the level of squares / circles and all the numerous connections between them.
He may not know all the technical nuances, but he understands the principles of interconnections between individual moving parts within the project, the degree of their influence on each other.
Also, the tester understands the business objectives - strategic and tactical, which are solved by specific iterations or releases (in the short term), and can somehow coordinate them, understanding when they are not coordinated. A modern tester should have a well-developed planning skill. He must understand that he really will go into release and with what he will need to work, and what exactly will not fit. With this in mind, he understands exactly how to expend energy.
In essence, a good tester is a filling material in a project that provides feedback between parts of the process. He is an internal product integrator between the heads of all participants, who always knows more about the real state of affairs than other team members.
By the way, about the programming skill - it is really necessary, if only to make life easier for yourself. I think that it’s not worthwhile to put an equal sign between the tester and the developer, but both of them are engineers, in essence.
- Does Russia (in the post-Soviet space) have its own specifics in the field of testing?
Nikita Makarov: Here you can remember the catch phrase that all families are happy equally and unhappy in different ways. All good testers look the same, and all bad testers look very individual.
The portrait of a good tester is international. Due to this, good testers are now moving rapidly between countries. The last two years I have been seeing a serious outflow of personnel to Europe and the USA. Testers and test automation engineers leave the same way the developers left before.
Do we have any specificity? In Russia, this specificity applies not only to testers, but also to all participants in the development process. It lies in the fact that all work under conditions of increased stress. Therefore, stress tolerance among our colleagues is much higher (against the background of people in the West).
- Stress due to the fact that the industry, relatively speaking, somewhere in a hurry?
Nikita Makarov: Yes, we are trying to keep up or even overtake. In Russia, some companies produce products for a month that would have spent years in the West.
In Russia, at the level of all development processes, there is a big misfortune with an understanding of customer needs (this is more a business problem). Here they are trying to make the feature "at random", while somewhere in Sweden or Germany they think long over it, analyze it, and then do it without haste, knowing that it will shoot exactly, having some confirmation. Perhaps this stress is from here.
- At every corner they shout that we need: “to increase the percentage of product automation up to 80%,” “to achieve code coverage of 99%,” and other crazy slogans. In your personal experience, how much can such slogans be beneficial or harmful?
Nikita Makarov: For a start, these slogans are very stupid. I personally participated in the project where the coverage of unit tests was 10%. We raised it to 30% and stayed on this, because it didn’t make any further sense. Each next percent is a lot of money that will turn into nothing, because we did not add this percentage to any quality product.
Such slogans seem to me stupid, because they are trying under a common comb to scrape together all types of projects, regardless of their specifics. And there is specificity. Every time you have to turn on your head.
- And as a motivation slogans work?
Nikita Makarov: As a motivation, maybe. But here we need to understand such a thing: you can chant slogans at daily and weekly rallies - “let's raise the code coverage”, “let's cover 80% of the product with autotests” - but from this meeting each developer will go to his own little corner, put on good headphones and work with the product itself. And until he himself feels a profit, which is expressed through these slogans, he will not begin to do something. Here, in addition to slogans, there should be another example.
And the English-language literature is very capaciously called “lead by example”. From what I observe, this is probably the only survival strategy - a real living example that can be observed in dynamics. Those.slogans alone are not enough.
- What can you wish to all who are engaged in testing?
Nikita Makarov: To listen to business and learn.
Thus, in recent years, the testing industry has changed dramatically, and now there is no room for indulgence. This is an independent IT-direction, without which it is difficult to imagine the development of a modern product. By the way, one of the indicators that the industry has “grown” to a certain level of maturity is the emergence of separate technical conferences, such as the Heisenbag 2017 Piter , which will be held in June in St. Petersburg.
- Please tell us a little about yourself and your work. How are you related to testing?
Nikita Makarov: At the moment I am in charge of testing at Odnoklassniki. I have come to the current position in several steps. Initially, I was an engineer, but at some point, at the initiative of the technical director, the introduction of test automation began and I was asked to take up this direction.
')
The first four years I have been working exclusively on testing automation. When we won the biggest problems in this area, I was given the management of delivery. Those. I have become not only the head of test automation, but also a delivery manager who determines who, in what sequence and by what rules, will break production.
A year later (from April 2016), I began to lead all testing at Odnoklassniki, retaining the previous positions as well. As a result, the scope of my duties is quite wide - most often I can call myself a psychoanalyst in IT.
- The market is pushing to release products faster. Does it often happen at the expense of testing? Is this justified?
Nikita Makarov: Indeed, the speed of release of releases is very often the determining factor. This is due to the fact that it is very difficult to find a fresh idea now and throw it onto the market. It is doubly difficult to do it alone, so that no one will know about it and try to copy it (copying ideas, using a slightly more fragmented niche audience can also be successful; a vivid example is Uber, Yandex.Taxi, GetTaxi and a bunch of similar services around the world). To enter the market faster, it is often necessary to shorten the testing phase.
In varying degrees in mass products, users are ultimate testers. About what you have done wrong, that the product works poorly, you can learn from feedback from users through the market, feedback forms, etc.
Is this justified? In terms of speed, yes. From the point of view of the audience of products - not always, not every product can be sold that way.
I focus on what we are talking about products for the mass user market - not life critical, not mission critical - about things like Instagram, Facebook, Uber, etc. In such products, in case of failure, someone may not lose very much money. In the areas of life critical solutions, testing, as well as the development itself, is very rigidly defined by the standards, so there it is not always possible to cut off something. And if it turns out, the cost of the error here can be very large.
One example is the case of the Knight Capital Group : within 45 minutes, the guys lost about half a billion dollars. A completely recent case is the DoDo Pizza, when inaccurately depot the service rolled back transactions for several million rubles.
How justified is this risk? Every business decides in its own way.
- That is, the amount of testing determines the testing department in a dialogue with the business? Who, in this case, should understand the interlocutor's language, putting forward arguments clear to him?
Nikita Makarov: The initiative is here on the testing side. The testing department, which does not understand the language of business, is obviously dead. The testing department in itself does not earn money (unless we are talking about selling this department to outsourcing). Accordingly, it should contribute to the reduction of losses or greater earnings. Therefore, the correct selection of arguments for business is one of the main qualities of a good tester, especially in product projects.
- How do you feel about the idea that the way in IT is easier to navigate through testing? They say that requirements are lower there, market demand is higher. And after a couple of years, you can do programming when you have mastered all the processes, entrenched in the profession?
Nikita Makarov: This is a stereotype. On the one hand he is faithful, on the other - not.
If you want "right off the bat", working as a phone seller, to get into IT, do it quickly and with some more or less intelligible money, now you most likely will not succeed. But through testing you can really get into the industry, because Demand for testers is high, and requirements for juniors are indeed lower. This confirms this stereotype.
And after the person got “inside”, the daily, hard work begins, and here the stereotypes fail - you either like to do it or not. And it happens in different ways - someone stays in testing for a long time, someone quickly realizes that this is not his, someone understands that IT is not for him at all, “here you have to work with your head ©”.
In the market, by the way, there are many case studies of joint projects of companies with higher education institutions. Within the framework of such projects there is a niche training for their personnel needs. Through them, in principle, it is also easy to enter IT, especially for a student. But we must understand that most often you sharpen to the needs of a particular company.
- It turns out that the next steps in the industry are questionable?
Nikita Makarov: If you continue to develop, the next step within the company can be quite good and definite. I have a lot of examples of how we took interns and they shot very cool. They are immersed in the culture of the company, they understand what is required of them, they themselves begin to develop. This is cool. But if you study in the educational project of one company and go to the market, this is not always fair. The needs of the market are much broader than the needs of any one company.
The fact that there is a big drift from testers to programmers and related professions is indeed true. This is evident even in the age statistics in IT: there are quite a lot of software engineers aged 35–40 years, which is not the case with testers. A lot of testers go either to programming or to project management (recently, more and more project management is going away).
- Is there an influx of personnel from related industries, such as technical support?
Nikita Makarov: There is an influx, and technical support is one of the places where people can really be taken. Technical support is located at the end of the product supply chain and its employees know the product based on communication with customers.
True, a good technical support engineer on a fairly complex product, which has been doing this for years, will not always go to testing for at least money. A good support engineer gets not much less, and maybe even more.
- Is it true that the cadres themselves in testing for “quality” are much inferior to developers, analysts, etc.? If so, why do you think this happens?
Nikita Makarov: It was observed earlier, in the mid-90s - early 00s, when we had an era of mass production of fairly standard enterprise-software. Then there really was a fashion to hire people in the hundreds and thousands to test standard things using well-defined scripted documents, and the tester seemed like a “monkey with a test plan.”
But now it is not so - to scale in public is much more difficult than on hardware or technology, not to mention the fact that it is more expensive.
Everything is changing very quickly. Those who think that they will work as testers now or 10 years from now, as they did in 1995 - 96 or at the beginning of zero, are mistaken. Now, at the input level, there is indeed a low frame quality. But if the tester has been working in the industry for 3-4 years (especially if he has worked in more than one place), his baggage will quickly begin to grow with a variety of knowledge and skills.
- Automated testing helps to cope with a personnel problem (for example, a shortage of personnel)? Or rather, generates new ones?
Nikita Makarov: The shortage of employees means that there is a problem at the organizational level (no one thought about hiring additional employees). And then automated testing is likely to impose more problems on top of what you already have.
For automated testing should be treated with caution. Probably, automated testing can really solve the problem. But it certainly can ruin a lot of things, primarily due to the fact that some scenarios will not be considered in principle. In addition, support for automated testing is 80% of its total cost, and people are needed there. And the cost of engineers for automated testing in the market is comparable to the cost of middle-developers. And very often it is cheaper just not to do it.
If there is a desire to engage in automated testing, it should not be due to personnel problems. You need to know the exact answer to the “why” question, what needs to be obtained, and somehow analyze in the process the achievement of goals.
Successful projects for the introduction of automated testing generally begin with the development department. Developers themselves roll up their sleeves and grow additional competence. Knowing their product and their code, they build quite specialized things around test automation around it.
- How do you improve your skills and qualifications of subordinates?
Nikita Makarov: I'll start with a story about myself. To somehow improve the qualifications, it must not be lost. I have long been engaged in automated testing and application development around it. Now, when I do only 90% of my time with managerial tasks, I still try to devote time to reviewing the code. On the one hand, not to forget how it all looks, but on the other - to write something yourself. In other words, I try to be a “playing coach” in order not to lose my hold and understand the context: what the automated testing team lives with, what problems it works with, everything is all right there.
And I improve my qualifications with a very large number of reading and viewing professional videos. Every day, when I can, I try to set aside at least an hour or two (usually on the way to work and back) to watch a video, study professional literature — articles, books, blogs.
- And as for subordinates?
Nikita Makarov: Everything is somewhat more complicated with subordinates. They must first be brought to the conclusion that the qualification does not correspond (or will cease to correspond in the near future) to the tasks that they are trying to solve. And then work out a constructive solution.
In Odnoklassniki, we periodically send employees to internal and external courses. It is very good when they themselves come and ask for it (this means that the employee has already shot the thought in his head that he does not know something, and he even looked for where he could pull up knowledge on this topic).
In general, I still work with a small number of people where you can communicate with each individually, find out the desires and needs, relate them to the goals of the company.
- What do you think, is it possible in such conditions to build training on a university-wide basis?
Nikita Makarov: I think it will not work. The processes developed in educational institutions are generated from fundamental areas that have not changed for at least dozens of years, and sometimes for centuries. And training in our industry should be exclusively applied in nature, since the industry is changing very fast.
When I first got to work on a project as a tester, no one has ever heard of session based test management or context driven testing, because they have not been invented yet. And that was 10 years ago. At this time, the first companies in Russia were just starting to work on agile methodology, and at the word scrum, people did not have any sighs or hatred, because no one knew what it was.
A lot has changed in 10 years. You can say everything.
- A modern tester should be able to program, often, no worse than a regular developer. That is, it is no longer a tester, but a developer who writes code for testing. So who is a modern tester?
Nikita Makarov: A modern tester is a person who helps a project not to make mistakes. He does not indicate the project to its mistakes, but helps them not to do in the process. He actively participates in the whole operational activity of the team, communicates perfectly with all the participants, regardless of their roles.
For example, a good tester can shake out the necessary information from the administrator, put it into some speculative (albeit not always true, but fairly orderly) form in his head and take it to the developer. Thus, the tester can provoke a dialogue between the developer and the administrator, or between any other roles within the project.
A modern tester understands very well the architecture of the project at the level of squares / circles and all the numerous connections between them.
He may not know all the technical nuances, but he understands the principles of interconnections between individual moving parts within the project, the degree of their influence on each other.
Also, the tester understands the business objectives - strategic and tactical, which are solved by specific iterations or releases (in the short term), and can somehow coordinate them, understanding when they are not coordinated. A modern tester should have a well-developed planning skill. He must understand that he really will go into release and with what he will need to work, and what exactly will not fit. With this in mind, he understands exactly how to expend energy.
In essence, a good tester is a filling material in a project that provides feedback between parts of the process. He is an internal product integrator between the heads of all participants, who always knows more about the real state of affairs than other team members.
By the way, about the programming skill - it is really necessary, if only to make life easier for yourself. I think that it’s not worthwhile to put an equal sign between the tester and the developer, but both of them are engineers, in essence.
- Does Russia (in the post-Soviet space) have its own specifics in the field of testing?
Nikita Makarov: Here you can remember the catch phrase that all families are happy equally and unhappy in different ways. All good testers look the same, and all bad testers look very individual.
The portrait of a good tester is international. Due to this, good testers are now moving rapidly between countries. The last two years I have been seeing a serious outflow of personnel to Europe and the USA. Testers and test automation engineers leave the same way the developers left before.
Do we have any specificity? In Russia, this specificity applies not only to testers, but also to all participants in the development process. It lies in the fact that all work under conditions of increased stress. Therefore, stress tolerance among our colleagues is much higher (against the background of people in the West).
- Stress due to the fact that the industry, relatively speaking, somewhere in a hurry?
Nikita Makarov: Yes, we are trying to keep up or even overtake. In Russia, some companies produce products for a month that would have spent years in the West.
In Russia, at the level of all development processes, there is a big misfortune with an understanding of customer needs (this is more a business problem). Here they are trying to make the feature "at random", while somewhere in Sweden or Germany they think long over it, analyze it, and then do it without haste, knowing that it will shoot exactly, having some confirmation. Perhaps this stress is from here.
- At every corner they shout that we need: “to increase the percentage of product automation up to 80%,” “to achieve code coverage of 99%,” and other crazy slogans. In your personal experience, how much can such slogans be beneficial or harmful?
Nikita Makarov: For a start, these slogans are very stupid. I personally participated in the project where the coverage of unit tests was 10%. We raised it to 30% and stayed on this, because it didn’t make any further sense. Each next percent is a lot of money that will turn into nothing, because we did not add this percentage to any quality product.
Such slogans seem to me stupid, because they are trying under a common comb to scrape together all types of projects, regardless of their specifics. And there is specificity. Every time you have to turn on your head.
- And as a motivation slogans work?
Nikita Makarov: As a motivation, maybe. But here we need to understand such a thing: you can chant slogans at daily and weekly rallies - “let's raise the code coverage”, “let's cover 80% of the product with autotests” - but from this meeting each developer will go to his own little corner, put on good headphones and work with the product itself. And until he himself feels a profit, which is expressed through these slogans, he will not begin to do something. Here, in addition to slogans, there should be another example.
And the English-language literature is very capaciously called “lead by example”. From what I observe, this is probably the only survival strategy - a real living example that can be observed in dynamics. Those.slogans alone are not enough.
- What can you wish to all who are engaged in testing?
Nikita Makarov: To listen to business and learn.
Thus, in recent years, the testing industry has changed dramatically, and now there is no room for indulgence. This is an independent IT-direction, without which it is difficult to imagine the development of a modern product. By the way, one of the indicators that the industry has “grown” to a certain level of maturity is the emergence of separate technical conferences, such as the Heisenbag 2017 Piter , which will be held in June in St. Petersburg.
Source: https://habr.com/ru/post/327362/
All Articles