“To achieve change, understand why people resist them”: Jim Holmes on the culture of testing

What could the army teach the tester? What are the two extremes in testing approaches? How to explain that technical debt payment is red? What do the previous questions have in common?

The general thing is that for all their difference, they are all close to one person. Jim Holmes has several decades of IT experience, which began in the 80s in the US Air Force - no wonder that he is ready to tell a lot. For him, the concept of “testing culture” is important, and we asked him questions that can vary greatly, but ultimately in one way or another connected with the culture of testing.

- Introductory question: tell us about yourself.
')
- My name is Jim Holmes, I am an independent consultant and owner of my own company, Guidepost Systems. I specialize in the quality of software delivery and in total I have been programming for about 20 years, before that I served in the army for a long time. I was seriously interested in quality about 15 years ago, while working on several not very successful projects. In general, I did a lot of test automation, unit tests, integration testing and, in particular, functional testing, UI testing. In particular, I worked for a company that wrote a great tool Telerik Test Studio. And in the last 5-8 years, I became involved in not only purely technical aspects, but also the quality of the delivery as a whole - that is, how well the communication between the supply team and the business is established. But at the same time, I still debug WebDriver scripts for a long time and solve problems like periodic failures due to incorrect asynchronous actions. I live in the city of Ashland in Oregon - note, not in Portland (when in the United States you say "Oregon", usually everyone thinks "Portland").

- Probably, for the tester the most unusual part of your biography is military service. What exactly did you do there?

- I served in the USAF because I was always inspired by airplanes and my father was a pilot. He took me on my first flight when I was only 4 years old. In the Air Force, I was very lucky, I served on the E-3 team - this is such a big plane with a huge disk on top, where I was responsible for the work of the radar and some communication systems. For eleven years I have been administering and debugging this large electronic system. And during the free flight, I administered the computers of our squadron. About then it became possible to create networks in the workplace. In addition, since I was a soldier, I was paid for my education, so I studied at an evening school in software development.

As for your question, the experience of military service is unusual not only for testers, but also for many other areas. I can say that at least they taught me discipline there.

- Yes, in the case of the army, the first thing that comes to mind is discipline. And with this, many testers and developers have problems. Do you think they have something to learn from the military?

- This is a very interesting question. The fact is that the discipline that I was taught was somewhat different than, say, the paratroopers, that is, there were no constant shouts and punishments. I was taught a disciplined approach to testing and, in general, to solving problems. Also discipline in the workplace was important - understanding what is a hierarchy; to learn how to work in it, you need to be able to communicate with the authorities. This kind of discipline helped me a lot.

I left the army 25 years ago - yes, I am an old man - and the work in the civilian sector contrasted very interestingly with my previous experience because of what priorities there were. In the army, I served on the plane, which was supposed to monitor the enemy over a large area and ensure their safety. Any mistake meant a risk to the lives of our soldiers. I don't want to exaggerate, but it really was. So, when in IT someone starts panicking and demanding that some problem be solved this very second, thanks to my discipline and my experience, I can always calm people down - we are not threatened that someone will die, and we will not lose hundreds of thousands of dollars a minute (and I have been in such situations). We very often panic and allow business or shareholders to intimidate us unnecessarily and create unreasonable tension, which does not bring anything good to anyone. Perhaps the best thing that the army taught me is the ability to call for calm, sensibly assess the situation and pre-plan their actions, and not rush to the task without thinking because of the artificially created panic.

- That is, you need to devote time to thinking through a strategy.

- Yes, and in IT this problem is already a very long time: constant demands to submit the project this very second. The main thing is to do it quickly, and what exactly is being done is secondary. Most often, this pressure can be controlled if the dialogue is conducted correctly. But for this it is important to be able to say that we will create more value if we spend a little more time and rationally approach the problem.

- Let's get to your current activity. You are familiar with the work with how differently they approach testing in different companies. Clearly, the approach may depend, for example, on the size of the company. But for sure there are many less obvious differences - can you tell about them?

- This is a great question. In addition to my independent work, I also collaborate with Pillar Technology from Columbus, Ohio. I have been acquainted with people from this company for a long time. Now there are about 250 people working there, and almost all of them work on the principle of extreme programming (XP): they have a lot of unit tests, and the development is very well thought out. When I was hired there three years ago, I was the only tester, and they created a very high-quality product. They approached testing only from the position of XP, from the standpoint of test-driven development. This is a great approach, and they used it quite successfully, and I, together with some other employees, managed to change some aspects of software delivery.

And you can compare this experience with another company in which I have been consulting for the last three years. This is an industrial company, it is in the top ten of the Fortune list, and employs about 200,000 people worldwide. They have a very large IT department for internal company needs. There are a lot of good people there, but their testing is stuck on the practices of the 1980s or 1990s. The company produces huge batches of exactly the same products, and many there are used to the fact that by producing, say, ball bearings, it is quite possible to estimate the number of expected defects. But the problem is that this kind of thinking they transfer to IT.

I had a conversation with four, in general, very smart employees who were engaged in collecting reports on defects from several projects and trying to predict the number of possible defects in the future. I told them that it is quite reasonable to do this in the industry, but how will they distinguish the indicators calculated for the mobile application from those of, say, web services? I was told that there is no difference. So I was lucky to see absolutely opposite points on the spectrum of different approaches and testing cultures.

In addition, I worked with some companies that could not manage to get out of the startup stage, when all the time you work without thinking, not trying to cover the whole situation. And then, after a few years, in complex projects, the problems that have arisen at this stage begin to be acutely felt, and I have to convince people that this approach is wrong and that now we need to stop and think about how to act. In general, my answer turned out to be somewhat extensive, I hope I have not completely lost touch with your question.

- And in your consulting practice, there are cases when, after the completion of work with the company, you were told that “nothing has become better”?

- I will tell you in secret that it’s hard with people, we are very difficult creatures. This is one of the reasons why I have so much gray hair (the second is my daughter). Changing one person is difficult, changing people in an organization is even more difficult. So yes, this situation I have quite often.

We even say about some consultants that he “burned out”. This is due to the fact that we are making a lot of efforts to change the culture that has developed in the organization, and we have to work a lot with problems on a personal level - people are afraid of changes, they constantly have to be convinced that it is necessary and look for the path that suits them. No matter how strong and loud I am, I can’t just come and say: do this and that. We need to come to a consensus. This work needs to be carried out over the years in order to at least achieve something, and when it seems that you have solved the problem and you leave to advise to another organization, then you meet there exactly the same problems as in the previous place. You again spend a huge amount of time and effort, and then it turns out that in that first company everything has already returned to its former condition.

People are so arranged, with us it is hard, we often return to our old habits. But despite this, there are examples of really successful work. There are organizations that manage to consolidate the results achieved with you. In general, I would say that one balances the other. I continue to do this work because these cases of success bring great satisfaction.

- In your blog you wrote about your professional principles, and there it was about the need to be open, always learn new things, listen more than speak, adapt and at the same time be kind to people. I am wondering if kindness towards people really helps in IT?

- Yes, it helps. I said that I served in the army, where we had a tough discipline - but we must understand that discipline and structuredness in no way interfere with good relations and empathy for other people. Do you know the Scottish chef Gordon Ramsay? He hosts Hell's Kitchen. I mention it because chefs in expensive restaurants very often behave disgustingly with employees - they shout, they insult. I hate this attitude to people. For me, a good attitude towards each other is important. If you want to make any changes, then you need to understand why people resist them, that is, you need empathy. And it will allow you to make the person himself want to change. This approach is much more effective than screaming at people and demanding that they listen and not ask questions. Each person has his own mind, his own soul, his own experience. I would not like to go into philosophy, but I believe that in the long run, good attitude and empathy will allow you to achieve great results. So yes, they help.

- You hold seminars, and at one of them you teach programming to people who cannot program. I have two questions. First, what problems most often prevent people from programming themselves? Second, do you think that automated testing will prevail over manual testing over the next year?

- In my experience, people are often not hampered by technical difficulties, but by fear. I can give an example from my experience in a company from Fortune 10, about which I have already spoken. I worked with a team of testers who were involved in manual testing, and we needed to automate using WebDriver. Of these, most could not even open Eclipse to start writing code. They were afraid to write a script for automation, because in their understanding it was not much different from writing a scalable web service or database with multithreading, that is, things that developers have been learning for years. This fear prevented them from doing in general uncomplicated things.

I am developing a course for Ministry of Testing based on a seminar in which I explain that you do not need years of practice to simply open a Java file, C # or a Ruby script, read it and understand the general structure, or write simple test on webdriver. Moreover, even if you do not fully understand everything that happens in the code, you can give a general assessment to it, because you know, for example, that one method should not occupy three screens, there should not be four nested if statements - they will be difficult test, you can not give variables the names of one letter, and so on. In my opinion, at the initial stage, the main difficulty is to overcome the fear that you will have to solve incredibly complex tasks. And it was always interesting to me to see how quickly my clients get rid of this fear - you just need to give the person a simple unit test and ask him to write arrange, act and assert. I think this human component is very important.

Most of the technologies we work with are not so complicated, few people do things like unmanned vehicles. I once held a seminar for a single client in India, and there was one manager without any programming experience. This manager was at first very angry, and the reason was just in the fear that I was talking about now, and this fear was imposed on the reluctance to seem silly. But by the end of the first lesson, he plunged into writing tests so much that I had to expose him from the audience an hour after the lesson was over - he sat with his head in the monitor, paired with another participant of the seminar and wrote simple tests for the wage algorithm fees. And the point here is not at all in my teaching skills - it just shows how much this fear plays or its absence. Here we are talking about things embedded in human nature.

Your second question related to the transition from manual to automated testing. I have a certain experience here, for three years I worked in a company that has been automating testing. I believe that the question should be put differently and should not be aimed at automating testing, but towards developing testers and gaining many technical skills, one of which should be automated testing. I would like to see testers who, having a user history, specification and requirements, could freely engage in dialogue with developers about quite specialized things and jointly look for solutions that will then be embodied in the code. This is somewhat different from the approach in which the tester teaches programming, just to be able to write WebDriver scripts. This skill, of course, is also important, but, in my opinion, it is not the most important. The most important task, in my opinion, is the ability to conduct a dialogue with the developer and encourage him to refer to the testing process when he writes code. Even TDD will already be a significant result - I believe that all organizations should be able to write like that.

The bottom line is that properly delivered testing is far from boiling down to unit tests or integration tests. Very often, people are satisfied with testing the typical code execution path - all tests are passed, ticks are everywhere, 100% code coverage is achieved. But after all, none of this guarantees the quality of the code, does it? Although this is also a necessary procedure, of course. So, in my understanding, a tester should not just write a few functional tests or tests in WebDriver, but also think about how to establish closer cooperation with developers and business analysts. You can, for example, try mob programming . In general, I believe that automation is a tool, not a goal.

- The words “close cooperation with developers” are very close to us as Heisenbug organizers - even the conference slogan is “about testing, but not only for testers”. And in connection with this collaboration, I would like to ask this question: what would you like as a person with extensive testing experience would like to say to our programmer readers?

- That we are not all evil! Testers by their own actions have earned themselves notoriety. They find fault with the smallest details at meetings, often communicate with bugs, not words. The developer comes to work in the morning and sees 50 bug reports about grammatical errors and unaligned pages. I have been in both roles, and I know how programmers react to this. This is part of the old school of testing, and I myself also once behaved like this. But testers who are more accustomed to the modern approach understand that than filling a programmer with bug reports, it is easier and better to just talk to him.

In my opinion, it is important for developers to understand that a good tester can be very valuable, and if you talk to him in advance, you can save yourself a lot of time and nerves. This is what the theory of constraints says. Suppose I am a developer, and I see that the tester found some problem. If, instead of corresponding with her in TFS or elsewhere, I just go to him and talk personally, then this conversation will help me reduce the number of defects in the code I am writing. In addition, in personal communication, most testers are usually far from being so terrible.In general, you, as a developer, will be able to improve the quality of your code if you get rid of common stereotypes about testers - which, unfortunately, often have a basis, and we also need to work on this.

Therefore, I really liked that in your conference you are trying to gather people with different specializations. This is very important, because for a long time all the major conferences, both professional and community-based, have severely limited themselves — either only testers or developers. If we start to communicate with each other, it will be a great conference.

, , : Heisenbug , Fortune 10. , ? 6-7 Heisenbug , .

— . «The leadership journey» . , : , , — . ?

- I think this is a common mood for IT: it does not matter whether you are involved in database administration, programming, testing, or project management. You have work that you do well, you feel comfortable, you have experience, you have influence in this role. Compared to this, the need to take on more responsibilities and, oh, horror, working with people, sounds much less comfortable and causes rejection. Therefore, at the very beginning of my book, I suggest ways to find out if you really need all this, and what it means to be a leader.

Some leader functions can be performed without moving up the hierarchy. A person who within the team has more experience and skills than others will be in this team informally playing the role of leader. So it's not necessarily about becoming a manager: perhaps you just think that you need to have a lot of influence in your team. Although in some organizations it is taken for granted that, having gained some experience, a person begins to occupy leadership positions. He is made a technical manager, and, in addition to writing code, he is now responsible for the qualifications of his team. And at higher levels, he generally stops writing code. So, in general, leadership can take many different forms. The specificity of our area is that we are afraid to occupy leadership positions,because we are afraid of losing our technical skills because we stop writing code or testing. In addition, if the tester is not engaged in the development, it will be terrible for him to lead programmers. All these problems are completely natural, overcoming them is an integral aspect of leadership in our industry.

- And one more question comparing developers and testers. Among the developers, there is an active discussion regarding remote work: some believe that this should be done a long time ago by default, while others object that live communication cannot be replaced with Skype. You work remotely - what is your opinion about the remote in testing?

“Of the 18 years since my daughter’s birth, I worked remotely at 13 or 14. So I really have an opinion on this issue.” In my opinion, remote work is an important tool, it allows you to expand the circle of people with whom you can potentially work. For example, with a person who lives in another state or on another continent. So I am a supporter of remote work, in some situations it is very useful.

At the same time, it is necessary to recognize that during remote work, there are significant problems with communication and productivity. Purely from a technical point of view, a team that works completely or mostly from a distance should have a good infrastructure. Each member of such a team should work on the principle of DevOps. There should not be a situation where we cannot contact the server administrator in days to get logs for debugging. In addition, you need to seriously approach the organization of means of communication. And if you decide that you have enough Skype and some messenger, did not take Skype for Business and use the free version of Google Hangouts, then you will have problems. Remote work requires good infrastructure. In addition, you are required more responsibility and discipline. Yes,you can work in your pajamas, but you have to work and not play on the Xbox for days on end - I’m ashamed to admit it, but I say this from my own experience.

In addition, conversations in the corridors and live communication are an integral part of any kind of work at all - people are just like that. And I must admit that here remote work has a significant drawback. Let you have hangout running on Slack and live video chat - this is all wrong. This problem is solved, but this requires time and discipline, and for each new team member this problem must be solved anew.

Over the past 20 years, I have worked at a distance of two times more than in the office, as I said, this is a great tool. But at the same time it is necessary to have a clear idea of ​​the advantages and disadvantages of such work. In addition, the organization needs to bring the entire team together for live communication at least four times a year in order to build connections that you cannot build over the Internet. Everyone needs to sit in the same room, work, argue, have fun. All this is very important for creating a well-coordinated team. If you think that remote work is just a way to reduce costs, you will be disappointed. I again had a rather long answer, but what to do, you ask interesting questions.

— «Technical debt: payment plan». « » , « » , . , , , , ?

- This is a very important question. I think that anyone who has worked at IT for a long time has necessarily faced the problem of technical debt. Here I need to thank Trish Khoo, she can be found on Twitter as hogfish. She is an excellent tester, I have known her for many years. She, in fact, wrote the post “Technical debt: payment plan”. I also wanted to write about it when I read this post, because for five years I made speeches at conferences where I expressed, in general, similar thoughts.

So, in our industry there is a technical problem for a long time, we all have to think how to avoid it and what to do with it when it does arise. In my opinion, your proposal to think about it as a financial debt has very important consequences. If you do the appropriate research, you will see that this technical debt does lead to financial costs. Due to technical debt, your project takes more time on X, more people work on it on Y, and they all need to pay a certain salary. That is, these costs are quite measurable, although they may not be so easy to evaluate. Or, suppose you had to fix 30 bugs within a month and a half due to problems with technical debt. This, again, leads to an increase in the price of the project in dollars. Moreover,we are talking not only about the costs in dollars, but also about the opportunity costs: for one and a half months you could do something else.

Many years ago I worked in a company where for every 10 days of work on new functionality, there were 3 days to correct the problems with quality and technical debt. And for every 3 days, there was a day of correction. There were problems that were rediscovered many times. It all cost money. I left this organization, so that during the time I worked there, I could not change the culture that had developed there and make them admit that such a situation was unacceptable. The last straw was my unsuccessful attempt to explain to the director that out of three years that I worked there, we lost a year due to the correction of problems related to technical debt. I was not able to convey to him my thought, he turned the conversation to another topic. I believe that people working in ITone must learn to better explain the significance of technical debt and the costs to which it leads. We cannot just sit in our corner, work and not communicate with anyone. We need to learn to speak a language that will be understood by business. A business understands the language of dollars and lost time. Business begins to think when sales or renewal of licenses are falling, and this is due to a drop in quality, which, in turn, is caused by technical debt.

The cause of technical debt is excessive pressure, attempts to save on important things and the fact that business does not understand the meaning of its decisions. Although our fault in this also happens when we do not have the appropriate skills, when we do not follow generally accepted recommendations, and when we do not know how to conduct a dialogue with business. Learning to formulate technical debt in terms of financial debt, in my opinion, is a very important skill that we all need to work on. I repeat, you asked a very good question.

— , , , . , , , .

- This problem has two sides. It is not clear to business why to refactor and why to write automated tests. They only see that people spend on writing tests twice as long as writing code, and this seems monstrous to them. This is already a matter of education, so it is necessary to explain that development is a process that has been established for decades. There are statistics and studies that show that attempts to save on testing directly lead to costs in dollars due to delays and corrections. Business does not understand this.

For our part, we did not learn how to properly communicate with business, we did not learn how to convince. In addition, we often lack perseverance to explain that some things are not discussed, they cannot be ignored or put off. There are safe and productive practices that have developed historically, and if you, managers, follow them, in the end, you will be happy, because a great value will be created. This is a tough conversation, and we, IT professionals, do not know how to conduct it.

“Therefore, it is always good when a manager or team leader has development experience - then they understand these things.” A good project is easier to create when management has technical experience.

- In that company from Fortune 10, about which I spoke above, we had exactly the same problem. The lower and middle managers did not understand why testing is so important. We managed to restart the project and reach an agreement on when the project should be considered ready, that is, we managed to include managers in the dialogue. We have determined that the user history will be considered finished when there is documentation, when the support is well acquainted with the new functionality, when the product the operator checked the fulfillment of the requirements, which he also wrote with the help of Acceptance Testing.

In addition, we have achieved the consent of the management to implement specific technical measures, which he did not understand. We agreed that the code should meet the standard indicators of static analysis (for example, analysis using the Sonar Cube), that is, it should not be too complicated, there should not be too many lines in the methods. Managers realized that in order to maintain a healthy code base, industry standards should be followed for how the code should look. In addition, in our definition of a finished project there was the phrase “necessary testing”, that is, we did not specify a certain number of tests to be performed, or a certain code coverage by automatic tests to be achieved. Thanks to this, the project team leader, the developer and the tester could determine with respect to each module,what testing will suffice. Each time it happened in the course of a dialogue, and managers for a long time could not understand that this again corresponds to generally accepted practices in the industry.

- At this our questions are over. Thank you very much for your detailed answers.

- Thank you, I liked your questions. I think it was obvious: you managed to raise a lot of topics that are very close to me personally.