Crossing Testing and Architecture: An Interview with Neil Ford

What does the QA architect post mean? And what does the completely incomprehensible post “meme wrangler” mean? From what moment do testers need to be connected when working on the architecture? How to change the processes in the organization so that people at the meeting with the first difficulty did not return to the old?

Neil Ford on his website presents three options: “ThoughtWorker” (an employee of ThoughtWorks, which many people know because of Martin Fowler), “Software Architect” and “Meme Wrangler”. Soon at our conference Heisenbug, he will talk about the creation of "evolutionary architectures" that can be changed by changing external circumstances. In the meantime, Mikhail xomyakus Druzhinin (a member of the Heisenbug program committee) asked Neal about how his presentation interferes with testing, and about many other things.

- Can you first talk about yourself and your career?
')
- Of course. I have been at ThoughtWorks for almost 14 years. Before that, he was the technical director at a small consulting and training firm of about 25 people, and there he began to try his hand at technologies like Java. The first three books I wrote before I joined ThoughtWorks. The very first was about Delphi, at that time quite popular in Russia and in Europe.

When I was the technical director, I got tired of being an alpha geek. When you know more than the people around you, you are not particularly developed. I started speaking at a lot of conferences, and I really enjoyed being among other speakers, because they were very smart and interested people. And I began to unconsciously look for a company in which really smart and interested people would work. And literally stumbled over ThoughtWorks, like many, on the Martin Fowler website. For things like CruiseControl and the NUnit library for testing, I noticed that they make a huge contribution to open source and do a lot of very interesting things. And so I inadvertently began the process of interviewing ThoughtWorks, at the end of which I received a job offer. It was a good step for me, because they are very smart and very enthusiastic developers.

I could be an independent consultant, but in this role I don’t like two things. The first is that you yourself must be involved in all business affairs: billing, chasing people for money, adjusting cash flows and all that. I am able to do this, but it does not fascinate me. My passion is software and development.

And second: if you are an independent consultant, then you rarely manage to work in collaboration with someone in a team. You're almost always on your own, and I really like team development. I even wrote several books together with other authors, including my latest book, Evolutionary Architecture. It seems to me that when working together, the result is greater than the sum of the individual components. When you collaborate in creative work, be it a writing project or a program, you can get more points of view and a more general idea, so the result will be better.

So in April, it will be 14 years already, like me at ThoughtWorks. I know this so well, because one of the interesting advantages of working at ThoughtWorks is that if you have been with the company for 10 years, you get 12 weeks of paid sabbatical leave, when you can do whatever you want. And after that, every 5 years you get a 6-week paid sabbatical, so I had a year left before the first 6-week. I really liked the 12-week week, and I look forward to the next one. So you always remember what a dozen years you have been here: they are measured by such pleasant vacations.

- This is a very tangible vacation length, cool.

- I was hired by ThoughtWorks for the position of architect and served this role for a while, until I switched to the director level. Most of my work now belongs to the field of software architecture, I spent a lot of time at the intersection of architecture and engineering practices (for example, continuous delivery) - I suspect that here my interests overlap with those of the Heisenbug audience.

In particular, what I have talked about a lot lately is the theme of my most recent book, the construction of evolutionary architecture. If you can prevent the appearance of bugs, then people who are engaged in testing the application will be easier. Evolutionary architecture includes protection techniques for critical architectural properties — such as deployment, testability, scalability, performance, and so on.

“What does Meme Wrangler mean?”

- According to the rules of ThoughtWorks, you can choose any job title, except for a few reserved ones like “CEO”. If you invent an interesting post for yourself, then please. And when the set of business cards is over, you can come up with a new one.

My first job was an application architect. This position reflected the essence of the work, but very often in large companies, it implies that you no longer bring any special benefit, that you spend more time drawing pictures than creating something.

And one of the advantages of custom development is that you can get acquainted with a large number of different projects. Software architects, I think, by nature are such “pattern matchers”: we try to apply patterns to everything we see, even to things from the real world. So, if you are an architect in the development of custom, you can see a lot of different projects, you see in them repeated patterns, see which ones work. And I realized that, in fact, my work is something of a collection of interesting ideas from the software ecosystem. From here went "Meme Wrangler".

Meme is a term coined by the writer Richard Dawkins; it is a common unit for designating ideas. Everyone is familiar with memes from the Internet - this is a kind of witty thing that catches and spreads like a virus. And the word “wrangle” has two useful meanings, the first is to act as a judge in a dispute, and the second meaning is “to drive into a herd”. And I chose the position of Meme Wrangler because it more accurately reflects what I am doing now. Moreover, now, when I am releasing another book, I tweet that I “lashed out another meme” and put it in this book.

- Can you explain what the software architect should do, what do you do as a software architect?

- Of course, I can try, but no one really can not give this definition. Martin Fowler - a man who is very cool at defining everything in the field of programming - publicly refused to define the term “software architect” in his text “Who needs an architect?” .

But if you look at the role of the software architect, then one of the things is that they are responsible for everything that is difficult to change later. This is all related to the structure of your software system: which underlying patterns you will use, which language or which frameworks. Because all these decisions have far-reaching consequences.

One of the things that architects really care about very much - what we call “architectural parameters” is also called “non-functional requirements”, but we don’t like this name. These are things such as performance, scalability, elasticity, deployment - all this is the responsibility of the architect. The architect can create a structure that will take into account the requirements for the program itself, and all these architectural parameters that should be provided in it.

Suppose you are an architect and you need to create a software system structure for registering students at the university. Suppose we have a thousand students and 10 courses for which they must enroll. And now, based on our knowledge of how universities are organized, what you think will happen: the students will be distributed evenly throughout the semester so that we have the same number of students in each course, or they will all pull until the last hour, and then rush recorded all at once?

And this is elasticity, this is an architectural parameter that you must ensure when you make such a system. Most likely, this is not indicated anywhere, but you know it, simply on the basis of the subject area. This is what makes the software system architecture so complex: you need to understand the subject area, as well as the technical capabilities and limitations of your company. You may be limited, for example, by the fact that this relational database has already been selected for the standard of our company, and we need to make sure that the productivity increases. How can we achieve these results?

This is the job of the software architect - to cope with all these important decisions related to the fact that it has very long-lasting consequences and that it is very difficult to change later. Many elements of the structure, such as the user interface, are fairly easy to develop, because you only change one layer of the application. And architecture is more about how all these layers are compared to each other, and it is usually much more difficult to change.

They say that microservices are now quite popular, and this architecture was created precisely with the expectation of constant changes. So, it is much easier to make serious changes to the microservice structure, but they were created from the beginning. And this is a very interesting topic for research - how to design an architecture that is easy to change, because for the industry this is exactly what has been the most difficult for a very long time.

- On the issue of architecture and positions: I increasingly see on the business cards and on LinkedIn the post “QA Architect”.

- This is one of the problems of the term “architect”: each company has to invent its own names for this. I met both “QA architects” and “data architects”, “system architects”, “solution architects”, “technical architects” - I met architects of all possible kinds. And this is a problem, because no one can give a clear definition, and give what you want.

What “QA Architect” can mean for a particular company, I don’t even know for sure. Does such a person develop a QA structure? As for me, it is closer to the function of an architect as a technical member, because often the architect is also a technical project manager. But the architect also deals with presentation and documentation. So, if I am an architect and I have a brilliant idea for a new architecture, but I cannot make a presentation and convince people with money that we have to do this, then we will not have a chance to implement my cool idea. This means communication skills.

These skills are applicable to the field of QA, and the person who does it can also be called an “architect”. You see, this is such a vague position. Many organizations just come to what they call the most senior engineer architect, because it sounds cool and they can not figure out what else to call it. Like, you know, senior-senior-senior developer - okay, architect. And I met a company where there is one type of architects, and I met a company where there are dozens of varieties of architects. In essence, these are fictional posts. My “meme wrangler” is better in this sense because it is obviously invented.

- Let's talk in the direction of your performance on Heisenbug. You are speaking at a testing conference - what is software quality for you?

- I personally consider the quality of the architectural components of the system. I know that the QA world regards software more as a black box, that is, it looks from the user's position if there are any errors or failures, whether it functions correctly. Of course, I am also interested in this, but I am more concerned about the root causes of errors: why is the application unreliable, why does it occasionally crash? Here I have to be the last line of defense, figuring out why this is happening. And there are things like performance and responsiveness. About them now there is a lot of talk in the UI world, they have clear indicators: if your mobile site loads visible content for longer than 3 seconds, users will go away and go somewhere else.

Great attention is paid to web productivity: what is the time before the first visible content is loaded, what is the total page load time. And all these are quality parameters, which are unconditionally visible from the point of view of an outside observer, and I am the one who should come up with how we build a system in which such indicators will be achieved. And this may lead to changes in the frontend regarding which web technologies will be used, but it can also lead to changes in the backend. Perhaps, to transfer information not in real time, but in batches, and in the middle make a caching mechanism? That's what quality looks like for me: determine what the requirements are, and then come up with a technical implementation that implements them.

- Can you give the most interesting case from the practice related to quality?

- It is difficult to say, all projects are different, so the best is always the last one I worked on!

Well, somehow I was working on a system where the requirements were partially reminiscent of Lotus Notes. Remember this ancient program? She was a terrible program, but she did one thing very, very well. This program was very well synchronized: you could download your mail and notes, then catch a taxi, go somewhere, answer all the letters at that time, and the next time you connect to the network, everything will automatically be unloaded, synchronized and work magically in a way.

And we had a client with a sales system who wanted the same working principle. Because salespeople are always on the road, and it was necessary that they could place orders without an Internet connection, and then everything would be synchronized and become as it should.

And we realized how difficult it is, because of the heaps of borderline cases and scenarios that need to be foreseen. First, we developed a system that worked without any connection to the Internet at all, then we started synchronization. And there was a lot of headaches - for example, the performance of the application in online compared to offline, when connecting, there was a noticeable delay, because at that time synchronization occurred. So, in this case, QA was a big splinter for us, because they found borderline cases that demolished the entire system.

And from the outside it seems like a simple task. Applications like Dropbox and Google Drive solved this problem so that it became invisible. And it seems easy. But when we tried to solve it, it turned out that there are a million border cases. So without a reliable QA, it is difficult to find them all, each of which must be returned for coordination with the structure of the application, in order to ensure that the overall structure still works.

Actually, while we were developing this system, making fundamental changes to the structure, we realized that borderline cases would be frequent and unacceptable. And this is a great illustration of how important it is to have a very well established feedback loop between architecture development and parts such as QA. Too many companies put it off for the very last moment, but if you do that, then in the end many things will be done wrong, and you will have to go back and spend a lot of time to redo it. If you establish close contact between the development of architecture and testing, you can find these borderline cases and change the structure much faster. Fortunately, in this project we were extremely flexible - since this was a ThoughtWorks project, we had a very fast cycle. And we had a very strong team of testers.

- Many people in testing ask how they can affect the architecture. For them, architecture is something in an ivory tower. What can you do about it?

- It seems to me that it would be useful for testers to come to the architects and explain to them that they are not doing their job well, and for testers to know better how to do it. People love it when they say that! Actually, no, they don't, nothing good will come of it.

I have a favorite mantra for such cases: "it is better to show than to discuss." You need to show how your world is different from theirs. If you are a testing engineer, you need to bring a user case, which will clearly demonstrate the lack of design. If you show this scenario to the architect, then this is no longer just a complaint about something, but a specific demonstration: “Here, your project will not work in such a scenario, so it needs to be changed.” If they disagree, it means that they simply lost touch with reality. And architects must be sensitive to things that happen in the real world.

There is a great little, as I call it, poetry from former Defense Secretary Donald Rumsfeld, which refers to "well-known known" and "unknown unknowns." So, in every software project there are “unknown unknowns”, so the development of any architecture must be iterative. Because no matter how much you think about, things that you could never have foreseen would just suddenly appear while you try to build this structure.

, , Docker. , , . , , .

. , , , — , .

, , ?

— .

- Of course! ! , , , . (QA). , , , .

, , QA , . , , , . -, . -, .

, — . , QA , . , . , .

— , ?

— . , , , - QA-, -, , data science, . , .

, . , , . - : « ». - : «, , , , , ». , , , . .

, ThoughtWorks, « » . , , deployment pipeline. , , , , , . , QA, , , , , , , . , -.

— . , «-», «QA» - ?

— . ThoughtWorks , , , . , , . , , -, . , . , .

— QA ?

— , , . — . , .

- , . , , , . - , . , , . , , , . , .

, . , — , 1975 , .

— 1975-?

— « -» , . , . ( ), « ». , . , , - , . , . , .

, , , — , QA . , . , , . , , .

— .

— , , Apple , . Wi-Fi , , . . UI, . QA, .

— , . , ThoughtWorks — ?

— , , « , ». , , , .

, . ? UI, , , — , . , , , : «! !» , , 50 Jira, .

. , : , , , : , , , . , .

. , . , , , .

, — « , ». , . , , . , , . , , — , . , .

. HR, . . — , , . , - ? , , -, , , ? , - :
- Hey! , - ? ?
- Yes.
— .

15- Jira, , , . , , , , . , , - Slack, , , , Slack.

, , . , 30 . How do you do this? , . , , , . -. , . , .

: , , , , . - QA: , , email Slack, , . , , , .

— . . , -, , . .

— , . , , . , , : , .

Slack! Slack — , . , , . « » , Slack. , « ». , , , , . - Slack, , , - .

, , .

-. , . — , , , . , . , - ?

— .

— ! , . , . Slack, , . , Slack, , , , , , , .

— -. - , , .

. , IT-, , , , . . , , - - ?

— . And that's why. - , , , -, . - , .

: - , . «, ». . , .

, . , , . ThoughtWorks, , , , . . , ThoughtWorks , .

, , , , , . « , ». , , - , — , . , - , . «»: , , , . . , , , .

, . -, . : «, , agile-, 1,3 , «», 30% ». , . : « , , , , ».

— , ?

— : ? ? , . , - — . ? - — , , .

, , . , , , . , -, QA , , , , . , . , , , — , , , .

— , , , - ?

— . , : , , -, , , , .

— , , , . , : « », , , .

— «architecture is abstract until operationalized», . , , ?

— . , , meme wrangler.

, DevOps-. , , . DevOps , , — , , .

, — , « » . . , . , : , , . , , ?

— .
— , , ! , , . — , , , , , . , , , , : Docker -, - . , , .

, , — , , .

— . !

Heisenbug , 17-18 , «Building evolutionary architectures: Fitness functions». , : , — .