How to distinguish good SCRUM from bad, using the approach of the founder of quantum computing
In 1985, David Deutsch was the first to describe the Turing quantum machine. Later, he combined the ideas of Popper, Dawkins, Everett and Turing in the theory of reasonable explanations. And recently, I found that I am improving the development processes of banking products based on his approach to the methodology of science.
Hey. My name is Dima Murzin.
By profession, I am a business analyst in the field of finance, I work with business stakeholders and with the development team in which I play the role of Product Owner. The third year I live with my family in New York.
')
When I lived in St. Petersburg, I was fortunate enough to work in a team in which the SCRUM process was very well set. Largely, this was due to the fact that the product manager on the customer side very deeply understood the principles of Agile, and also knew how SCRUM should be arranged in terms of all practices and ceremonies. At One Big European Bank (for which I worked) it was a rarity. Also, I should note my direct management by the contractor company. They hired a good coach, who dedicated us to the process and followed us for several more weeks, gave us complete freedom of action and responsibility for the product, and after that they intervened only if serious problems arose, which rarely happened. In addition, they were engaged in hiring and selected people who were suitable for work in conditions of constant change.
After I moved to the USA, closer to the customer (and got into a completely different team, which Agile didn’t smell of), I tried to analyze why my previous team was so effective and productive and why, in general, I knew that is the team so good? From the point of view of the business result, this was not at all obvious. The product we made was infrastructure (that is, it itself did not bring profit), and the One Big European Bank lost almost half the price of the stock while I was working in it, so evaluate the benefits for business it was difficult.
Maybe I was just fooling myself, the team was ordinary, and there was no super-productivity? However, I had a satisfying feeling that this was not so, and I tried to figure out where it came from. At that time, I read David Deutsch’s book “The Beginning of Infinity” and found in it a theoretical basis explaining why the team was productive (in my future activities, I tried to determine whether this theory was true by experiment).
Below in the text, I use Uppercase Letters to indicate Important Concepts.
For myself, I briefly call this theoretical approach David Deutch's Theory of Intelligent Explanations (TRODD). I will try to describe it below, but because I’m not a theoretician, but a practitioner, then I urge everyone who wants a more accurate academic presentation to familiarize themselves with the original source ( David Deutsch - The Beginning of Infinity ).
The main position of the TRODD is the Principle of Search for the Best Reasonable Explanation (PPNRO).
To fully understand PPNRO, you must first understand several definitions from the theory of scientific knowledge of Karl Popper.
Karl Popper believed that some theories are better than others. Most of all, he did not respect theories that were impossible to falsify, for example, those theories that can be very easily varied and explain anything to them. In order to separate such theories and not think about them at all, Popper proposed a way of demarcation, i.e. a way to distinguish scientific knowledge from unscientific. Scientific theory must be fundamentally falsified , i.e. to allow the possibility that something will happen (experiment), and it will become clear that the theory is wrong. So, for example, the theory “Everything happens by the will of fate” is not scientific, because there is no event at all that could refute it.
After Popper separated unscientific theories from scientific, he began to deal with scientific. Most scientific theories could easily be falsified by some simple experiment and discarded as false. However, it turned out that there were some theories that were not so easily falsified. Such theories were much better than incorrect ones. But were they the right theories? In the history of science there were a huge number of examples when it seemed that the theory was true, but then new observations proved that it was wrong (the canonical example - the particular and general theories of relativity replaced the classical physics of Newton).
Then Popper put forward another principle - the principle of fallibilism - which claims that any scientific knowledge is only hypothetical in nature and error-prone. In fact, he said that there is no ideal theory that would explain all observable facts and exclude all unobservable facts. Thus, Popper proposed a scale for evaluating theories, according to which unscientific theories had zero points, ideal theories were somewhere in infinity, and all available scientific theories hung somewhere between them, depending on how many facts the theory explained and ruled out .
In the book of Deutsch, the term “Theory” is replaced by the term “Reasonable Explanation”. In one of the chapters, he even says that he would gladly use the term “Delusion” instead of these two terms to emphasize that any of our theory is in some way wrong.
Deutsch used the Karl Popper scale to propose a method or strategy of actions that must be followed in order to get an answer to a question or solution to a problem. This strategy is outlined in the TRODD and briefly described in the Principle of Finding the Best Reasonable Explanation.
It reads roughly as follows:
Those. to find the Best Reasonable Explanation is impossible, but you have to try, because in the search process you can find a better explanation than a working one. Then the working Explanation is recognized as false, and the best Explanation becomes working.
Also, Deutsch introduces the concepts of scope and explanatory power. The scope is the set of observable facts that the theory is trying to explain. The areas of applicability of different theories may not overlap at all or partially overlap. Explanatory power is a measure of a multitude of facts that are explained or excluded by the Intelligent Explanation.
Deutsch writes that in most cases it is not possible to achieve free competition between different Intelligent Explanations, and this is due to the fact that there are many different kinds of Wrong Philosophy. Philosophy is untenable when it does not allow to implement the strategy described in PPNRO, i.e. either impedes the free generation of ideas, or impedes the tradition of criticism and experimental refutation.
Deutsch describes many areas of Failing Philosophy, but I want to give two examples in the article, because I have to deal with them very often in working practice. The first example is a reference to authority instead of Explanation (“I’m here a manager, I know better!”). This philosophical position blocks any attempts at criticism, and also negatively affects creativity - the process of creating new Explanations. The second example is Postmodernism, or the idea that since if the correct Explanation does not exist (the principle of fallibilism), then all Explanations are equally reasonable (rejection of the principle of falsifiability). Postmodernism gives a certain legitimacy to even the most harmful and unreasonable Explanations. At the same time, the process of creativity flourishes, but each of the interested parties considers its explanation as the most reasonable, as a result of which criticism does not lead to the fact that the false explanation is discarded.
Approximately in this form, I learned for myself the theory of Intelligent Explanations by David Deutsch.
I realized that SCRUM is a very convenient framework for organizing processes according to PPNRO. And the key processes in our team were arranged in this way. We solved the problem “ How to maximize the benefits for the business? " More specific problems that stemmed from the main problem were:
What do we have to do? What to do next? - The main issue of backlog prioritization.
How to do it? - I.e. What should be the solution, what solution should the architecture have?
How to work? What should be the process?
To answer each of these questions, a process was organized in which several people put forward their version of the Sensible Explanation of why this is necessary. Further discussion took place, during which one of the versions was accepted as a working one and verified using an experiment.
The backlog prioritization process was very simple, according to the canonical SCRUM, in which participated: product manager, business analyst on the part of the customer, 4 business analysts on the part of the contractor, project manager on the part of the contractor and architect. All participants (except the project manager, who was there to keep abreast of events) really influenced the results of prioritization. The role of the product was to make final decisions, but most decisions were still made by consensus. The highest priority stories went to work in the next sprint and were shown on the Demo. Prioritization took place every two weeks, therefore, a feedback cycle arose when the results of the implementation of some Histories influenced further prioritization.
Thus, both necessary, according to the principle of the Search for the Best Explanation, process took place: the creative creation of new Explanations (why do we need to do this now) and the tradition of criticism in order to falsify the explanation (i.e., an attempt to explain what else needs to be done now )
The decision-making process related to the implementation of the decision was pretty tricky. In our team, he was in the form of a regular rally (3 times for a two-week sprint), which was called Product Backlog Refinement. In the SCRUM guide there is no particular specification of how the Product Backlog Refinement should be arranged (it is not described as a rally at all, but as a process of continuous grooming of a backlog). Apparently, the coach decided to arrange a rally in this way. The goal was to have time to discuss 3 medium-sized stories per rally, each in 20 minutes. In 10 minutes I gave a presentation about business value and the necessary technical details. After 5 minutes there was a quick discussion, then an attempt to score at Story Points using the Planning Poker technique. The last 5 minutes were reserved for discussion if the assessment did not agree. Then an attempt was made to re-evaluate the History, and if again it did not work out, then the discussion of the History was postponed to the next meeting. For a second discussion, action points were set up: prepare additional information or conduct an experiment. Thus, here again both key processes were carried out according to the TRODD: the team proposed various solutions (and this option is always based on an explanation of why this should be done), criticized them and tried to refute, including using prototyping.
SCRUM has a Retrospective rally, which was created specifically to find the answer to this question. And this meeting was also organized according to the principle of finding the Best Solution: the team tried to identify existing problems, identify the necessary changes in the process, execute them during the sprint, and at the next Retrospective evaluate their benefits and decide whether to abandon them or leave them.
There were a lot of problems and they were not resolved immediately. But, because Retrospective practice was not an empty formality, but really worked, as a result, the team got the so-called (me) endlessly improving process. It happens, in many ways, with the help of automation of routine tasks, so the team focuses on key tasks. I present this process as follows: there are two machines (development teams) that initially move at approximately the same speed. One of the cars begins to improve gradually, while nothing changes in the other. And then two weeks passed, and the first car drove 1 km / h faster than the other due to the fact that some problems were solved. And this happens every two weeks, the first car as if rides with acceleration relative to the second. And at a distance of two years, it turns out that the first car didn’t just drive more.
It turns out that due to the constant habit of identifying and solving problems in the process, the team made a quantum leap. At some point in a team in which an endlessly improving process works, synergistic effects begin when the system becomes more than just the sum of the components. Such a team does not so much more than others (although this too), but does better, more complex and more necessary things. Such a team can throw any idea, it will be discussed, criticized and verified by experiment.
In practice, in my current work, I am guided by the PPNRO to determine whether a process is set up to solve a particular problem, whether it is good or bad. If the necessary conditions according to PPNRO are not fulfilled, more often than not, this means that the process is set poorly and needs to be adjusted. In any case, he will definitely not improve endlessly.
My experience, of course, is extremely limited, because I have always been engaged only in corporate custom development, and mainly for banking products. However, it seems to me that TRODD is a universal theory that can have many areas of application. Agile in the field of software development, Lean Startup in the field of Product Discovery, as well as such an approach as Design Thinking in the field of design, all of them repeat the same two main points from the PPNRO: the need for creative thinking (creativity) and the tradition of criticism (falsification).
I hope that people who practice these approaches will hear enough “bells and whistles” in the process of reading this article to make them want to refer to the original source. This is an exciting read.
Hey. My name is Dima Murzin.
By profession, I am a business analyst in the field of finance, I work with business stakeholders and with the development team in which I play the role of Product Owner. The third year I live with my family in New York.
')
When I lived in St. Petersburg, I was fortunate enough to work in a team in which the SCRUM process was very well set. Largely, this was due to the fact that the product manager on the customer side very deeply understood the principles of Agile, and also knew how SCRUM should be arranged in terms of all practices and ceremonies. At One Big European Bank (for which I worked) it was a rarity. Also, I should note my direct management by the contractor company. They hired a good coach, who dedicated us to the process and followed us for several more weeks, gave us complete freedom of action and responsibility for the product, and after that they intervened only if serious problems arose, which rarely happened. In addition, they were engaged in hiring and selected people who were suitable for work in conditions of constant change.
After I moved to the USA, closer to the customer (and got into a completely different team, which Agile didn’t smell of), I tried to analyze why my previous team was so effective and productive and why, in general, I knew that is the team so good? From the point of view of the business result, this was not at all obvious. The product we made was infrastructure (that is, it itself did not bring profit), and the One Big European Bank lost almost half the price of the stock while I was working in it, so evaluate the benefits for business it was difficult.
Maybe I was just fooling myself, the team was ordinary, and there was no super-productivity? However, I had a satisfying feeling that this was not so, and I tried to figure out where it came from. At that time, I read David Deutsch’s book “The Beginning of Infinity” and found in it a theoretical basis explaining why the team was productive (in my future activities, I tried to determine whether this theory was true by experiment).
Below in the text, I use Uppercase Letters to indicate Important Concepts.
For myself, I briefly call this theoretical approach David Deutch's Theory of Intelligent Explanations (TRODD). I will try to describe it below, but because I’m not a theoretician, but a practitioner, then I urge everyone who wants a more accurate academic presentation to familiarize themselves with the original source ( David Deutsch - The Beginning of Infinity ).
Theory of Intelligent Explanations by David Deutsch
The main position of the TRODD is the Principle of Search for the Best Reasonable Explanation (PPNRO).
To fully understand PPNRO, you must first understand several definitions from the theory of scientific knowledge of Karl Popper.
Karl Popper believed that some theories are better than others. Most of all, he did not respect theories that were impossible to falsify, for example, those theories that can be very easily varied and explain anything to them. In order to separate such theories and not think about them at all, Popper proposed a way of demarcation, i.e. a way to distinguish scientific knowledge from unscientific. Scientific theory must be fundamentally falsified , i.e. to allow the possibility that something will happen (experiment), and it will become clear that the theory is wrong. So, for example, the theory “Everything happens by the will of fate” is not scientific, because there is no event at all that could refute it.
After Popper separated unscientific theories from scientific, he began to deal with scientific. Most scientific theories could easily be falsified by some simple experiment and discarded as false. However, it turned out that there were some theories that were not so easily falsified. Such theories were much better than incorrect ones. But were they the right theories? In the history of science there were a huge number of examples when it seemed that the theory was true, but then new observations proved that it was wrong (the canonical example - the particular and general theories of relativity replaced the classical physics of Newton).
Then Popper put forward another principle - the principle of fallibilism - which claims that any scientific knowledge is only hypothetical in nature and error-prone. In fact, he said that there is no ideal theory that would explain all observable facts and exclude all unobservable facts. Thus, Popper proposed a scale for evaluating theories, according to which unscientific theories had zero points, ideal theories were somewhere in infinity, and all available scientific theories hung somewhere between them, depending on how many facts the theory explained and ruled out .
In the book of Deutsch, the term “Theory” is replaced by the term “Reasonable Explanation”. In one of the chapters, he even says that he would gladly use the term “Delusion” instead of these two terms to emphasize that any of our theory is in some way wrong.
Deutsch used the Karl Popper scale to propose a method or strategy of actions that must be followed in order to get an answer to a question or solution to a problem. This strategy is outlined in the TRODD and briefly described in the Principle of Finding the Best Reasonable Explanation.
It reads roughly as follows:
- When people are looking for an answer to a question or solution to a problem, the search for the Best Reasonable Explanation is the right strategy.
- You need to use creativity or “Creativity” to come up with different Intelligent Explanations (1st ingredient).
- Next, you need to try to refute these explanations with the help of experiments (including mental ones), this is called the “Tradition of criticism” (2nd ingredient).
- If it is not possible to refute, then the Explanation is considered working and is used.
- But attempts to find other, better Reasonable Explanations do not stop, because according to the principle of fallibilism, not one Explanation can be deemed conclusively correct.
Those. to find the Best Reasonable Explanation is impossible, but you have to try, because in the search process you can find a better explanation than a working one. Then the working Explanation is recognized as false, and the best Explanation becomes working.
Also, Deutsch introduces the concepts of scope and explanatory power. The scope is the set of observable facts that the theory is trying to explain. The areas of applicability of different theories may not overlap at all or partially overlap. Explanatory power is a measure of a multitude of facts that are explained or excluded by the Intelligent Explanation.
Deutsch writes that in most cases it is not possible to achieve free competition between different Intelligent Explanations, and this is due to the fact that there are many different kinds of Wrong Philosophy. Philosophy is untenable when it does not allow to implement the strategy described in PPNRO, i.e. either impedes the free generation of ideas, or impedes the tradition of criticism and experimental refutation.
Deutsch describes many areas of Failing Philosophy, but I want to give two examples in the article, because I have to deal with them very often in working practice. The first example is a reference to authority instead of Explanation (“I’m here a manager, I know better!”). This philosophical position blocks any attempts at criticism, and also negatively affects creativity - the process of creating new Explanations. The second example is Postmodernism, or the idea that since if the correct Explanation does not exist (the principle of fallibilism), then all Explanations are equally reasonable (rejection of the principle of falsifiability). Postmodernism gives a certain legitimacy to even the most harmful and unreasonable Explanations. At the same time, the process of creativity flourishes, but each of the interested parties considers its explanation as the most reasonable, as a result of which criticism does not lead to the fact that the false explanation is discarded.
Approximately in this form, I learned for myself the theory of Intelligent Explanations by David Deutsch.
And then here is SCRUM?
I realized that SCRUM is a very convenient framework for organizing processes according to PPNRO. And the key processes in our team were arranged in this way. We solved the problem “ How to maximize the benefits for the business? " More specific problems that stemmed from the main problem were:
What do we have to do? What to do next? - The main issue of backlog prioritization.
How to do it? - I.e. What should be the solution, what solution should the architecture have?
How to work? What should be the process?
To answer each of these questions, a process was organized in which several people put forward their version of the Sensible Explanation of why this is necessary. Further discussion took place, during which one of the versions was accepted as a working one and verified using an experiment.
What do we have to do? What to do next?
The backlog prioritization process was very simple, according to the canonical SCRUM, in which participated: product manager, business analyst on the part of the customer, 4 business analysts on the part of the contractor, project manager on the part of the contractor and architect. All participants (except the project manager, who was there to keep abreast of events) really influenced the results of prioritization. The role of the product was to make final decisions, but most decisions were still made by consensus. The highest priority stories went to work in the next sprint and were shown on the Demo. Prioritization took place every two weeks, therefore, a feedback cycle arose when the results of the implementation of some Histories influenced further prioritization.
Thus, both necessary, according to the principle of the Search for the Best Explanation, process took place: the creative creation of new Explanations (why do we need to do this now) and the tradition of criticism in order to falsify the explanation (i.e., an attempt to explain what else needs to be done now )
How to do it? What should be the architecture?
The decision-making process related to the implementation of the decision was pretty tricky. In our team, he was in the form of a regular rally (3 times for a two-week sprint), which was called Product Backlog Refinement. In the SCRUM guide there is no particular specification of how the Product Backlog Refinement should be arranged (it is not described as a rally at all, but as a process of continuous grooming of a backlog). Apparently, the coach decided to arrange a rally in this way. The goal was to have time to discuss 3 medium-sized stories per rally, each in 20 minutes. In 10 minutes I gave a presentation about business value and the necessary technical details. After 5 minutes there was a quick discussion, then an attempt to score at Story Points using the Planning Poker technique. The last 5 minutes were reserved for discussion if the assessment did not agree. Then an attempt was made to re-evaluate the History, and if again it did not work out, then the discussion of the History was postponed to the next meeting. For a second discussion, action points were set up: prepare additional information or conduct an experiment. Thus, here again both key processes were carried out according to the TRODD: the team proposed various solutions (and this option is always based on an explanation of why this should be done), criticized them and tried to refute, including using prototyping.
How to work?
SCRUM has a Retrospective rally, which was created specifically to find the answer to this question. And this meeting was also organized according to the principle of finding the Best Solution: the team tried to identify existing problems, identify the necessary changes in the process, execute them during the sprint, and at the next Retrospective evaluate their benefits and decide whether to abandon them or leave them.
There were a lot of problems and they were not resolved immediately. But, because Retrospective practice was not an empty formality, but really worked, as a result, the team got the so-called (me) endlessly improving process. It happens, in many ways, with the help of automation of routine tasks, so the team focuses on key tasks. I present this process as follows: there are two machines (development teams) that initially move at approximately the same speed. One of the cars begins to improve gradually, while nothing changes in the other. And then two weeks passed, and the first car drove 1 km / h faster than the other due to the fact that some problems were solved. And this happens every two weeks, the first car as if rides with acceleration relative to the second. And at a distance of two years, it turns out that the first car didn’t just drive more.
It turns out that due to the constant habit of identifying and solving problems in the process, the team made a quantum leap. At some point in a team in which an endlessly improving process works, synergistic effects begin when the system becomes more than just the sum of the components. Such a team does not so much more than others (although this too), but does better, more complex and more necessary things. Such a team can throw any idea, it will be discussed, criticized and verified by experiment.
In practice, in my current work, I am guided by the PPNRO to determine whether a process is set up to solve a particular problem, whether it is good or bad. If the necessary conditions according to PPNRO are not fulfilled, more often than not, this means that the process is set poorly and needs to be adjusted. In any case, he will definitely not improve endlessly.
My experience, of course, is extremely limited, because I have always been engaged only in corporate custom development, and mainly for banking products. However, it seems to me that TRODD is a universal theory that can have many areas of application. Agile in the field of software development, Lean Startup in the field of Product Discovery, as well as such an approach as Design Thinking in the field of design, all of them repeat the same two main points from the PPNRO: the need for creative thinking (creativity) and the tradition of criticism (falsification).
I hope that people who practice these approaches will hear enough “bells and whistles” in the process of reading this article to make them want to refer to the original source. This is an exciting read.
Source: https://habr.com/ru/post/460891/
All Articles