How to build a community. Translation of the book "Social Architecture": Chapter 6. Living Systems

"Living System" is a system that develops in the natural environment, independently adapting to new conditions. Living Systems can exist for quite a long time, easily adapting to any changes, thus being extremely effective. In contrast, “Planned Systems” are, as a rule, unstable, reacting poorly to changes and, as a result, short-lived. In this article I will talk about the Living System on the example of software and society, and also tell you how to create such a system.

Why “Living Systems”

According to Wikipedia , “Living Systems” are entities consisting of self-organizing elements that actively interact with the environment. These systems are supported by the flow of information, energy and substances. ”This term was proposed by psychologist James Greer Miller to refer to the concepts of life.

I want to use this term to create a new metaphor for software systems and organizations involved in them - two types of systems that are of greatest interest to me. These two systems are not just alike. Software is a product created by a group of people, and, as Conway noted, the structure of the software system reflects the structure of the organization that develops this system. I want to say that “the psychology of software is the psychology of people”.

Today, most software products are well planned, but they do not become “Living Systems”. They inexorably fail at the delivery stage, being sold by force or deception. In order for software to become a “Living System”, it must be used by the organization that develops it, and then it “lives” or “dies” along with this organization. An “organization” can be something more than a company or a team. It can include thousands of teams, businesses, customers, and suppliers who have unexplained but really significant connections.
')
Nothing demonstrates this more clearly than the Internet, which is the “Living System” of software, people, enterprises, and other organizations. The organization that created the Internet is nothing but human society itself. There are many “Living Systems”, just look around. This is a surprisingly simple truth: the better the quality of the large-scale software systems we create, the more they resemble the reality around us.

There are also planned systems that are the complete opposite of Living Systems. It is much easier to plan a system than to grow it. However, plans are inevitably based on false assumptions and short-sighted decisions. Planned Systems in some sense look attractive and effective, however, they inevitably suffer defeat. In life, you can find many similar examples, for example, a cooperative economy, planned cities, Microsoft Windows 8, and so on.

In the software business, this division into live and planned is best reflected in the confrontation between free and closed software. Free software (and its brother open source code) is usually generated with actual use, while closed source code is usually planned. This is the main reason why I do not work with closed source: its “death” is soon and expected. I prefer that my work be kept as long as possible.

I will make some harsh statements, starting with: the most successful large-scale software systems are “Living Systems”. Thus, in a competitive market, the living system will certainly win the planned one. It is much faster, cheaper and more accurate to identify and solve serious problems. If your business is based on a Planned System, then it is vulnerable because you cannot cope with the attacks of the Living System.

The second statement is that all of the above also applies to organizations. If your company is a Planned System - it is already dead. While if your company operates as a Living System, it will take a dominant position in the market. It is interesting that when two Living Systems intersect, they do not conflict. Rather, they specialize in different areas, and then merge to form a single Living System. Competition and conflict usually work for the benefit of Living Systems, even if its individual components are affected.

Let me develop the theme of conflict and competition. Of course, competition, sometimes even tough, is normal for people. This is our biological need. However, we also have a need for cooperation, which is often a much more successful strategy. The living system involves competition between people and is preserved in the event of the loss of individual components. In general, it depends on the process of competition and failure. The planned System, in fact, tries to act individually and cannot tolerate internal competition or the loss of individual components.

What are Living Systems

The live system consists of loosely coupled components. It is out of space (thus “distributed”) and time (thus “asynchronous”). This means that many processes occur in unexpected places, at unpredictable times. For the chief planner, this is a dangerous chaos.

In the Planned System, on the contrary, the time and place of the expected events as if prescribed by the script. The focus is on “management and control”, where decisions are made centrally and communicating with the structure. Planned Systems are always hierarchical, since such a structure is the most optimal way to quickly disseminate information about decisions taken from the top to the bottom.

We build the planned System, and the living system develops itself. My goal is to understand and teach how to artificially create a living system. I study developmental factors, care patterns, and the driving forces of a self-developing system. In fact, I am talking about creating artificial life and an artificial mind, but in a form unusual for researchers of artificial intelligence. I believe that individual components - including you and me - cannot be “smart”, if only in a narrow and superficial sense: after all, intelligence is a property inherent only in systems.

Living Systems are characterized by a lack of central planning or decision making. Look at the software development project and ask “who is the designer?” If there is a specific designer (and there is almost always a designer) - whether it is an individual or an organization, then this is a Planned System. In the Living System there are neither designers, nor work plans, nor any definite plans for the future, except for “survive and develop”.

The Living System is more like Adam Smith’s free market than Stalin’s five-year plan. Economics, politics, psychology are just as important - perhaps even more important - in the development of the Living System, as are technologies. A free market depends on key provisions such as: clear laws, standards, contracts, and fair governance. The work of the Living System depends on the same provisions.

The governing body adopts laws that define a fair market, and then implements them. Units, currencies, contracts, and more. In software systems, such laws, for example, are a license for source code or a contribution policy. A fair market allows anyone to create a new enterprise and compete with others. To create a real competition (i.e. free choice of customers), customers can request clear contracts, which in the software are documented application programming interfaces and protocols.

Living Systems DNA is essentially a set of regulated contracts. For example, the Internet is evolving from a set of working proposals (RFCs (protocols called Requests for Comments)), which are regulated by the Internet Engineering Task Force (IETF). “Living Cities” develops thanks to criminal and civil law, established standards for water, energy and waste, transportation, and so on.

If all strategies were honest, there would be no need for management, regulation at all. However, every Living System is vulnerable to fraud. There is a certain group of people who cheat - systematically or depending on the situation. Knowing how the market works, they will always try to turn the situation in their favor, even if it is bad for the rest. They will lie, steal, deceive, intimidate and coerce, etc.

Without showing resistance to such fraud, the market will suffer and the entire system will die sooner or later. Centralized power is one way to protect against fraud. However, it is also significantly vulnerable: fraudsters can and even often seize power themselves. In Living Systems, they can only capture managers, which happens all the time, in fact.

When scammers seize the managers of a real Living System, the usual reaction is to dismiss, if possible. In open source software systems it is possible to create a branch and continue to work under better management. That is why the branch provides substantial freedom, and does not mean defeat. Because the branch can also be used as a capture method, free licenses (GPL and others) are best for Live Systems in terms of software.

The growth of living systems is a constant and natural process. This is their main distinguishing feature - the absence of large expenditures of effort on their creation. Here, however, you will observe small changes. This may seem boring and unpromising. However, this is the best method of survival. The living system must do two things. First, it must address a number of issues regarding profit. Secondly, over time, it must change and adapt, following the changes taking place in the surrounding world.

As for the Planned System, it is very difficult, often impossible, to tune it for the changing world around it. Resources define power. Therefore, the Planned Systems actively and aggressively oppose changes, deny them, and when they cannot do without changes, they cease to exist.

The living system only benefits from changes. For her, it does not matter when studying the landscape - “today” or “tomorrow. It develops through continuous learning. To really destroy it, you must do great damage to it, which is hard to do if the Living System is already successful and highly developed.

For her, working with minor problems is no different from normal activities. In fact, the Living System develops due to difficult situations only if they are not very heavy, insurmountable. A difficult situation is what helps components compete with each other and develop better solutions. That which does not kill the Living System only makes it stronger.

So, since Living Systems learns everything and pours into new spheres much faster and to their advantage, they will strive to thrive and dominate, destroying any competitive Planned Systems. They respond quickly, moving resources to areas where they are needed. And since they do not need any instructions for action, they can be resized to any size. Lack of coordination means unlimited scale.

Components of the Living System

Let's turn to the individual components of the Living System. Remember that the Live System is similar to the market where components compete for the provision of certain services. The components of a living system have certain features that distinguish them from the components of a planned system. Each component of the Living System has a certain group of owners and investors, and each component is assigned a separate group (while in the Planned System each component has the same owners). Components are combined in a network of suppliers and customers, data, names and addresses of which are always available for the convenience of the client. An easy way to cheat is to replace the high-quality component with low-grade. Therefore, the governing body may have to enforce the profile identification and protect the identification data.

Components are as independent as possible from their location. This fact creates a larger and more efficient free market. This means that we strive to ensure that our Living System is independent. This is contrasted with the Planned System, where location plays a very important role, and competition between components is either very small or non-existent.

Also components can appear and disappear completely arbitrarily. There are no guarantees that the component on which we depend today will still exist or be available tomorrow. It probably seems unreliable, but in fact it is reasonable and reasonable. We do not depend on certain components, we rely on contracts. If we really need something, we will have many alternatives. If one of them disappears, another will replace it. If you miss one taxi, you will definitely catch another.

Components are as independent as possible. This means that they exist and change at their own pace, in their own direction. A change in one component is almost imperceptible to another, except through an open interface. This freedom is necessary for a free market driven by specialization and trade. So, one component can focus on speed, and the other on safety.

Since there is no generally accepted decision about which components exist, or who creates them, they will be heterogeneous, and this variety of components is important for understanding the entire system. The set of various components involved in the free market will cope with the solution of the problem faster and more successfully than a solid, monolithic Planned System.

The components are abstracted, which means that they can themselves be whole systems. For example, a web address may be a separate, small piece of software (one web server) or a large infrastructure (internet business). In turn, only from the owners of each group depends on which system they will create - Live or Planned. The Living System will be able to safely take in the components of the Planned System. The reverse process, however, is impossible.

Components avoid prior agreement known as the general changing state. Each component has certain knowledge that it can share with others, but they all do it asynchronously. So, although the Living System is a large holistic knowledge base, there is no guaranteed consistency between the components. It seems to be paradoxical. But does, say, every member of the assembly agree with the agenda of the day?

In fact, meetings with their agendas and protocols are the embodiment of the overall changing state on which the Planned System depends. Planned Systems cannot function without systematic prior agreement. With parallel software design, we use “locks” to achieve a similar result. It has been proven that a software system using locks to share the state of the components will not evolve. You can try to create distributed software like the Scheduled System: at first everything works well, but almost does not grow at all. While the launch of the Living System takes a little more time, its subsequent growth is limitless.

Ultimately, the components are “lazy” and situationally conditioned. They work only when there are tasks that need to be solved, and grow and develop only when there are all new, profitable opportunities for this. This means that the components can be simple and minimalist. In addition, they can solve the “problem of the landscape” much more accurately, without unnecessary obstacles and prejudices. In the Planned System, on the contrary, the components are created in advance, based on the prediction of future problems or, at best, the experience of the past.

Example: at a scheduled conference, the organizers select certain topics based on the experience of the past year. Now, a month before the conference, a very important event attracted the interest of the public to a completely different problem. How quickly will the conference organizers react? A conference run by participants may change in real time, while a scheduled conference will take almost a year to somehow respond to this.

Live System Protocols

There are certain connections between the components of the Living System. Each link is a combination of a stream of information, knowledge or references, in both directions. The best way to model relationship data is discrete events or “messages” that carry a certain set of relationships, relationships, which we call “protocols”. In natural Living Systems, we can also observe messages and protocols. Cells, for example, communicate with each other through chemical messages. We humans communicate using a set of protocols that underlie our speech. For example, hierarchies in which men occupy a dominant position are a characteristic feature of human society, indicating that the management and control protocols on which these hierarchies are based are embedded in our minds and not recognized. I can even assume that the male mind, guided by the need of the ancestors to organize hunting campaigns, is responsible for the Planned Systems.

Protocols have much in common with each other. We see broadcast protocols, where one component transmits a signal to many listeners. This protocol is usually one-way. Usually, the return signal, the signal from the listeners, is not received.

We also see one-to-one protocols, where the two components exchange knowledge, tasks, requests, and so on. Such protocols are more formal and ideally completely asynchronous. Less formal protocols take longer to form, thus creating a total “delay”. If I, for example, cook pizza and I have to learn about each ingredient, naturally, it will take longer. “Do you like mushrooms?”, “What about garlic?”, “Well, what kind of cheese do you prefer?”.

Ideal relationships, relationships are aimed at reducing the “delay”, since the “delay” in the entire system is the sum of the “delays” of its entire supply chain. So, if I cook myself a meal, I need to spend a minute trying to solve some questions about the pizza, which will add a minute to the total cooking time. In an asynchronous dialogue with a small delay, I will immediately ask all the questions, and I will think about the answers later, when they will come to me one by one.

To create efficient asynchronous systems, we need queues and strategic scheduling. Ideally, we always encounter queues when we are waiting for a message and try to redirect them to the recipient as soon as possible in order to avoid delays. We need strategies for working with full queues (the space is not infinite): you can simply delete old messages or pause the sending of messages (only this works for one-to-many dialogs, and not for one-to-many). We may need a queue of incoming messages, one per stream, and the ability to wait for messages on this queue.

Protocols are an integral part of the Living System. They carry out official contracts. If I ask “Do you like garlic?”, As a reply, I expect either yes or no. Talking about the weather in this case will be a breach of contract. When we develop our Living Systems, we need to fix the protocols in order to study it and reassure. And the simpler and clearer it is, the better. Complicated, ambiguous protocols are difficult to study and implement and do not fit into the concept of a free market.

Some Living Systems rely on trust and identification numbers, and do not look at whether a contract is certified. This is permissible, but for a very short time, especially when sharing knowledge, because they are also vulnerable to fraudsters. Alternatively, you can secure the process of confirming each contract using meta-contracts. This practice is often even more productive for trading. Any taxi driver is good, until he drives us to the right address and does not ask for too high a price. However, we go to receive news from trusted sources.

As soon as we have contacts that can be easily verified, we will be able to cope with violations. If one strategy fails, there is always another. Another one refuses, you can try the next one. However, after breaking contact, you are unlikely to want to continue this way, as this may cause more significant damage.

Case study: ZeroMQ library and community

The ZeroMQ community is the Living People System, which builds the Live Software System (a collection of software under the same name). Although I originally developed the ZeroMQ community with most of the properties of the Living System, it came out only in 2012, refusing the services of its main planners.

This community consists of loosely coupled projects with a common goal, which is to provide queues or messages for other software systems. I argued and still believe that only the Live System can be optimally applied with ZeroMQ.

ZeroMQ projects are connected in supply chains with official relations based on API and wire protocols. Not only the design of these APIs and protocols, but also the control of their effectiveness takes a lot of time. In fact, we usually do not document internal components, but only external APIs.

There is no central planning or coordination in it. However, each project develops organically, as users contribute to their development and improve them. In order to make this process simpler, the ZeroMQ collaboration agreement was created, which ensures that the organization expands to include all of its competent users.

Anyone can start a new ZeroMQ project or create a new branch for competing and experimenting. We encourage this, so we have several different types of competition at different levels. It works well in practice. Basic licenses are LGPL v3 or MPL v2, ensuring that branches are always protected (development can be done in both directions).

The management team in the ZeroMQ community is a group led by iMatix, the company that developed the first software. There is no need to specifically manage, in principle, except to stop abusing the name “ZeroMQ”. Clear documentation of the protocols is enough for customers to check their suppliers.

ZeroMQ scales very well. The cost of adding a new project is close to zero, not counting the cost of prospecting. Projects are asynchronous, they use items from GitHub and requests to include code. Coordination is minor or non-existent. We check the code after the fact, and fix the bad code in the next development process, rather than discussing it.

The complete transformation of ZeroMQ into the Living System turned out to be a difficult process, since initially there was no chance of success. The bulk of free software projects still depend on careful planning. Violation of standard procedures seemed very strange, if not insane. The loss of major contributors — who provided the authority on which central planning was based — seemed, in prospect, disastrous.

However, ZeroMQ expanded rapidly in space and flourished. We have refuted the theory that central planning is very important for quality. In fact, we found out that without central planning, the software improved in quality and accuracy. Prior to this, ZeroMQ was extremely unstable, experimental and did not meet the needs of users, it became quite stable, reliable and close to what users want.

Today ZeroMQ is an example of how the Live System should work properly. It provides great value as a data warehouse, as there have been numerous attempts to replace it with both the previous main planners and other teams. It is noteworthy that every Planned System that claimed to be “better than ZeroMQ” failed, while every Living System that began to compete with ZeroMQ eventually became part of it.

Transformation into a Living System

Is it possible to turn the Planned System into Live? Suppose that we have a technical right (an agreement from a sufficient number of participants or a legal right - having a license); what are the practical requirements then?

The most difficult thing is getting the right size of components. This means that you have to drop existing components and create new ones. This can be a disaster if you do this with all the components at once. Therefore, with more components, you have to start in one area, perform a redesign, and then develop the resulting culture.

The size of the components usually depends on the people, so it’s appropriate that “some people could work with”. The scale of the Living System is due to the fact that more and more components are added there that can use and replace each other as you please, without increasing their size. A component is too small when it cannot by itself provide something or anyone, and too large when it cannot focus on one thing.

And finally, you need contracts. We obtained good results for software systems by simply accepting the ZeroMQ C4.1 contract to use it along with the programming style manual and software license.

For several reasons, I strongly recommend such a general license as the LGPL (my theory is: if you use a weak license like Apache or BSD, for example, you will not be able to create the Living System).

Previously, the launch of such a living system was complicated by the fact that self-organizing software ecosystems were not properly reflected in the documents, and were generally poorly accepted. We lacked empirical data demonstrating that processes such as C4.1 can work, not to mention that they can work so well. As far as I know, that contract was the first contract in Live Systems software.

Economics of Living Systems

How to make money on free software? I often get asked this question. I always give a different answer, depending on who I deal with - with an individual, a small firm, a large firm.

The key to understanding Living Systems is that they, in general, are an economy. No component is in the system just like that. However, the choice between egoism and altruism is a false dilemma. At the heart of the Living System is both. This is the basic theory of economics: as egoists in specialization and commerce, we create common well-being. This is a person's superpower: large-scale specialization and trade between individuals, families, generations, villages, cities and entire regions.

A Living System belongs to each of its participants, so its value is much more difficult to measure, while the Planned System, which is owned by several people from the “top”, represents a certain visible value both for its owners and for outside observers. However, the total value of the Living System will always surpass any Planned System competing with it. The living system can bring incredible profits, which is divided among all its participants.

Here is the first answer: The Living System can destroy competing Planned Systems and thereby assign some of the previously hidden values. : , .

, , . : , . .

. , , , . , . , , .

Conclusion

, . . , , , . . , .

, , . . , , .

, , , ( ) . . , , - , , . .

, , , . , . , — .

, , . , , , , , .

To create a large-scale Live System in software, create the same system of people. They will cooperate, develop, function correctly and thus dominate in any market. While competing Planned Systems will be defeated, collapsing, functioning separately, competing Living Systems will strive to specialize in different areas, and as a result merge into one large single Living System.

Translation of the book "Social Architecture":

• Preface. Crowd wisdom
• Chapter 1. Toolkit
• Chapter 2
  - Emotional burnout of volunteers
  - How to capture / protect an open-source project
  - The myth of individual intelligence
  
• Chapter 6. Living Systems

---