How to build a community. Translation of the book “Social Architecture”: Chapter 4. Protocol for the C4 Collaboration

"This is the essence of thirty years of software development experiences."

ZeroMQ process: C4

When we talk about ZeroMQ, we sometimes mean libzmq - the main library. In early 2012, we synthesized the libzmq process into a formal and reusable collaboration protocol, which we called the “Collective Code Development Contract” or C4. You can view this as a layer above the license (for example, MPLv2). These are our rules, and I will explain the reasons for the emergence of each of them.

C4 is an evolution of the GitHub Fork + Pull model. You might think that I'm a fan of git and github. And that's for sure: these two tools have had a positive impact on our work in recent years, especially when it comes to creating a community.

Tongue

The keywords “MUST”, “NOT SHOULD”, “REQUIRED”, “MUST”, “NOT SHOULD”, “FOLLOWING”, “NOT SUBJECT”, “RECOMMENDED”, “CAN” and “ADDITIONAL” in this document should be interpreted as as described in RFC 2119.

Beginning with RFC 2119, the C4 text clearly states that it intends to act as a protocol, and not as a randomly written set of recommendations. A protocol is an agreement between the parties that defines the rights and obligations of each party. They may be familiar on the web, or they may be strangers working in the same project.

I think that C4 is the first example of how someone tried to codify community standards as a formal and reusable protocol specification. Previously, our rules were distributed on several wiki pages and were very specific to libzmq . But experience teaches us that the more formal, precise, and reusable the rules, the easier it becomes for people who are unfamiliar with each other to cooperate. Fewer disagreements mean a more scalable community. During C4, we also had some disagreement in the libzmq project on what process we used. Not everyone felt bound by the same rules. Let's just say, some people thought they had a special status, and this created conflict with the rest of the community. Thus, the specification made things clearer.
')
Using C4 is easy: just place your project on GitHub, get another person to join and open the pull request function. In your README, place a link to C4, and that's it. We did this in a number of projects, and it seems to be working. I have often experienced a pleasant surprise when applying these rules to my own work, for example in the CZMQ project. None of us is incomparable enough to work without the others.

Goals

The C4 specification is designed to provide an optimal, repeatable collaboration model for open source projects.

In short, the reason for creating C4 was to put an end to the differences in the libzmq collaboration process. Dissenters went to other places. The ZeroMQ community blossomed smoothly and easily , as I predicted. Most people were surprised by this, but also satisfied. There was no real criticism of C4, except for its branching policy, which I will discuss later, since it deserves a separate discussion.

There is a reason why I consider the creation story here: as the founder of the community, you ask people to invest in your property, trademark and branding. In turn, as we do in the case of ZeroMQ, you can use this branding to set the bar for quality. When you download a product labeled “ZeroMQ”, you know that it was released according to certain standards. This is the basic rule of quality: write down your process; otherwise you cannot improve it. Our processes are not perfect, they never will be. But any flaw in them can be corrected and tested.

Therefore, it is very important to make C4 reusable. To learn more about the best possible process, we need to collect the results of the widest possible range of projects.

All of this has the following specific goals: expanding the community formed around the project, reducing the entry threshold for new members and creating a scalable partnership model with positive feedback.

The number one goal is the size and viability of the community — not technical quality, not profit, not productivity, not market share. The goal is simply the number of people who contribute to the project. The science here is simple: the larger the community, the more accurate the results.

Reduce dependence on key individuals by distributing the required skill sets to different specialists so that any area has a high level of competence.

Perhaps the worst problem we encountered in libzmq was dependence on people who could understand the code, manage the GitHub branches and make clean releases — all at the same time. This is similar to the search for athletes who can run marathons and sprints, swim, and also lift weight. We humans can be very good by observing specialization. To ask us to be really good in two contradictory things is not the best idea: it will drastically reduce the number of candidates, which is bad for any project. We had this problem in libzmq in 2009 or so, and it was solved by dividing the role of Meynteiner into two: one person makes patches, and the other releases releases.

Providing faster and more accurate project development by developing a decision-making process.

This theory is not fully proven, but not falsified. The larger the community and the number of people who can participate in discussions without fear of being criticized or dismissed, the faster and more accurately the software is developed. The speed here is quite subjective. A quick move in the wrong direction is not just useless, it is very detrimental to the project (and we experienced a lot of things in libzmq before moving to C4).

Maintaining the life cycles of project versions from experimental to stable, by conducting secure experiments, quickly responding to problems, and isolating stable code.

This is an entertaining influence on the process: the git master branch is usually always ideally stable. This is due to the size of changes and the waiting time, i.e. period from writing someone code to its full use by someone else. Yet the normal process of learning design is usually repeated in the projects, until it becomes stable and unshakable.

Reducing the intrinsic complexity of repositories, leading to easier participation of contributors and a reduction in the amount of errors

A curious observation: people who succeed in difficult situations like to increase the degree of entanglement, because then they retain their high value. This is the Cobra effect (google it). Git made the branches light and left us with the too common syndrome “git is simple if you understand that the git branch is just a folded five-dimensional lepton space that has a disconnected history without intermediate cache”. Developers should not feel stupid because of their tools. I have seen too many high-profile developers tangled in repository structures and not accepting common wisdom about git branches. We will be back soon to deal with the git-branches, dear reader.

Ensuring the status of collective ownership of the project, which increases the financial motivation of participants and reduces the risk of plagiarism from hostile organizations.

Ultimately, we all are economic beings, and the feeling that “we own it and our work can never be used against us” makes life much easier for people to invest in an open source project such as ZeroMQ. And it cannot be just a feeling, it must be a reality. There are a number of aspects for creating collective ownership, we will look at them one by one when we go through C4.

Main provisions

The project MUST use the git distributed version control system.

Git has its drawbacks. His command-line API is terribly inconsistent, and he has a complex, sloppy internal model, which he pokes you in the face for any reason. But, despite the fact that it is doing everything possible to make its users feel stupid, git does its work very well indeed. More pragmatically, I found that if you stay away from certain areas (branches!), People quickly learn git and do not make many mistakes. This is suitable for me.

The project MUST be posted on github.com or its equivalent, here called Platform.

I am sure that one day a large company will buy GitHub and break it, and another platform will take its place. Until now, Github serves an almost perfect set of minimal, fast, simple tools. I sent hundreds of people there, and all of them are still there, like flies stuck in a saucer with honey.

A project MUST use a project management system.

We made a mistake in libzmq by switching to Jira, because we had not yet learned how to use the GitHub tracker correctly. Jira is a great example of how to turn something useful into a tangled mess, because a business depends on selling more “functions.” But even without criticizing Jira, saving the task tracker on the same platform means that there will be less user interface learning, less logging, less login, smooth integration between projects and patches.

The project MUST have clearly documented code style guidelines.

This is a protocol plugin: insert code style rules here. If you are not documenting the style of code you are using, you have no reason to reject patches besides biases.

A “Contributor” is a person who wants to provide a patch, which is a set of commits that solve clearly defined problems. "Maintainer (Maintainer)" - a person who combines the patches in the project. Maintainers are not developers; their job is to follow the development process.

We now turn to the definitions of the parties and the separation of roles that have saved us from the pernicious structural dependence on rare people. This works well in libzmq , but, as you will see, it depends on the rest of the process. C4 is not a self-made tablecloth, you will need the whole process (or something very similar) so that everything does not fall apart.

Participants MUST NOT be able to commit to the repository, unless they are also Maintainers. Maintainers MUST be able to commit to the repository.

What we wanted to avoid was that people push their changes directly into the master branch. It was the biggest source of problems in libzmq historically: large masses of raw code, which would take months or years to stabilize. In the end, we followed other ZeroMQ projects, such as PyZMQ, using download requests. We went further and indicated that all changes should follow the same path. No exceptions for "special people."

Everyone, without distinction or discrimination, MUST have an equal right to become a Participant in accordance with the terms of this contract.

We should have pointed it out. It used to be this way: libzmq supporters refused to give patches simply because they didn’t like it. Now it may seem reasonable for the author of the library (although libzmq not written by one person), but let us recall our goal of creating work that belongs to as many people as possible. Saying “I don’t like your patch, so I’m going to reject it,” is equivalent to saying: “I say that I own it, and I think that I’m better than you and I don’t trust you.” These are toxic messages for those who are thinking of becoming your co-investors.

I think that this struggle between individual experience and collective intelligence is played out in other areas. She created Wikipedia, and still continues to do so, already a decade after having surpassed everything that a small group of experts could do. For me, this is how we make software, slowly synthesizing the most accurate knowledge, just as we do Wikipedia articles.

Licensing and property

The project MUST use the same license as MPLv2, or the GPLv3 variant (GPL, LGPL, AGPL).

I have already explained how full remixability (the ability to re-work with material) creates a better scale, and why MPLv2 or the GPL and their variants seem like an optimal contract for remixing software. If you are a big business aimed at dumping a code in the market, you do not need C4, but then you don’t have to do with the community.

All contributions to the project source code (“patches”) MUST use the same license as for the project.

This eliminates the need for any specific license or agreement to participate in the development of patches. You fork MPLv2 or GPL code, post your reworked version on GitHub, and you or anyone else can send it as a fix to the source code. BSD does not allow this. Any work containing a BSD code may also contain an unlicensed proprietary code, so you need permission from the author of the code before you can redo it.

All patches belong to their authors. No copyright assignment process MUST be present.

Here we come to the main reason that people are confident in their contribution to ZeroMQ: it is logically impossible to buy copyrights to create a competitor with closed source code for ZeroMQ. iMatix can't do that either. And the more people send patches, the harder it becomes. ZeroMQ is not just free and open today - this feature will allow it to stay that way forever. Please note that this does not apply to all MPLv2 / GPL projects, many of which still require the return of copyrights to their maintainers.

Each Participant MUST be responsible for identifying themselves in the List of Project Participants.

In other words, the maintainers are not karma accountants. Anyone who wants approval is required to declare it.

Patch requirements

In this section, we define the obligations of the Participant: in particular, what constitutes a “fit” patch, so that the Meinteners have rules according to which they can make decisions about accepting or rejecting the patch.

Maintainers and Participants MUST have an account on the Platform and MUST use their real name or a known nickname.

In the worst case, when someone placed a harmful code (patented or owned by someone else), we should be able to keep track of who did it and when so that we could remove the code. Specifying the real name or a well-known pseudonym is a theoretical strategy to reduce the risk of dummy patches. We do not know if this works, because we have not had any problems with this yet.

The patch MUST be the minimum solution to a specific identified and agreed problem.

This is a realization of the Principle of simplicity-oriented development, which I will discuss in this chapter later. One obvious problem is one minimal solution, application, testing, repetition.

The patch MUST adhere to the project style code rules (style guidelines), if defined.

This is just sanity. I spent the time cleaning other people's patches, because they insisted on putting else next to if, and not lower, as the Universe requires. Serial code looks healthy.

The patch MUST adhere to the guidelines for “Developing Public Interfaces” defined below.

Ah, pain, pain. I'm not talking about the time when I was eight years old, and I stepped on a board with a 4-inch nail sticking out of it. It was nothing yet. I’m talking about 2010-2011, when we had several parallel ZeroMQ releases, each of which had different incompatible APIs or wired protocols. These were exercises in bad rules, meaninglessly followed, which still hurt us today. The rule was: "If you change the API or protocol, you MUST create a new basic version." Pierce my foot with a nail, it is less painful.

One of the big changes we made with C4 is the prohibition of such authorized sabotage. Surprisingly, it is not even difficult. We just don’t allow us to violate existing public contracts, and that’s it, unless everyone agrees with this, then yes. As Linus Torvalds said on December 23, 2012: “WE DO NOT VIOLATE USER SPACE!”

A patch MUST NOT include non-trivial code from other projects, unless the Participant is originally the author of this code.

This rule has two effects. First, it forces people to make minimal decisions, because they cannot simply import samples of existing code. From what I have observed in other cases, this always leads to poor results, unless the imported code is very clearly separated. Secondly, it resolves licensing disputes. You write a patch, you can publish it as LGPL, and we can accept it. But if you find a 200-line snippet on the Internet and try to insert it, we will refuse.

The patch MUST clearly compile and self-test the project, at least on the main target platform.

- , , .

( 50 ) , («: ...»), , («: ...») ).

, . ( , — ).

« » — , .

, .

Development process

.

.

. : , , . ZeroMQ . . , open-source. , , , , .

, .

, . , () .

, .

«: X. : » — . «: A B, . : X» . Since , - , , : , .

.

, , . « A B, C . : C ».

, , , .

, . , , - . , , . , - , . — .

, , .

, GitHub , . . , GitHub, , , ZeroMQ, .

, , .

GitHub fork + pull request, (4), .

, Pull Request .

GitHub , git. , , git , , , — git, . , … .

.

, , , . , , .

, Pull Request' .

, GitHub , , , . Wikidot, . , - .

, .

- GitHub , Pull Request' . , . , .

.

, , : . , , , , . , ( ). , , , . « » . -, , C4, , . , , . -, , , - .

.

, : , . : , , . , -, , .

.

, , . — . Pull Request' , , ( ).

: () , () , () .

, , « », , .

( , ) – , (CI), , , . , / . ( ) « , » « ».

, . . , - .
, , .

, :

• , : «, ». , , , . . , .
• , . . . , . , , .
• . - : , , . : , .
• . --. - . - . , . - . , .
• . , . « ». , . , . .
• , open source. , .

, (). , . open source :

1. , , .
2. , , .
3. , , .
4. -, .

, , (4). , (2).

:

1. : , . . : .
2. : , , . : , . , -. .
3. : , . : . , . , .
4. : , . -- . . : . . , . git- .

, .

( ) . ZeroMQ, .

, , .

, , . , .

, , .

, , , «» «». , . , « », . libzmq , , , . , .

, .

.

C4, . : . , .

, .

(«»), , .

, - , . - ( ), - .

«topic branch» - . «topic branch».

. ( «tl;dr», ), , – , .

, . -.

« » API . 2011 libzmq . (.. « ») libzmq , , . 2012 « », .

, . «» ( , , ) — API , . , ZeroMQ 2.0, 3.0 4.0, .

(API ) .

, -, , . — . C4 , , . « ». . (, - C4, ).

.

, . , , , . ( ) .

, , , .

API, . , , . « » . ZeroMQ v3.x ZMQ_NOBLOCK ZMQ_DONTWAIT? , POSIX recv(), ? . : « ».

, , ( ).

ZeroMQ , ( – , -). ZeroMQ v.3.0 «ROUTER», , «ROUTER» 2.. , , ? : , , .

«» («draft»), .
«» («deprecated»).

, , . «» , « , ». , « , ». , , , . «» , « ».

«» («deprecated») .
, .

. , , , , API.

.

, , , ZeroMQ v3.x API (zmq_send[3] zmq_recv[3]) , ( , , ). , , , , , , , . , , ! , .

.

- , , .

, .

, . .

, , , .

, , . .

, , .

: . , ( , ).

« », , . , . « » — , , , , , .

. . C4 . -, , , . -, . , ( , , , ).

« »:

• Preface.
• 1.
• Chapter 2
  - Emotional burnout of volunteers
  - How to capture / protect an open-source project
  - The myth of individual intelligence
• Chapter 3
  - ZeroMQ Community
  - Psychology of software architecture
  - Importance of contracts
  - 4 steps to a self-governing community
• Chapter 5
  - Design, development, innovation
  - Stratagem for success open source projects
• 6.

---

/ , .
25 !

Telegram
, PhilTech- .

#philtech news
#philtech .

Facebook
Philtech-, .


.