Why am I closing my work on Debian

From the translator: this text is a translation of an entry in Michael Stapelberg’s personal blog (Michael Stapelberg) of a prominent open source developer ( GitHub profile ) who made a significant contribution to the development of Debian.

---

This post was difficult to write from an emotional point of view, but I didn’t limit myself to “a short letter, because I didn’t have time.” Please, before reading this text, please note that I am writing it with the best intentions and do not set myself the goal of demotivating or downplaying the contribution of someone from the developers. Rather, I want to explain why my level of frustration has exceeded all acceptable values.
')
Debian has been a part of my life for 10 years.

A few weeks ago, at a Debian meeting in Zurich, I met with my old friends whom I had not seen for many years. When I was riding my bike home, it dawned on me that all the topics we were discussing in one way or another came down to what we discussed with them last time. We discussed the merits of systemd, which once again attracted the attention of participants in the open source community, touched upon the topic of processes in Debian. The culmination was the discussion of democracy as such and the corresponding theoretical and practical errors. But, in fact, this is a purely Swiss theme.

This is not a review of the past mitap, I just want to explain what made me think about my current attitude towards Debian and whether it suits me.

So, I made a decision that I should have taken a long time ago: I minimize my participation in the development of Debian.

What does it mean?

In the coming weeks, the following will happen:

• I will pass important packets to a maintained command;
• remove myself from Uploaders packages that have other maintainers;
• Packages in which I was the only maintainer will become "orphans."

I will try to continue to support the manpages.debian.org pages and the codesearch.debian.net service, but I highly appreciate any help in this area.

If you have any questions or any problems, consider that I am on indefinite leave. I will try to take part in some administrative issues (for example, transfer of permissions) and tasks addressed to me personally, but only if it does not take a lot of time.

Why?

When I first joined the work on Debian, I was still studying, that is, I had a lot of free time. Now, after five years of working full time, I learned a lot - both in terms of how and what works in large software development projects, and in terms of understanding my personal preferences in computer systems. Now I am clearly aware of what I spend on that small part of my free time and what I have left at the end of the day.

Each of the following sections will be devoted to things that hurt me as a developer. This list is given in random order. Some of these problems affect each other, for example, if the changes to the system were more qualitative, we would have a chance that the packages will be easier to handle by the machine.

Debian change process

For the past few years, my current team has been working on various reorganizations across the entire code base (affecting thousands of projects), so we received many valuable lessons on how to make these changes effectively. It annoys me that Debian works in almost every way in almost every way. I appreciate the fact that each project is different, but I think that many of the following items apply to all of Debian.

In Debian, development of packages is channeled to the right path using a document called the “Debian Policy” or its software version “Lintian” .

Despite the fact that lint is very convenient for receiving fast direct or local / autonomous feedback, it is even better not to require using such a tool at all. For example, the C ++ command, which introduces a new security flag for all packages, should be able to make its work transparent for me ( judging by the GitHub profile, the main language of Michael is Go, - approx. Lane ).

Instead, now all packages are made “dirty” (orig. - lint-unclean): all maintainers should read what this new thing is, how it can break, whether it affects their work in general, and if so, how. Then you need to manually run some tests and, finally, discard the changes. All this - a lot of overhead and manually performed mechanical operations between packages.

In the Debian model, the decision to roll out new products rests on the accompanying packages, and not on the developers. At my main job, we found that it is more efficient to do the opposite: if a written change affects a significant part of users, then it is the developers who must make the decision about the need for its implementation. This approach makes project development more efficient and reduces time and labor costs. Of course, there are exceptions, for example, in large areas where language chips are used, which should be taken care of by the respective owner-curators, but what is important is that by default everything should be different than it is now.

There are no major changes in Debian: it is programmatically difficult to work with fragmented packages and repositories (more on this in the section below). The closest event to the default “sending a change to a review” is the process of opening a bug report with a patch attached to it. It seemed to me that the existing process of adopting bug fix was too complicated, and I started working on the merge bot, but only Guido showed interest in him and no one else ( apparently, the author talks about Guido agx Gunther , another Debian developer, - Lane comment .

If expressed literary, the reaction to the fluff and, accordingly, receiving feedback are slow. There are simply no deadlines. Sometimes I get emails notifying me that the patch I sent several years ago (!!!) has finally failed. This stretches weekly projects for many years, which for me is a powerful demotivator.

It is curious to note, but the practice of online turtle activity is projected onto the offline culture, and I don’t want to discuss the advantages of systemd 10 years after I first heard about it.

Finally, any changes can be blocked by those who disagree, who ultimately refuse to cooperate. My canonical example of this is rsync. The curator declined my patches, which add support for debhelper to the package, solely out of my own convictions.

Providing such a degree of personal freedom to individual maintainers prevents all of us, the project participants, from raising the level of abstraction of the Debian build, which in turn complicates the development tools.

What would it all look like in a perfect world?

1. As a project, we must strive for unification. Indeed, unification does not exclude experiments; it simply changes the current compromise between simpler experiments and more complex automation; a compromise between more complex experiments and simpler automation.
2. Our development culture should move away from the paradigm of “this package is my farm, how dare you touch it,” a common sense of ownership, in which any participant can easily make or check changes, even without involving specific facilitators.

To understand what successful large changes (patches) can look like, I recommend reading the presentation by my colleague Hiram Wright, “Large-scale Changes at Google: Lessons Learned from Five-Year Mass Migrations . ”

Fragmented workflow and infrastructure

Debian usually prefers decentralized approaches rather than centralized ones. For example, individual packages are stored in separate repositories, and each repository can use any SCM (usually git and svn) or not at all. Plus, each repository can be placed on any site. Of course, the contents of each repository also differs slightly from team to team, and even within the team.

In practice, non-standard hosting options are rarely used, since they do not justify their cost, but often enough to cause great pain when trying to automate the process of making changes to packages. And instead of using the GitLab API to create requests for merge, you need to develop a completely different, more complex system that works with periodically (or constantly!) Inaccessible repositories, abstracts differences in the delivered patches (error reports, merge requests , retrieval requests) and so on and so forth ...

Radically different development processes are not just a waste of working time. I participated in long conversations about the various git development processes during DebConf 13 and understood that similar discussions took place then.

Personally, I can’t remember enough details about various development methodologies. Every time I touch a package that does not work like mine, I get very upset because I have to re-learn aspects of its work.

I noticed a similar workflow fragmentation in my own Go packaging team and tried to fix it by making suggestions for changes to the workflow , but could not implement them. The lack of effective automation and the low rate of change in the toolkit, despite my willingness to waste their own time and energy, killed any motivation.

Legacy infrastructure: loading packages

When you want to make a package available in Debian, you upload GPG signed files via anonymous FTP. There are several types of tasks (the queue daemon, unchecked, dinstall, and others) that run on a fixed schedule (for example, dinstall runs at 01:52 UTC, 07:52 UTC, 13:52 UTC, and 19:52 UTC).

I calculated that depending on the time of day you can wait more than 7 hours (!!!) before your package is installed.

But the worst thing for me is that the feedback is asynchronous with respect to the load. I like to do something, get it over with and go on to the next task. The current setup requires a long wait and, in fact, costly switching between tasks without good technical reasons. You may notice that a few minutes do not matter much, but when all the time in a day I can spend on Debian is measured in minutes, it is of great importance for perceiving my own productivity and satisfaction with the work done.

The last message I can find about the acceleration of this process is the post of George Ganneff Jaspirt from 2008.

What would it all look like in a perfect world?

1. Anonymous FTP would be replaced by a web service that accepts my package and returns a formal decision about accepting or rejecting it in its response.
2. For received packages there is a page that displays the status of the assembly and the time when the package will be available through a network of mirrors.
3. Packages must be available within a few minutes after the build is completed.

Outdated infrastructure: bugtracker

I'm afraid to interact with the Debian error tracker. Debbugs is a piece of code straight from 1994 that is currently used only by Debian and the GNU project.

Debbugs processes are based on the use of e-mail, that is, they are asynchronous and cumbersome. Despite the fact that it runs on the fastest machines that we have on the Debian project (well, so I was told when this topic was last raised), its web interface loads extremely slowly.

It is noteworthy that the web interface on bugs.debian.org is read-only. Setting up a working email zone for reportbug (1) or manually working with attachments is a pretty serious challenge.

For some reason I don’t understand, every interaction with debbugs leads to the creation of a multitude of chains of letters .

Besides the technical implementation, I also can’t remember the different ways in which Debian uses generation pseudo-packages for errors and processes. I need them too rarely so that I finally put it all in my head and remember exactly how they function and are used, but I occasionally come across them, so this variety is annoying.

What would it all look like in a perfect world?

1. Debian will switch from a non-standard way of tracking errors to (any) already established.
2. In Debian, automation of these processes appears. The main interface should be more convenient (for example, a web form).

Outdated Infrastructure: Email Archives

I’m confused by the fact that in 2019 we still don’t have a handy tool for viewing archived conversations in the mail. As widely as in Debian, Email and conversations are not used anywhere else, so this is even somewhat ironic. Using Gmane seemed to solve this problem, but its availability over the past few years was, to put it mildly, abrupt (now, at the time of writing this post, it does not work at all).

I tried to create a multi-threaded archive, but our list-masters were not impressed and refused to support the project.

It’s hard for machines to work with Debian

Although it is obvious that you can programmatically work with Debian packages, this experience can hardly be called pleasant. Everything seems slow and cumbersome. I have selected three small examples to illustrate my position on this issue.

debiman needs help from piuparts in analyzing the alternative mechanism of each package for displaying documentation pages, for example, PSQL (1) . This was required, since the scripts modify the alternative database by invoking shell scripts. But without actually installing the package, you won’t know what changes it makes.

pk4 should maintain its own cache to search for package metadata based on its name. Other tools analyze the apt database from scratch with every call. The correct database format, or at least the binary exchange format, will be of great importance for this process.

Debian Code Search wants to get new packages as quickly as possible. Previously, the fedmsg instance was used , but it no longer exists. It is not clear where to get new package notifications and where to get them from.

Difficult build stack

Here you can simply read my post "Debian package building tools" . I am really worried about the fact that the increase in the number of tools is not considered a problem.

Debian is a painful developer experience.

Most of the questions I have raised relate to the Debian development experience, but, as I recently described in my post on Debian Debagging , the experience gained from developing using Debian also leaves much to be desired.

I have more ideas

The article was quite voluminous, but I hope you got a rough idea of ​​my motivation.

Although I described a number of specific flaws above, the final nail in the coffin is the lack of a positive prognosis for the future. I have other ideas that seem really convincing to me, but, based on how my previous projects were moving forward, I don’t think I can implement any of them within the Debian project.

I intend to publish a few more posts about specific ways to improve operating systems on my blog. So drop in.

Finally, I hope that this post will inspire someone, ideally a group of people, to make the life of Debian developers better.