Thinking out loud about protocol X

Two years ago, while working on Awesome , I joined the development of XCB , which is part of the Freedesktop initiative. I had to learn the secrets of the X11 protocol and the whole ancient and mysterious world surrounding it.

Over the past few months, I have finally washed away all this dirt from myself, and now I feel the need to share my thoughts on all this mess that has been going on for decades.


')

When I was not in the world ...

... the Toto group released their famous song "Africa" , and a few smart guys were working on a window system - the X Window System (this is the full name), which also has a (too) long story. The final protocol version (11th) was developed in the 80s. You can find out the details of the story in an article on Wikipedia .

In 2010, we still listen to disco, and still use various protocols developed in the 80s (and even earlier than X). Music evolves, protocols change, and X11 does not stand still.

The problem is that X11 does not evolve correctly. The guys from MIT-and-any-other-places-with-wise men created the first version of X in 1984 and made changes up to the 11th version of the protocol. It was released in 1987, and we use it today. Eleven versions were released in 3 years in full compliance with the “early release, frequent updates” model. I do not know why, but this approach was no longer applied over the next 23 years (although the developers added (and removed) many protocol extensions).

I do not know what changes were made in the first 11 major versions of Protocol X, but I am more than confident that in the past 20+ years there is a great need to make major changes to it.

In my humble opinion, X11 was not designed with the expectation that he would live for 23 years. True, I do not blame anyone: I was a 4-year-old child and played Lego when the latest version of Protocol X was released, so I had very few chances to do something better.

We will not fix it. We will adapt

This is perhaps the main slogan of protocol X in recent years. Just do not get me wrong: I'm not going to make a spanking.

The X11 protocol grew older over the years, and developers began to add extensions to it. Over the years, tons of extensions have been added. In strict accordance with one of the initial principles of X :

It is important to clearly define what the system is and what it is not. Do not try to serve the needs of the whole world; instead, the system should be made expandable so that additional needs can be satisfied in an appropriate way.

All of them, without exception, were added because (alas, X) the X11 protocol did not provide for all things that have appeared in the last 23 years, such as video, OpenGL, multiple monitors, or the pleasure of drawing oval windows. Some of these extensions are still in use, and some have already sunk into oblivion.

Protocol extensions are not something bad, but attempts to fix the protocol with extensions like XFixes seem like a bad idea to me, despite all the good intentions that Keith Packard could have guided.

Actually, it's still worse than you think.

The X11 protocol (without extensions) defines about 120 types of queries: window creation, window movement, and so on.

Nowadays, at least 25% of them are completely useless: the use of server-side fonts or the drawing of squares and polygons are not used by any modern application or toolkit. All of them are redefined in requests from extensions, such as, for example, XRender .

Working with multiple monitors is completely confusing. X11 was designed to work in Zaphod mode (independent monitors). But Xinerama , and nowadays XRandR has completely supplanted it - the latest versions of server X (from about 2007) no longer support the Zaphod mode, even though it is part of the core of the X11 protocol.

Even worse: many requests contain limitations or flaws in the design, for example, those described in the document by researchers from DEC: Why X is not our ideal window system .

We pile on top of even more broken standards.

Following its original principles , X does not define policies, but only provides mechanisms that look right.

As a result, the developers got together and started writing specifications to define a set of rules and dogmas: ICCCM . That was 22 years ago, in 1988. It is useless to add that many items of this specification are already hopelessly outdated, because they did not take into account many modern technologies.

I am not the only one who thinks so. The authors of the projects, which became the two main desktop environments, KDE and GNOME , saw this already in the 90s, when I was just learning to count. And they wrote EWMH , another standard that perched on top of the ICCCM, complementing it with a number of nice features (maximization, full screen mode, and others).

The problem is that this standard was also written by not very far-sighted people who were simultaneously working on their own environments (GNOME, KDE, and maybe some others). These desktop environments had then and still have a number of strong concepts that determine how the desktop should work: “it should have workspaces”, “a window can exist only on one workplace”, “we can only see one workplace at a time, ”“ we do not have multiple screens, ”and so on.

Dude, don't worry: we have tulkits!

The idea of ​​how the desktop should work nowadays is carved in marble for each application or library that implements EWMH, including GTK + and Qt .

As a result, now everyone just forgot about the standards. Tulkits implement them independently, bypassing the limitations and flaws of the X11 protocol, and no one wants to look back.

And with these toolkits, people are treated as carelessly as with standards. This sometimes leads to unpleasant side effects. For example, Openoffice as a switch jobs .

We do not want to look back? Worse, we forgot where we came from!

The desktop with all these poorly designed standards continues to evolve for over a decade. New standards continue to be added to it, the most recent of which are based on D-Bus, for example, the Desktop Notification Specification or the new Status Notifier Specification , developed in KDE.

Status Notifier is a new implementation of the good old system tray, based on XEmbed , but now using D-Bus mechanisms instead of X11, and adding the ability to show something more than just icons in the system tray.

The draft specification contains a serious design flaw, which Wolfgang Draxfinger noticed in his appeal to the XDG mailing list . Wolfgang points out that X is a network-centric protocol, and D-Bus is not. Therefore, using the Status Notifier D-Bus specification to send system tray events is a bad idea: running a graphics application from computer A on computer B will update the tray on the wrong host!

From the reading of the correspondence it is obvious that this does not frighten some KDE developers :

Of course, you should not worry much about this bizarre particular case. At the very least, you will consider him as such until you come across him yourself (just testing something or setting up some sophisticated kiosk). "

What Oswald describes as a special case for many of us is the most typical situation. In general, as anyone lucky.

In my opinion, this is a step back in the wrong direction. But we can also conclude that the network part of X has become useless, at least for KDE.

I prefer to believe in XCB

When I joined Freedestop, I had to work on the XCB library, XC Binding. XCB has a nice, clean, 21st century-based API for working with the X11 protocol. Its code is generated automatically based on an XML file with a description of the protocol.

For comparison, Xlib consists of a muddy code of the 80s codes, practically uncommented and full of hard-coded things. Only a few are able to understand some of its nooks, such as internationalization or the implementation of XKB. And all his code is synchronous .

Let me explain to those who are not yet up to date: X is a network protocol in which you must send a request (like GET to HTTP) in order to get an answer. Xlib causes the application to wait for a response to the request, so the application is blocked until server X sends an answer. XCB does not block the application, which allows you to send a series of requests, do something useful during the waiting time, and then get answers.

Imagine that your browser sends to the web server one request at a time and makes you wait for the page to display until it downloads the latest image.

In the case when X and all its clients are located on the same computer, the delay is small and inconspicuous, so the benefit of asynchronous XCB is small. However, on a slow network, the performance gains can be huge, as Peter Harris proved by rewriting xlsclients to XCB .

One of the long-term goals of XCB developers is to finally get rid of Xlib in order to increase speed and reduce the response time of X11 applications. To do this, you need to port a lot of libraries, because almost none of them (with the exception of Cairo ) support XCB.

From my point of view, now it is a rather useless work. The desktop world is trusted to GNOME and KDE projects, that is, GTK + and Qt. At the same time, it seems that none of these toolkits are interested in working either on XCB or on protocol X. They rather make great efforts to circumvent the existing limitations and flaws of X, and still sit atop a barrow of crutches and implementations of perverse by design standards. It seems to me that no one wants to look back on all these levels and see how they can be improved.

They climbed too high to look down and see what recoil could be.

Enlightenment with its EFL was the first toolkit to have an XCB backend (thanks to the work of Vincent Torri). Unfortunately, this backend is no longer supported, and no one cares about it. The last time I took it, it did not even compile.

X12?

The Freedesktop wiki has a page called X12 that lists all the things that need to be fixed. Unfortunately, this list is only growing, and no one stutters about working on X12.

On the other hand, there are several people who are working on XKB2 in their free time, the second version of the “ let's -again-try-fix-keyboard-part-protocol-written-23-year-back” extension.

In general, it does not appear that X12 will appear in the coming decade.

Alternatives?

Do we have an alternative to X? There is a wayland , but for now it is rather useless. There is also DirectFB , but it doesn’t port well. In my opinion, there are no candidates to replace X yet.

In any case, none of the main toolkits support such an alternative. GTK + once supported DirectFB, but as far as I know, now nothing works (as noted by Josselin Mouette ). That is why the latest versions of the Debian installer have moved to using X in the graphical part (thanks to the work of Cyril Brulebois).

Conclusion

XCB has existed for more than five years, but very few have become interested in it. As far as I can see, no one is interested in using the X protocol, everyone is just trying to encapsulate it with some high-level API in order to stop seeing it as soon as possible. This leads to poorly written applications and toolkits full of ugly hacks.

All this also means that writing new applications and XCB-based toolkits should be a very interesting project, but this will have to spend too much time to figure out how to bypass the flaws of Protocol X, introduced by many predecessors over many years, including Qt and GTK + .

The main toolkits practically do nothing to win at the expense of returning to the dark waters of X. I think most of their developers prefer to work on beautiful 3D effects based on geolocation than on redefining the best base for everyone.

There is too little labor in world X. The lack of maintainers of X in Debian is a simple consequence of this situation. Of course, there are very competent and qualified X developers, as you can easily see by reading the blogs on Planet Freedesktop (I don't count). Unfortunately, their number is not enough to cover the entire scope of X: input devices, graphic devices, specifications of new protocol extensions, and so on. Server X is a fairly late development, and most developers are more interested in working on it, and not on the protocol itself. This can be understood.

I wonder where we will come with all this in a few years. By now, I have been cooking in a pot called X for 3 years now, and I feel that sooner or later all the alternatives to KDE and GNOME will become extinct. The time when you could choose between a dozen “modern” window managers has passed.

In the end, maybe this is simple Darwinism as applied to computer software.

Translator's note: The author of this article is French programmer Julien Danjou, the developer of the Awesome window manager. The original article is located here . This is my translation. I have not practiced translations for a long time, so I will be grateful for corrections and clarifications.