Can a simple user participate in the development of the Linux kernel? Why it is needed and what causes
Not so long ago, a topic was published on Habré - Brain Fuck Scheduler - set in 5 minutes. It was about installing an alternative scheduler in the Linux kernel, using the example of Ubuntu. The author of the scheduler, Con Kolivas, was for some time a fairly well-known developer of kernel patches. But then he stopped his activity. I was curious to know what kind of person is behind this name. It turned out that Kolivas, in his main job, is not an anesthetist programmer, but an anesthesiologist. This pushed curiosity even further. As a result, an article was found from his interview, true two years ago, in which he deals not only with the core issues, but also with the development of the computer industry as a whole. The article seemed to me so interesting that I wanted to translate it. Something I have shortened, but I think that the essence is stated correctly.
Original - here
Translation - under the cut
')
Linux is full of corporate junk, and this makes it weak in the role of the desktop system, says Kernel hacker Con Colivas, who announced his retirement after several years of work on improving the Linux kernel.
Kon Kolivas is a notable kernel hacker, an adherent of using Linux as a desktop operating system. But why did he decide to leave the development of the kernel?
It is rather difficult to get information from the “first hand”, since the main type of its activity is the work of the anesthesiologist at the hospital, and the development of the Linux kernel is just a hobby.
Despite this, Kon Kolivas is a well-known name in the light of the development of the Linux kernel, and there are good reasons for this. Its focus on the performance of the operating system kernel on desktop systems led to the emergence of a crowd of fans and its kernel patches (marked as -ck) had a noticeable effect on the development of the kernel as a whole. Some of its changes were directly incorporated into the kernel, while others inspired other developers to change, which continues to be made to the kernel to this day.
The interview gives the reasons why Kolivas left development, as well as his point of view about the future of the Linux kernel.
The death of innovation on the PC and how Con Kolivas began to notice the problems of productivity on the desktop Linux system.
APC: How did you get involved in kernel development? Did you get pleasure from it and what motivated you at all?
In fact, this question must be followed by a direct answer. But since I have the opportunity, I will try to answer the question more deeply. And if I succeed, it will be the answer to all the questions you may have. This will be my view on the history of personal computers.
My first computer came to me 24 years ago. Then the story of the PC was just beginning and it was interesting in which direction it would move. All were in anticipation of what was to come, waiting for new developments.
The end of the 80s was the golden era of personal computers. Many developers came to this market and offered their new components, their own operating systems with unique functions. This gave the opportunity to choose and led to healthy competition. It is exciting to remember that at that time in Australia the Amiga was the leading computer in terms of the number of users and sales. Anyone who had a computer at that time could tell their own, different story, how he ran into computers and how the computer turned out to be at his home. Attempts to relive those exciting moments often ended in failure, but not in the case of Amiga. My point of view regarding Amiga is that its radical hardware design is a feat for developers to improve their software. During modern systems this is not.
At that time, IBM-compatible personal computers were clumsy, expensive, bombastic DOS machines for text editors. Their owners have always upgraded their video and sound cards, trying to reach the level that was originally in Amiga and Atari computers.
The priority of "hardware", moving the computer development, sometimes led to failures due to poor marketing, developer errors, and many other problems. And now software has become the king of the mountain and the iron just tries to match it.
We all know which operating system is a de facto standard at a particular point in time. And it is clear that for hardware that does not work with such an operating system, there is simply no market. As soon as such an operating system appears, hardware manufacturers targeting other operating systems experience a reduction in the number of programs that support it.
When this happens, the iron begins to perform a maintenance function in relation to the operating system. This state of affairs happened in 1994 and really to this day ... It is even worse that iron producers slowly absorb each other, reducing the choice for the user. Simply faster and more powerful versions of what was before are being released. As a result, a situation has arisen when faster processors, large amounts of RAM, disk space, and powerful video cards are needed only for servicing operating systems. Just the development of innovative iron is no longer perceived by the market. There is no money for it. There is no market for this. Computers have become boring.
Go to Linux. We all know that it began as a hobby. We all know that he grew up in something so much more than anyone could imagine. It can be said that Linux is now one of the most important of a very small number of competing operating systems on the players, while the market has a de facto standard - MS Windows (which I consider to be unfair). If the development of innovative hardware was still the driving force behind the development and competition of operating systems, Windows would not have so many users, developers, and most importantly, time to become what it is now - a standard. Iron has changed very little during this time. PCs of course have become incredibly more powerful compared to those that existed in 1991, when Linux was first loaded. But this is a consequence of increased speed, not functionality or innovation.
So what's wrong with the PC? I believe that the PC died from all points of view 20 years ago. Now we know that PC is rubbish and in the foreseeable future all the environment, methods of information processing, communication will remain the same. Internet has strengthened the position of the PC.
Linux was created for personal computers. Those who are looking for joy and fun to use their computer, the presence of the operating system, which is the de facto standart, is not an obstacle. We want to improve, have full control and authority over the operating system. Or, alternatively, we believe that this can be achieved by freedom of action. And we use Linux. This is exactly the reason why I was connected to him, I wanted something to use on my home PC.
However, PC is trash. This is rubbish. In all areas of application that are important to us, everything is overloaded and too slow. Today we have personal computers that are equal in performance to supercomputers a decade ago. Ten years ago, we had personal computers equal to twenty years old supercomputers, and so on. So why is everything so slow? If computers are getting faster, why does it take longer to start up the system and launch applications? No, when they are engaged in pure processing of information (such as video decoding) they are amazing. But otherwise, today's computers are incredibly slow. They can be 1000 times faster than they were a decade ago, and everything will still work slowly.
The usual argument that people give me in response - but now, computers do a lot more than before, so this is not a fair comparison. Okay, but they are 10 times slower instead of being 1000 times faster, therefore they have to do more than 10,000 times. Obviously, those 10,000 times what they do, they do somewhere not where needed.
APC: So the performance issues in Linux on the desktop system have become your main motivator?
Yes. I started by improving Linux on the desktop, thinking “if I can control all the software, I can make it work faster”. There must be a way to customize, optimize and achieve better performance. Opportunities to improve something in the area of user experience were very limited when I started working with Linux. In general, he barely worked on the desktop, and the attempt to improve something with the purpose of acceleration led to the fact that nothing worked at all. The legacy of Unix was evident. We adapted the operating system not intended for the desktop and this process was painful.
Gradually, I noticed that changes in the core lead to an increase in speed. The changes were not huge, but they entailed slightly noticeable changes in things like processor behavior under load and the like. The first patch that I brought to the public did not contain my own code and was for the 2.4.18 kernel. It was somewhere around February 2002. I didn’t even know what C code was because I had never studied any computer science. Because of this, I was stuck in place until the 2.6 kernel was in development. I watched her move and I will be honest - I was horrified! Those core developers that I respected worked on some kind of corporate junk that absolutely no users need.
And worst of all, although I like to see Linux on a machine with 1024 processors and 1000 hard drives, the idea that the possibility included in the kernel can kill the performance of the desktop system is disgusting to me.
If quantitative indicators are all that we know, measuring performance, then indeed, it is better than ever. But we launch the audio player and it stutters. Stutters We have the devil knows how many megahertz, and we can not get to play audio normally?
Or start writing a large file to disk and at that time only the cursor will move, and everything else on the desktop will be dead, and the windows will not be updated for a minute.
I felt ready to cry.
I remember we sent a bug-report and one developer responded that he could not reproduce our problem on his four-processor computer with four gigabytes of memory, the hard drives of which are combined into a raid array. And think about what kind of hardware average users had 4 years ago.
All developers are doing something that has nothing to do with the desktop system. They are all employees of large companies who do not care about the problems of ordinary users. All they care about is 1% extra percentage of productivity obtained in its benchmark database.
Linux won. We are a huge competitor in the server and database market and all big companies are taking care of Linux. Money flows to developers and they improve system performance in areas where customers need it.
Users lost. The usual desktop computer for which Linux was originally developed was put aside. Productivity, in the form in which it is understood by ordinary users, no longer exists. Worse, we can't even measure it to prove something. The only place where I found an opportunity to improve performance is the core. Development of which also began to move in the opposite direction from the user's interests.
APC: And what did you do to try to convince kernel developers to focus their efforts on the needs of ordinary users?
I decided to develop a benchmark to try to get a numerical value that reflects problems in Linux performance. The first benchmark I created was terribly difficult to use, and the result was difficult to interpret. But at least I tried. That helped. Some changes were made as a result of his testimony, and they were mainly related to the operation of the disk subsystem. But I again faced the problem of performance scheduler.
I already had experience creating patches for the previous kernel and, ironically, they were associated with the scheduler. Although I never studied programming, the code before my eyes gradually began to fill with meaning.
I stopped telling people about what I was thinking: in which part of the code there was a problem. After complaints, reproaches and instructions, I decided that I should start digging into the code myself and try to do everything on my own.
After several unsuccessful experiments, I began to write code that helped ... it really helped. As it turned out, people paid attention to it, and my code was included in the kernel. I didn’t like how the scheduler runs interactive applications, but at least it worked on the desktop.
Not being satisfied with the deeper mechanism of the scheduler, I decided to redo everything from the very beginning, and so the second generation of “-ck” patches was born. This time they were almost entirely made up of my own code. This is actually the story of how I started writing my code for the kernel.
Following this path, I found that I was writing code that would be useful for many ordinary users, but I had no way of measuring improvements. There is always a placebo effect element that makes it very difficult for the end user to say whether there is an improvement or not.
Nevertheless, I myself tried to be resistant to this placebo effect and the fact that my site is already close to one million hits, says that people also see the difference.
APC: And what of your code was included in the official core?
Looking back at the time when my patches were heard, I can say that in reality very little of my code was included in the official core, although this aroused interest in the development of other people. For the most part, my code touched the scheduler in terms of improvements in properties such as interactivity, fairness, SMP user fairness, hyperthread fairness, and some others. There were also a lot of smaller changes in different areas - in the virtual memory subsystem, deferred calls, scheduling of the i / o subsystem, and random bug fixes.
My latest patches concern those areas where, in my opinion, the main problems for desktop users are hidden. And, despite my attempts, these problems only get bigger every year. However, they do not receive the attention of professional developers. They are concerned only when the problem becomes so obvious that people included in the mailing list of kernel developers complain about it.
APC: What was the payback for your participation in the development?
When I was immersed in the business with my head, I did not sleep at night, thinking about solving another problem. But it could affect my work and family life. I tried very hard not to sacrifice these things for the sake of participation in the development of the kernel, but at times it turned out that I was pushing them away quite far. I hold the opinion not to regret what I did, but now, when I am “tied up” with the development, I treat that period of my life favorably.
APC: What was the main force that led to the abandonment of the developer keyboard and the “-ck” patch sets.
This is one of the questions about which I want to say - “it’s a pity that I don’t get a coin every time I get it asked”
Like most people, I had three reasons that motivated me to participate in the development, but this did not concern my normal career and life.
The first reason is fun. I have always been a computer geek and I liked to spend hours in front of a computer doing ... yes, no matter what. And so spending time doing something that is your passion is an obvious source of pleasure.
The second is an intellectual test. There were things that I thought should be done in the core, and there were people who did not consider it necessary to do this. I wanted to face them and do something that I had never done, especially in the field of “high programming. I learned a lot about the development of operating systems, I got acquainted with the basics of computer science, which most of the students of relevant faculties may consider to be very boring.
It was all new to me. At the same time, I brought little to the research of operating systems. There are a lot of innovations in Linux, but they do not relate to the ways in which I solved the existing problems. All studies have already been conducted during the confrontation of software and iron. Many people accused me of claiming to have invented an “honest planner.” But speaking directly - I have never claimed this.
Everything I did was known and I simply used it in relation to the existing gland and core. But at the time when the problem and its solution were known for a long time, there was a difference between its hardware and software start, because of which the problem appeared. My innovation was only in the embodiment of existing ideas. Academic achievement tends to be inapplicable in the real world. On the other hand, pure hacking has the nature of a disaster in the long run, even if at the moment it can quickly solve the problem. Finding the right balance between hacking and not ignoring long years of academic research is what is needed.
APC: And you found this balance?
Not. But this is what I was trying to achieve. I think there are only a few kernel developers who could. But I think that if one of them starts making decisions which code is “correct” and which is not, based on purely human motives, it will be impossible to recognize.
Well, the third reason is the ego. If I corrected something that worried me, it usually turned out that there were a lot of people who cared about it as well as me. The reason for this was that I was a regular user who, at the same time, claims to be a kernel hacker. And if I introduced changes that affected the same ordinary users, then this worried a lot of people.I remember one famous developer, whom I respect very much - he was joking about my “fans”, asking how can I entice the crowd so much? He contributed to the kernel perhaps 1000 times more lines of code, and I had “followers”.
But I only attracted users because I was working on what I was concerned about. The thing that made me develop was just a “thank you” that I received from users of -ck patches.
But I still have not answered the question of what made me stop working on the kernel. I think that the explanation of the motives why I did will help explain why I stopped.
An intellectual test - yes, it still was.
Fun, pleasure? No, it disappeared. In my public statement, sent by e-mail, that I was leaving, I slightly touched this point. My patches were originally for my own experiments. But when the changes became more widespread, the improvement became more serious and more noticeable. This led to the fact that more and more people were beginning to ask me when they would be included in the main kernel version. The more such questions there were, the less often my answers became, that I don’t want to touch on the issues of including my code in the kernel — to participate in the mainstream development. Once I posted some of my patches for example in the mailing list of kernel developers. This caused even more interest and questions about the inclusion of my code in the kernel. In the end, I was tired and had to do it.
APC: And what was the “last straw” for you?
The main failure to include my code into the kernel, which I received, concerned my scheduler. He was much better at interactivity, although he was also not perfect. As I continued to work on it, the focus of the main maintainers shifted from the scheduler to other issues. And the reason Alan Morton refused to include my code was that there are more urgent issues and bugs that need to be fixed in the core. It really was. But there are always a lot of subsystems in the kernel that are constantly being rewritten and flaws are detected in them. But their constant rewriting is more important than improvements concerning desktop issues, right?
But the main problem was that there was no clear way to prove that my scheduler is much better when working on the desktop. Just user messages were not enough. What a benchmark was not. And the impossibility of proving the unfounded assertion of users simply angered the mainteiners with their subjectivity. I tried again to write my benchmark. It was better than the previous one, but it’s almost impossible to prove the benefits of a scheduler based only on a chain of numbers.
With the help of William Lee Irwin, I published my scheduler, which could be connected to the kernel as a module, so that I could choose to use which scheduler I wanted to use. This is very similar to the plug-in I / O subsystem scheduler that is used in the current kernel. But this was rejected by Linus and Ingo (the scheduler manntaineer), since they believed that there should be only one scheduler in the kernel, as good as possible.
The goal of Plugshed was to integrate our scheduler into kernel code. I do not know if this is correct, but many people continue to go this way.
Then work began on the swap prefetch function. I spent a lot of time improving it. They were included in the core 18 months ago and since then I have been working on it.
Andrew did not believe that this feature is useful and believed that it could cause negative consequences. But over the past 9 months there have been no bug reports or complaints about performance loss due to it. I wrote a benchmark that showed how it works. But Linus asked me, “Yes, but does it really help?” User reports and benchmark indicators were not enough either. Well at least they just did not throw the function out of the kernel.
Now about the Staircase scheduler Deadline CPU. First, work on it began as a branch from the Staircase CPU. But I soon realized that it provides excellent interactivity, and it is even possible to solve problems with "honesty", since the improvement of interactivity is often the cause of the deterioration of "honesty". Well, honesty is paramount for servers and systems with a large number of users.
Many users sent requests to “push” the scheduler into the main code, because it corrected many flaws ... and there were even users who required Linus to do this by releasing a bug fix for the main kernel as quickly as possible. In the meantime, I improved the code and fixed some bugs.
In the end, Ingo decided that the idea was good ... and he wrote his planner, who would be both honest and provide good interactivity. And with the code he was helped by a man who refused to take the solutions that I proposed.
I was lying and wondering what I was doing and why. But I decided to bring work on the Staircase Deadline scheduler. And then I left work forever.
Original - here
Translation - under the cut
')
Why I left.
Linux is full of corporate junk, and this makes it weak in the role of the desktop system, says Kernel hacker Con Colivas, who announced his retirement after several years of work on improving the Linux kernel.
Kon Kolivas is a notable kernel hacker, an adherent of using Linux as a desktop operating system. But why did he decide to leave the development of the kernel?
It is rather difficult to get information from the “first hand”, since the main type of its activity is the work of the anesthesiologist at the hospital, and the development of the Linux kernel is just a hobby.
Despite this, Kon Kolivas is a well-known name in the light of the development of the Linux kernel, and there are good reasons for this. Its focus on the performance of the operating system kernel on desktop systems led to the emergence of a crowd of fans and its kernel patches (marked as -ck) had a noticeable effect on the development of the kernel as a whole. Some of its changes were directly incorporated into the kernel, while others inspired other developers to change, which continues to be made to the kernel to this day.
The interview gives the reasons why Kolivas left development, as well as his point of view about the future of the Linux kernel.
The death of innovation on the PC and how Con Kolivas began to notice the problems of productivity on the desktop Linux system.
APC: How did you get involved in kernel development? Did you get pleasure from it and what motivated you at all?
In fact, this question must be followed by a direct answer. But since I have the opportunity, I will try to answer the question more deeply. And if I succeed, it will be the answer to all the questions you may have. This will be my view on the history of personal computers.
My first computer came to me 24 years ago. Then the story of the PC was just beginning and it was interesting in which direction it would move. All were in anticipation of what was to come, waiting for new developments.
The end of the 80s was the golden era of personal computers. Many developers came to this market and offered their new components, their own operating systems with unique functions. This gave the opportunity to choose and led to healthy competition. It is exciting to remember that at that time in Australia the Amiga was the leading computer in terms of the number of users and sales. Anyone who had a computer at that time could tell their own, different story, how he ran into computers and how the computer turned out to be at his home. Attempts to relive those exciting moments often ended in failure, but not in the case of Amiga. My point of view regarding Amiga is that its radical hardware design is a feat for developers to improve their software. During modern systems this is not.
At that time, IBM-compatible personal computers were clumsy, expensive, bombastic DOS machines for text editors. Their owners have always upgraded their video and sound cards, trying to reach the level that was originally in Amiga and Atari computers.
The priority of "hardware", moving the computer development, sometimes led to failures due to poor marketing, developer errors, and many other problems. And now software has become the king of the mountain and the iron just tries to match it.
We all know which operating system is a de facto standard at a particular point in time. And it is clear that for hardware that does not work with such an operating system, there is simply no market. As soon as such an operating system appears, hardware manufacturers targeting other operating systems experience a reduction in the number of programs that support it.
When this happens, the iron begins to perform a maintenance function in relation to the operating system. This state of affairs happened in 1994 and really to this day ... It is even worse that iron producers slowly absorb each other, reducing the choice for the user. Simply faster and more powerful versions of what was before are being released. As a result, a situation has arisen when faster processors, large amounts of RAM, disk space, and powerful video cards are needed only for servicing operating systems. Just the development of innovative iron is no longer perceived by the market. There is no money for it. There is no market for this. Computers have become boring.
Go to Linux. We all know that it began as a hobby. We all know that he grew up in something so much more than anyone could imagine. It can be said that Linux is now one of the most important of a very small number of competing operating systems on the players, while the market has a de facto standard - MS Windows (which I consider to be unfair). If the development of innovative hardware was still the driving force behind the development and competition of operating systems, Windows would not have so many users, developers, and most importantly, time to become what it is now - a standard. Iron has changed very little during this time. PCs of course have become incredibly more powerful compared to those that existed in 1991, when Linux was first loaded. But this is a consequence of increased speed, not functionality or innovation.
So what's wrong with the PC? I believe that the PC died from all points of view 20 years ago. Now we know that PC is rubbish and in the foreseeable future all the environment, methods of information processing, communication will remain the same. Internet has strengthened the position of the PC.
Linux was created for personal computers. Those who are looking for joy and fun to use their computer, the presence of the operating system, which is the de facto standart, is not an obstacle. We want to improve, have full control and authority over the operating system. Or, alternatively, we believe that this can be achieved by freedom of action. And we use Linux. This is exactly the reason why I was connected to him, I wanted something to use on my home PC.
However, PC is trash. This is rubbish. In all areas of application that are important to us, everything is overloaded and too slow. Today we have personal computers that are equal in performance to supercomputers a decade ago. Ten years ago, we had personal computers equal to twenty years old supercomputers, and so on. So why is everything so slow? If computers are getting faster, why does it take longer to start up the system and launch applications? No, when they are engaged in pure processing of information (such as video decoding) they are amazing. But otherwise, today's computers are incredibly slow. They can be 1000 times faster than they were a decade ago, and everything will still work slowly.
The usual argument that people give me in response - but now, computers do a lot more than before, so this is not a fair comparison. Okay, but they are 10 times slower instead of being 1000 times faster, therefore they have to do more than 10,000 times. Obviously, those 10,000 times what they do, they do somewhere not where needed.
APC: So the performance issues in Linux on the desktop system have become your main motivator?
Yes. I started by improving Linux on the desktop, thinking “if I can control all the software, I can make it work faster”. There must be a way to customize, optimize and achieve better performance. Opportunities to improve something in the area of user experience were very limited when I started working with Linux. In general, he barely worked on the desktop, and the attempt to improve something with the purpose of acceleration led to the fact that nothing worked at all. The legacy of Unix was evident. We adapted the operating system not intended for the desktop and this process was painful.
Gradually, I noticed that changes in the core lead to an increase in speed. The changes were not huge, but they entailed slightly noticeable changes in things like processor behavior under load and the like. The first patch that I brought to the public did not contain my own code and was for the 2.4.18 kernel. It was somewhere around February 2002. I didn’t even know what C code was because I had never studied any computer science. Because of this, I was stuck in place until the 2.6 kernel was in development. I watched her move and I will be honest - I was horrified! Those core developers that I respected worked on some kind of corporate junk that absolutely no users need.
And worst of all, although I like to see Linux on a machine with 1024 processors and 1000 hard drives, the idea that the possibility included in the kernel can kill the performance of the desktop system is disgusting to me.
If quantitative indicators are all that we know, measuring performance, then indeed, it is better than ever. But we launch the audio player and it stutters. Stutters We have the devil knows how many megahertz, and we can not get to play audio normally?
Or start writing a large file to disk and at that time only the cursor will move, and everything else on the desktop will be dead, and the windows will not be updated for a minute.
I felt ready to cry.
I remember we sent a bug-report and one developer responded that he could not reproduce our problem on his four-processor computer with four gigabytes of memory, the hard drives of which are combined into a raid array. And think about what kind of hardware average users had 4 years ago.
All developers are doing something that has nothing to do with the desktop system. They are all employees of large companies who do not care about the problems of ordinary users. All they care about is 1% extra percentage of productivity obtained in its benchmark database.
Linux won. We are a huge competitor in the server and database market and all big companies are taking care of Linux. Money flows to developers and they improve system performance in areas where customers need it.
Users lost. The usual desktop computer for which Linux was originally developed was put aside. Productivity, in the form in which it is understood by ordinary users, no longer exists. Worse, we can't even measure it to prove something. The only place where I found an opportunity to improve performance is the core. Development of which also began to move in the opposite direction from the user's interests.
APC: And what did you do to try to convince kernel developers to focus their efforts on the needs of ordinary users?
I decided to develop a benchmark to try to get a numerical value that reflects problems in Linux performance. The first benchmark I created was terribly difficult to use, and the result was difficult to interpret. But at least I tried. That helped. Some changes were made as a result of his testimony, and they were mainly related to the operation of the disk subsystem. But I again faced the problem of performance scheduler.
I already had experience creating patches for the previous kernel and, ironically, they were associated with the scheduler. Although I never studied programming, the code before my eyes gradually began to fill with meaning.
I stopped telling people about what I was thinking: in which part of the code there was a problem. After complaints, reproaches and instructions, I decided that I should start digging into the code myself and try to do everything on my own.
After several unsuccessful experiments, I began to write code that helped ... it really helped. As it turned out, people paid attention to it, and my code was included in the kernel. I didn’t like how the scheduler runs interactive applications, but at least it worked on the desktop.
Not being satisfied with the deeper mechanism of the scheduler, I decided to redo everything from the very beginning, and so the second generation of “-ck” patches was born. This time they were almost entirely made up of my own code. This is actually the story of how I started writing my code for the kernel.
Following this path, I found that I was writing code that would be useful for many ordinary users, but I had no way of measuring improvements. There is always a placebo effect element that makes it very difficult for the end user to say whether there is an improvement or not.
Nevertheless, I myself tried to be resistant to this placebo effect and the fact that my site is already close to one million hits, says that people also see the difference.
APC: And what of your code was included in the official core?
Looking back at the time when my patches were heard, I can say that in reality very little of my code was included in the official core, although this aroused interest in the development of other people. For the most part, my code touched the scheduler in terms of improvements in properties such as interactivity, fairness, SMP user fairness, hyperthread fairness, and some others. There were also a lot of smaller changes in different areas - in the virtual memory subsystem, deferred calls, scheduling of the i / o subsystem, and random bug fixes.
My latest patches concern those areas where, in my opinion, the main problems for desktop users are hidden. And, despite my attempts, these problems only get bigger every year. However, they do not receive the attention of professional developers. They are concerned only when the problem becomes so obvious that people included in the mailing list of kernel developers complain about it.
APC: What was the payback for your participation in the development?
When I was immersed in the business with my head, I did not sleep at night, thinking about solving another problem. But it could affect my work and family life. I tried very hard not to sacrifice these things for the sake of participation in the development of the kernel, but at times it turned out that I was pushing them away quite far. I hold the opinion not to regret what I did, but now, when I am “tied up” with the development, I treat that period of my life favorably.
APC: What was the main force that led to the abandonment of the developer keyboard and the “-ck” patch sets.
This is one of the questions about which I want to say - “it’s a pity that I don’t get a coin every time I get it asked”
Like most people, I had three reasons that motivated me to participate in the development, but this did not concern my normal career and life.
The first reason is fun. I have always been a computer geek and I liked to spend hours in front of a computer doing ... yes, no matter what. And so spending time doing something that is your passion is an obvious source of pleasure.
The second is an intellectual test. There were things that I thought should be done in the core, and there were people who did not consider it necessary to do this. I wanted to face them and do something that I had never done, especially in the field of “high programming. I learned a lot about the development of operating systems, I got acquainted with the basics of computer science, which most of the students of relevant faculties may consider to be very boring.
It was all new to me. At the same time, I brought little to the research of operating systems. There are a lot of innovations in Linux, but they do not relate to the ways in which I solved the existing problems. All studies have already been conducted during the confrontation of software and iron. Many people accused me of claiming to have invented an “honest planner.” But speaking directly - I have never claimed this.
Everything I did was known and I simply used it in relation to the existing gland and core. But at the time when the problem and its solution were known for a long time, there was a difference between its hardware and software start, because of which the problem appeared. My innovation was only in the embodiment of existing ideas. Academic achievement tends to be inapplicable in the real world. On the other hand, pure hacking has the nature of a disaster in the long run, even if at the moment it can quickly solve the problem. Finding the right balance between hacking and not ignoring long years of academic research is what is needed.
APC: And you found this balance?
Not. But this is what I was trying to achieve. I think there are only a few kernel developers who could. But I think that if one of them starts making decisions which code is “correct” and which is not, based on purely human motives, it will be impossible to recognize.
Well, the third reason is the ego. If I corrected something that worried me, it usually turned out that there were a lot of people who cared about it as well as me. The reason for this was that I was a regular user who, at the same time, claims to be a kernel hacker. And if I introduced changes that affected the same ordinary users, then this worried a lot of people.I remember one famous developer, whom I respect very much - he was joking about my “fans”, asking how can I entice the crowd so much? He contributed to the kernel perhaps 1000 times more lines of code, and I had “followers”.
But I only attracted users because I was working on what I was concerned about. The thing that made me develop was just a “thank you” that I received from users of -ck patches.
But I still have not answered the question of what made me stop working on the kernel. I think that the explanation of the motives why I did will help explain why I stopped.
An intellectual test - yes, it still was.
Fun, pleasure? No, it disappeared. In my public statement, sent by e-mail, that I was leaving, I slightly touched this point. My patches were originally for my own experiments. But when the changes became more widespread, the improvement became more serious and more noticeable. This led to the fact that more and more people were beginning to ask me when they would be included in the main kernel version. The more such questions there were, the less often my answers became, that I don’t want to touch on the issues of including my code in the kernel — to participate in the mainstream development. Once I posted some of my patches for example in the mailing list of kernel developers. This caused even more interest and questions about the inclusion of my code in the kernel. In the end, I was tired and had to do it.
APC: And what was the “last straw” for you?
The main failure to include my code into the kernel, which I received, concerned my scheduler. He was much better at interactivity, although he was also not perfect. As I continued to work on it, the focus of the main maintainers shifted from the scheduler to other issues. And the reason Alan Morton refused to include my code was that there are more urgent issues and bugs that need to be fixed in the core. It really was. But there are always a lot of subsystems in the kernel that are constantly being rewritten and flaws are detected in them. But their constant rewriting is more important than improvements concerning desktop issues, right?
But the main problem was that there was no clear way to prove that my scheduler is much better when working on the desktop. Just user messages were not enough. What a benchmark was not. And the impossibility of proving the unfounded assertion of users simply angered the mainteiners with their subjectivity. I tried again to write my benchmark. It was better than the previous one, but it’s almost impossible to prove the benefits of a scheduler based only on a chain of numbers.
With the help of William Lee Irwin, I published my scheduler, which could be connected to the kernel as a module, so that I could choose to use which scheduler I wanted to use. This is very similar to the plug-in I / O subsystem scheduler that is used in the current kernel. But this was rejected by Linus and Ingo (the scheduler manntaineer), since they believed that there should be only one scheduler in the kernel, as good as possible.
The goal of Plugshed was to integrate our scheduler into kernel code. I do not know if this is correct, but many people continue to go this way.
Then work began on the swap prefetch function. I spent a lot of time improving it. They were included in the core 18 months ago and since then I have been working on it.
Andrew did not believe that this feature is useful and believed that it could cause negative consequences. But over the past 9 months there have been no bug reports or complaints about performance loss due to it. I wrote a benchmark that showed how it works. But Linus asked me, “Yes, but does it really help?” User reports and benchmark indicators were not enough either. Well at least they just did not throw the function out of the kernel.
Now about the Staircase scheduler Deadline CPU. First, work on it began as a branch from the Staircase CPU. But I soon realized that it provides excellent interactivity, and it is even possible to solve problems with "honesty", since the improvement of interactivity is often the cause of the deterioration of "honesty". Well, honesty is paramount for servers and systems with a large number of users.
Many users sent requests to “push” the scheduler into the main code, because it corrected many flaws ... and there were even users who required Linus to do this by releasing a bug fix for the main kernel as quickly as possible. In the meantime, I improved the code and fixed some bugs.
In the end, Ingo decided that the idea was good ... and he wrote his planner, who would be both honest and provide good interactivity. And with the code he was helped by a man who refused to take the solutions that I proposed.
I was lying and wondering what I was doing and why. But I decided to bring work on the Staircase Deadline scheduler. And then I left work forever.
Source: https://habr.com/ru/post/69622/
All Articles