The story of how Linux brought in Windows
All the time I work at Microsoft, I have been creating tools for Linux developers. I started work in August 2016, after graduating from the University of Virginia, where I studied computer science and management. During my studies, I programmed mainly in C ++. My main operating system was Linux.
It may seem that my experience does not quite correspond to what Microsoft might need, but at that time the company was undergoing a major shift, both in terms of technology and in terms of culture. The company was moving into a new state in which all operating systems, including Linux, were important to it.
My work at Microsoft began on the Linux team. I was doing SQL Server product there. I was invited to join this team, hoping that I would bring my experience in Linux to it. I was greatly impressed by the fact that, although I had just left out, I could be of value to the team because of my experience.
')
A few years ago, the idea of ​​developing a variant of SQL Server for Linux would only be an April Fools joke, but in 2016 it was completely real. I joined the team shortly after they released the first version of the product. I began to improve the SQL Server toolkit, particularly for administrators. This toolkit was aimed at managing Linux servers and applications. Using SQL Server on Linux required bringing the command line tools to the look Linux users were used to.
In addition, I had the opportunity to design the first version of the SQL Server GUI for Linux. I started with a fork of Visual Studio Code, which today is called Azure Data Studio. This is an Electron-based application that, regardless of the operating system, can work with any kind of SQL Server.
I learned a lot from SQL Server for Linux in my first year at Microsoft. In this knowledge, I can include the development of an approach to managing the development and maintenance of projects based on a combination of technology and business thinking.
WSL Team, Chocolatey, and Boxstarter at Microsoft Build 2018
In August 2017, I joined the WSL (Windows Subsystem for Linux, Windows Subsystem for Linux) team as a project manager. The first time I heard about WSL was during the announcement at Microsoft Build 2016. Related video from Channel 9, “ Running Bash on Ubuntu on Windows! ", Immediately after the release became viral. It clearly interested the audience more than many other announcements made at the conference. WSL technology was briefly, literally within a couple of minutes, discussed in the framework of voicing the main theses of the conference. However, the audience from this directly went crazy, not to mention the journalists. There was a time when the Channel9 support team feared that the high level of user interest in this video clip was actually due to a DDoS attack! Microsoft spokesperson launching Bash on Ubuntu running inside Windows ... This action was an instant hit.
The first team to discover user needs for WSL was the one that worked on the Windows Console. During development, they have heard over and over that people want something like Linux's Bash. As a result, the team came to the following thought: “Why do something resembling Bash, if you can just make the Bash shell run right on Windows?”
This is not to say that it was easy to do. The creation of WSL required a combination of in-depth knowledge of Windows, which the development team at the core of the system possessed, and a technology developed at Microsoft Research called picoprocess. It is interesting that picoprocesses, in addition, are the very technology that makes SQL Server work on Linux.
The picoprocess was supposed to provide unmodified Linux implementation in user mode. The Windows kernel team has been developing shims designed to connect Linux system calls to Windows. In other words, thanks to WSL, it is now possible to run code compiled for Linux on the Windows NT kernel. There was no need to recompile the code or use virtual machines.
We did not create then only our own Linux distribution. The fact is that there were many such distributions. The first Linux version available in WSL was Ubuntu. We contacted Canonical experts to find out if they would like to help us work on the WSL. They were enthusiastic about our idea. This led to the fact that Ubuntu appeared in the Microsoft Store. By the way, the previous sentence, in itself, sounds quite unusual. Now in the Microsoft Store you can count 6 distributions. I wonder what other application stores available on certain operating systems have other operating systems?
Microsoft Store now has 6 Linux distributions that can be used in WSL
The code we wrote then was Linux-compatible kernel system calls, which served as the interface between the Linux processes and the Windows kernel. Linux has about 340 system calls. The question was to decide which system calls to implement first. As with any operating system, on Linux new system calls are added as new versions of the OS are released, but old calls are never deleted, which ensures backward compatibility. At first, processing of a certain set of system calls was implemented, and everything else was wrapped in events like "not yet implemented." This allowed the WSL team to begin analyzing exactly what system calls Linux users need.
The answer to the question of which system calls should be implemented in the first place meant the need to establish communication with those people who would use WSL. The message about this technology at the Build conference was aimed at making people start using WSL and give feedback. In order to acquire WSL, you had to be a member of the Windows Insider Program. Anyone can connect to this program. You might think that the program participants were only those who were interested in Windows, but then, among more than ten million subscribers, there were people with a wide variety of interests. They were interested not only in Windows, but also, for example, games, and new features of Bluetooth, and WSL.
One of the user groups interested in making the Bash shell run on Windows was the web developer involved in building web applications that run on Linux servers. The whole process of assembling their applications was often a sequence of Bash commands. In addition, if someone decides to ask for help in the development of web applications, say, on Stack Overflow, most of the code samples that he can find will be designed to run on Linux. This is not particularly good for those who use Windows as their computer for web development. Often, it’s easiest for such developers to simply switch to the Mac platform. On macOS, similar code examples run without any problems.
In the first couple weeks of WSL's presence on Windows, one enterprise user was able to run XEyes. This program was run on an X11 windowing system running in WSL. XEyes is a simple program. She displays cartoon eyes that follow the mouse cursor. But this demonstration blew up social networks.
XEyes runs on Windows through WSL
We discussed a lot about exactly how we want to collect user reviews about WSL. The traditional tool used to collect feedback was UserVoice. We had a UserVoice-site designed for WSL, which collected hundreds of ideas and thousands of votes on various issues. The real question came up to us when it came to whether we should use GitHub. Since web developers were one of the first user groups interested in WSL, the question of GitHub was very important. But WSL was not an open source project. Placing such a project on GitHub looks strange. We decided to meet the developers' requirements and created a page on GitHub for reporting problems, for feedback and discussions. Since then, we have received thousands of messages regarding many issues related to using Linux on Windows.
Thousands of people have left error messages in the WSL GitHub repository . Each such message was reviewed and discussed by the WSL team, and comments were given on each of them. After that, a decision was made on further actions. If in order to realize a certain opportunity or fix some kind of error, it was necessary to write new code - this code was created and added to the WSL project, after which it got into all Windows assemblies distributed under the Windows Insider program. The whole cycle, from receiving an error message to correcting it, did not take much time - about a couple of weeks.
As a result, thanks to the operational response of the WSL team to user requests, it was possible to say that the community was involved in the process of creating WSL. People reported problems or their wishes through UserVoice or GitHub, the team reviewed all this and made changes to the project, which then appeared in the builds of the Windows Insider project.
When I joined the WSL team as a project manager, I focused on getting WSL out of beta. User complaints mostly related to compatibility and performance. But, in my opinion, if users worry about such things, it means that they use our development seriously. Performance is taken care of only by those who solve some big problems with the help of a certain software system. As a result, although we still had a lot to do, we did it for a reason, so that people would be able to solve more problems using WSL, and so that they could solve their problems faster.
As the capabilities of WSL began to expand, we made efforts to bring WSL closer to developers, and not just to those users who traditionally work using the Microsoft ecosystem. It was very interesting to attend events like PyCon and OSCON. The developers who were present there were surprised that Microsoft representatives also took part in these events. Developers were distrustful of the idea of ​​running Linux in a Windows environment. At these events, I showed SQL Server, WSL, and Visual Studio Code.
Showcase WSL at various events
I responded to their skeptical comments with a proposal to try what I showed them. When doubters began to execute their own commands, small scripts and snippets, I always met with a violent reaction to what was happening: “Wait a minute, and this is really Linux. How did you do that? Why didn’t I know about this? ” Often they came to the conclusion that we have created something that meets their needs, something that looks very interesting.
We took into account user complaints regarding WSL compatibility and performance, and released a new system architecture - WSL 2 . It provides full compatibility through the inclusion of the Linux kernel in Windows and provides a 20-fold increase in speed. I got an interesting experience creating the foundation of WSL 2 and watching the announcement of this technology at the Build conference, which was held in May 2019. Today, the WSL project has already outgrown the beta version and got to version 2.
In addition, I worked at Microsoft with other teams, trying to ensure that WSL could be used with their products. One significant example of this use of WSL is Visual Studio Code. This is the most popular code environment used in developing JavaScript and Node.js projects. I became interested in Visual Studio Code when I realized that developers using this editor can get significant benefits from WSL. At first, it was not about the need to write huge amounts of code. The main objective was to simplify the debugging of Node.js programs running in the WSL environment. This would give developers the opportunity to develop programs designed for the Linux version of Node.js on a Windows computer using WSL. Debugging such programs would look exactly like debugging them on Linux.
First attempt to integrate Visual Studio Code with WSL and Node.js
After this turned out to be possible for JavaScript and Node.js, we began to receive a lot of requests for something similar to be done for other languages, for example, for C ++ and Python. I was fascinated by this example of the integration of WSL and VS Code, I found that I was all extremely interested in this. This led me to my new role in building tools for Linux developers. Now I focus on tools for C ++ developers in VS Code. In this work, of course, I focus on Linux. It was nice to see a demonstration of Visual Studio Code Remote Development at the PyCon event this year when the corresponding WSL extension was released. At the same time, an extension for C ++ was introduced, which my team was developing.
Despite the fact that I did not spend so much time at Microsoft, I am glad that I was able to participate in the creation of many tools for Linux developers. This database and support for Linux on Windows, and tools for writing and debugging code. I plan to continue to work on Linux and create tools that developers around the world will enjoy using.
Dear readers! Do you use WSL?
It may seem that my experience does not quite correspond to what Microsoft might need, but at that time the company was undergoing a major shift, both in terms of technology and in terms of culture. The company was moving into a new state in which all operating systems, including Linux, were important to it.
My work at Microsoft began on the Linux team. I was doing SQL Server product there. I was invited to join this team, hoping that I would bring my experience in Linux to it. I was greatly impressed by the fact that, although I had just left out, I could be of value to the team because of my experience.
')
A few years ago, the idea of ​​developing a variant of SQL Server for Linux would only be an April Fools joke, but in 2016 it was completely real. I joined the team shortly after they released the first version of the product. I began to improve the SQL Server toolkit, particularly for administrators. This toolkit was aimed at managing Linux servers and applications. Using SQL Server on Linux required bringing the command line tools to the look Linux users were used to.
In addition, I had the opportunity to design the first version of the SQL Server GUI for Linux. I started with a fork of Visual Studio Code, which today is called Azure Data Studio. This is an Electron-based application that, regardless of the operating system, can work with any kind of SQL Server.
I learned a lot from SQL Server for Linux in my first year at Microsoft. In this knowledge, I can include the development of an approach to managing the development and maintenance of projects based on a combination of technology and business thinking.
WSL Team, Chocolatey, and Boxstarter at Microsoft Build 2018
In August 2017, I joined the WSL (Windows Subsystem for Linux, Windows Subsystem for Linux) team as a project manager. The first time I heard about WSL was during the announcement at Microsoft Build 2016. Related video from Channel 9, “ Running Bash on Ubuntu on Windows! ", Immediately after the release became viral. It clearly interested the audience more than many other announcements made at the conference. WSL technology was briefly, literally within a couple of minutes, discussed in the framework of voicing the main theses of the conference. However, the audience from this directly went crazy, not to mention the journalists. There was a time when the Channel9 support team feared that the high level of user interest in this video clip was actually due to a DDoS attack! Microsoft spokesperson launching Bash on Ubuntu running inside Windows ... This action was an instant hit.
The first team to discover user needs for WSL was the one that worked on the Windows Console. During development, they have heard over and over that people want something like Linux's Bash. As a result, the team came to the following thought: “Why do something resembling Bash, if you can just make the Bash shell run right on Windows?”
This is not to say that it was easy to do. The creation of WSL required a combination of in-depth knowledge of Windows, which the development team at the core of the system possessed, and a technology developed at Microsoft Research called picoprocess. It is interesting that picoprocesses, in addition, are the very technology that makes SQL Server work on Linux.
The picoprocess was supposed to provide unmodified Linux implementation in user mode. The Windows kernel team has been developing shims designed to connect Linux system calls to Windows. In other words, thanks to WSL, it is now possible to run code compiled for Linux on the Windows NT kernel. There was no need to recompile the code or use virtual machines.
We did not create then only our own Linux distribution. The fact is that there were many such distributions. The first Linux version available in WSL was Ubuntu. We contacted Canonical experts to find out if they would like to help us work on the WSL. They were enthusiastic about our idea. This led to the fact that Ubuntu appeared in the Microsoft Store. By the way, the previous sentence, in itself, sounds quite unusual. Now in the Microsoft Store you can count 6 distributions. I wonder what other application stores available on certain operating systems have other operating systems?
Microsoft Store now has 6 Linux distributions that can be used in WSL
The code we wrote then was Linux-compatible kernel system calls, which served as the interface between the Linux processes and the Windows kernel. Linux has about 340 system calls. The question was to decide which system calls to implement first. As with any operating system, on Linux new system calls are added as new versions of the OS are released, but old calls are never deleted, which ensures backward compatibility. At first, processing of a certain set of system calls was implemented, and everything else was wrapped in events like "not yet implemented." This allowed the WSL team to begin analyzing exactly what system calls Linux users need.
The answer to the question of which system calls should be implemented in the first place meant the need to establish communication with those people who would use WSL. The message about this technology at the Build conference was aimed at making people start using WSL and give feedback. In order to acquire WSL, you had to be a member of the Windows Insider Program. Anyone can connect to this program. You might think that the program participants were only those who were interested in Windows, but then, among more than ten million subscribers, there were people with a wide variety of interests. They were interested not only in Windows, but also, for example, games, and new features of Bluetooth, and WSL.
One of the user groups interested in making the Bash shell run on Windows was the web developer involved in building web applications that run on Linux servers. The whole process of assembling their applications was often a sequence of Bash commands. In addition, if someone decides to ask for help in the development of web applications, say, on Stack Overflow, most of the code samples that he can find will be designed to run on Linux. This is not particularly good for those who use Windows as their computer for web development. Often, it’s easiest for such developers to simply switch to the Mac platform. On macOS, similar code examples run without any problems.
In the first couple weeks of WSL's presence on Windows, one enterprise user was able to run XEyes. This program was run on an X11 windowing system running in WSL. XEyes is a simple program. She displays cartoon eyes that follow the mouse cursor. But this demonstration blew up social networks.
XEyes runs on Windows through WSL
We discussed a lot about exactly how we want to collect user reviews about WSL. The traditional tool used to collect feedback was UserVoice. We had a UserVoice-site designed for WSL, which collected hundreds of ideas and thousands of votes on various issues. The real question came up to us when it came to whether we should use GitHub. Since web developers were one of the first user groups interested in WSL, the question of GitHub was very important. But WSL was not an open source project. Placing such a project on GitHub looks strange. We decided to meet the developers' requirements and created a page on GitHub for reporting problems, for feedback and discussions. Since then, we have received thousands of messages regarding many issues related to using Linux on Windows.
Thousands of people have left error messages in the WSL GitHub repository . Each such message was reviewed and discussed by the WSL team, and comments were given on each of them. After that, a decision was made on further actions. If in order to realize a certain opportunity or fix some kind of error, it was necessary to write new code - this code was created and added to the WSL project, after which it got into all Windows assemblies distributed under the Windows Insider program. The whole cycle, from receiving an error message to correcting it, did not take much time - about a couple of weeks.
As a result, thanks to the operational response of the WSL team to user requests, it was possible to say that the community was involved in the process of creating WSL. People reported problems or their wishes through UserVoice or GitHub, the team reviewed all this and made changes to the project, which then appeared in the builds of the Windows Insider project.
When I joined the WSL team as a project manager, I focused on getting WSL out of beta. User complaints mostly related to compatibility and performance. But, in my opinion, if users worry about such things, it means that they use our development seriously. Performance is taken care of only by those who solve some big problems with the help of a certain software system. As a result, although we still had a lot to do, we did it for a reason, so that people would be able to solve more problems using WSL, and so that they could solve their problems faster.
As the capabilities of WSL began to expand, we made efforts to bring WSL closer to developers, and not just to those users who traditionally work using the Microsoft ecosystem. It was very interesting to attend events like PyCon and OSCON. The developers who were present there were surprised that Microsoft representatives also took part in these events. Developers were distrustful of the idea of ​​running Linux in a Windows environment. At these events, I showed SQL Server, WSL, and Visual Studio Code.
Showcase WSL at various events
I responded to their skeptical comments with a proposal to try what I showed them. When doubters began to execute their own commands, small scripts and snippets, I always met with a violent reaction to what was happening: “Wait a minute, and this is really Linux. How did you do that? Why didn’t I know about this? ” Often they came to the conclusion that we have created something that meets their needs, something that looks very interesting.
We took into account user complaints regarding WSL compatibility and performance, and released a new system architecture - WSL 2 . It provides full compatibility through the inclusion of the Linux kernel in Windows and provides a 20-fold increase in speed. I got an interesting experience creating the foundation of WSL 2 and watching the announcement of this technology at the Build conference, which was held in May 2019. Today, the WSL project has already outgrown the beta version and got to version 2.
In addition, I worked at Microsoft with other teams, trying to ensure that WSL could be used with their products. One significant example of this use of WSL is Visual Studio Code. This is the most popular code environment used in developing JavaScript and Node.js projects. I became interested in Visual Studio Code when I realized that developers using this editor can get significant benefits from WSL. At first, it was not about the need to write huge amounts of code. The main objective was to simplify the debugging of Node.js programs running in the WSL environment. This would give developers the opportunity to develop programs designed for the Linux version of Node.js on a Windows computer using WSL. Debugging such programs would look exactly like debugging them on Linux.
First attempt to integrate Visual Studio Code with WSL and Node.js
After this turned out to be possible for JavaScript and Node.js, we began to receive a lot of requests for something similar to be done for other languages, for example, for C ++ and Python. I was fascinated by this example of the integration of WSL and VS Code, I found that I was all extremely interested in this. This led me to my new role in building tools for Linux developers. Now I focus on tools for C ++ developers in VS Code. In this work, of course, I focus on Linux. It was nice to see a demonstration of Visual Studio Code Remote Development at the PyCon event this year when the corresponding WSL extension was released. At the same time, an extension for C ++ was introduced, which my team was developing.
Despite the fact that I did not spend so much time at Microsoft, I am glad that I was able to participate in the creation of many tools for Linux developers. This database and support for Linux on Windows, and tools for writing and debugging code. I plan to continue to work on Linux and create tools that developers around the world will enjoy using.
Dear readers! Do you use WSL?
Source: https://habr.com/ru/post/461663/
All Articles