Penguin in the window: the potential and prospects for WSL2
Hi, Habr!
While the summer sale is going on with might and main, we would like to invite you to discuss one of the most ambitious topics that we have been working on recently - the interaction between Windows and Linux, related in particular to the development of the WSL system. WSL 2 is already on the way, and you are invited to a brief overview of the features that await us in this subsystem, as well as a forecast of further integration of Windows and Linux.
In May of this year, Microsoft announced that WSL2, the latest version of the Windows subsystem on Linux, will run on a full-fledged Linux kernel built by the company.
Thus, for the first time Microsoft has included the Linux kernel as one of its components in Windows. Microsoft also introduces a command line to Windows that will expand PowerShell and WSL.
')
Both the Linux kernel for WSL2, created by Microsoft, and the new Windows command line are interesting, first of all, for developers.
“This is the strongest move in the game against AWS,” notes Joshua Schwartz, head of digitalization programs at consulting firm AT Kearney.
The future of Microsoft is not related to the PC market, although it will continue to firmly hold positions in this segment. Much more important will be to gain a foothold in the cloud market, one of the components of which in the future can become desktop PCs.
WSL2 is the latest Windows subsystem infrastructure for Linux. It allows you to dramatically improve the performance of the file system and provides full compatibility with system calls.
One of the main requests of the WSL-community was related to the improvement of the functional. WSL2 runs much more tools under Linux than WSL, in particular, Docker and FUSE.
WSL2 handles intensive file operations, in particular, git clone, npm install, apt update and apt upgrade. The actual increase in speed depends on the particular application and how it interacts with the file system.
The first tests showed that WSL2 is about 20 times faster than WSL1 copes with unpacking tar from zip. When using git clone, npm install and cmake in various projects, the system showed a performance increase of two to five times.
In essence, Microsoft is seeking to gain recognition and trust in the developer community, taking on the development of its own version of the Linux kernel to support WSL2 processes, says Cody Swann, CEO at Gunner Technology.
“Apart from developing strictly under Windows, creating all other applications — cloud, mobile, web applications — on a PC was extremely inconvenient, which is why the developer somehow had to download the Linux distribution package in parallel with the Windows OS. Microsoft recognized this and proposed a solution, ”he concludes.
It is unlikely that the introduction of its own Linux kernel will seriously affect the work with the system from the point of view of a regular user. In this case, however, opportunities open up for closer interaction between Microsoft services and the Linux operating system.
Such a move on the part of Microsoft is really very literate, because it helps to penetrate deeper into the developer community, and also actively use products that someone else is developing - that is, connecting to the open source, says Swann.
The trend towards the creation and support of the Linux kernel “specifically for Windows” reflects the decisive direction of development towards open-source, which is promoted by the CEO Satya Nadella. Microsoft is no longer the one with Gates and Balmer, when everything was kept behind a proprietary palisade, and no one thought about interoperability.
“Satya has completely transformed Microsoft into a much more modern platform, and this strategy has paid off handsomely. Hi, capitalization is trillions of dollars, ”says Schwartz.
According to Charles King, chief analyst at Pund-IT, Microsoft’s two main strengths are related to efficiency and security.
“By actively using its own serious developments - resources and tools - the company can guarantee customers that the kernel will be fully up to date and equipped with the latest patches and patches that ensure complete security,” he adds.
Linux binaries perform many functions using system calls, such as accessing files, requesting memory, and creating processes. WSL1 relies on a translation layer that interprets many of these system calls and allows them to interact with the Windows NT kernel.
The most difficult thing is to implement all system calls. Since this was not done in WSL1, some applications could not work there. In WSL2, there are many new applications that work normally in this environment.
New architecture allows Microsoft to bring the latest optimizations to the Linux kernel much faster than with WSL1. Microsoft can update the WSL2 core, and not re-implement all restrictions.
Microsoft’s own Linux kernel development has been the culmination of many years of work done by the Linux Systems Group, as well as many other teams operating throughout Microsoft, says Jack Hammons, program manager for the Linux Systems Group, Microsoft.
The kernel provided for WSL2 will be completely open source, and Microsoft will post on GitHub instructions on how to build such a kernel. The company will interact with developers who want to help the project and stimulate upward changes.
Microsoft developers created WSL2 using continuous integration and continuous delivery systems operating in the company. This software will be serviced through the Windows update system and will be completely transparent to the user. The kernel will remain relevant and include all the features of the newest stable Linux branch.
To ensure the availability of source code, the company mirrors the repositories locally, as well as constantly monitors the contents of the Linux mailing list on security issues, and also collaborates with several companies that support work with databases in the corporate virtual environment (CVE). In this way, it is guaranteed that the latest updates will be taken into account in the Microsoft Linux kernel and all emerging threats will be eliminated.
Microsoft ensures that all kernel changes will propagate in an upstream direction — an important aspect of the Linux philosophy. Support for downstream patches comes with added complexity; in addition, this practice is not generally accepted in the free development community.
The goal of Microsoft actively using Linux is to become a disciplined member of this community and deliver changes to the community order. In order to achieve stability of branches related to long-term support, some patches — for example, those containing new features — can only be included in new kernel versions, and not be ported to the current version of LTS in backward compatibility mode.
When the WSL kernel sources are available, they will consist of references to a set of patches and from the long-playing stable part of the sources. Microsoft expects that this list will shrink over time as the patches propagate upstream, and new local patches will be added to support the latest WSL features.
Microsoft also announced the release of the upcoming "winter" version of Windows Terminal - a new application for users working with command-line tools and shells, in particular, Command Prompt, PowerShell and WSL.
Terminal windows
Windows Terminal 1.0 offers a lot of settings and configuration options that give you more control over the appearance of the terminal window, as well as over the shells / profiles that should open as new tabs.
The settings will be saved in a structured text file, making them easy to configure and arrange the terminal window to your taste.
Microsoft stops updating the existing Windows console, and creates a new one from scratch, deciding to take a fresh approach. Windows Terminal is installed and running in parallel with the existing Windows Console application supplied out of the box.
When a Windows 10 user directly starts Cmd / PowerShell / etc, the process attached to the normal Console instance is triggered. The new terminal configuration mechanism allows Windows users to create multiple profiles for all desired shells / applications / tools, whether in PowerShell, the command line, Ubuntu, or even SSH connections to Azure or Internet of Things devices.
In these profiles, you can provide your own combinations of design and size of the font, color themes, blur levels or background transparency. In addition, users will be able to choose a new fixed-width font so that the terminal window looks more modern and cooler. Programmer ligatures are provided in this font, it will be laid out in open access and will be stored in its own repository.
The main advantages of the new Windows command interface are a lot of tabs and beautiful text. Support for multiple tabs was considered the most requested request for terminal development. Beautiful text is obtained through the rendering engine based on DirectWrite / DirectX, equipped with GPU-accelerated.
The engine displays text icons, glyphs and special characters available in fonts, including Chinese, Japanese and Korean ideograms (CJK), emoji, powerline symbols, icons, and programmer ligatures. In addition, this engine displays text much faster than GDI, previously used in the console.
Backward compatibility remains in good order, although, if desired, you can try Windows Terminal.
Microsoft will provide Windows Terminal through the Microsoft Store in Windows 10 and update it regularly. Thus, users will always work with the latest versions and the latest updates - almost without any extra effort.
Microsoft plans to launch a new terminal next winter. After Microsoft rolls out Windows Terminal 1.0, developers will continue to pursue the many features already put back into backlog.
The source code for Windows Terminal and the Windows Console is already available on GitHub.
The possibility that Microsoft will use its own Linux kernel for other purposes, for example, to develop its Linux distribution, today seems somewhat hypothetical.
Probably, the outcome depends on whether Microsoft will be able to detect a noticeable demand for such a product, and what kind of commercial opportunities could potentially open up such developments, says Charles King.
He thinks that in the foreseeable future, the company will focus its main work on ensuring the ever closer compatibility between Windows and Linux and their mutual complementarity.
Joshua Schwartz believes that in this case it will be necessary to weigh what investments will be made in this work, and what will be the return from it. If today Microsoft was a very young company, it probably would have done everything based on Linux. However, porting all the developments that Microsoft already has to the native Linux architecture is now a costly and difficult project that hardly pays for itself well. Linux lovers will get their own Linux, and the main architecture will remain untouched.
When Apple re-invented Mac OS in 2000, this operating system was based on BSD Unix, which is more similar to Linux than DOS. Today, a new version of Microsoft Windows is created based on Linux.
The Microsoft Linux kernel can open the way for closer interaction between Windows services and the Linux operating system. In essence, these developments of Microsoft indicate that Microsoft itself already understands: today there are almost no customers left who prefer to exist in a world where Windows is everywhere.
It is much more reasonable to use heterogeneous technologies and systems that best meet business requirements and specific practical situations.
The more important strategic question is, what new strategic opportunities does this move open for the Microsoft platform itself?
Azure - Microsoft's cloud-based ecosystem - already provides tremendous support for Linux. Previously, Windows was well supported by Linux using virtual machines.
The principal changes taking place today are due to the fact that now Linux processes will be natively executed on the Windows kernel, which means that working with Linux from Windows will go much faster than on virtual machines. It is likely that as a result, Azure will be enriched by a whole layer of engineers using Linux on an industrial scale.
While the summer sale is going on with might and main, we would like to invite you to discuss one of the most ambitious topics that we have been working on recently - the interaction between Windows and Linux, related in particular to the development of the WSL system. WSL 2 is already on the way, and you are invited to a brief overview of the features that await us in this subsystem, as well as a forecast of further integration of Windows and Linux.
In May of this year, Microsoft announced that WSL2, the latest version of the Windows subsystem on Linux, will run on a full-fledged Linux kernel built by the company.
Thus, for the first time Microsoft has included the Linux kernel as one of its components in Windows. Microsoft also introduces a command line to Windows that will expand PowerShell and WSL.
')
Both the Linux kernel for WSL2, created by Microsoft, and the new Windows command line are interesting, first of all, for developers.
“This is the strongest move in the game against AWS,” notes Joshua Schwartz, head of digitalization programs at consulting firm AT Kearney.
The future of Microsoft is not related to the PC market, although it will continue to firmly hold positions in this segment. Much more important will be to gain a foothold in the cloud market, one of the components of which in the future can become desktop PCs.
What WSL2 does
WSL2 is the latest Windows subsystem infrastructure for Linux. It allows you to dramatically improve the performance of the file system and provides full compatibility with system calls.
One of the main requests of the WSL-community was related to the improvement of the functional. WSL2 runs much more tools under Linux than WSL, in particular, Docker and FUSE.
WSL2 handles intensive file operations, in particular, git clone, npm install, apt update and apt upgrade. The actual increase in speed depends on the particular application and how it interacts with the file system.
The first tests showed that WSL2 is about 20 times faster than WSL1 copes with unpacking tar from zip. When using git clone, npm install and cmake in various projects, the system showed a performance increase of two to five times.
Will it win the trust of the developers?
In essence, Microsoft is seeking to gain recognition and trust in the developer community, taking on the development of its own version of the Linux kernel to support WSL2 processes, says Cody Swann, CEO at Gunner Technology.
“Apart from developing strictly under Windows, creating all other applications — cloud, mobile, web applications — on a PC was extremely inconvenient, which is why the developer somehow had to download the Linux distribution package in parallel with the Windows OS. Microsoft recognized this and proposed a solution, ”he concludes.
It is unlikely that the introduction of its own Linux kernel will seriously affect the work with the system from the point of view of a regular user. In this case, however, opportunities open up for closer interaction between Microsoft services and the Linux operating system.
Such a move on the part of Microsoft is really very literate, because it helps to penetrate deeper into the developer community, and also actively use products that someone else is developing - that is, connecting to the open source, says Swann.
Welcome to New Microsoft
The trend towards the creation and support of the Linux kernel “specifically for Windows” reflects the decisive direction of development towards open-source, which is promoted by the CEO Satya Nadella. Microsoft is no longer the one with Gates and Balmer, when everything was kept behind a proprietary palisade, and no one thought about interoperability.
“Satya has completely transformed Microsoft into a much more modern platform, and this strategy has paid off handsomely. Hi, capitalization is trillions of dollars, ”says Schwartz.
According to Charles King, chief analyst at Pund-IT, Microsoft’s two main strengths are related to efficiency and security.
“By actively using its own serious developments - resources and tools - the company can guarantee customers that the kernel will be fully up to date and equipped with the latest patches and patches that ensure complete security,” he adds.
Developers also win
Linux binaries perform many functions using system calls, such as accessing files, requesting memory, and creating processes. WSL1 relies on a translation layer that interprets many of these system calls and allows them to interact with the Windows NT kernel.
The most difficult thing is to implement all system calls. Since this was not done in WSL1, some applications could not work there. In WSL2, there are many new applications that work normally in this environment.
New architecture allows Microsoft to bring the latest optimizations to the Linux kernel much faster than with WSL1. Microsoft can update the WSL2 core, and not re-implement all restrictions.
Fully open source tool
Microsoft’s own Linux kernel development has been the culmination of many years of work done by the Linux Systems Group, as well as many other teams operating throughout Microsoft, says Jack Hammons, program manager for the Linux Systems Group, Microsoft.
The kernel provided for WSL2 will be completely open source, and Microsoft will post on GitHub instructions on how to build such a kernel. The company will interact with developers who want to help the project and stimulate upward changes.
Microsoft developers created WSL2 using continuous integration and continuous delivery systems operating in the company. This software will be serviced through the Windows update system and will be completely transparent to the user. The kernel will remain relevant and include all the features of the newest stable Linux branch.
To ensure the availability of source code, the company mirrors the repositories locally, as well as constantly monitors the contents of the Linux mailing list on security issues, and also collaborates with several companies that support work with databases in the corporate virtual environment (CVE). In this way, it is guaranteed that the latest updates will be taken into account in the Microsoft Linux kernel and all emerging threats will be eliminated.
Ascending changes become mandatory.
Microsoft ensures that all kernel changes will propagate in an upstream direction — an important aspect of the Linux philosophy. Support for downstream patches comes with added complexity; in addition, this practice is not generally accepted in the free development community.
The goal of Microsoft actively using Linux is to become a disciplined member of this community and deliver changes to the community order. In order to achieve stability of branches related to long-term support, some patches — for example, those containing new features — can only be included in new kernel versions, and not be ported to the current version of LTS in backward compatibility mode.
When the WSL kernel sources are available, they will consist of references to a set of patches and from the long-playing stable part of the sources. Microsoft expects that this list will shrink over time as the patches propagate upstream, and new local patches will be added to support the latest WSL features.
More pleasant window decoration
Microsoft also announced the release of the upcoming "winter" version of Windows Terminal - a new application for users working with command-line tools and shells, in particular, Command Prompt, PowerShell and WSL.
Terminal windows
Windows Terminal 1.0 offers a lot of settings and configuration options that give you more control over the appearance of the terminal window, as well as over the shells / profiles that should open as new tabs.
The settings will be saved in a structured text file, making them easy to configure and arrange the terminal window to your taste.
Microsoft stops updating the existing Windows console, and creates a new one from scratch, deciding to take a fresh approach. Windows Terminal is installed and running in parallel with the existing Windows Console application supplied out of the box.
How it works
When a Windows 10 user directly starts Cmd / PowerShell / etc, the process attached to the normal Console instance is triggered. The new terminal configuration mechanism allows Windows users to create multiple profiles for all desired shells / applications / tools, whether in PowerShell, the command line, Ubuntu, or even SSH connections to Azure or Internet of Things devices.
In these profiles, you can provide your own combinations of design and size of the font, color themes, blur levels or background transparency. In addition, users will be able to choose a new fixed-width font so that the terminal window looks more modern and cooler. Programmer ligatures are provided in this font, it will be laid out in open access and will be stored in its own repository.
The main advantages of the new Windows command interface are a lot of tabs and beautiful text. Support for multiple tabs was considered the most requested request for terminal development. Beautiful text is obtained through the rendering engine based on DirectWrite / DirectX, equipped with GPU-accelerated.
The engine displays text icons, glyphs and special characters available in fonts, including Chinese, Japanese and Korean ideograms (CJK), emoji, powerline symbols, icons, and programmer ligatures. In addition, this engine displays text much faster than GDI, previously used in the console.
Backward compatibility remains in good order, although, if desired, you can try Windows Terminal.
Chronology: how it will be
Microsoft will provide Windows Terminal through the Microsoft Store in Windows 10 and update it regularly. Thus, users will always work with the latest versions and the latest updates - almost without any extra effort.
Microsoft plans to launch a new terminal next winter. After Microsoft rolls out Windows Terminal 1.0, developers will continue to pursue the many features already put back into backlog.
The source code for Windows Terminal and the Windows Console is already available on GitHub.
What can we expect in the future?
The possibility that Microsoft will use its own Linux kernel for other purposes, for example, to develop its Linux distribution, today seems somewhat hypothetical.
Probably, the outcome depends on whether Microsoft will be able to detect a noticeable demand for such a product, and what kind of commercial opportunities could potentially open up such developments, says Charles King.
He thinks that in the foreseeable future, the company will focus its main work on ensuring the ever closer compatibility between Windows and Linux and their mutual complementarity.
Joshua Schwartz believes that in this case it will be necessary to weigh what investments will be made in this work, and what will be the return from it. If today Microsoft was a very young company, it probably would have done everything based on Linux. However, porting all the developments that Microsoft already has to the native Linux architecture is now a costly and difficult project that hardly pays for itself well. Linux lovers will get their own Linux, and the main architecture will remain untouched.
When Apple re-invented Mac OS in 2000, this operating system was based on BSD Unix, which is more similar to Linux than DOS. Today, a new version of Microsoft Windows is created based on Linux.
Perhaps a new door is opening before us?
The Microsoft Linux kernel can open the way for closer interaction between Windows services and the Linux operating system. In essence, these developments of Microsoft indicate that Microsoft itself already understands: today there are almost no customers left who prefer to exist in a world where Windows is everywhere.
It is much more reasonable to use heterogeneous technologies and systems that best meet business requirements and specific practical situations.
The more important strategic question is, what new strategic opportunities does this move open for the Microsoft platform itself?
Azure - Microsoft's cloud-based ecosystem - already provides tremendous support for Linux. Previously, Windows was well supported by Linux using virtual machines.
The principal changes taking place today are due to the fact that now Linux processes will be natively executed on the Windows kernel, which means that working with Linux from Windows will go much faster than on virtual machines. It is likely that as a result, Azure will be enriched by a whole layer of engineers using Linux on an industrial scale.
Source: https://habr.com/ru/post/459844/
All Articles