I would like to write something about Microsoft Singularity. This is a very cool thing, and in IT today everyone is talking about it. Here is a review of Singularity for those who do not want to read official publications.

What it is

Singularity is an operating system research project. This is a team of smart people who were told: "How would you make an operating system that was originally focused on reliability?".

People on popular forums like Slashdot or OSNews have been dreaming for years about Microsoft abandoning Windows and starting from scratch - to overcome the problems of unreliability or malicious code (malware). Usually, the dreams of all these people revolve around the idea of ​​porting Windows to some UNIX - but this is a shitty idea, it would not solve the main problems. If you want to solve the problems caused by the fundamental features of the design, you should start with the design. What is being done in Singularity.

Reliability is a rather broad question. At a minimum, this means that applications do not crash and that they are safe. But despite the fact that the development of Singularity affects many areas, developers do not have the task to make a full-fledged OS - for example, they are not engaged in GUI development.

What is the difference?

Actually, this is what I will talk about here.

Singularity is a micronuclear design, as well as high performance, a single address space, static type verification and flexible access rights management.

Oh yes, in the previous sentence there was a whole bunch of frightening academic terms. Let's try to sort them out. This will not be easy, because Singularity is rather far from standard OS designs from textbooks.

Singularity is highly productive . Actually, performance was not the goal of the project, but the developers turned out to be smart enough to realize that without this, their development would “fly off into the clouds” and become completely non-commercialized. This indirectly indicates that Microsoft plans to someday use this development in real products, so it needs to be quick (more on that a bit later).

And Singularity is a microkernel. Let's focus now on what it is. We will talk about the rest of things later. Scroll down if you are already aware of all this - in this case, we assume that you absolutely know what a microkernel is and why it is considered slow.

Uh-uh ... micro what?

Historically, there are two ways to design an operating system, two ways to do the same thing, using different cost-benefit ratios: a microkernel or a monolithic core. Notice, we are now talking about a low-level design ... this does not affect whether you use the taskbar or the dock in your UI.

In micronuclear design, subsystems like the file system or network drivers work as more or less normal programs outside the kernel itself (which differs from other programs in that it works in a special processor mode). The kernel is only responsible for creating processes / threads, sending messages between them and for some other small things, such as allocating CPU resources. True micronuclei today almost anywhere you will not meet. Most likely, you used it without even knowing it. For example, QNX is an operating system designed for embedded applications, such as Cisco routers. QNX is a pure microkernel.

Mistress note

Here is a quick overview of virtual memory. When your code reads something from memory, the CPU internally converts your address from a virtual address to a physical address, which it can feed to the memory controllers. On a 32-bit CPU, they are both 32-bit pointers, and you will most likely never see the immediate physical address unless you are a nuclear developer. This transformation is done by a processor component called MMU (memory management unit) - it also implements access control. The memory is divided into “pages” of 4 kilobytes (in Intel / AMD chips) and a mapping is created for each page. The display page has access bits — read / write / execute — just like files on UNIX.

This memory mapping is the basis of security in all operating systems. It does not allow a buggy program to spoil into the memory of other programs, and - since only the kernel can update the memory page tables, plus all access to the hardware occurs through the kernel - such a program operating in the user mode of the processor cannot do anything "interesting" until the kernel does not allow to do it. And since the MMU does not allow you to read the memory of the kernel, you cannot access it. It also means that we can use swap files to use the disk as a RAM chip — just unload the part of the process address space, catch the read error from it — and load the display again.

Virtual memory is a great thing, one of the biggest improvements in computer reliability over the past 13 years. Windows 3.1 did not use it, unlike Windows 95 - and this is why so many people have updated their OS. The advantages of the microkernel are obvious here - the buggy components of the kernel cannot send the computer to BSOD aka "blue screen of death" - as they do today. If your file system crashes - just restart it!

In monolithic design, file systems, drivers, and even web servers load directly into the kernel and run in privileged processor mode. The kernel still provides a message-passing system for user-mode processes — but this is not specifically used anywhere. Today, any popular server or desktop OS - Windows, Linux and MacOS - is monolithic. Notice that Linux has always been monolithic, Windows NT was initially micronuclear, and MacOS — based on Mach — is theoretically micronuclear. However, I do not know anyone who would believe in it.

It is probably difficult to say whether any operating system is microkernel or monolithic, because it is not binary “yes / no” logic — for example, in Linux, the graphics subsystem works in a separate process (X Server), whereas in Windows when it was so - but not today. One way or another, no one argues that Linux is not a microkernel. A good evaluation criterion is whether the file system works in kernel mode or not — graphical systems may be in a gray area, but not file systems.

In any case, the fact that Singularity employs a microkernel design looks rather strange, because historically it has won over the microkernel in an academic environment, but the monolithic core has always been defeated in the market, mainly due to performance problems. These disputes flared up in the 80s - here you can read the famous dispute between Torvalds and Tanenbaum . So, at first glance it may seem that Singularity is another academic development of theoretically pure, but practically non-usable things ... but it is not.

Why is the microkernel slower?

Microkernels are typically slower than monolithic kernels - due to the overhead of switching between processor modes (user-mode to kernel-mode and back). In addition, there are also costs for switching the processor between two user-mode processes (context switching).

These costs are small, but real, and when you make a stop hundred of such switchings per second, this leads to the fact that all the work of the processor is reduced to switching back and forth, and there is no time left for real work. And it’s quite difficult to measure these costs, although the Singularity team did it.

The reason why these switches take up valuable time is that the CPU has to do an unusual job for it, and since the processor spends most of the time without doing this work, it is not very well optimized. This has changed in the latest generations of x86 chips, but overall, everything remains the same.

For example, by doing syscall to force the kernel to do something, you use a special processor instruction. This is usually “int 80” on Linux, but today you can use the “sysenter” opcode on the kernels and processors that support it (almost everyone can). In this case, control passes to the kernel. This is pretty fast on modern computers, but this was not always the case - for example, earlier versions of Windows used incorrect processor instructions, as the developers found that a CPU exception was a quicker way to get into kernel mode than using an interrupt (“official” way). Intel fixed it :)

Context switching is more expensive; firstly, because it obviously causes a switch to kernel mode, and also because reconfiguring the memory mapping tables is not a quick thing.

Not quick because it is, again, an unusual operation (for this you need to use special registers in x86 chips), but mainly because it requires resetting the so-called “translation lookaside buffers” (TLB). These buffers store the result of MMU requests. After all, even if the MMU is hardware specialized for its task, its use is not free - and the translation of memory addresses must be done every time the code is accessed to memory (which happens all the time) - and here caching is necessary.

Because of this, it is difficult to measure how expensive context switching is. We know that this is worth something there - due to the fundamental design features of the CPU. But the real cost is spread over the code of the running process. Immediately after the context switch, your computer runs a bit slower and picks up progress as you fill out the TLB.

So we have two conflicting priorities. On the one hand, using virtual memory to separate address spaces can improve the reliability of separating programs from each other (which is good), but, on the other hand, it costs us hard to measure performance loss - which is bad. Even worse, even though processors become faster from time to time - they become faster in the execution of the code, but not in manipulating the address space, so we cannot rely on the help of Moore's law this time.

Message exchange

Micro-cores are based on the idea of ​​forwarding messages between processes in different address spaces. So, to read the file, you need to send a message from your program to the file system server. You form a message in your memory (this is fast), you do a syscall “send message” (not so fast anymore), then the kernel copies the message to its own address space (rather slowly), switches the context to the file system server (slowly), and then copies the message to the memory of the file system server before leaving the kernel mode.

When the file system reads the file you are looking for, you need to repeat all this tin in the reverse order, this time copying the data into the return message ... and since the cost of sending the message grows with its size - this is even slower than the initial request!

It looks completely different in a monolithic design: you form your request (it is fast), you do syscall (not so fast), wait until the file system receives your data, then the kernel copies the data directly to your address space (or if you use DMA), returning control to user mode ... wow, it's easier and faster! The disadvantage here is that if the file system is booted up, the entire OS will crash into the BSOD and you will lose everything.

About 80% of crashes in Windows are caused by crooked drivers. So, if we could prevent system crashes due to drivers in the same way as we prevented crashes because of buggy programs, we would prevent 80% of all BSODs in the world! This is pretty cool! It also means that security mechanisms can work with drivers. If you install a third-party file system today, who knows what you will actually get? Until you view and compile the code yourself - you have to trust the one who gave it to you. Even if this is all right, a bug in the new driver can open a hole for the rootkit, bending the entire security system: (

Probably, it is not surprising that academics preferred a slow, but reliable solution - while desktop OS developers preferred a quick but unstable one. But do not think that they did not try to do otherwise! Windows NT was developed as a purely microkernel solution, but even with a super-optimized interprocess communication, they once gave up and “moved” to the kernel along with the GUI - for which they received a lot of criticism, but this made Windows responsive and the users happy.

Singularity

Singularity manages to "sit down and eat" - to get all the advantages of a microkernel and even better performance than a monolithic core. Cool!

How it works?

Singularity is impressive in that it has a clean micronuclear design, ~ 30% faster than the traditional approach and 10% faster than the monolithic (in the benchmark heavy file I / O). How does she do it?

The trick is very simple - they just gave up hardware memory protection, completely. In Singularity, everything works in kernel mode and everything works in a single address space. In other words, the MMU does nothing. And there are no “processes” in the traditional sense.

Of course, if this were the only thing they did, it would not be so interesting. Eighty percent of the crash of Windows is caused by buggy drivers, and not by errors in the kernel (I suspect that the remaining 20% ​​are caused by glitches in the hardware). Numerous vulnerabilities that allow escalation of privileges are caused by glitches in the drivers. Each such vulnerability - a gift to "bad guys." Protection against software errors is what makes microkernels useful.

Programs in Singularity are isolated from each other — but isolation is done entirely and programmatically, using type theory, not silicon. This is possible because Singularity programs are written in a C # descendant named Sing # (theoretically, you can use any language for .NET — Singularity uses quite a few features added to C #, so any other language will require only these few changes) .

You probably already know that in most modern languages ​​- like Java or C # - you do not have direct memory access. Roughly speaking, you cannot write a similar C-code in Java:

*((char *)0x1234) = ‘X’; // 1234

, . Java/C# -, , , . , , JVM/MSIL « », , , , / . — !

, . . , C , — Java/.NET ( C++) .

, , reflection () , .

JVM- . , JRE — .

Singularity — C#, C++ . . Microsoft — , .

Singularity — SIP, software isolated process « ». Singularity , -, , — .

SIP (heap), ( GC, , GC ) . SIP , ( KaffeOS), . , SIP. , SIP — , . , , GC , .

CPU — , , «». Singularity «» , SIP' , , SIP' . , / , C++ . SIP ( ), , , .

SIP' , . , GC SIP' — . syscall .

SIP' «». — , UNIX' , . , , — - . . SIP' ( , -) .

, . ? SIP' ?

, SIP (, CD), MSIL-. (just-in-time, JIT) , Java .NET. , . Singularity — . .

MSIL CPU , « I/O » « », — - , . , , , . , — CPU .

OS - . : , iTunes , , iPod. , -, , — , -, .

SIP' , ( ) SIP'. — ? , ?

Singularity . , .NET, XML-, SIP' , . . , , .

Singularity

, :

1. 80% Windows .
    
2. Linux , 7 , .
    
3. , « » , — — .
    
4. , . « ».

Singularity - , , — Sing#- (Sing# «» C#), MSIL-. SIP, . , C# — , «overflow» .

MSIL , , - . , - DLL, , , . DLL (Singularity.DriverRuntime) , Microsoft «trusted computing base», - DRM, . DLL, , . IoPortRange, IoIrqRange ( ..) - .

Singularity , . . , — , ? ?

Singularity , , . .NET — , . XML- ( , ) (compile-time transform), , . — HAL (hardware abstraction layer).

, Singularity , . , ( ) — . , , .

?

, - ( ?). , , DMA- , .

DMA — Direct Memory Access ( ) — , RAM, CPU. , , , . , DMA CPU, MMU, , DMA- . , . Singularity .

, DRM! , ! CPU, Intel AMD, -, ( ) IOMMU. , MMU — , / DMA . IOMMU, DMA. ( DRM-). , , , .

, , DMA- - «».

?

2001- Microsoft . , . — , , , . Microsoft unit-, , . Microsoft. ?

, , . , . , , . $250 , «OK» . .

. , — .

, , : «, , “ ”, DMA, ( , , )». , ? . , , . BSOD, . , , — , .

, ( Singularity , SIP'), — . ? , . ( DoS' - ? ). , , - , , . , Microsoft, .