Exo-Core: Application-Level Resource Management Operating System Architecture
Hi, Habrayuzer!
Today, I want to please you with the translation of a part of an article by Dawson Engler, Frans Kaashoek and James O'Tulla Jr. from the MIT computer science laboratory (Dawson R. Engler, M. Frans Kaashoek, James O'Toole Jr., MIT Laboratory of Computer Science ) about exonuclear operating systems. The article itself is quite voluminous, so I will translate it in parts, if, of course, you are interested.
Traditional operating systems limit the performance, flexibility, and functionality of applications using fixed interfaces and providing operating system abstractions such as interprocess communication and virtual memory. The architecture of exoderic operating systems solves these problems by shifting hardware resource management issues to the application level. When using such an architecture, a small OS kernel safely transfers all hardware resources to an untrusted operating system library through low-level interfaces. Operating System Library This library implements system objects and policies. Separating the protection of hardware resources from managing them allows you to implement operating system-dependent abstractions of the operating system, expanding, specializing, or even replacing system libraries.
We made a prototype of an exunuclear operating system. Studies show that the simplest operations, such as intercepting exceptions or transferring controls, are 10 to 100 times faster than the absolutely monolithic version of UNIX, compared to Ultrix. In addition, we have demonstrated that the ekzoyadro allows applications to manage machine resources in a way that is simply unacceptable in traditional operating systems. For example, virtual memory and interprocess communication were implemented using application level libraries. Studies show that the primitives of virtual memory and interprocess interaction of the user level work from 5 to 40 times faster than similar kernel level primitives of Ultrix OS. Comparing with the data given in the literature, the prototype of an exunuclear system at least 5 times faster than traditional operating systems solves such problems as interception of exceptions and interprocess communication.
Operating systems define the interface between applications and hardware resources. Unfortunately, tightly defined interfaces can significantly reduce performance, as well as ease of application development. Traditionally, operating systems hide information about machine resources behind such high-level abstractions as processes, files, address space, and interprocess communication. Such abstractions, in fact, define a virtual machine, within which the application runs - they cannot be replaced or modified by untrusted applications. The rigid definition of such abstractions is unsatisfactory for three main reasons: it deprives applications of the advantages of domain-specific optimization (domain-specific optimization), it does not allow making changes to existing implementations of abstractions, and this reduces the flexibility in developing applications, since new abstractions can be implemented only using existing ones (if at all possible).
We believe that these problems can be solved by transferring resource management to the application level, i.e. make it untrusted. To prove this, we developed a new kernel architecture, an exo-core, in which such traditional operating system abstractions as virtual memory (VM), interprocess communication (IPC) are implemented at the application level with untrusted software. When using this architecture, the minimum core safely multiplexes the available hardware resources. The operating system libraries that run above the exo-core infrastructure implement high-level abstractions. Application authors choose the libraries they need, or write their own. Using new libraries is implemented by simply linking executable files.
Our research has shown that applications can greatly benefit from the fact that there will be a control mechanism over how abstractions are implemented that control hardware resources. Apple and Lee [5] argue that existing general-purpose primitives that implement access to virtual memory reduce the performance of garbage collectors and systems with distributed shared memory. Cao (Cao) [10] says that transferring control over file caching to the application level can reduce application operation time by 45%. Hartley and Cheriton [26] and Kruger [30] show that application-dependent virtual memory policies can increase the speed of application execution. Stonebreaker [47] proves that inappropriate decisions when implementing file systems can significantly affect the productivity of databases. Takkot and Levy [50] show that exceptions can be processed an order of magnitude faster if we move signal processing to the application level.
To give applications control over hardware resources, the exo-kernel defines a low-level interface. The architecture of the exo-core is based on one old and simple observation: the lower the level of the primitive, the more efficiently it can be used, and the more free it provides for the implementation of high-level abstractions.
To realize the lowest possible interface (ideally, the hardware interface), the architects of the exo-nucleus set one goal for themselves: to separate protection from management. For example, an ekzoyadro should protect framebuffers, not knowing about the window system, and protect the disk, having no idea about the file system. One solution is to provide each application with its own virtual machine. As we will show later in section 8, virtual machines can reduce performance. In addition, the ekzoyadro does not emulate hardware resources, but provides them, which allows to use an effective and simple solution. The exoducleus provides three ways to securely provide resources. First, it is a secure binding - applications can be safely associated with hardware resources, and handle events. Second, visible resource revocation. And third, using the cancel protocol, the exo-core can break the connection between a malfunctioning application and hardware resources.
We developed a prototype of an exonuclear system based on secure binding, visible resource pinching and a cancel protocol. It contains exocon (Aegis), and untrusted operating system libraries (ExOS). We use this system to demonstrate several important properties of an ex-nuclear architecture:
In practice, our prototype of an exonuclear system provides greater flexibility in developing applications, as well as providing greater performance than systems with a monolithic or microkernel. The low-level Aegis kernel interface allows application-level software to use resources more efficiently. Secure Aegis core control transfer is almost 7 times faster than any of the claimed implementations. Interception exceptions occur five times faster.
Our core also provides ExOS (and all other software running at the application level) flexibility not available in microkernel systems. For example, VMs implemented at the application level can easily be integrated with a system of distributed shared memory and garbage collectors. The control transfer protocol used in Aegis allows applications to implement a large number of IPC primitives, choosing between performance and usability. For example, micronuclear systems such as Amoeba [48], Chorus [43], Mach [2], V [15] do not allow untrusted applications to implement IPC primitives, since virtual memory and messaging services are implemented in the kernel by trusted servers. Also, abstractions such as the structure of the table of pages of memory, abstractions of processes, cannot be modified in the microkernel. After all, a host of hardware resources, such as a network, an information output system, and hard drives, are encapsulated into heavyweight servers that cannot be bypassed or optimized for application needs. Such heavyweight servers, in fact, are kernel subsystems operating at the application level.
This article focuses on the architecture and design of the exo-core, as well as on how it can be safely and effectively implemented. The second part describes the exo-nucleus in more detail. In the third discussion of the problems that we encountered in the design and architecture. In the fourth, we describe the state of the prototype and how we tested it. Parts 5 and 6 represent the implementation. The seventh part contains the report of the experiment, in which we demonstrate the flexibility of the exunuclear architecture. Part 8 describes the work done, and part 9 summarizes the results.
You can find the original here .
Today, I want to please you with the translation of a part of an article by Dawson Engler, Frans Kaashoek and James O'Tulla Jr. from the MIT computer science laboratory (Dawson R. Engler, M. Frans Kaashoek, James O'Toole Jr., MIT Laboratory of Computer Science ) about exonuclear operating systems. The article itself is quite voluminous, so I will translate it in parts, if, of course, you are interested.
annotation
Traditional operating systems limit the performance, flexibility, and functionality of applications using fixed interfaces and providing operating system abstractions such as interprocess communication and virtual memory. The architecture of exoderic operating systems solves these problems by shifting hardware resource management issues to the application level. When using such an architecture, a small OS kernel safely transfers all hardware resources to an untrusted operating system library through low-level interfaces. Operating System Library This library implements system objects and policies. Separating the protection of hardware resources from managing them allows you to implement operating system-dependent abstractions of the operating system, expanding, specializing, or even replacing system libraries.
We made a prototype of an exunuclear operating system. Studies show that the simplest operations, such as intercepting exceptions or transferring controls, are 10 to 100 times faster than the absolutely monolithic version of UNIX, compared to Ultrix. In addition, we have demonstrated that the ekzoyadro allows applications to manage machine resources in a way that is simply unacceptable in traditional operating systems. For example, virtual memory and interprocess communication were implemented using application level libraries. Studies show that the primitives of virtual memory and interprocess interaction of the user level work from 5 to 40 times faster than similar kernel level primitives of Ultrix OS. Comparing with the data given in the literature, the prototype of an exunuclear system at least 5 times faster than traditional operating systems solves such problems as interception of exceptions and interprocess communication.
1. Introduction
Operating systems define the interface between applications and hardware resources. Unfortunately, tightly defined interfaces can significantly reduce performance, as well as ease of application development. Traditionally, operating systems hide information about machine resources behind such high-level abstractions as processes, files, address space, and interprocess communication. Such abstractions, in fact, define a virtual machine, within which the application runs - they cannot be replaced or modified by untrusted applications. The rigid definition of such abstractions is unsatisfactory for three main reasons: it deprives applications of the advantages of domain-specific optimization (domain-specific optimization), it does not allow making changes to existing implementations of abstractions, and this reduces the flexibility in developing applications, since new abstractions can be implemented only using existing ones (if at all possible).
We believe that these problems can be solved by transferring resource management to the application level, i.e. make it untrusted. To prove this, we developed a new kernel architecture, an exo-core, in which such traditional operating system abstractions as virtual memory (VM), interprocess communication (IPC) are implemented at the application level with untrusted software. When using this architecture, the minimum core safely multiplexes the available hardware resources. The operating system libraries that run above the exo-core infrastructure implement high-level abstractions. Application authors choose the libraries they need, or write their own. Using new libraries is implemented by simply linking executable files.
Our research has shown that applications can greatly benefit from the fact that there will be a control mechanism over how abstractions are implemented that control hardware resources. Apple and Lee [5] argue that existing general-purpose primitives that implement access to virtual memory reduce the performance of garbage collectors and systems with distributed shared memory. Cao (Cao) [10] says that transferring control over file caching to the application level can reduce application operation time by 45%. Hartley and Cheriton [26] and Kruger [30] show that application-dependent virtual memory policies can increase the speed of application execution. Stonebreaker [47] proves that inappropriate decisions when implementing file systems can significantly affect the productivity of databases. Takkot and Levy [50] show that exceptions can be processed an order of magnitude faster if we move signal processing to the application level.
To give applications control over hardware resources, the exo-kernel defines a low-level interface. The architecture of the exo-core is based on one old and simple observation: the lower the level of the primitive, the more efficiently it can be used, and the more free it provides for the implementation of high-level abstractions.
To realize the lowest possible interface (ideally, the hardware interface), the architects of the exo-nucleus set one goal for themselves: to separate protection from management. For example, an ekzoyadro should protect framebuffers, not knowing about the window system, and protect the disk, having no idea about the file system. One solution is to provide each application with its own virtual machine. As we will show later in section 8, virtual machines can reduce performance. In addition, the ekzoyadro does not emulate hardware resources, but provides them, which allows to use an effective and simple solution. The exoducleus provides three ways to securely provide resources. First, it is a secure binding - applications can be safely associated with hardware resources, and handle events. Second, visible resource revocation. And third, using the cancel protocol, the exo-core can break the connection between a malfunctioning application and hardware resources.
We developed a prototype of an exonuclear system based on secure binding, visible resource pinching and a cancel protocol. It contains exocon (Aegis), and untrusted operating system libraries (ExOS). We use this system to demonstrate several important properties of an ex-nuclear architecture:
- exo-nuclei can be very effective due to the small number of primitives that need to be implemented;
- low-level secure multiplexing of resources may be provided with very low latency;
- traditional abstractions, such as VM and IPC, can be effectively implemented at the application level, where they can be extended, specialized, or replaced;
- Applications can create more specialized implementations of abstractions, depending on their needs.
In practice, our prototype of an exonuclear system provides greater flexibility in developing applications, as well as providing greater performance than systems with a monolithic or microkernel. The low-level Aegis kernel interface allows application-level software to use resources more efficiently. Secure Aegis core control transfer is almost 7 times faster than any of the claimed implementations. Interception exceptions occur five times faster.
Our core also provides ExOS (and all other software running at the application level) flexibility not available in microkernel systems. For example, VMs implemented at the application level can easily be integrated with a system of distributed shared memory and garbage collectors. The control transfer protocol used in Aegis allows applications to implement a large number of IPC primitives, choosing between performance and usability. For example, micronuclear systems such as Amoeba [48], Chorus [43], Mach [2], V [15] do not allow untrusted applications to implement IPC primitives, since virtual memory and messaging services are implemented in the kernel by trusted servers. Also, abstractions such as the structure of the table of pages of memory, abstractions of processes, cannot be modified in the microkernel. After all, a host of hardware resources, such as a network, an information output system, and hard drives, are encapsulated into heavyweight servers that cannot be bypassed or optimized for application needs. Such heavyweight servers, in fact, are kernel subsystems operating at the application level.
This article focuses on the architecture and design of the exo-core, as well as on how it can be safely and effectively implemented. The second part describes the exo-nucleus in more detail. In the third discussion of the problems that we encountered in the design and architecture. In the fourth, we describe the state of the prototype and how we tested it. Parts 5 and 6 represent the implementation. The seventh part contains the report of the experiment, in which we demonstrate the flexibility of the exunuclear architecture. Part 8 describes the work done, and part 9 summarizes the results.
You can find the original here .
')
Source: https://habr.com/ru/post/127207/
All Articles