Specification of system calls of the Chameleon operating system (be careful, lots of pictures!)
Health to all readers!
We offer you a description of the system calls the microkernel operating system Chameleon aka Xameleon. My chameleon has not yet hatched from his egg and is gaining strength in a virtual machine. But he is very lonely and the little lizard wants to get to know the inhabitants of Habr.
The “micro-core vs monolith” dispute has been going on for many years, but do both arguing parties represent the architecture of a micro-core system? Perhaps this topic will shed some light on the architecture of micro-core systems.
The document has a long history - once I realized that I had given too much time to Chameleon, from which I got problems in my main job. Therefore, it was decided to kill two birds with one stone and to write an article advertising the report generator and, finally, to create documentation. At first she was in broken English, but at some point it was decided not to disgrace and rewrite to a slightly better Russian.
Actually, the document revision history:
Let's get down to business. The Chameleon operating system can be displayed as follows:
It is based on the L4 Pistachio microkernel. The microkernel consists of two modules: the L4Ka Pistachio microkernel itself and the L4 Sigma0 basic memory manager. The best way to learn more about Pistachio is from this document: L4 Version X.2 Reference Manual (Latest snapshot, July 19 2010) .
')
The Chameleon operating system is what is above the microkernel. The system is organized in the form of several processes working on top of the microkernel and interacting with each other and with the microkernel through IPC (Inter Process Communication) —specific system calls that implement synchronous message transmission. The basic rule of the system is that any interaction between an application and a system is (with a few exceptions) based on IPC. The requesting task generates a message in a special way, then calls the IPC, which sends the message to the receiving task. Some IPCs are blocked until a response is received, while others are blocked until they are received by the receiving party. Synchronicity imposes a requirement for successful transmission of a message — the receiving party must listen to them. Hence the conclusion that some tasks must constantly listen to messages. Such tasks in Chameleon are Supervisor, File System, Network Stack, device drivers. The good news for us is the fact that a task in the state of waiting for a message does not consume the computational resources of the processor and gives its time to other tasks. The most interesting thing for us is that the L4 concept uses one primitive for sending and receiving messages - this is an IPC message. The message at the kernel level is organized by one system call, which may have several combinations of the transmission and reception phases.
The specification describes the following types of data, of which the first five types are represented by the microkernel:
And now let's take a close look at Chameleon’s basic service - Supervisor. From the point of view of the microkernel, the Supervisor is nothing but the first user task, so any other task, be it an application program, service or driver, can get the Supervisor's ThreadID via KIP (Kernel Interface Page - the basic concept of the L4 microkernel) ThreadID of other subsystems (using the GetDeviceHandle system call).
Twenty-two system calls for resource management. This is quite enough to allocate and free virtual memory, start a process prepared in memory in advance, start a protocol driver / implementation, send a signal to a process, create a program flow (execution thread), wait for the process to finish, set a process timer and perform several other functions. .
Consider some of the system calls in more detail. Since the Supervisor is able to launch a new process for execution, the memory of this process must be prepared in advance. Accordingly, we need a memory block where we put the new process before its execution. You can request memory for a process using the following system call:
As I said above, the first version of the documentation was in broken English. Unfortunately, I did not have time to Russify the entire specification, so the description of the message body remained on the "ruglish".
The symmetric system call for working with allocated memory is called ReferencingSegment. In order to release a previously allocated memory segment, you need to reset the reference counter for this segment using the ReferencingSegment call. At the same time, the segment itself and the virtual memory pages occupied by it are available for subsequent allocation using AllocateSegment:
A process can also terminate itself using the ExitProcess call:
Unlike previous messages, this message has no reception phase. This is due to the fact that the process after the transfer of this message is completed, so the Supervisor does not send an acknowledgment. The message conveys one parameter - the return code. This code will be passed to a task waiting for the process to terminate using the ProcessWait system call:
Since the IPC microkernel uses synchronous calls, in many cases, the limitations imposed by synchronism can be circumvented using multithreading. Any process can create a program flow (execution thread) in its address space. In this case, all threads share common resources. The following system call creates a new program flow:
If threads are created, then there must be a way to stop them and free resources. To do this, use the ExitThread system call:
A bit of humor: "What is common to the process of the Chameleon system and amoebas - they multiply by division." But seriously, in this case the bike was not invented, and the POSIX fork () was taken as the basis:
Implementing the fork () function based on the L4 primitives is a non-trivial, but doable, task. The chip of this call is that it calls one process, and the answer is received by two processes - the caller and the newly created copy of the calling process. Of course, both processes, the initiator and its copy, will be executed in different address spaces.
The process can clone itself and we come to an interesting challenge - creating a new task from a prepared image. Pay attention to MR2 (Message register 2) - it passes the handle to the memory block of the process being started. This memory block should be created using the AllocateSegment system call described above and a prepared image of the process being started should be stored in it. Also, the system call sets the basic process parameters - the entry point, the beginning and the size of the code, data and BSS segments.
Note the last two parameters passed in the ExecProcess message. You can see the data types not yet discussed above - Compound. In the message body, not the elements themselves, their descriptors are transmitted. In the case of the ExecProcess message, these descriptors point to two memory blocks - the first memory block transmits char ** argv, the second memory block transmits char ** envp - analogs of the arguments of the function int main (int argc, char ** argv, char ** envp) .
The supervisor is a basic service and it provides an interface to access other registered services. For example, a certain task wants to work directly with the serial port driver, without using the file system. You can learn the ThreadId of the registered serial port driver using the GetDeviceHandle system call:
Using the thread ID of the ThreadID process obtained by GetDeviceHandle, we can send messages to it and receive responses from it. The message format depends on the type of device and can be described by one of the following protocols:
Download the full (pre-alpha) version of the specification in PDF format at the link: Xameleon protocol family specification . The document is still very raw, but I hope that your comments will give strength to correct it further.
You can discuss the system call, clarify, point out an error or ask a question not only on Habré, but also on this page: fotki.yandex.ru/users/almandrykin/album/164974 - select a picture and ask your question in the comments.
It so happened that the description of even several system calls of the Supervisor is too big. If interest is noticed, then the description of the remaining system calls of the Supervisor is possible, as well as the description of the system calls of the Falova system, the Network subsystem, and also the system calls of the drivers of block, character, and network devices.
Finally, a few words about what development tools can be used to write programs for the Chameleon system. For development, you can use any programming language that can create executable files in Elf or PE format - gcc, g ++, Visual C ++, various Pascal compilers. In the general case, the compilation process is similar to compilation on any other system, but the main rule is not to use the standard libraries supplied with the compiler. Chameleon's current implementation uses the original Elf-based libc and the original crt0, which provide a subset of POSIX functions. Almost nothing prevents to compile this libc with a compiler that generates object code in PE format and to assemble MS Visual C ++ programs with a compiler. It is also possible to implement system calls in the Pascal language and then no one else will say that this language is not created for system programming.
ps When creating the documentation, the FastReport report generator was used.
pps My English is really bad. Please treat us indulgently. I promise to translate the entire document into Russian.
We offer you a description of the system calls the microkernel operating system Chameleon aka Xameleon. My chameleon has not yet hatched from his egg and is gaining strength in a virtual machine. But he is very lonely and the little lizard wants to get to know the inhabitants of Habr.
The “micro-core vs monolith” dispute has been going on for many years, but do both arguing parties represent the architecture of a micro-core system? Perhaps this topic will shed some light on the architecture of micro-core systems.
The document has a long history - once I realized that I had given too much time to Chameleon, from which I got problems in my main job. Therefore, it was decided to kill two birds with one stone and to write an article advertising the report generator and, finally, to create documentation. At first she was in broken English, but at some point it was decided not to disgrace and rewrite to a slightly better Russian.
Actually, the document revision history:
Let's get down to business. The Chameleon operating system can be displayed as follows:
It is based on the L4 Pistachio microkernel. The microkernel consists of two modules: the L4Ka Pistachio microkernel itself and the L4 Sigma0 basic memory manager. The best way to learn more about Pistachio is from this document: L4 Version X.2 Reference Manual (Latest snapshot, July 19 2010) .
')
The Chameleon operating system is what is above the microkernel. The system is organized in the form of several processes working on top of the microkernel and interacting with each other and with the microkernel through IPC (Inter Process Communication) —specific system calls that implement synchronous message transmission. The basic rule of the system is that any interaction between an application and a system is (with a few exceptions) based on IPC. The requesting task generates a message in a special way, then calls the IPC, which sends the message to the receiving task. Some IPCs are blocked until a response is received, while others are blocked until they are received by the receiving party. Synchronicity imposes a requirement for successful transmission of a message — the receiving party must listen to them. Hence the conclusion that some tasks must constantly listen to messages. Such tasks in Chameleon are Supervisor, File System, Network Stack, device drivers. The good news for us is the fact that a task in the state of waiting for a message does not consume the computational resources of the processor and gives its time to other tasks. The most interesting thing for us is that the L4 concept uses one primitive for sending and receiving messages - this is an IPC message. The message at the kernel level is organized by one system call, which may have several combinations of the transmission and reception phases.
The specification describes the following types of data, of which the first five types are represented by the microkernel:
And now let's take a close look at Chameleon’s basic service - Supervisor. From the point of view of the microkernel, the Supervisor is nothing but the first user task, so any other task, be it an application program, service or driver, can get the Supervisor's ThreadID via KIP (Kernel Interface Page - the basic concept of the L4 microkernel) ThreadID of other subsystems (using the GetDeviceHandle system call).
Twenty-two system calls for resource management. This is quite enough to allocate and free virtual memory, start a process prepared in memory in advance, start a protocol driver / implementation, send a signal to a process, create a program flow (execution thread), wait for the process to finish, set a process timer and perform several other functions. .
Consider some of the system calls in more detail. Since the Supervisor is able to launch a new process for execution, the memory of this process must be prepared in advance. Accordingly, we need a memory block where we put the new process before its execution. You can request memory for a process using the following system call:
As I said above, the first version of the documentation was in broken English. Unfortunately, I did not have time to Russify the entire specification, so the description of the message body remained on the "ruglish".
The symmetric system call for working with allocated memory is called ReferencingSegment. In order to release a previously allocated memory segment, you need to reset the reference counter for this segment using the ReferencingSegment call. At the same time, the segment itself and the virtual memory pages occupied by it are available for subsequent allocation using AllocateSegment:
A process can also terminate itself using the ExitProcess call:
Unlike previous messages, this message has no reception phase. This is due to the fact that the process after the transfer of this message is completed, so the Supervisor does not send an acknowledgment. The message conveys one parameter - the return code. This code will be passed to a task waiting for the process to terminate using the ProcessWait system call:
Since the IPC microkernel uses synchronous calls, in many cases, the limitations imposed by synchronism can be circumvented using multithreading. Any process can create a program flow (execution thread) in its address space. In this case, all threads share common resources. The following system call creates a new program flow:
If threads are created, then there must be a way to stop them and free resources. To do this, use the ExitThread system call:
A bit of humor: "What is common to the process of the Chameleon system and amoebas - they multiply by division." But seriously, in this case the bike was not invented, and the POSIX fork () was taken as the basis:
Implementing the fork () function based on the L4 primitives is a non-trivial, but doable, task. The chip of this call is that it calls one process, and the answer is received by two processes - the caller and the newly created copy of the calling process. Of course, both processes, the initiator and its copy, will be executed in different address spaces.
The process can clone itself and we come to an interesting challenge - creating a new task from a prepared image. Pay attention to MR2 (Message register 2) - it passes the handle to the memory block of the process being started. This memory block should be created using the AllocateSegment system call described above and a prepared image of the process being started should be stored in it. Also, the system call sets the basic process parameters - the entry point, the beginning and the size of the code, data and BSS segments.
Note the last two parameters passed in the ExecProcess message. You can see the data types not yet discussed above - Compound. In the message body, not the elements themselves, their descriptors are transmitted. In the case of the ExecProcess message, these descriptors point to two memory blocks - the first memory block transmits char ** argv, the second memory block transmits char ** envp - analogs of the arguments of the function int main (int argc, char ** argv, char ** envp) .
The supervisor is a basic service and it provides an interface to access other registered services. For example, a certain task wants to work directly with the serial port driver, without using the file system. You can learn the ThreadId of the registered serial port driver using the GetDeviceHandle system call:
Using the thread ID of the ThreadID process obtained by GetDeviceHandle, we can send messages to it and receive responses from it. The message format depends on the type of device and can be described by one of the following protocols:
- exchange format with the service;
- exchange format with a block device;
- exchange format with a character device;
- exchange format with a network device.
Download the full (pre-alpha) version of the specification in PDF format at the link: Xameleon protocol family specification . The document is still very raw, but I hope that your comments will give strength to correct it further.
You can discuss the system call, clarify, point out an error or ask a question not only on Habré, but also on this page: fotki.yandex.ru/users/almandrykin/album/164974 - select a picture and ask your question in the comments.
It so happened that the description of even several system calls of the Supervisor is too big. If interest is noticed, then the description of the remaining system calls of the Supervisor is possible, as well as the description of the system calls of the Falova system, the Network subsystem, and also the system calls of the drivers of block, character, and network devices.
Finally, a few words about what development tools can be used to write programs for the Chameleon system. For development, you can use any programming language that can create executable files in Elf or PE format - gcc, g ++, Visual C ++, various Pascal compilers. In the general case, the compilation process is similar to compilation on any other system, but the main rule is not to use the standard libraries supplied with the compiler. Chameleon's current implementation uses the original Elf-based libc and the original crt0, which provide a subset of POSIX functions. Almost nothing prevents to compile this libc with a compiler that generates object code in PE format and to assemble MS Visual C ++ programs with a compiler. It is also possible to implement system calls in the Pascal language and then no one else will say that this language is not created for system programming.
ps When creating the documentation, the FastReport report generator was used.
pps My English is really bad. Please treat us indulgently. I promise to translate the entire document into Russian.
Source: https://habr.com/ru/post/119283/
All Articles