Hummingbirds for Phantom
Summary: development of the Kolibri OS Compatibility Module inside the Unix OS Compatibility Module (Phantom OS)
Inside the OS Phantom there is a little simple Unix. POSIX subsystem. In principle, the Phantom itself is optional and rather incomplete - Unix Quake was able to be built under it, and, for example, the Apache will not gather almost certainly. Nevertheless - it is.
To continue, you need to understand what a Colibri OS is. Hummingbirds are a Russified western project of micro-OSes on an assembler. Actually, this description is quite complete. Fans of programming on the x86 assembler are working on Hummingbirds. It is, therefore, intolerable and, unfortunately, very poorly designed. Very - it is disastrous. To understand the scale of a disaster, there is no general mechanism for determining the success or failure of a system call. For some, it is simply impossible to determine non-success, some calls return their own set of error codes, some - some other.
')
Why is it nevertheless curious to implement a compatibility layer with this OS? There are several reasons for this:
For the curious, code , heder .
Actually, the development of the compatibility layer is divided into several obvious parts:
The Hummingbird Image Downloader is trivial, except for the implementation of running packed programs. Unfortunately, the implementation of unpacking on the available fragmentary specimens did not lead to success, and it was not possible to find someone who understands. Unpacked programs are very simple, so that the detection and loading of the image complemented the existing elf loader without problems. By the way, under the Phantom of the program for the Hummingbird you can compile the usual elf-ovsky toolchain - regardless of which bootloader downloaded the program, it can refer to both the POSIX system calls and the Hummingbird. Can be mixed)
The entry point also does not represent anything unexpected - calls are made via a software interrupt, the usual IDT gate and a small assembler springboard solve the problem.
The main work, of course, is the implementation of system calls. The work is laborious for two reasons - firstly, the calls are schematically documented, and secondly, their creators really did not bother themselves with any consistency. For example, calling the process information request in one of the fields returns ... the number of another process whose window is located at a given z-position on the screen.
Call implementation was made according to the “TDD type” principle: instead of implementing a system call, we simply write an error to the log, launch application programs, if an unimplemented system call is called, implement what was caused. So if the call is not made, then I did not reach the application program that needs it.
In general, calls are divided into several groups.
Everything is simple with them. Also because some of the information can be given in constant or fake form. A good example is
The hummingbird is a system with almost no protection, the process can be very much, including modifying the processor registers, which are usually available only to the kernel. Most of these calls, of course, are simply not implemented.
The Hummingbird file operations are more or less consistent (at least, the error codes are the same), but their implementation is very expensive - they all store almost no state in the kernel and, de facto, are reduced to the open / do / close group. That is, reading from the file for each call performs the opening and closing of the file. In principle, you can try to optimize it through the hash map of the file name and keep the pool of open descriptors, but this is not included in the weekend project skoup. I did not do it. Otherwise, the implementation is more or less trivial.
Here I expected problems. They were, but mostly not as creepy as they could have happened. However, there is a feeling that the presentation of the authors of the Hummingbird graphics subsystem to the end I still did not comprehend - some programs work fine, some - strange. Alas, without actually reading the source code of the original implementation, probably progress is impossible here, and the source code, I remind you, is in assembly language. I didn’t find enough courage in myself, and I stopped at the fact that some subset of programs was working fine.
The main difference between Hummingbird graphics is that the core implements some of the traditionally applied components of a window — in particular, buttons. There was no ready mate in the window system of the Phantom, I had to write. Obviously, the Hummingbird application code is rigidly tied to specific fonts used in the kernel - the fonts had to be extracted and converted into the format of the Phantom window subsystem, plus a few modifications to the font rendering code. There were other little things, in general, insignificant.
Hummingbird supports simple IPC mechanism - mailboxes. It is implemented schematically, but not subjected to detailed testing. The launch of additional threads of the process is also implemented - also without serious testing. By the way, it is not quite clear which part of the state of the kernel should relate to the process, and which part to the thread.
Fully implemented memory mapping control functions — currently, the Phantom core only works with one address space (Unix and Colibri processes are supported through the x86 segment addressing mechanism), so there’s nothing to display them. Due to this fact, it is difficult (although possible) to implement a change in the size of the address space of the process. Support for the DLL Colibri DLL itself is not implemented very inactive, and the plan was to make the minimum necessary. However, this is not difficult.
It is pretty obvious that to continue the work you need:
In general, you need a hero from the Kolibri OS team. I need a hero :)
Inside the OS Phantom there is a little simple Unix. POSIX subsystem. In principle, the Phantom itself is optional and rather incomplete - Unix Quake was able to be built under it, and, for example, the Apache will not gather almost certainly. Nevertheless - it is.
To continue, you need to understand what a Colibri OS is. Hummingbirds are a Russified western project of micro-OSes on an assembler. Actually, this description is quite complete. Fans of programming on the x86 assembler are working on Hummingbirds. It is, therefore, intolerable and, unfortunately, very poorly designed. Very - it is disastrous. To understand the scale of a disaster, there is no general mechanism for determining the success or failure of a system call. For some, it is simply impossible to determine non-success, some calls return their own set of error codes, some - some other.
')
Why is it nevertheless curious to implement a compatibility layer with this OS? There are several reasons for this:
- It is very compact. Looking ahead - the first program for the Hummingbird in the Phantom was able to start after four hours of work.
- This mini-project has become a driver for the development of some native subsystems of the Phantom,
in particular - window. - The main thing - the entire state of the Hummingbird process, known to the core, fits into a small structure. Many (almost all!) Challenges are stateless, that is, they do not rely on any knowledge
stored in the kernel. This is an ideal candidate for implementing persistent (undergoing OS restart) binary (not written in bytecode-language) processes in Phantom.
For the curious, code , heder .
Actually, the development of the compatibility layer is divided into several obvious parts:
- Executable image loader
- System Call Entry Point
- "Gaskets" for system calls, converting them into the semantics of existing components of the system
- Development of missing functionality
The Hummingbird Image Downloader is trivial, except for the implementation of running packed programs. Unfortunately, the implementation of unpacking on the available fragmentary specimens did not lead to success, and it was not possible to find someone who understands. Unpacked programs are very simple, so that the detection and loading of the image complemented the existing elf loader without problems. By the way, under the Phantom of the program for the Hummingbird you can compile the usual elf-ovsky toolchain - regardless of which bootloader downloaded the program, it can refer to both the POSIX system calls and the Hummingbird. Can be mixed)
The entry point also does not represent anything unexpected - calls are made via a software interrupt, the usual IDT gate and a small assembler springboard solve the problem.
The main work, of course, is the implementation of system calls. The work is laborious for two reasons - firstly, the calls are schematically documented, and secondly, their creators really did not bother themselves with any consistency. For example, calling the process information request in one of the fields returns ... the number of another process whose window is located at a given z-position on the screen.
Call implementation was made according to the “TDD type” principle: instead of implementing a system call, we simply write an error to the log, launch application programs, if an unimplemented system call is called, implement what was caused. So if the call is not made, then I did not reach the application program that needs it.
In general, calls are divided into several groups.
Informational
Everything is simple with them. Also because some of the information can be given in constant or fake form. A good example is
st->eax = dummy++; // TODO n of context switches st->eax = dummy++; // TODO n of context switches - increases, and it is fine. Where possible, of course, the information is given real.
Low-level
The hummingbird is a system with almost no protection, the process can be very much, including modifying the processor registers, which are usually available only to the kernel. Most of these calls, of course, are simply not implemented.
File operations
The Hummingbird file operations are more or less consistent (at least, the error codes are the same), but their implementation is very expensive - they all store almost no state in the kernel and, de facto, are reduced to the open / do / close group. That is, reading from the file for each call performs the opening and closing of the file. In principle, you can try to optimize it through the hash map of the file name and keep the pool of open descriptors, but this is not included in the weekend project skoup. I did not do it. Otherwise, the implementation is more or less trivial.
Graphics
Here I expected problems. They were, but mostly not as creepy as they could have happened. However, there is a feeling that the presentation of the authors of the Hummingbird graphics subsystem to the end I still did not comprehend - some programs work fine, some - strange. Alas, without actually reading the source code of the original implementation, probably progress is impossible here, and the source code, I remind you, is in assembly language. I didn’t find enough courage in myself, and I stopped at the fact that some subset of programs was working fine.
The main difference between Hummingbird graphics is that the core implements some of the traditionally applied components of a window — in particular, buttons. There was no ready mate in the window system of the Phantom, I had to write. Obviously, the Hummingbird application code is rigidly tied to specific fonts used in the kernel - the fonts had to be extracted and converted into the format of the Phantom window subsystem, plus a few modifications to the font rendering code. There were other little things, in general, insignificant.
Implemented but not tested
Hummingbird supports simple IPC mechanism - mailboxes. It is implemented schematically, but not subjected to detailed testing. The launch of additional threads of the process is also implemented - also without serious testing. By the way, it is not quite clear which part of the state of the kernel should relate to the process, and which part to the thread.
Missing
Fully implemented memory mapping control functions — currently, the Phantom core only works with one address space (Unix and Colibri processes are supported through the x86 segment addressing mechanism), so there’s nothing to display them. Due to this fact, it is difficult (although possible) to implement a change in the size of the address space of the process. Support for the DLL Colibri DLL itself is not implemented very inactive, and the plan was to make the minimum necessary. However, this is not difficult.
Project development
It is pretty obvious that to continue the work you need:
- A set of regression tests. Preferably in the form of a) internal unit tests for components;
b) special test application programs; and c) running existing programs with monitoring their performance. - Code review by experienced Hummingbird kernel developer for compliance with the original implementation
- Review of the code on the subject of detailed verification of all entry points of the Humming-Phantom regarding the admissibility and meaningfulness of the parameters.
In general, you need a hero from the Kolibri OS team. I need a hero :)
Source: https://habr.com/ru/post/333982/
All Articles