Linux and baremetal OS launch
Recently, I posted a small project on a network of 8 kilo C lines in the BSD licensed network. Officially, this is a collection of benchmarks for my clients - vendors of industrial automation. The code is very specific, and, at first glance, of little use outside the narrow area of the PLC and motion control . But there is a small zest, on which I did not particularly focus in the article on the IDZ . The supply of benchmarks includes baremetal environment for their execution. In this post I will describe what it is and how it can be used.
What is baremetal OS? This is a small code that initializes the kernel / kernels and transfers control to the user program, which for some reason needs to be run without a normal OS and even without an RTOS . Naturally, the code of this program has to statically link libraries and drivers for all the devices it needs ... This spring, Habré already had a series of articles on the creation of baremetal OS. Therefore, I will not repeat and describe the loading procedure. However, for a complete understanding of this post, it is advisable to re-read the two key articles of that series. This article describes how to build, load and run an arbitrary program instead of the OS directly from GRUB. And here - how to run your code on several kernels at once. I can also recommend English-language primary sources - osdev and the most famous examples - tutorials from Bran and from James Molloy .
The second article describes in detail how to run the initialization code for a new kernel. To do this, send Init IPI (Inter-Processor-Interrupt) and then two more Startup IPIs to it. An IPI is initiated by simply writing a few bytes to a specific physical address. And what will happen if you write a small Linux user mode bootloader - a program that copies a binary to a specific address and sends it to another Init IPI kernel, 2xSIPI, so that it boots into this binary?
Do you think there will be a kernel panic? After all, on this kernel, Linux already performs user processes, an idle process, or an interrupt. In fact, everything will not fall right away, but the system will start to work unstable and will soon hang. And what will happen if you load Linux with only one kernel, or zafflining the rest of the kernels on a running system? (For example, like this:
And repeat for each core, except 0.
')
In this case, there is no need to fear competition with the task scheduler. But all available physical memory Linux mm when loading already cut into bucket'y and rightly considers its. Fortunately, there is a work round. Even two — you can write a kernel module, which politely asks for a bit of continuous physical memory, to change the linker script to put the .text and .data areas of the binary into this memory. And you can make it even easier - ask Linux to reserve a little physical memory at a certain address when loading. For example, when booting the kernel with the memmap = 0x80M $ 0x18000000 option, the OS will allocate 128 megabytes of continuous physical memory starting at address 0x18000000. Unfortunately, Linux kernels starting from 3.1 sometimes crash at the very beginning of the boot with this parameter. Now just trying to find the cause.
Voilà, now we can keep Linux on several kernels, and arbitrary baremetal programs on all others. But how can this come in handy? Suppose we already have the task to run some code without an OS. For example, I needed benchmarks running on different x86 platforms and measuring the variation between average code execution time and worst case time. x86 is not a model of determinism at all, but if you add an operating system ... Even if it’s a compact and smart RTOS, it’s still not immediately obvious where latency and jitter come from. In addition, each of my clients has their own RTOS, some slightly harder than baremetal. Or if there is a code from the microcontroller that is sensitive to delays, which already works without the OS, and you just have to port it to one of the x86 processor cores. If you simply turn on its kernel thread on the kernel, and disable everything else on it, it will still be interrupted. With the RT-Linux patch, everything turns out better, but it adds latency and is sometimes not compatible with other useful patches.
What about Linux and baremetal together better than a clean baremetal boot boot image (GRUB)? And with Linux it is more convenient! Compare the restart of the entire piece of hardware or just a couple of shell commands in the command line of the Linux host. Similarly, with debugging - from Linux, you can perfectly read-write memory baremetal OS, diagnose hardware in the process. (Of course, if you are careful!) There are some drawbacks to this approach. To run baremetal OS with Linux, you need to know the ACPI ID of the kernel to which the IPI is sent. If you think that the first core has number 0, the second has 1, and so on, then you are an optimist. I saw very different options :). You have to either do enumeration, this is described in one of the posts to which I referred, or select options. (And do not forget to [turn off] Hyperthreading!). Another possible ambush is non-determinism. If you disable all kernels except the first one in Linux and do not run anything serious on it, then it is impossible to measure the influence of Linux on the performance of kernels that execute baremetal code. But there is still no 100% guarantee - if desired, or accidentally, it is possible through shared resources (cache, memory controller) to greatly affect the performance of baremetal code. For example, try switching the VGA console (Ctrl-Alt-F1, Ctrl-Alt-F2), measuring the performance of baremetal OS.
If anyone is interested in how it all works, or if there are ideas on how it can be practically used (the BSD license allows), download the source code from IDZ or github (it's fresh there). The baremetal OS itself is in tools / src / baremetal. In the first directory - phymem is a tool for reading and writing physical memory. And in the second - smallos - runtime itself. There, everything is sharpened to run the benchmarks, so you have to use the file to cut them off - just throw out references to bench / *. O from smallos / Makefile, and replace the benchmark call code with your code in smallos / apps / mwaitTester.c. Linux usermode bootloader lies in smallos / linux_boot. Good luck!
What is baremetal OS? This is a small code that initializes the kernel / kernels and transfers control to the user program, which for some reason needs to be run without a normal OS and even without an RTOS . Naturally, the code of this program has to statically link libraries and drivers for all the devices it needs ... This spring, Habré already had a series of articles on the creation of baremetal OS. Therefore, I will not repeat and describe the loading procedure. However, for a complete understanding of this post, it is advisable to re-read the two key articles of that series. This article describes how to build, load and run an arbitrary program instead of the OS directly from GRUB. And here - how to run your code on several kernels at once. I can also recommend English-language primary sources - osdev and the most famous examples - tutorials from Bran and from James Molloy .
The second article describes in detail how to run the initialization code for a new kernel. To do this, send Init IPI (Inter-Processor-Interrupt) and then two more Startup IPIs to it. An IPI is initiated by simply writing a few bytes to a specific physical address. And what will happen if you write a small Linux user mode bootloader - a program that copies a binary to a specific address and sends it to another Init IPI kernel, 2xSIPI, so that it boots into this binary?
Do you think there will be a kernel panic? After all, on this kernel, Linux already performs user processes, an idle process, or an interrupt. In fact, everything will not fall right away, but the system will start to work unstable and will soon hang. And what will happen if you load Linux with only one kernel, or zafflining the rest of the kernels on a running system? (For example, like this:
$ sudo echo 0 > /sys/devices/system/cpu/cpu1/onlineAnd repeat for each core, except 0.
')
In this case, there is no need to fear competition with the task scheduler. But all available physical memory Linux mm when loading already cut into bucket'y and rightly considers its. Fortunately, there is a work round. Even two — you can write a kernel module, which politely asks for a bit of continuous physical memory, to change the linker script to put the .text and .data areas of the binary into this memory. And you can make it even easier - ask Linux to reserve a little physical memory at a certain address when loading. For example, when booting the kernel with the memmap = 0x80M $ 0x18000000 option, the OS will allocate 128 megabytes of continuous physical memory starting at address 0x18000000. Unfortunately, Linux kernels starting from 3.1 sometimes crash at the very beginning of the boot with this parameter. Now just trying to find the cause.
Voilà, now we can keep Linux on several kernels, and arbitrary baremetal programs on all others. But how can this come in handy? Suppose we already have the task to run some code without an OS. For example, I needed benchmarks running on different x86 platforms and measuring the variation between average code execution time and worst case time. x86 is not a model of determinism at all, but if you add an operating system ... Even if it’s a compact and smart RTOS, it’s still not immediately obvious where latency and jitter come from. In addition, each of my clients has their own RTOS, some slightly harder than baremetal. Or if there is a code from the microcontroller that is sensitive to delays, which already works without the OS, and you just have to port it to one of the x86 processor cores. If you simply turn on its kernel thread on the kernel, and disable everything else on it, it will still be interrupted. With the RT-Linux patch, everything turns out better, but it adds latency and is sometimes not compatible with other useful patches.
What about Linux and baremetal together better than a clean baremetal boot boot image (GRUB)? And with Linux it is more convenient! Compare the restart of the entire piece of hardware or just a couple of shell commands in the command line of the Linux host. Similarly, with debugging - from Linux, you can perfectly read-write memory baremetal OS, diagnose hardware in the process. (Of course, if you are careful!) There are some drawbacks to this approach. To run baremetal OS with Linux, you need to know the ACPI ID of the kernel to which the IPI is sent. If you think that the first core has number 0, the second has 1, and so on, then you are an optimist. I saw very different options :). You have to either do enumeration, this is described in one of the posts to which I referred, or select options. (And do not forget to [turn off] Hyperthreading!). Another possible ambush is non-determinism. If you disable all kernels except the first one in Linux and do not run anything serious on it, then it is impossible to measure the influence of Linux on the performance of kernels that execute baremetal code. But there is still no 100% guarantee - if desired, or accidentally, it is possible through shared resources (cache, memory controller) to greatly affect the performance of baremetal code. For example, try switching the VGA console (Ctrl-Alt-F1, Ctrl-Alt-F2), measuring the performance of baremetal OS.
If anyone is interested in how it all works, or if there are ideas on how it can be practically used (the BSD license allows), download the source code from IDZ or github (it's fresh there). The baremetal OS itself is in tools / src / baremetal. In the first directory - phymem is a tool for reading and writing physical memory. And in the second - smallos - runtime itself. There, everything is sharpened to run the benchmarks, so you have to use the file to cut them off - just throw out references to bench / *. O from smallos / Makefile, and replace the benchmark call code with your code in smallos / apps / mwaitTester.c. Linux usermode bootloader lies in smallos / linux_boot. Good luck!
Source: https://habr.com/ru/post/183576/
All Articles