Systemicus Part 3: OMFS3 and Liberte
In this part, I would like to share the experience of creating the OMFS3 file system for Systemicus OS. The main purpose of its creation is strong data encryption.
')
Previous parts:
First public performance of RTOS Systemicus
Systemicus Part 2: GUI
I just want to point out why I decided to write my own FS, and not to fasten, for example, FAT, NTFS (fork NTFS Kolibri code) or ext2 / 3. I wanted it initially to be simple to implement, yet extensible and secure.
Simplicity It consists in the absence of journaling at a basic level. Everything is arranged quite simply - there are large clusters (64 kilobytes each, since malloc in OS Systemicus also allocates 64k), in some clusters there is information about files and nested directories, in others - the data itself. The first cluster is reserved for partition information + plenty of space for future extensions (now it only uses the first 64 bytes).
I took the second cluster under the kernel code of my OS. This was done because of the concern for the system itself. Firstly, it greatly reduces the bootloader code (since you do not need to write a file search throughout the system), secondly, it hides the kernel from the user, which means that neither it nor the user program can damage the kernel code (because . formally this cluster does not enter the visible space of the file system). + this does not affect the versatility, the 2nd cluster may not be busy, but it is always alone and there is code or not - it does not matter.
There is another system cluster (or clusters, if the disk size is large). Its address is no longer fixed, but is indicated in the section parameters. This is a partition byte map, where each byte indicates whether the corresponding cluster is free or not. Why not a bit? So it's easier + from experience I know that in the future something may come in handy, for example, to specify other than 0 and 1 other states, for example, damaged, only for reading, etc. One cluster with a byte card provides coverage of 65 536 (number of records in the card on one cluster) * 65 536 (cluster size) = 4 294 967 296 bytes, i.e. 4 gigabytes of disk space. As for me, it is quite acceptable.
How is data storage organized? I didn’t make any special structures, just 2 dwords are indicated at the end of each cluster: the size of meaningful data in the current cluster (in principle, one could do without this value, since the size of the entire file is known through its inode record) and address of the next cluster with the current file data If the address is 0, then this is the last cluster in the data chain. Everything is simple and logical - when reading a file, we turn to the disk sequentially, without switching each time to a separate data location structure.
Most delicious
Actually, because of what their own FS was written. So, encryption in the FS is at a low level, i.e. each cluster is encrypted separately, not the file. Those. at the file system driver level. Encryption occurs in 2 stages with two different keys.
To begin with, from the user's password, we get through the 256-bit version of GOST 34.11-2012 a 256-bit key (or rather, a pair, one for the password). The first key is to encrypt the cluster using the GOST 28147-89 algorithm (using the “replacement” table, the one used by the Central Bank of the Russian Federation for substitutions). With the second key, the cluster is again encrypted, but the algorithm is set by the user when creating the file system itself (formatting) - to choose RC4, RC6, IDEA (slow infection) or Blowfish-256.
IMPORTANT! The first two clusters are encrypted, read above why (for what they are used).
Further, it is separately possible to encrypt a single file at a high level, but still at the file system level. Those. on top of these two encryption, which remind you apply not to the file, but to the cluster, it is possible to encrypt the file. This opportunity is spelled out, but so far I have not implemented it (a problem of time, not of complexity).
About extensibility
First, the main parameters are stored in 64-bit variants, i.e. there will be no problems with the sizes (and considering that addressing goes on sector-by-sector, and on clusters of 64 kilobytes, you can address a lot :-)), as well as with time stamps (also 64 bits).
There is a place for additional file flags (access, permissions, ...). Also, there is a lot of space (see 1st cluster) for other extensions.
In principle, it is possible in the future to implement journaling, I do not see anything contradictory, the journal can be written in a separate file special or chain of clusters.
With encryption too, the type of algorithm is stored in a byte variable, i.e. in addition to these 4, you can still tie 252 other encryption and hashing algorithms)
Bun
I have long wanted to make my analog LeanFS GUI , because compiling the section each time manually is inconvenient somehow ... and there is no encryption in the FS itself except GOST + RC4 (and those are temporarily disabled during development). Therefore, it is necessary to write.
Chose Delphi (do not be angry, do not like C / C ++, or do not know how ;-)). For 2.5 days I managed it, I did the fasm functions as a separate dll and just screwed it to my application. It should be noted that with Blowfish and IDEA so far weird creepy things happen, so I turned them off in this version, I personally have enough GOST + RC6.
So, I called the program Liberte and in its essence it turned out not only my file system viewer, but also a cryptocontainer :-) I use it myself, I did this for myself earlier - Wolfram , but there was encryption of individual files and it was not convenient. Now for my important data I use Liberte.
Do not complain about the encryption speed - I myself am in shock, but everything is ahead.
I give a link to download - nebka.ru/files/Liberte_0.1c.7z
No, here's another one, but in the last article they cursed that the server gave something wrong: http://yadi.sk/d/1up9km3cSBPvE
Scold) because I want to bring it to mind. The container format is stable, and the program can be completed.
')
Previous parts:
First public performance of RTOS Systemicus
Systemicus Part 2: GUI
I just want to point out why I decided to write my own FS, and not to fasten, for example, FAT, NTFS (fork NTFS Kolibri code) or ext2 / 3. I wanted it initially to be simple to implement, yet extensible and secure.
Simplicity It consists in the absence of journaling at a basic level. Everything is arranged quite simply - there are large clusters (64 kilobytes each, since malloc in OS Systemicus also allocates 64k), in some clusters there is information about files and nested directories, in others - the data itself. The first cluster is reserved for partition information + plenty of space for future extensions (now it only uses the first 64 bytes).
I took the second cluster under the kernel code of my OS. This was done because of the concern for the system itself. Firstly, it greatly reduces the bootloader code (since you do not need to write a file search throughout the system), secondly, it hides the kernel from the user, which means that neither it nor the user program can damage the kernel code (because . formally this cluster does not enter the visible space of the file system). + this does not affect the versatility, the 2nd cluster may not be busy, but it is always alone and there is code or not - it does not matter.
There is another system cluster (or clusters, if the disk size is large). Its address is no longer fixed, but is indicated in the section parameters. This is a partition byte map, where each byte indicates whether the corresponding cluster is free or not. Why not a bit? So it's easier + from experience I know that in the future something may come in handy, for example, to specify other than 0 and 1 other states, for example, damaged, only for reading, etc. One cluster with a byte card provides coverage of 65 536 (number of records in the card on one cluster) * 65 536 (cluster size) = 4 294 967 296 bytes, i.e. 4 gigabytes of disk space. As for me, it is quite acceptable.
How is data storage organized? I didn’t make any special structures, just 2 dwords are indicated at the end of each cluster: the size of meaningful data in the current cluster (in principle, one could do without this value, since the size of the entire file is known through its inode record) and address of the next cluster with the current file data If the address is 0, then this is the last cluster in the data chain. Everything is simple and logical - when reading a file, we turn to the disk sequentially, without switching each time to a separate data location structure.
Most delicious
Actually, because of what their own FS was written. So, encryption in the FS is at a low level, i.e. each cluster is encrypted separately, not the file. Those. at the file system driver level. Encryption occurs in 2 stages with two different keys.
To begin with, from the user's password, we get through the 256-bit version of GOST 34.11-2012 a 256-bit key (or rather, a pair, one for the password). The first key is to encrypt the cluster using the GOST 28147-89 algorithm (using the “replacement” table, the one used by the Central Bank of the Russian Federation for substitutions). With the second key, the cluster is again encrypted, but the algorithm is set by the user when creating the file system itself (formatting) - to choose RC4, RC6, IDEA (slow infection) or Blowfish-256.
IMPORTANT! The first two clusters are encrypted, read above why (for what they are used).
Further, it is separately possible to encrypt a single file at a high level, but still at the file system level. Those. on top of these two encryption, which remind you apply not to the file, but to the cluster, it is possible to encrypt the file. This opportunity is spelled out, but so far I have not implemented it (a problem of time, not of complexity).
About extensibility
First, the main parameters are stored in 64-bit variants, i.e. there will be no problems with the sizes (and considering that addressing goes on sector-by-sector, and on clusters of 64 kilobytes, you can address a lot :-)), as well as with time stamps (also 64 bits).
There is a place for additional file flags (access, permissions, ...). Also, there is a lot of space (see 1st cluster) for other extensions.
In principle, it is possible in the future to implement journaling, I do not see anything contradictory, the journal can be written in a separate file special or chain of clusters.
With encryption too, the type of algorithm is stored in a byte variable, i.e. in addition to these 4, you can still tie 252 other encryption and hashing algorithms)
Bun
I have long wanted to make my analog LeanFS GUI , because compiling the section each time manually is inconvenient somehow ... and there is no encryption in the FS itself except GOST + RC4 (and those are temporarily disabled during development). Therefore, it is necessary to write.
Chose Delphi (do not be angry, do not like C / C ++, or do not know how ;-)). For 2.5 days I managed it, I did the fasm functions as a separate dll and just screwed it to my application. It should be noted that with Blowfish and IDEA so far weird creepy things happen, so I turned them off in this version, I personally have enough GOST + RC6.
So, I called the program Liberte and in its essence it turned out not only my file system viewer, but also a cryptocontainer :-) I use it myself, I did this for myself earlier - Wolfram , but there was encryption of individual files and it was not convenient. Now for my important data I use Liberte.
Do not complain about the encryption speed - I myself am in shock, but everything is ahead.
I give a link to download - nebka.ru/files/Liberte_0.1c.7z
No, here's another one, but in the last article they cursed that the server gave something wrong: http://yadi.sk/d/1up9km3cSBPvE
Scold) because I want to bring it to mind. The container format is stable, and the program can be completed.
Source: https://habr.com/ru/post/225145/
All Articles