Inferno Part 0: namespaces
FAQ: What is OS Inferno and why is it needed?
I promised, but have not yet written a post on how to install and run Inferno, and do incredible things; I have blown it up to a series of articles, of which this is the first, introductory part. I'm going to talk a little bit about namespaces. This is a critical part of the OS, and one of those things that strongly distinguishes Plan 9 and Inferno from what is currently running on your computer.
Inferno is little known, and I do not intend to evangelize it. I want to help bridge the gap between history and practice to explain how to use Inferno, and why it works the way it works.
')
I will have to deviate slightly to the side. I'm going to talk about the evolution of namespaces, not quite in chronological order, rather in the order of development of possibilities, and then show how to use the capabilities of Inferno.
Remember DOS? You had A: and B :, your floppy drives. Was C :, your hard drive, etc. You had a directory hierarchy on each of these disks. (Let us omit the extra details.) You can think of this as a flat, high-level, single-letter, namespace bound to the gland. One disk - one letter.
In Unix, however, there was a top-level directory, "/". The root disk was mounted there, and all other disks were mounted in subdirectories of the root directory, in subdirectories of subdirectories where you would like to see this disk. You did not need to know what partition or disk, unless you were engaged in setting up the system or free space was running out on one of the partitions. You can think of it as a hierarchical namespace common to the entire OS.
This is a much more flexible and powerful approach. Programs and users do not need to know what is on what disk.
Later, a new type of file system appeared in Unix: synthetic. For example, / proc. On Linux, there is also / sys, and it is not rare that / dev is also a synthetic file system. There is no physical disk on which files of these file systems would be stored, but, nevertheless, programs do not need to know anything about it. Files are visible in the namespace just like any other files, and programs do not need to be modified so that they can read or write files in these file systems.
In a sense, between them there are file systems that, although they store files on the disk as usual, but the user cannot know for sure or prove this. For example, NFS is a remote service containing a hierarchy of files and directories.
Next is FUSE , which provides file systems in user space. Using FUSE to interact with the local OS kernel, and having realized what you were going to write a driver for, you can create a service for any synthetic file system — without having to write a real driver for the OS kernel. (I keep quiet about BSD; there seem to be fewer features from Plan 9, although DragonFly developed some interesting ideas.)
On the other hand, since the namespace is common to the entire system, you need to be root in order to change it. Imagine a random user who has mounted something on top of / sbin, having even killed the umount command. Custom mount is a minor exception, in a sense, a crutch that does not fundamentally change anything: you can mount the FUSE file system in any _ your_ directory. (By the way, Linux has a driver for 9P2000, the protocol used by Plan 9 and Inferno.)
Previously, the file system could only be mounted in one place. Linux Bind mounts provide a bit more flexibility - if the file system (or part of it) is already available somewhere in the namespace, it can be connected to another location via
Plan 9 and Inferno took several large chunks of the historical semantics of Unix and threw them out of the window, threw out some of the key assumptions of the OS, assumptions that were largely implied and implicit. I'm talking about Inferno now, but almost all of this also applies to (and first appeared in) Plan 9. When I say “in Inferno” you can safely add in brackets “and in Plan 9”. These assumptions are not specified in any particular order, I simply numbered them for convenience.
The first assumption that was easiest to break, and perhaps the most obvious in retrospect, is that the files and directories in the namespace represent objects on the disk.
The second assumption is that there is a one-to-one connection between file systems and mount points. Those. that this file system is connected only in one place, and that only this one file system is connected to this place. Getting rid of this assumption means that the file system and the place where it is connected are independent of each other, and that any number of file systems can be connected to the same place.
The third assumption is that there is a one-to-one relationship between the operating system and the namespace. If namespace changes are visible only to the process that has changed the namespace and its descendants, there will be no need to protect users from namespace changes made by other users, and the restriction of who can install what and where will be removed. It is enough for the user to have read access to the file system in order to mount it.
Throwing out these assumptions about namespaces allowed us to get a much more powerful system, and this difference may not be so obvious at first glance, but in fact is as big as the difference between DOS and Unix namespaces.
These three assumptions were violated to varying degrees in Unix-like operating systems, and with varying degrees of success. The first assumption was successfully violated in Unix. You have / proc, for example, in almost all Unix. FUSE allowed users to mount file systems, even to hide them (not perfect, of course) from other users, which gave the semblance of a personal namespace, almost breaking the third assumption. Bind mounts made it possible for one part of the namespace to appear in somewhere in another part, approaching a violation of the second assumption, but requiring root. Union mounts also made their way to Unix, it seems. But, with the exception of the first assumption, all these solutions are not complete. What if we start from the beginning without dragging Unix luggage?
Since the namespace is now for each process its own (more precisely, any process can, but is not required to fork the namespace), anyone can mount anything anywhere, or connect (bind) anything anywhere, without creating any problems security. Since the mount point can contain any number of mounted systems, and all will be simultaneously available, it becomes possible to overlay one file system to another. Since any file system can be “real” or “synthetic” as long as it uses the appropriate protocol (9P2000 in the case of Inferno), anything can be a file system and any program that works with files can use it.
Many things that are either hacks or curved solutions suddenly disappear, since they are no longer necessary. Obviously, much of what has been delegated to root is no longer needed, as is sudo. chroot is no longer needed. Symbolic links are not needed. A huge number of drivers built into the OS kernel are not needed. NSF and / etc / exports are not needed (good riddance).
One of the things that annoys me in the documentation for Inferno (and Plan 9) is that there are research articles, man pages, source code, but almost no textbooks or howto documents. Maybe I'm spoiled by the Linux that comes with the HOWTO how to make coffee . Getting started is not an easy task, especially if you don’t know enough about the system to use the system to study yourself. You try, but there is an unfamiliar shell that launches unfamiliar commands, and the man pages written as a reference book do not really help.
So I can evaluate the document trying to answer the question “So? I heard tempting promises before. How can I use it? ”And I'm going to combine all this with a few concrete examples, explaining what's going on.
Remember all these things that are no longer needed? Each of them was replaced by something better, and some of them by several things, each one better. I’m going to (for simplicity) ignore authentication in these examples, but I’ll say that you have a choice of authentication methods and encryption in Inferno.
It's pretty simple:
This mounts the prog file system (analogous to / proc) to / n / prog. By the way, you can manage, debug, and do everything normally with running processes through the / prog file system, just like in Unix. The difference is that if you locally mount / prog of a remote machine, you can do all these things with remote processes without any ssh and even without running a debugger on the remote machine. Yes, you can execute
I have a bunch of .iso files on my computer. Suppose they are in / n / cds, and I want to connect one of them:
Now the current process and its descendants can see the contents of this CD. Nothing much is required; 9660srv (4) knows how to read the ISO-9660 file system on the one hand and supports the 9P2000 protocol on the other.
Inferno also comes bundled with, for example, tarfs (4) , which treats the tar archive as a read-only file system.
Well, most. Those that do not just export the file system, and finally you can forget about ioctl (). The drivers provided by the kernel can be seen in the / dev / drivers file, and this is a rather short list, more concise than the output of
Where you need a kernel driver to support the new file system on most operating systems, Inferno does not need modprobe (and thus root) and does not touch the kernel to add a new file system. You can get ext2 support (read-only) by downloading ext2fs , just like the kernel did not need to know about ISO-9660 in the example above.
Since the process can create a namespace for itself, everything that can be dangerous to show to a non-trusted process can be unmounted from the namespace, and the pctl (2) system call can be used to prevent the process from mounting the drivers. An extreme example that unmounts everything (and therefore will not be able to do anything except use the shell built-in functionality):
Of course, you need to act a little more carefully than just unmount everything if you want to get something useful, but the idea is simple.
You can also make a temporary sandbox, if you want, by mounting the file system in memory over the root directory:
And, of course, / asdf will not appear anywhere. (It should be noted that this only applies to the creation of new files.)
bind solves this problem by providing the ability to put some directories on top of others. For example, I like the tinytk icon set more than the default icons, so I overlap the default icons with them before running the window manager:
You can even put one file on top of another:
Since symlinks are not needed and are missing in Inferno, the problem of circular references in the file system almost disappears.
I hate messing with $ PATH. Large chunks of my .bashrc are dedicated to figuring out who goes first, and not adding redundant paths or paths that point to binaries for other architectures. I have Plan 9 Port installed, but I don't want the Plan 9 man version to conflict with the Linux version, for example.
In Inferno, everything you want to run is in / dis (/ bin in Plan 9). If you want to run programs from another location, you simply connect it before or after / dis.
So if you want to install something, but you cannot write to / dis, install it wherever you want, and connect it over / dis. sudo is not required. The same goes for / lib and / module.
I'm going to ignore authentication (the -A flag) and encryption, so we will export something unimportant : memfs (4) again. First car:
Second car:
Since most file server commands can work via stdin / stdout instead of connecting to a mount point, the file system does not even need to be locally mounted to make it accessible over the network:
And on the second car:
The listen (1) and dial (1) commands are simply convenient wrappers for interacting with the / net file system, which is the interface to the network for Inferno. Sockets in Unix are handled by the BSD sockets library (with the corresponding system calls) and are much more difficult to use. You can think of listen and dial as analogous to netcat. styxlisten is the same, only expects to transmit the 9P2000 protocol (also called Styx until it was accepted into Plan 9) instead of a stream of bytes.
All this is achieved by several short commands, which themselves are just wrappers around Inferno system calls. You have bind (1), mount (1), and unmount (1) . If you need introspection, ns (1) will help you.
Representation of all OS objects as file systems means that programs like cat, sed and echo can be used for, for example, network access using the ip (3) file system. ftpfs (4) allows you to use ls and cp without an interactive client, which makes scripting easier (no need to redirect through an FTP client). In fact, your scripts do not need to know at all that they work with an FTP server and not with a regular file on disk. zipfs allows you to mount a .zip file. By combining them, you can mount .iso, which is inside .zip, which is on the FTP server, and then copy individual files from it.
We are already at an interesting point. Most of the population owns at least a laptop and a smartphone, and, of them, most also own a home computer, have a computer at work, a tablet, etc. In the heyday of Unix, there was one computer and many users. This seems obvious, but we are rapidly approaching the point when the general case will be many computers for one user and not vice versa.
Managing all these resources (disk and CPU, in particular) is already a big problem. File synchronization, contact synchronization, etc. If your phone does not have enough resources to process something, you need to go to a regular computer (physically or via ssh). If you want to watch a movie on your phone, you need to copy it there. What do you do if you read the article on the phone and decide to continue reading on the laptop? Do you send email or IM? (This is still better than DBUS, but this is a digression.)
I do not think that this will continue. As a temporary solution, you can give all your data to Google, Facebook, or any “cloud” service, but this solution leaves much to be desired (and listing the reasons is another digression).
Software solution to the problem of managing computational resources and combining all these machines (local and remote) is inevitable. At least at the moment, I see Inferno as the best candidate for the platform that will make all this possible. If this does not fully explain why I suddenly fell in love with this system, then at least I do it partially.
This description is very superficial. “Everything is a file system” is only one of the great ideas on which Inferno is built.
I promised, but have not yet written a post on how to install and run Inferno, and do incredible things; I have blown it up to a series of articles, of which this is the first, introductory part. I'm going to talk a little bit about namespaces. This is a critical part of the OS, and one of those things that strongly distinguishes Plan 9 and Inferno from what is currently running on your computer.
Inferno is little known, and I do not intend to evangelize it. I want to help bridge the gap between history and practice to explain how to use Inferno, and why it works the way it works.
')
I will have to deviate slightly to the side. I'm going to talk about the evolution of namespaces, not quite in chronological order, rather in the order of development of possibilities, and then show how to use the capabilities of Inferno.
Content
- Dos
- Unix, Linux
- Plan 9, Inferno
- … Well?
- Finally, something that you can dial
- Powerful primitives mean you will never regret
- Somewhere in the future
- To be continued
Dos
Remember DOS? You had A: and B :, your floppy drives. Was C :, your hard drive, etc. You had a directory hierarchy on each of these disks. (Let us omit the extra details.) You can think of this as a flat, high-level, single-letter, namespace bound to the gland. One disk - one letter.
Unix, Linux
In Unix, however, there was a top-level directory, "/". The root disk was mounted there, and all other disks were mounted in subdirectories of the root directory, in subdirectories of subdirectories where you would like to see this disk. You did not need to know what partition or disk, unless you were engaged in setting up the system or free space was running out on one of the partitions. You can think of it as a hierarchical namespace common to the entire OS.
This is a much more flexible and powerful approach. Programs and users do not need to know what is on what disk.
Later, a new type of file system appeared in Unix: synthetic. For example, / proc. On Linux, there is also / sys, and it is not rare that / dev is also a synthetic file system. There is no physical disk on which files of these file systems would be stored, but, nevertheless, programs do not need to know anything about it. Files are visible in the namespace just like any other files, and programs do not need to be modified so that they can read or write files in these file systems.
In a sense, between them there are file systems that, although they store files on the disk as usual, but the user cannot know for sure or prove this. For example, NFS is a remote service containing a hierarchy of files and directories.
Next is FUSE , which provides file systems in user space. Using FUSE to interact with the local OS kernel, and having realized what you were going to write a driver for, you can create a service for any synthetic file system — without having to write a real driver for the OS kernel. (I keep quiet about BSD; there seem to be fewer features from Plan 9, although DragonFly developed some interesting ideas.)
On the other hand, since the namespace is common to the entire system, you need to be root in order to change it. Imagine a random user who has mounted something on top of / sbin, having even killed the umount command. Custom mount is a minor exception, in a sense, a crutch that does not fundamentally change anything: you can mount the FUSE file system in any _ your_ directory. (By the way, Linux has a driver for 9P2000, the protocol used by Plan 9 and Inferno.)
Previously, the file system could only be mounted in one place. Linux Bind mounts provide a bit more flexibility - if the file system (or part of it) is already available somewhere in the namespace, it can be connected to another location via
mount --bind . This is a relatively recent addition, and before the 2.6.26 kernel it was impossible to attach the same file system to one place in read-write mode and at the same time to another in read-only mode.Plan 9, Inferno
Plan 9 and Inferno took several large chunks of the historical semantics of Unix and threw them out of the window, threw out some of the key assumptions of the OS, assumptions that were largely implied and implicit. I'm talking about Inferno now, but almost all of this also applies to (and first appeared in) Plan 9. When I say “in Inferno” you can safely add in brackets “and in Plan 9”. These assumptions are not specified in any particular order, I simply numbered them for convenience.
The first assumption that was easiest to break, and perhaps the most obvious in retrospect, is that the files and directories in the namespace represent objects on the disk.
The second assumption is that there is a one-to-one connection between file systems and mount points. Those. that this file system is connected only in one place, and that only this one file system is connected to this place. Getting rid of this assumption means that the file system and the place where it is connected are independent of each other, and that any number of file systems can be connected to the same place.
The third assumption is that there is a one-to-one relationship between the operating system and the namespace. If namespace changes are visible only to the process that has changed the namespace and its descendants, there will be no need to protect users from namespace changes made by other users, and the restriction of who can install what and where will be removed. It is enough for the user to have read access to the file system in order to mount it.
Throwing out these assumptions about namespaces allowed us to get a much more powerful system, and this difference may not be so obvious at first glance, but in fact is as big as the difference between DOS and Unix namespaces.
These three assumptions were violated to varying degrees in Unix-like operating systems, and with varying degrees of success. The first assumption was successfully violated in Unix. You have / proc, for example, in almost all Unix. FUSE allowed users to mount file systems, even to hide them (not perfect, of course) from other users, which gave the semblance of a personal namespace, almost breaking the third assumption. Bind mounts made it possible for one part of the namespace to appear in somewhere in another part, approaching a violation of the second assumption, but requiring root. Union mounts also made their way to Unix, it seems. But, with the exception of the first assumption, all these solutions are not complete. What if we start from the beginning without dragging Unix luggage?
Since the namespace is now for each process its own (more precisely, any process can, but is not required to fork the namespace), anyone can mount anything anywhere, or connect (bind) anything anywhere, without creating any problems security. Since the mount point can contain any number of mounted systems, and all will be simultaneously available, it becomes possible to overlay one file system to another. Since any file system can be “real” or “synthetic” as long as it uses the appropriate protocol (9P2000 in the case of Inferno), anything can be a file system and any program that works with files can use it.
Many things that are either hacks or curved solutions suddenly disappear, since they are no longer necessary. Obviously, much of what has been delegated to root is no longer needed, as is sudo. chroot is no longer needed. Symbolic links are not needed. A huge number of drivers built into the OS kernel are not needed. NSF and / etc / exports are not needed (good riddance).
mount -o loop not needed. mount --bind not needed. FUSE is not needed. Even $ PATH is no longer needed, and with it the need to be root to install software.… Well?
One of the things that annoys me in the documentation for Inferno (and Plan 9) is that there are research articles, man pages, source code, but almost no textbooks or howto documents. Maybe I'm spoiled by the Linux that comes with the HOWTO how to make coffee . Getting started is not an easy task, especially if you don’t know enough about the system to use the system to study yourself. You try, but there is an unfamiliar shell that launches unfamiliar commands, and the man pages written as a reference book do not really help.
So I can evaluate the document trying to answer the question “So? I heard tempting promises before. How can I use it? ”And I'm going to combine all this with a few concrete examples, explaining what's going on.
Finally, something that you can dial
Remember all these things that are no longer needed? Each of them was replaced by something better, and some of them by several things, each one better. I’m going to (for simplicity) ignore authentication in these examples, but I’ll say that you have a choice of authentication methods and encryption in Inferno.
root is not needed for mounting
It's pretty simple:
bind '#p' /n/prog This mounts the prog file system (analogous to / proc) to / n / prog. By the way, you can manage, debug, and do everything normally with running processes through the / prog file system, just like in Unix. The difference is that if you locally mount / prog of a remote machine, you can do all these things with remote processes without any ssh and even without running a debugger on the remote machine. Yes, you can execute
kill 15 locally and this will kill process 15 on the remote machine.mount -o loop and FUSE are not needed
I have a bunch of .iso files on my computer. Suppose they are in / n / cds, and I want to connect one of them:
; 9660srv /n/cds/dfly-i386-3.0.2_REL.iso /n/dfly ; ls /n/dfly Now the current process and its descendants can see the contents of this CD. Nothing much is required; 9660srv (4) knows how to read the ISO-9660 file system on the one hand and supports the 9P2000 protocol on the other.
Inferno also comes bundled with, for example, tarfs (4) , which treats the tar archive as a read-only file system.
Drivers in the kernel are not needed
Well, most. Those that do not just export the file system, and finally you can forget about ioctl (). The drivers provided by the kernel can be seen in the / dev / drivers file, and this is a rather short list, more concise than the output of
lsmod on a typical Ubuntu machine or even Arch.Where you need a kernel driver to support the new file system on most operating systems, Inferno does not need modprobe (and thus root) and does not touch the kernel to add a new file system. You can get ext2 support (read-only) by downloading ext2fs , just like the kernel did not need to know about ISO-9660 in the example above.
chroot is not needed
Since the process can create a namespace for itself, everything that can be dangerous to show to a non-trusted process can be unmounted from the namespace, and the pctl (2) system call can be used to prevent the process from mounting the drivers. An extreme example that unmounts everything (and therefore will not be able to do anything except use the shell built-in functionality):
; load std ; pctl newns ; pctl nodevs ; unmount / ; ls / sh: ls: './ls' file does not exist Of course, you need to act a little more carefully than just unmount everything if you want to get something useful, but the idea is simple.
You can also make a temporary sandbox, if you want, by mounting the file system in memory over the root directory:
; mount -bc {memfs -s} / ; echo Mad with power! > /asdf And, of course, / asdf will not appear anywhere. (It should be noted that this only applies to the creation of new files.)
Simlinks are not needed
bind solves this problem by providing the ability to put some directories on top of others. For example, I like the tinytk icon set more than the default icons, so I overlap the default icons with them before running the window manager:
; bind -bc /icons/tinytk /icons/tk You can even put one file on top of another:
; echo Hello > some-file ; bind some-file /NOTICE ; cat /NOTICE Hello Since symlinks are not needed and are missing in Inferno, the problem of circular references in the file system almost disappears.
$ PATH, $ LDPATH and root privileges are not needed to install software
I hate messing with $ PATH. Large chunks of my .bashrc are dedicated to figuring out who goes first, and not adding redundant paths or paths that point to binaries for other architectures. I have Plan 9 Port installed, but I don't want the Plan 9 man version to conflict with the Linux version, for example.
In Inferno, everything you want to run is in / dis (/ bin in Plan 9). If you want to run programs from another location, you simply connect it before or after / dis.
; bind -bc $home/dis /dis So if you want to install something, but you cannot write to / dis, install it wherever you want, and connect it over / dis. sudo is not required. The same goes for / lib and / module.
NFS is not needed
I'm going to ignore authentication (the -A flag) and encryption, so we will export something unimportant : memfs (4) again. First car:
; mount -c {memfs -s} /n/mem ; echo something > /n/mem/asdf ; listen -A 'tcp!*!1234' { export /n/mem & } Second car:
; mount -A 'tcp!thefirstmachine!1234' /n/remotememfs ; cat /n/remotememfs/asdf something Since most file server commands can work via stdin / stdout instead of connecting to a mount point, the file system does not even need to be locally mounted to make it accessible over the network:
; styxlisten -A 'tcp!*!1234' {memfs -s} And on the second car:
; mount -Ac 'tcp!thefirstmachine!1234' /n/remotememfs The listen (1) and dial (1) commands are simply convenient wrappers for interacting with the / net file system, which is the interface to the network for Inferno. Sockets in Unix are handled by the BSD sockets library (with the corresponding system calls) and are much more difficult to use. You can think of listen and dial as analogous to netcat. styxlisten is the same, only expects to transmit the 9P2000 protocol (also called Styx until it was accepted into Plan 9) instead of a stream of bytes.
Powerful primitives mean you will never regret
All this is achieved by several short commands, which themselves are just wrappers around Inferno system calls. You have bind (1), mount (1), and unmount (1) . If you need introspection, ns (1) will help you.
Representation of all OS objects as file systems means that programs like cat, sed and echo can be used for, for example, network access using the ip (3) file system. ftpfs (4) allows you to use ls and cp without an interactive client, which makes scripting easier (no need to redirect through an FTP client). In fact, your scripts do not need to know at all that they work with an FTP server and not with a regular file on disk. zipfs allows you to mount a .zip file. By combining them, you can mount .iso, which is inside .zip, which is on the FTP server, and then copy individual files from it.
Somewhere in the future
We are already at an interesting point. Most of the population owns at least a laptop and a smartphone, and, of them, most also own a home computer, have a computer at work, a tablet, etc. In the heyday of Unix, there was one computer and many users. This seems obvious, but we are rapidly approaching the point when the general case will be many computers for one user and not vice versa.
Managing all these resources (disk and CPU, in particular) is already a big problem. File synchronization, contact synchronization, etc. If your phone does not have enough resources to process something, you need to go to a regular computer (physically or via ssh). If you want to watch a movie on your phone, you need to copy it there. What do you do if you read the article on the phone and decide to continue reading on the laptop? Do you send email or IM? (This is still better than DBUS, but this is a digression.)
I do not think that this will continue. As a temporary solution, you can give all your data to Google, Facebook, or any “cloud” service, but this solution leaves much to be desired (and listing the reasons is another digression).
Software solution to the problem of managing computational resources and combining all these machines (local and remote) is inevitable. At least at the moment, I see Inferno as the best candidate for the platform that will make all this possible. If this does not fully explain why I suddenly fell in love with this system, then at least I do it partially.
To be continued
This description is very superficial. “Everything is a file system” is only one of the great ideas on which Inferno is built.
Source: https://habr.com/ru/post/146076/
All Articles