WASI Standard: Running WebAssembly Outside the Web
March 27, we in Mozilla announced the start of standardization of WASI, the WebAssembly system interface (WebAssembly system interface).
Why: developers started using WebAssembly outside of the browser, because WASM provides a fast, scalable, secure way to run the same code on all machines. But we do not yet have a solid foundation for such development. Outside the browser, you need a way to communicate with the system, that is, a system interface. And the WebAssembly platform does not have it yet.
What: WebAssembly is an assembler for a conceptual, not a physical machine. It works on different architectures, so the system interface is needed for the conceptual OS to work on different operating systems.
')
This is what WASI is: this is the system interface for the WebAssembly platform.
We strive to create a system interface that will become a true companion for WebAssembly with maximum portability and security.
Who: in the framework of the WebAssembly development group, we organized a subgroup that will be engaged in WASI standardization. We have already gathered interested partners and are looking for new ones.
Here are some reasons why we, our partners and supporters consider this important:
Sean White, Mozilla's R & D director:
Tyler McMullen, CTO Fastly:
Miles Borins, Technical Director of the Node Steering Committee:
Laurie Voss, co-founder of npm:
So this is a big event!
Currently there are three WASI implementations:
WASI demonstration in action:
Next, we will talk about the Mozilla proposal, how this system interface should work.
Many say that languages ​​like C give direct access to system resources. But it is not so. On most systems, these languages ​​do not have direct access to such things as opening or creating files. Why not?
Because these system resources — files, memory, and network connections — are too important for stability and security.
If one program accidentally spoils the resources of another, it may cause a crash. Worse, if a program (or user) specifically invades other people's resources, it can steal confidential data.
Therefore, we need a way to control which programs and users can access resources. System developers for a long time have come up with a way to provide such control: rings of protection.
With OS protection rings, it essentially sets a security barrier around system resources. This is the core. Only it can perform operations such as creating a file, opening a file, or opening a network connection.
User programs run outside the kernel in what is called user space. If a program wants to open a file, it must ask the kernel.
This is where the concept of a system call arises. When a program needs to ask the kernel for some operation, it sends a system call. The kernel checks the accessing user and sees if he has rights to access this file.
On most devices, the only way to access system resources is through system calls.
The operating system provides access to system calls. But if each OS has its own system calls, don't they need to write different versions of the code? Fortunately, no. The problem is solved with the help of abstraction.
Most languages ​​have a standard library. When coding, the programmer does not need to know for which system he is writing. It just uses the interface. Then, when compiling, your tool chain chooses which interface implementation to use for which system. This implementation uses functions from the operating system API, so it is specific to it.
This is where the concept of a system interface appears. For example, if you compile
However, this creates a problem for WebAssembly. Here we do not know for which OS to optimize the program even at compile time. Thus, you cannot use the system interface of any single OS within the implementation of the standard library on WebAssembly.
I have already said that WebAssembly is an assembler for a conceptual machine , not a real machine. Similarly, WebAssembly needs a system interface for a conceptual, not a real OS.
But there are already runtimes that can run WebAssembly outside of the browser, even without this system interface. How do they do it? Let's get a look.
The first tool to generate WebAssembly code was Emscripten. It emulates a certain OS system interface on the web - POSIX. This means that a programmer can use functions from the standard C library (libc).
For this, Emscripten uses its own libc implementation. It is divided into two parts: the first is compiled into the WebAssembly module, and the other is implemented in the “JS-glue” code. This JS glue sends calls to the browser that talks to the OS.
The bulk of the early WebAssembly code is compiled with Emscripten. Therefore, when people began to want to run WebAssembly without a browser, they started to run Emscripten code.
So in these rantaymah should create own implementations for all functions which were in the JS-glue code.
But there is a problem. The interface provided by the JS glue code was not designed as a standard or even a public interface. For example, for a call like
The first parameter
The second parameter,
For Emscripten in the browser, this is normal. But now runtimes see this as a de facto standard, implementing their own versions of JS glue. They emulate the internal details of the POSIX emulation layer.
This means that they re-implement the code (for example, pass arguments as heap values), which made sense given the Emscripten restrictions, but in these runtimes there are no such restrictions.
If we build the WebAssembly ecosystem for decades to come, it needs a solid foundation, not crutches. This means that our actual standard cannot be an emulation emulation.
But what principles apply in this case?
Two fundamental principles of WebAssembly:
We go beyond the browser, but keep these key principles.
However, the POSIX approach and the Unix access control system do not give us the desired result. Let's see what the problem is.
POSIX provides source code portability. You can compile the same source code with different versions of libc for different computers.
But WebAssembly must go beyond that. We need to compile once to run on a whole bunch of different systems. We need portable binaries.
This simplifies the distribution of the code.
For example, if the native Node modules are written in WebAssembly, then users do not need to run node-gyp when installing applications with native modules, and developers do not need to configure and distribute dozens of binary files.
When code asks the operating system to do input or output, the OS must evaluate the security of this operation, usually using an access control system based on ownership and groups.
For example, the program asks to open the file. A user has a specific set of files to which he has access.
When a user starts a program, the program starts on behalf of that user. If the user has access to the file — either he is its owner, or belongs to a group that has access to the file — then the program has the same access.
This protects users from each other, which made sense in the old days, when many people worked on the same computer, and administrators controlled the software. Then the main threat was other users looking into your files.
Everything has changed. Currently, systems, as a rule, are single-user, but use third-party code of unknown reliability. Now the main threat comes from the code that you run yourself.
For example, for the library in your application, a new maintainer started (as often happens in open source). He may be a sincere activist ... or an attacker. And if it has access to your system — for example, the ability to open any file and send it over the network — then this code can cause great damage.
Suspicious application : I work for user Bob. Can I open his Bitcoin wallet?
Core : For Bob? Of course!
Suspicious app : Great! What about the network connection?
This is why using third-party libraries is dangerous. In WebAssembly, security is assured otherwise — through a sandbox. Here the code cannot talk directly to the OS. But how then to address to system resources? The host (browser or runtime wasm) sandboxes the functions that the code can use.
This means that the host programmatically limits the functionality of the program, not allowing you to simply act on behalf of the user, causing any system calls with full user rights.
Having a sandbox in and of itself does not make the system safe — the host can still transfer full functionality to the sandbox, in which case it does not provide any protection. But the sandbox gives at least a theoretical opportunity for hosts to build a more secure system.
WA : Please, here are some safe toys for interacting with the OS (safe_write, safe_read).
Suspicious application : Oh damn ... and where is my network access?
In any system interface you need to adhere to these two principles. Portability facilitates the development and distribution of software, and tools to protect the host and users are absolutely essential.
Given these two key principles, what should the WebAssembly system interface be?
This we will find out in the process of standardization. However, we have a proposal to start:
What will happen in wasi-core? These are the basics needed by all programs. The module will cover most of the POSIX functionality, including files, network connections, clocks, and random numbers.
Much of the basic functionality will require a very similar approach. For example, a POSIX file-based approach is provided with open, close, read, and write system calls, and the rest is the additions from above.
But wasi-core will not cover all POSIX functionality. For example, the concept of a process does not fit well into a WebAssembly. In addition, it is clear that each WebAssembly engine must support process operations, such as
Languages ​​like Rust will use wasi-core directly in their standard libraries. For example,
For C and C ++, we created wasi-sysroot , which implements libc in terms of the wasi-core functions.
We expect compilers, such as Clang, to interact with the WASI API, and complete tool chains, such as the Rust compiler and Emscripten, will use WASI as part of their system implementations.
How does custom code call these WASI functions?
The runtime in which the code is executed transfers the wasi-core function by placing the object in the sandbox.
This provides portability, because each host can have its own wasi-core implementation specifically for its platform: from WebAssembly runtimes, such as Mozilla Wasmtime and Fastly Lucet, to Node or even a browser.
It also provides reliable isolation, because the software-based host selects which functions of the wasi-core to sandbox, that is, which system calls to allow it. This is safety.
WASI allows you to enhance and expand security by introducing a power-based protection concept into the system.
Usually, if the code needs to open a file, it calls
In the case of WASI, when calling a function to access a file, you must pass a file descriptor to which permissions are attached for the file itself or for the directory containing the file.
Thus, you cannot have code that accidentally asks you to open
This allows you to safely allow different system calls to isolated code, since the capabilities of these system calls are limited.
And so in every module. By default, the module does not have access to file descriptors. But if the code in one module has a file descriptor, it can transfer it to functions called in other modules. Or create more limited versions of the file descriptor for transfer to other functions.
Thus, the runtime passes file descriptors that an application can use in top-level code, and then file descriptors are distributed to the rest of the system as needed.
This brings WebAssembly closer to the principle of least privilege, where the module gets access only to the minimum set of resources needed to perform its work.
This concept comes from authority-based security, as in CloudABI and Capsicum. One of the problems of these systems is difficult code portability. But we believe that this problem can be solved.
If the code is already using
If the code uses
We believe that wasi-core is a good start. It preserves the portability and security of WebAssembly, providing a solid foundation for an ecosystem.
But after the full standardization of the wasi-core, other issues need to be resolved, including:
This is just the beginning, so if you have ideas, get involved !
Why: developers started using WebAssembly outside of the browser, because WASM provides a fast, scalable, secure way to run the same code on all machines. But we do not yet have a solid foundation for such development. Outside the browser, you need a way to communicate with the system, that is, a system interface. And the WebAssembly platform does not have it yet.
What: WebAssembly is an assembler for a conceptual, not a physical machine. It works on different architectures, so the system interface is needed for the conceptual OS to work on different operating systems.
')
This is what WASI is: this is the system interface for the WebAssembly platform.
We strive to create a system interface that will become a true companion for WebAssembly with maximum portability and security.
Who: in the framework of the WebAssembly development group, we organized a subgroup that will be engaged in WASI standardization. We have already gathered interested partners and are looking for new ones.
Here are some reasons why we, our partners and supporters consider this important:
Sean White, Mozilla's R & D director:
“WebAssembly is already changing the way new kinds of attractive content are delivered to people. It helps developers and content creators. Until now, everything worked through browsers, but with WASI the advantages of WebAssembly will get more users and more devices in different places. ”
Tyler McMullen, CTO Fastly:
“We see WebAssembly as a platform for fast and secure code execution on the edge cloud. Despite the different environments (edge ​​and browsers), thanks to WASI, you don’t have to port the code to each platform. ”
Miles Borins, Technical Director of the Node Steering Committee:
“WebAssembly can solve one of Node’s biggest challenges: how to get close to native speed and reuse code written in other languages, such as C and C ++, while maintaining portability and security. Standardizing WASI is the first step to this. ”
Laurie Voss, co-founder of npm:
“Npm is extremely excited by the potential of WebAssembly for the npm ecosystem, since it makes it much easier to get native code to run in server-side JavaScript applications. We look forward to the results of this process. ”
So this is a big event!
Currently there are three WASI implementations:
- wasmtime , Mozilla WebAssembly runtime
- Lucet , Fastly WebAssembly Runtime
- polyfill browser
WASI demonstration in action:
Next, we will talk about the Mozilla proposal, how this system interface should work.
What is a system interface?
Many say that languages ​​like C give direct access to system resources. But it is not so. On most systems, these languages ​​do not have direct access to such things as opening or creating files. Why not?
Because these system resources — files, memory, and network connections — are too important for stability and security.
If one program accidentally spoils the resources of another, it may cause a crash. Worse, if a program (or user) specifically invades other people's resources, it can steal confidential data.
Therefore, we need a way to control which programs and users can access resources. System developers for a long time have come up with a way to provide such control: rings of protection.
With OS protection rings, it essentially sets a security barrier around system resources. This is the core. Only it can perform operations such as creating a file, opening a file, or opening a network connection.
User programs run outside the kernel in what is called user space. If a program wants to open a file, it must ask the kernel.
This is where the concept of a system call arises. When a program needs to ask the kernel for some operation, it sends a system call. The kernel checks the accessing user and sees if he has rights to access this file.
On most devices, the only way to access system resources is through system calls.
The operating system provides access to system calls. But if each OS has its own system calls, don't they need to write different versions of the code? Fortunately, no. The problem is solved with the help of abstraction.
Most languages ​​have a standard library. When coding, the programmer does not need to know for which system he is writing. It just uses the interface. Then, when compiling, your tool chain chooses which interface implementation to use for which system. This implementation uses functions from the operating system API, so it is specific to it.
This is where the concept of a system interface appears. For example, if you compile
printf for a Windows machine, it will use the Windows API. If compiled for Mac or Linux, it uses POSIX.However, this creates a problem for WebAssembly. Here we do not know for which OS to optimize the program even at compile time. Thus, you cannot use the system interface of any single OS within the implementation of the standard library on WebAssembly.
I have already said that WebAssembly is an assembler for a conceptual machine , not a real machine. Similarly, WebAssembly needs a system interface for a conceptual, not a real OS.
But there are already runtimes that can run WebAssembly outside of the browser, even without this system interface. How do they do it? Let's get a look.
How does WebAssembly work now outside the browser?
The first tool to generate WebAssembly code was Emscripten. It emulates a certain OS system interface on the web - POSIX. This means that a programmer can use functions from the standard C library (libc).
For this, Emscripten uses its own libc implementation. It is divided into two parts: the first is compiled into the WebAssembly module, and the other is implemented in the “JS-glue” code. This JS glue sends calls to the browser that talks to the OS.
The bulk of the early WebAssembly code is compiled with Emscripten. Therefore, when people began to want to run WebAssembly without a browser, they started to run Emscripten code.
So in these rantaymah should create own implementations for all functions which were in the JS-glue code.
But there is a problem. The interface provided by the JS glue code was not designed as a standard or even a public interface. For example, for a call like
read in a normal API, the JS-glue code uses the _system3(which, varargs) call _system3(which, varargs) .The first parameter
which is an integer that always coincides with the number in the name (in our case 3).The second parameter,
varargs lists the arguments. It is called varargs , because we can have a different number of arguments. But WebAssembly does not allow a variable number of arguments to be passed to the function. Therefore, they are transmitted through linear memory, which is not safe and slower than through registers.For Emscripten in the browser, this is normal. But now runtimes see this as a de facto standard, implementing their own versions of JS glue. They emulate the internal details of the POSIX emulation layer.
This means that they re-implement the code (for example, pass arguments as heap values), which made sense given the Emscripten restrictions, but in these runtimes there are no such restrictions.
If we build the WebAssembly ecosystem for decades to come, it needs a solid foundation, not crutches. This means that our actual standard cannot be an emulation emulation.
But what principles apply in this case?
What principles should the WebAssembly system interface follow?
Two fundamental principles of WebAssembly:
- portability
- security
We go beyond the browser, but keep these key principles.
However, the POSIX approach and the Unix access control system do not give us the desired result. Let's see what the problem is.
Portability
POSIX provides source code portability. You can compile the same source code with different versions of libc for different computers.
But WebAssembly must go beyond that. We need to compile once to run on a whole bunch of different systems. We need portable binaries.
This simplifies the distribution of the code.
For example, if the native Node modules are written in WebAssembly, then users do not need to run node-gyp when installing applications with native modules, and developers do not need to configure and distribute dozens of binary files.
Security
When code asks the operating system to do input or output, the OS must evaluate the security of this operation, usually using an access control system based on ownership and groups.
For example, the program asks to open the file. A user has a specific set of files to which he has access.
When a user starts a program, the program starts on behalf of that user. If the user has access to the file — either he is its owner, or belongs to a group that has access to the file — then the program has the same access.
This protects users from each other, which made sense in the old days, when many people worked on the same computer, and administrators controlled the software. Then the main threat was other users looking into your files.
Everything has changed. Currently, systems, as a rule, are single-user, but use third-party code of unknown reliability. Now the main threat comes from the code that you run yourself.
For example, for the library in your application, a new maintainer started (as often happens in open source). He may be a sincere activist ... or an attacker. And if it has access to your system — for example, the ability to open any file and send it over the network — then this code can cause great damage.
Suspicious application : I work for user Bob. Can I open his Bitcoin wallet?
Core : For Bob? Of course!
Suspicious app : Great! What about the network connection?
This is why using third-party libraries is dangerous. In WebAssembly, security is assured otherwise — through a sandbox. Here the code cannot talk directly to the OS. But how then to address to system resources? The host (browser or runtime wasm) sandboxes the functions that the code can use.
This means that the host programmatically limits the functionality of the program, not allowing you to simply act on behalf of the user, causing any system calls with full user rights.
Having a sandbox in and of itself does not make the system safe — the host can still transfer full functionality to the sandbox, in which case it does not provide any protection. But the sandbox gives at least a theoretical opportunity for hosts to build a more secure system.
WA : Please, here are some safe toys for interacting with the OS (safe_write, safe_read).
Suspicious application : Oh damn ... and where is my network access?
In any system interface you need to adhere to these two principles. Portability facilitates the development and distribution of software, and tools to protect the host and users are absolutely essential.
What should such a system interface look like?
Given these two key principles, what should the WebAssembly system interface be?
This we will find out in the process of standardization. However, we have a proposal to start:
- Creating a modular set of standard interfaces
- Let's start with the standardization of the main module wasi-core
What will happen in wasi-core? These are the basics needed by all programs. The module will cover most of the POSIX functionality, including files, network connections, clocks, and random numbers.
Much of the basic functionality will require a very similar approach. For example, a POSIX file-based approach is provided with open, close, read, and write system calls, and the rest is the additions from above.
But wasi-core will not cover all POSIX functionality. For example, the concept of a process does not fit well into a WebAssembly. In addition, it is clear that each WebAssembly engine must support process operations, such as
fork . But we also want to make fork standardization possible.Languages ​​like Rust will use wasi-core directly in their standard libraries. For example,
open from Rust is implemented when compiling into WebAssembly by calling __wasi_path_open .For C and C ++, we created wasi-sysroot , which implements libc in terms of the wasi-core functions.
We expect compilers, such as Clang, to interact with the WASI API, and complete tool chains, such as the Rust compiler and Emscripten, will use WASI as part of their system implementations.
How does custom code call these WASI functions?
The runtime in which the code is executed transfers the wasi-core function by placing the object in the sandbox.
This provides portability, because each host can have its own wasi-core implementation specifically for its platform: from WebAssembly runtimes, such as Mozilla Wasmtime and Fastly Lucet, to Node or even a browser.
It also provides reliable isolation, because the software-based host selects which functions of the wasi-core to sandbox, that is, which system calls to allow it. This is safety.
WASI allows you to enhance and expand security by introducing a power-based protection concept into the system.
Usually, if the code needs to open a file, it calls
open with the path name in the string. The OS then checks if the code has the right to such an action (based on the rights of the user running the program).In the case of WASI, when calling a function to access a file, you must pass a file descriptor to which permissions are attached for the file itself or for the directory containing the file.
Thus, you cannot have code that accidentally asks you to open
/etc/passwd . Instead, the code can only work with its directories.This allows you to safely allow different system calls to isolated code, since the capabilities of these system calls are limited.
And so in every module. By default, the module does not have access to file descriptors. But if the code in one module has a file descriptor, it can transfer it to functions called in other modules. Or create more limited versions of the file descriptor for transfer to other functions.
Thus, the runtime passes file descriptors that an application can use in top-level code, and then file descriptors are distributed to the rest of the system as needed.
This brings WebAssembly closer to the principle of least privilege, where the module gets access only to the minimum set of resources needed to perform its work.
This concept comes from authority-based security, as in CloudABI and Capsicum. One of the problems of these systems is difficult code portability. But we believe that this problem can be solved.
If the code is already using
openat with relative file paths, compiling the code will simply work.If the code uses
open and the openat-style migration is too sharp, WASI will provide an incremental solution. With libpreopen, you create a list of file paths to which the application has legitimate access. Then use open , but only with these paths.What's next?
We believe that wasi-core is a good start. It preserves the portability and security of WebAssembly, providing a solid foundation for an ecosystem.
But after the full standardization of the wasi-core, other issues need to be resolved, including:
- asynchronous I / O
- file monitoring
- file lock
This is just the beginning, so if you have ideas, get involved !
Source: https://habr.com/ru/post/446764/
All Articles