Upspin: Google's new global file system

Recently it became known about the new project Upspin , which is being developed by a group of engineers from Google. The main task of the project participants is to create a framework to ensure secure file sharing. Upspin, in short, defines a set of protocols, interfaces, and other software components, which allows you to link together various data, including file systems and storage services. Written framework in the language of Go. It is distributed under the BSD license. It is worth noting that the project itself is not the official development of Google, but a “hobby” of employees.

Regardless of where the files are located, Upspin unifies access to them, so you can get data from almost anywhere in the global network. According to the developers, their development is an attempt to solve the problem of the current fragmentation of various program elements for sharing files. As a result, a user who works with different platforms needs to spend time performing intermediate downloads, saving and repacking files.

To better understand the problem itself, you can imagine the situation when a user wants to post a photo that is uploaded to Facebook to another social network, for example, in Vkontakte. In this case, you have to upload the image first to a PC or phone, and then resave to another social network. To transfer a file to another user or transfer from a smartphone to a remote PC, you need to upload information to the “cloud” of any available file service. The user's control over the file disappears immediately after the data reaches the server of the selected service. To preserve the security of the data in this case you have to encrypt.
')
There are three main components in the system:

• Key server that stores the public keys of all users of the system. Private keys are stored on the user side;
• The storage server where the user data is located. Each user can start one or more storage servers;
• A directory server that provides information about data types on storage servers. The directory server can be launched together with the storage server.

In order to identify the file, Upspin developers suggest using a scheme like “ann@example.com/dir/file”, which consists of a user's email and a virtual path. If the global identifier of the file is known, then any user, having gained access, will be able to access this file from almost any local application. This will work like accessing a regular file in your own file system or any of their compatible with the online services framework.

The advantage of the project can also be called the provision of access to dynamically changing information, whether it be data from sensors or the results of a query to network services. To attach the global namespace to the local file system, the user receives a FUSE module. The second option is the upspin program with a set of typical commands like "cf". In order to make a connection to the global file system, you just need to install one of the Upspin implementations, create keys for email and register them in the keystore.

Access can be controlled by creating a special file in the exported directory, called Access, which lists all the permissions granted to other users. As a result, by adding a rule like “read: joe@here.com, mae@there.com” to the file, users joe@here.com and mae@there.com can read the files in the current directories and subdirectories. In addition, the framework provides the ability to provide access to a record or view the contents of a directory, delete, create a file. You can create individual group policies using templates or you can define individual rules for access to different files.

According to the developers, one of the main goals is to ensure the maximum level of security. As for performance, it plays a minor role here. To ensure the security of user data, methods are used to identify users by their public keys. One of them is Key Transparency with a centralized key.upspin.io key server. Initially, content verification using a digital signature is applied, and all data is transmitted in encrypted form using end-to-end shipper. This allows a lot, including deploying your own Upspin server, storing files in any cloud storage.