Tale about how GOST-disk encryption in Android implemented
Introduction
In the summer of 2015, we, the developers of the Security Code, were faced with the task of implementing secure storage for Android with encryption according to standards recognized by Russian law. Before that, we already had a solution on a tablet for which there were Android sources. And it allowed us to release updated nuclear encryption (dm-crypt) under the support of GOST 89, add the GOST library to the / system / lib, patch the cryptofs subsystem of the vold daemon. As a result, we had a solution that was only suitable for a specific tablet model, and was not universal. Having learned that in Android version 4.4 (API level 19), an API appeared that allows accessing data after registering and implementing its custom DocumentsProvider, we decided to create a solution using GOST-encryption in userspace using this API, which would not depend on the model devices.
For those who are interested, welcome under cat.
How Android data is encrypted
')
I will briefly describe the process of encrypting an Android device. Encryption is activated as follows: Settings → Security → Encrypt tablet / phone, implemented by means of linux kernel → dev-mapper → dm-crypt. When activating this functionality, the device will ask for a password, as well as ask to save and encrypt data from / data (or simply delete all data as desired by the user). After the device is rebooted, a password entry window will appear, in the system settings the variable ro.crypto.state will be set to encrypted.
Standard partition encryption in Android is based on the FDE (full-disk encryption) model. The model is the usual Linux kernel subsystem - device mapper and provides transparent encryption. Access to data (for example, to the / data section) is provided via the virtual block device / dev / block / dm-0, which each io-request encrypts or decrypts in AES-CBC mode on a 128-bit sector key, which is calculated from IV ( initialization vector).
Fig. 1: Android encryption logic
The diagram shows the sequence of requesting a Pin from the key container and setting the master-key via syscall ioctl to the dm-crypt core.
Access framework (SAF)
To use the application in the java-space, we use the Storage Access Framework (SAF), or the storage access platform. The site developer.android.com describes in some detail all the properties and nuances of using this platform, here we only mention the main points.
So, SAF appeared in Android version 4.4 (API level 19) and it makes it easier for users to search and open documents, images and other files in the repositories of all suppliers. We can become a supplier of these files by implementing the DocumentsProvider class and using several of its methods (see below).
The entire SAF platform consists of the following elements:
1) "Client" - an application that wants to access the file or create a new file. For the entire Android system, this is possible using intents with the ACTION_OPEN_DOCUMENT and ACTION_CREATE_DOCUMENT flags, respectively.
2) “Document Provider” is our application, which, as mentioned above, is a subclass of the DocumentsProvider class and implements access to files via the API (more on this later).
3) “Element of choice” - a system user interface that provides users with access to the files of all providers that meet the search criteria.
Now more. The document provider provides one or more root directories that are starting points for traversing the document tree. Each root directory has a unique identifier COLUMN_ROOT_ID and points to a document (directory) representing content at a level below the root. Root directories are dynamic in their design, which provides support for various use cases: multiple accounts, temporary storage on USB drives, and login / logout / logout capabilities.
Each storage server displays individual files and directories, referring to them using a unique identifier COLUMN_DOCUMENT_ID (in our case, this is the full path from the root of the file system to the file itself). Document identifiers must be unique and not changed after assignment, since they are used to issue constant URIs that are not dependent on device reloads.
A document is either a file being opened (having a specific MIME type) or a directory containing other documents (with the MIME_TYPE_DIR MIME type). Each document may have different properties, described by the flags COLUMN_FLAGS, such as FLAG_SUPPORTS_WRITE, FLAG_SUPPORTS_DELETE and FLAG_SUPPORTS_THUMBNAIL.
Fig. 2: Document provider data model
The document provider data model is based on the traditional file hierarchy. However, the physical method of data storage is left to the discretion of the developer, provided that they can be accessed through the DocumentsProvider API. For example, you can use tag-based cloud storage for data.
Note the following figure:
Fig. 3: Document provider and customer interaction model
On the SAF platform, suppliers and customers do not interact directly. The client requests permission to interact with the files (to read, edit, create or delete them).
The interaction begins when the application (in our example, the processing of the photo) activates the intention ACTION_OPEN_DOCUMENT or ACTION_CREATE_DOCUMENT. The intention may include filters to refine the criteria, for example, “provide openable files with a MIME image type”.
When the intent is triggered, the system selection item goes to each registered provider and shows the user the root directories with content matching the request. The selection element provides the user with a standard interface, even if document providers vary widely.
In Android version 4.3 and below, an application to activate a file from another application must activate the intent, for example, ACTION_PICK or ACTION_GET_CONTENT. After that, the user must select one of the applications, and it must provide the user with an interface with which he can select and receive files.
Fig. 4: Document provider selection dialog
Fig. 5: Dialogue to choose the recipient of the document
Register as a document provider
The next step in developing your own document provider is to create a subclass of the abstract DocumentsProvider class. At a minimum, you must implement the following methods:
queryRoots ()
queryChildDocuments ()
queryDocument ()
OpenDocument ()
The implementation of these methods is strictly required, however, to implement a full-featured application, you will have to implement others. Details are provided in the description of the DocumentsProvider class.
Jni
The next logical part is the JNI implementation of the layer between our SAF implementation and the open file system library for FATFS embedded systems. The latter was patched to read-write to a file (in our case, it is a volume) block by block. Thus, for an IO request, we get the sector number and the number of sectors, from here we can calculate the IV and sector key, and then encrypt and decrypt a specific block on the file. As a result, we have an analogue of dm-crypt in userspace.
Fig. 6: Forwarding SAF methods to the native part and back
Next, we implemented methods for working with volumes and files. To work with volumes we use the following functions:
Java_com_securitycode_fatfslib_FatFs_createVolume (...);
Creates a volume file through the file system library and creates an empty encrypted FS inside this file.
Fig. 7: Creating Secure Storage
Java_com_securitycode_fatfslib_FatFs_getVolumeFreeSpace (...);
Passing a specific volume to the function, we get the amount of free space in this volume.
Java_com_securitycode_fatfslib_FatFs_mount (...);
When mounting the volume, we install the master key with which the encryption-decryption of the FS blocks will occur.
Figure 8: Mounting secure storage
Java_com_securitycode_fatfslib_FatFs_unMount (...);
Cleaning up resources and completing I / O operations.
To work with files:
Java_com_securitycode_fatfslib_FatFs_createFile (...);
An empty file object is created on our file system with the specified size and type (required for SAF).
Java_com_securitycode_fatfslib_FatFs_getFile (...);
This function is required for the SAF method queryDocument (...).
In the function, we return the file id, its type, the root node containing the current file and the file name.
Java_com_securitycode_fatfslib_FatFs_readFile (...);
This function creates a Java stream, which is used in SAF and passed as a URI to the consumer.
Java_com_securitycode_fatfslib_FatFs_writeFile (...);
Similar to the previous function, only in this case we become the consumer.
Figure 9: Application General Screen
Figure 10: Storage Content
Conclusion
As a result of all the above actions, we received a cryptodisk with support for GOST encryption and electronic signatures in userspace.
We will be happy to answer your questions and tell you in more detail about all the stages of the project.
Sources:
1) Nikolay Elenkov. Android Security Internals, Chapter 10: Device Security.
2) Disk Encryption in Android
source.android.com/security/encryption
3) Documents Provider and SAF
developer.android.com/intl/ru/guide/topics/providers/document-provider.html
4) FatFs - Generic FAT File System Module
elm-chan.org/fsw/ff/00index_e.html
In the summer of 2015, we, the developers of the Security Code, were faced with the task of implementing secure storage for Android with encryption according to standards recognized by Russian law. Before that, we already had a solution on a tablet for which there were Android sources. And it allowed us to release updated nuclear encryption (dm-crypt) under the support of GOST 89, add the GOST library to the / system / lib, patch the cryptofs subsystem of the vold daemon. As a result, we had a solution that was only suitable for a specific tablet model, and was not universal. Having learned that in Android version 4.4 (API level 19), an API appeared that allows accessing data after registering and implementing its custom DocumentsProvider, we decided to create a solution using GOST-encryption in userspace using this API, which would not depend on the model devices.
For those who are interested, welcome under cat.
How Android data is encrypted
')
I will briefly describe the process of encrypting an Android device. Encryption is activated as follows: Settings → Security → Encrypt tablet / phone, implemented by means of linux kernel → dev-mapper → dm-crypt. When activating this functionality, the device will ask for a password, as well as ask to save and encrypt data from / data (or simply delete all data as desired by the user). After the device is rebooted, a password entry window will appear, in the system settings the variable ro.crypto.state will be set to encrypted.
Standard partition encryption in Android is based on the FDE (full-disk encryption) model. The model is the usual Linux kernel subsystem - device mapper and provides transparent encryption. Access to data (for example, to the / data section) is provided via the virtual block device / dev / block / dm-0, which each io-request encrypts or decrypts in AES-CBC mode on a 128-bit sector key, which is calculated from IV ( initialization vector).
Fig. 1: Android encryption logic
The diagram shows the sequence of requesting a Pin from the key container and setting the master-key via syscall ioctl to the dm-crypt core.
Access framework (SAF)
To use the application in the java-space, we use the Storage Access Framework (SAF), or the storage access platform. The site developer.android.com describes in some detail all the properties and nuances of using this platform, here we only mention the main points.
So, SAF appeared in Android version 4.4 (API level 19) and it makes it easier for users to search and open documents, images and other files in the repositories of all suppliers. We can become a supplier of these files by implementing the DocumentsProvider class and using several of its methods (see below).
The entire SAF platform consists of the following elements:
1) "Client" - an application that wants to access the file or create a new file. For the entire Android system, this is possible using intents with the ACTION_OPEN_DOCUMENT and ACTION_CREATE_DOCUMENT flags, respectively.
2) “Document Provider” is our application, which, as mentioned above, is a subclass of the DocumentsProvider class and implements access to files via the API (more on this later).
3) “Element of choice” - a system user interface that provides users with access to the files of all providers that meet the search criteria.
Now more. The document provider provides one or more root directories that are starting points for traversing the document tree. Each root directory has a unique identifier COLUMN_ROOT_ID and points to a document (directory) representing content at a level below the root. Root directories are dynamic in their design, which provides support for various use cases: multiple accounts, temporary storage on USB drives, and login / logout / logout capabilities.
Each storage server displays individual files and directories, referring to them using a unique identifier COLUMN_DOCUMENT_ID (in our case, this is the full path from the root of the file system to the file itself). Document identifiers must be unique and not changed after assignment, since they are used to issue constant URIs that are not dependent on device reloads.
A document is either a file being opened (having a specific MIME type) or a directory containing other documents (with the MIME_TYPE_DIR MIME type). Each document may have different properties, described by the flags COLUMN_FLAGS, such as FLAG_SUPPORTS_WRITE, FLAG_SUPPORTS_DELETE and FLAG_SUPPORTS_THUMBNAIL.
Fig. 2: Document provider data model
The document provider data model is based on the traditional file hierarchy. However, the physical method of data storage is left to the discretion of the developer, provided that they can be accessed through the DocumentsProvider API. For example, you can use tag-based cloud storage for data.
Note the following figure:
Fig. 3: Document provider and customer interaction model
On the SAF platform, suppliers and customers do not interact directly. The client requests permission to interact with the files (to read, edit, create or delete them).
The interaction begins when the application (in our example, the processing of the photo) activates the intention ACTION_OPEN_DOCUMENT or ACTION_CREATE_DOCUMENT. The intention may include filters to refine the criteria, for example, “provide openable files with a MIME image type”.
When the intent is triggered, the system selection item goes to each registered provider and shows the user the root directories with content matching the request. The selection element provides the user with a standard interface, even if document providers vary widely.
In Android version 4.3 and below, an application to activate a file from another application must activate the intent, for example, ACTION_PICK or ACTION_GET_CONTENT. After that, the user must select one of the applications, and it must provide the user with an interface with which he can select and receive files.
Fig. 4: Document provider selection dialog
Fig. 5: Dialogue to choose the recipient of the document
Register as a document provider
The next step in developing your own document provider is to create a subclass of the abstract DocumentsProvider class. At a minimum, you must implement the following methods:
queryRoots ()
queryChildDocuments ()
queryDocument ()
OpenDocument ()
The implementation of these methods is strictly required, however, to implement a full-featured application, you will have to implement others. Details are provided in the description of the DocumentsProvider class.
Jni
The next logical part is the JNI implementation of the layer between our SAF implementation and the open file system library for FATFS embedded systems. The latter was patched to read-write to a file (in our case, it is a volume) block by block. Thus, for an IO request, we get the sector number and the number of sectors, from here we can calculate the IV and sector key, and then encrypt and decrypt a specific block on the file. As a result, we have an analogue of dm-crypt in userspace.
Fig. 6: Forwarding SAF methods to the native part and back
Next, we implemented methods for working with volumes and files. To work with volumes we use the following functions:
Java_com_securitycode_fatfslib_FatFs_createVolume (...);
Creates a volume file through the file system library and creates an empty encrypted FS inside this file.
Fig. 7: Creating Secure Storage
Java_com_securitycode_fatfslib_FatFs_getVolumeFreeSpace (...);
Passing a specific volume to the function, we get the amount of free space in this volume.
Java_com_securitycode_fatfslib_FatFs_mount (...);
When mounting the volume, we install the master key with which the encryption-decryption of the FS blocks will occur.
Figure 8: Mounting secure storage
Java_com_securitycode_fatfslib_FatFs_unMount (...);
Cleaning up resources and completing I / O operations.
To work with files:
Java_com_securitycode_fatfslib_FatFs_createFile (...);
An empty file object is created on our file system with the specified size and type (required for SAF).
Java_com_securitycode_fatfslib_FatFs_getFile (...);
This function is required for the SAF method queryDocument (...).
In the function, we return the file id, its type, the root node containing the current file and the file name.
Java_com_securitycode_fatfslib_FatFs_readFile (...);
This function creates a Java stream, which is used in SAF and passed as a URI to the consumer.
Java_com_securitycode_fatfslib_FatFs_writeFile (...);
Similar to the previous function, only in this case we become the consumer.
Figure 9: Application General Screen
Figure 10: Storage Content
Conclusion
As a result of all the above actions, we received a cryptodisk with support for GOST encryption and electronic signatures in userspace.
We will be happy to answer your questions and tell you in more detail about all the stages of the project.
Sources:
1) Nikolay Elenkov. Android Security Internals, Chapter 10: Device Security.
2) Disk Encryption in Android
source.android.com/security/encryption
3) Documents Provider and SAF
developer.android.com/intl/ru/guide/topics/providers/document-provider.html
4) FatFs - Generic FAT File System Module
elm-chan.org/fsw/ff/00index_e.html
Source: https://habr.com/ru/post/280468/
All Articles