FreeBSD plans to upgrade to ZFSonLinux

Last week, one of the FreeBSD developers Matt Macy put forward a proposal to use the ZFS implementation from the ZFSonLinux project. Thus, all the required changes for FreeBSD will be immediately integrated into the ZFSonLinux codebase. This offer has not been completed, a fork has already been created in which the required improvements are made, and also the approval of ZFSonLinux core developers to include improvements for FreeBSD into a single code base has been obtained.

Is this news positive? Of course! The essence is in the concentration of all the developers involved in the OpenZFS project in one place, in one code base.

Until now, OpenZFS offered as a reference point the repository of the project Illumos, through which the code should be exchanged between all participants of the collaboration (ZFSonLinux, FreeBSD, Illumos, ZFSonOSX). It is worth noting that the compatibility between implementations was maintained at the proper level. But, unfortunately, the porting of changes under the condition that the code is not 100% identical is a very laborious operation.

The main motivation is the de facto full transition of key developers to the ZFSonLinux project. Previously, the main organization supporting the Illumos code base was Delphix. However, at the beginning of this year they decided to migrate to Linux .

The main life takes place in (oddly enough) the ZFSonLinux project. For several years, many new features have appeared in it, the main ones:

• Built-in encryption (with the ability to verify the integrity without a key!)
• Consistent "scrub" - now checking data integrity is carried out in the best way
• Metadata allocation class - can be allocated to separate media (SSD / NVME):
  
  • Metadata
  • Deduplication data
  • Blocks smaller than a given size (for example, files up to 4KB in size)
• Large dnodes - optimizations for systems that generate a large number of metadata (designed specifically for Luster)
• The ARC caching mechanism is redesigned to minimize fragmentation (using the scatter / gather mechanism)
• Vectorization of the RAIDZ code, vectorization of checksums

It should be noted that, along with the new functionality, the FreeBSD developers have noted the inclusion of many stabilization edits and deadlocks in ZFSonLinux, which have also not been ported to other projects.

Hopefully, this step will allow us to further improve the product, as well as concentrate all resources in one single code base.

Cheers, comrades!

PS Why you might be interested in ZFS.