Samba4 + GlusterFS

Many have already encountered such a thing as GlusterFS, someone is still not in the know, today I invite you all to get acquainted with this project, or rather the possibility of using GlusterFS together with Samba4 to build a distributed, linearly scalable file storage with protection against failures .

This article will discuss several possibilities of using GlusterFS with Samba4, as well as analyzing the performance of each option.

I use SLES11 SP3, all the examples will be based on it, but I think in other distributions everything is done in much the same way. GlusterFS can be downloaded from the official website of the project, there are already compiled assemblies for all major Linux server distributions.

---

The example uses GlusterFS 3.4.2 and Samba 4.1.6.

---

With samba4, it's not that simple. You can build samba4 yourself, but then do not forget to build a package with the vfs glusterfs module when building. I went a simpler way, namely I recompiled the assembly from Sernet, since In this case, it is not necessary to write init scripts, and the structure of the samba files is right, it is not required when compiling to specify the script what and where to put. The assembly can be obtained on the Sernet portal after registration.

And so, we need to ultimately get samba4 with the vfs glusterfs module, install sernet-samba-4.1.6-7.src.rpm, edit the spec file sernet-samba-4.1.spec, in which you need to do the following:
Looking for a string

%define vfs_modules_common vfs_audit,vfs_cap,vfs_catia,vfs_cacheprime,vfs_expand_msdfs,vfs_extd_audit,vfs_fake_perms,vfs_netatalk,vfs_recycle,vfs_streams_depot,vfs_aio_fork 

add at the end

 ,vfs_glusterfs 

Looking for a column

 CONF_OPTS="\ 

Add to it

 --enable-glusterfs \ 

Next, run the rpm build on our spec file.

 rpmbuild -bb sernet-samba-4.1.spec 

At the output, we get the rpm sernet-samba we need, which already include the module we need.
')
Samba4 can be used with glusterfs in any role, be it a domain controller or just a classic file server. There are several ways to use these two products in conjunction, with one of the two paths being for some reason much more popular, although slower than the other.

I will not consider in this material a step-by-step plan for creating a storage facility for glusterfs, this has already been done more than once before me at Habré, and there is nothing difficult in this process. You can refer to the official instructions or search for material on habrahabr . We assume that you already have a local volume called samba4_volume.

And so, the first option, the easiest, but also the slowest, just mount the glusterfs volume allow in / data and point the ball to this path in smb.conf.
In the file / etc / fstab add

 localhost:/samba4_volume /data glusterfs defaults,_netdev 0 0 

How to add the ball in smb.conf I think no need to explain.
In this case, by the way, you don’t need a vfs module at all, and what is described at the beginning of the article can be avoided.

Option two, more productive.
Use the vfs module and connect the volume directly to samba4.
An example of the smb.conf part we need

 [data] comment = For samba export of volume test vfs objects = glusterfs glusterfs:volfile_server = localhost glusterfs:volume = samba4_volume kernel share modes = no path = / read only = no guest ok = yes 

Accordingly, if you have samba4 compiled without this module, this option will not work.

It's time to check the speed of each option. At first, I checked the speed of copying over the network to the ball of one large file, then I checked the copying of the windows folder (a bunch of small files), I did not notice the percentage difference in speed between different options, so I decided not to take screenshots, steady speed when copying one large file more visual in my opinion.

First we test the speed of work directly with the ext4 file system, without glusterfs.



The speed of about 50 MB / s on a gigabit network is quite a normal result.

Check the speed when using the first option. Volume glusterfs is simply mounted via / etc / fstab



The speed immediately collapsed twice from the standard, a terrible result in general ...

Check the speed when using the vfs module



The speed sags one and a half times from the standard.

In fact, as we see a problem with the performance of GlusterFS at the moment there are anyway, if you decide to transfer your corporate storage to GlusterFS having one or more replication nodes, bear in mind that the performance of such storage will in any case decrease. Yes, the speed can be raised if you add Striped nodes, but this requires iron resources, which not everyone has. In my case, I wanted to use replication to another server for fault tolerance, seeing the state of affairs while I paused this project.

The developers promise to increase productivity in the new version 3.5, it remains only to wait ...