Optimization of mail storage in the Zimbra Collaboration Suite

In one of our previous articles on infrastructure planning when deploying to the Zimbra Collabortion Suite enterprise, it was said that the main limitation with the operation of this solution is the I / O speed of disk devices in mail storages. And indeed, at the time when several hundred employees of the enterprise simultaneously access the same mail storage, the width of the channel for recording and reading information from hard disks may not be enough for responsive service. And if for small Zimbra installations this does not become a particular problem, then in the case of large enterprises and SaaS providers all this can lead to unresponsive e-mail and, as a result, a decrease in employee efficiency, as well as a violation of the SLA. That is why, when designing and operating large-scale Zimbra installations, special attention should be paid to optimizing the performance of hard disks in mail storage. Let's look at two cases and try to figure out which methods to optimize the load on disk storage can be applied in each of them.



1. Optimization in the design of large-scale installation Zimbra
')
At the design stage of a high-loaded Zimbra installation, its administrator will have to choose which storage system to use. In order to decide on this issue, you should be aware that the main load on hard drives is created by the MariaDB DBMS that is part of the Zimbra Collaboration Suite, the Apache Lucene search system, and the BLOB storage. That is why for the operation of these software products under high loads, you must use high-speed and reliable equipment.

Under normal conditions, Zimbra can be installed both on RAID from hard drives and on storage connected via the NFS protocol. In cases of very small installations, you can install Zimbra on a regular SATA disk. However, in the conditions of large installations, all these technologies demonstrate various shortcomings in the form of reduced write speed or low reliability, which is unacceptable neither for large enterprises, nor, especially, for SaaS providers.

That is why in the conditions of large-scale infrastructures Zimbra would be best to use SAN. It is she who is currently capable of providing the greatest throughput for storage devices and at the same time, due to the possibility of connecting a large amount of cache, its use practically carries no significant risks to the enterprise. A good idea would be to use NVRAM, which is used in many SANs to speed up work during recording. But it is better to disable caching of the recorded data on the disks themselves, since it can lead to irreparable damage to the media and data loss in the event of power problems.

With regard to the choice of the file system, the optimal choice would be to use standard for Linux Ext3 / Ext4. The main nuance associated with the file system is that it should be mounted with the -noatime option . This option will disable the function of fixing the time of the last access to files, which means that it will greatly reduce the load on reading and writing. In general, when creating an ext3 or ext4 file system for Zimbra, use the following mke2fs utility parameters:

-j - To create a journal file system Create a file system with an ext3 / ext4 journal.
-L TITLE - To create a volume name, then use it in / etc / fstab
-O dir_index - To use a hashed search tree to speed up the search for files in large directories
-m 2 - To reserve 2% of the volume in large file systems under the root directory
-J size = 400 - To create a large log
-b 4096 - To determine block size in bytes
-i 10240 - For a message store, this parameter should correspond to the average message size. You should be attentive to this parameter, since subsequently its value cannot be changed.

In addition, it is recommended to enable dirsync for blob storage, Lucene search meta data storage and MTA queue storage. This should be done for the reason that Zimbra usually uses the fsync utility to ensure that the data blob is written to disk. However, when a Zimbra mail repository or MTA creates new files during message delivery, it becomes necessary to write changes to the disk in the appropriate folders. That is why even in the case when the file is already written to disk using fsync , the record of its addition to the directory may not have time to write to the disk and as a result may be lost due to a sudden server failure. By using dirsync, these problems can be avoided.

2. Optimization with running Zimbra infrastructure

It often happens that after several years of operation Zimbra the number of its users increases significantly and the work of the service becomes less and less responsive every day. The way out of this situation is obvious: you just need to add new servers to the infrastructure so that the service works again as quickly as before. Meanwhile, it is far from always possible to immediately add new servers to the infrastructure in order to increase its speed. Often, IT managers have a long time to coordinate the acquisition of new servers with the accounting department or security department, in addition, suppliers often fail, who can bring a new server late or not deliver what they need.

Of course, the best thing is to build your Zimbra infrastructure with a margin to always have a margin for its expansion and not to depend on anyone, but if the error has already been made, the IT manager can only smooth out its consequences as much as possible. For example, an IT manager can achieve small productivity gains by using a temporary shutdown of Linux system services, which, when running, regularly access hard drives and, as a result, can adversely affect the speed of Zimbra. So, for a while you can turn off:

autofs, netfs - Remote File System Discovery Services
cups - Print Service
xinetd, vsftpd - Built-in * NIX services that you probably won't need
portmap, rpcsvcgssd, rpcgssd, rpcidmapd - Remote procedure call services that are commonly used in conjunction with network file systems
dovecot, cyrus-imapd, sendmail, exim, postfix, ldap - Duplicates of the main utilities that make up the Zimbra Collaboration Suite
slocate / updatedb - Since Zimbra stores each message in a separate file, the updatedb service starts every day, it can cause problems, and therefore you can do it manually during the lowest load on the servers

Saving system resources as a result of disabling these services will not be very significant, but even this can be very useful in conditions close to force majeure. After the new server is added to the Zimbra infrastructure, it is recommended to re-enable previously disabled services.

You can also optimize the work of Zimbra due to the removal of the syslog service on a separate server so that it does not load the mail storage hard drives during operation. For these purposes, almost any computer is suitable, up to the cheap Raspberry Pi single-board device.