Migrate to Zimbra with imapsync
In addition to such long-awaited features as the Hierarchical Address Book and official support for Ubuntu 18.10 LTS, the release of Zimbra 8.8.12 brought us news of the impending withdrawal from support of a number of utilities needed for migration from other mail systems to the Zimbra Collaboration Suite. In particular, support for the ZCS Migration Wizard for Domino has been completely discontinued, and the ZCS Migration Wizard for Exchange will now only support the import of PST files from Outlook. To carry out a full-fledged migration, the Zimbra Collaboration Suite developers offer to use paid online services, but not every company is ready to provide their mailbox data to third parties for security reasons, and not every company can afford the additional costs associated with migration to Zimbra. Especially for such users, there is the imapsync program, which is able to provide reliable incremental migration from any mail system to Zimbra.
Imapsync is a free and free utility available in the repositories of any officially supported Zimbra Linux distribution. The purpose of this program is to transfer e-mail between different mail servers and it copes with it perfectly. Due to the flexibility of the settings, mail transfer can be done incrementally and in fully automatic mode. Let's see how you can use imapsync to migrate from Lotus Domino and MS Exchange to the Zimbra Collaboration Suite.
')
In order for email migration to succeed, you need to complete a number of preparatory activities. First of all, we need to create on the Zimbra server the same users as on the MS Exchange server with the same passwords. Carrying out this task can be very useful auto-tuning users Zimbra using AD. The process of such autotune is described in one of our previous articles.
After all accounts from AD or Exchange are migrated, you need to install the imapsync package on the server with Zimbra. If you are using Ubuntu as the OS for Zimbra, then you can install the program using the sudo apt-get install imapsync -y command , and CentOS and RHEL will be installed using the sudo yum install imapsync -y command .
When the installation of the package and all dependencies is complete, proceed to the configuration of the Exchange server. On it, we will need to enable the IMAP4 service. This is done by running the services.msc file, after which you need to go to Tools> Services -> IMAP4 Microsoft Exchange -> Start. After that, you will need to give the administrator access to user mailboxes in order to avoid using user passwords during the migration, and also disable or configure speed limiting policies so that they do not interfere with the migration. Now you can start transferring mail from server to server.
In our infrastructure, MS Exchange will be located on the exchange.company.ru host, and Zimbra will be located on the zimbra.company.ru host. Users will be migrated one by one using the command line, in which we will execute the imapsync utility with a number of parameters necessary for migration. First of all, it should be noted that in Zimbra, as in MS Exchange, administrator accounts have access to the contents of all user mailboxes, which means that to migrate to a new mail server, the system administrator will not need to know passwords from regular user accounts. That is why the migration of each user from MS Exchange to Zimbra will be carried out using the following command:
In this command, we specify the source host ( --host1 ) from which imapsync will pick up account data, as well as the host ( --host2 ) to which this data will be recorded. We also specify the username ( --user1 ) on the first host and the corresponding username ( --user2 ) on the second host. In order to get access to its data, we specify the name of the administrator account ( --authuser1 ) and the administrator password ( --password1 ) of the first host, and in order to record the transmitted data, the name of the administrator account ( --authuser2 ) and password administrator ( --password2 ) of the second host.
In an ideal world, such a command would be enough to transfer user data between servers running MS Exchange and Zimbra, but, as is often the case, the differences between solutions make their own adjustments and, therefore, simply transferring data from MS Exchange will cause monstrous chaos in the mailboxes of Zimbra . The stumbling block in this case are the different names of folders in mailboxes. In order to avoid confusion when transferring data, you need to bring our team to something like the following:
Note that this is not a universal version of the script that will work flawlessly in any infrastructure. Much of it may depend on the various features of your infrastructure. That is why, before running a full-scale migration, be sure to test data transfer between systems on a number of non-business-critical accounts.
Let the abundance of parameters with which imapsync runs does not scare you. Most of them are needed precisely so that when transferring folders from the mailbox there is no confusion and all items from MS Exchange are transferred to Zimbra. For example, the --subscribe_all parameter allows imapsync to get all the account folders to work with them in the future. The --f1f2 parameter allows you to change the name of the folder during synchronization. That is, when synchronizing, for example. The Sent items folder will change its name to Sent, which will allow you to perfectly fit its contents into the Zimbra Collaboration Suite folder structure. The --regextrans2 parameter also allows you to change folder names and is based on the regex mechanism of the Perl programming language, so those familiar with this language will easily master this parameter.
In order to explain what the --addheader and --useheader parameters do, it will be necessary to delve into how exactly the synchronization of letters in imapsync occurs. The fact is that to synchronize imapsync, it is necessary to identify letters on both servers in order to decide whether to synchronize them or not. To verify them, this utility uses parameters such as Message-Id: and Received: (identification number and time of receipt), which are not available for such e-mails, such as drafts. In order to synchronize them properly, we need the --addheader command. It allows you to add an identification number to them in order to somehow identify and compare such letters during synchronization. The parameter - useheader allows you to compare letters on only one of the two parameters during synchronization.
After writing the perfect script that will allow us to transfer the perfect transfer of accounts from MS Exchange to Zimbra, a significant problem remains that is connected with the fact that we still have to transfer users between servers one by one. This is not very optimal, especially in those companies where many employees and the number of mailboxes are measured in thousands. However, this process can be significantly optimized by creating a list of accounts, and then writing a small script that automatically transfers mail based on account names. To do this, we will create the imap_users file and add all the necessary data to it in the following form:
After that, let's write the actual script. We did something like this:
Since data transfer occurs incrementally, you can first transfer the bulk of data between servers, and then, within a small technical break, transfer the small amount of data that has accumulated since the last synchronization, transfer all incoming mail to Zimbra and make it the main mail system. We remind you that before migrating to a new mail system, you need to notify employees about the upcoming transition, conduct training with them, and also prepare memos about where their usual functions are located in Zimbra. Thanks to this, the migration will go more smoothly and not create problems for the business.
In many ways, the migration process from the IBM mail system using imapsync repeats the migration process from MS Exchange and any other mail system. However, during the direct migration, many administrators have an error with an IMAP connection failure and, accordingly, termination of synchronization. In this case, no error message appears in the logs. If during the migration from Lotus / Domino you encounter this problem, then the following steps will help to resolve this issue:
After completing all these steps, the transfer of mail from Lotus / Domino to Zimbra via imapsync should be done without any problems.
In addition to the utilities for migration, Zimbra clients for desktop computers and mobile devices were left without support. Also, HTML web client was removed from support, which was used to ensure compatibility with older browsers and low-power devices. All of them can be easily replaced by third-party clients, due to the fact that Zimbra supports almost all open standards and protocols.
Recall that the migration between Zimbra servers can be done using the free utility Zextras Migration Tool, which you can download on the official website of Zextras .
Imapsync is a free and free utility available in the repositories of any officially supported Zimbra Linux distribution. The purpose of this program is to transfer e-mail between different mail servers and it copes with it perfectly. Due to the flexibility of the settings, mail transfer can be done incrementally and in fully automatic mode. Let's see how you can use imapsync to migrate from Lotus Domino and MS Exchange to the Zimbra Collaboration Suite.
')
Migration from MS Exchange to Zimbra
In order for email migration to succeed, you need to complete a number of preparatory activities. First of all, we need to create on the Zimbra server the same users as on the MS Exchange server with the same passwords. Carrying out this task can be very useful auto-tuning users Zimbra using AD. The process of such autotune is described in one of our previous articles.
After all accounts from AD or Exchange are migrated, you need to install the imapsync package on the server with Zimbra. If you are using Ubuntu as the OS for Zimbra, then you can install the program using the sudo apt-get install imapsync -y command , and CentOS and RHEL will be installed using the sudo yum install imapsync -y command .
When the installation of the package and all dependencies is complete, proceed to the configuration of the Exchange server. On it, we will need to enable the IMAP4 service. This is done by running the services.msc file, after which you need to go to Tools> Services -> IMAP4 Microsoft Exchange -> Start. After that, you will need to give the administrator access to user mailboxes in order to avoid using user passwords during the migration, and also disable or configure speed limiting policies so that they do not interfere with the migration. Now you can start transferring mail from server to server.
In our infrastructure, MS Exchange will be located on the exchange.company.ru host, and Zimbra will be located on the zimbra.company.ru host. Users will be migrated one by one using the command line, in which we will execute the imapsync utility with a number of parameters necessary for migration. First of all, it should be noted that in Zimbra, as in MS Exchange, administrator accounts have access to the contents of all user mailboxes, which means that to migrate to a new mail server, the system administrator will not need to know passwords from regular user accounts. That is why the migration of each user from MS Exchange to Zimbra will be carried out using the following command:
imapsync --host1 exchange.company.ru --user1 "xxxxx" --authuser1 "yyyyy" --password1 "zzzzz" --nosslcheck --host2 zimbra.company.ru --user2 "XXXXX" --authuser1 "YYYYY" --password2 "ZZZZZ" In this command, we specify the source host ( --host1 ) from which imapsync will pick up account data, as well as the host ( --host2 ) to which this data will be recorded. We also specify the username ( --user1 ) on the first host and the corresponding username ( --user2 ) on the second host. In order to get access to its data, we specify the name of the administrator account ( --authuser1 ) and the administrator password ( --password1 ) of the first host, and in order to record the transmitted data, the name of the administrator account ( --authuser2 ) and password administrator ( --password2 ) of the second host.
In an ideal world, such a command would be enough to transfer user data between servers running MS Exchange and Zimbra, but, as is often the case, the differences between solutions make their own adjustments and, therefore, simply transferring data from MS Exchange will cause monstrous chaos in the mailboxes of Zimbra . The stumbling block in this case are the different names of folders in mailboxes. In order to avoid confusion when transferring data, you need to bring our team to something like the following:
imapsync --subscribe_all --maxmessagespersecond 4 --addheader --f1f2 "Inbox=INBOX" \ --f1f2 "Sent Items=Sent" --f1f2 "Sent items=Sent" --exclude "^Journal|^Tasks|^Trash|Contacts|Calendar|RSS Feeds" \ --useheader "Message-ID" --buffersize 8192000 --nosyncacls --allowsizemismatch --usecache --syncinternaldates --skipsize --nofoldersizes \ --tls1 --port1 143 --user1 "xxxxxxxxxx" --authuser1 "besadmin" --password1 "XXXXXXX" \ --host2 'xxxxxxx' --user2 "xxxxxx" --authuser2 "XXXXXX" --password2 "XXXX" --tls2 --port2 7143 \ --regextrans2 "s/Inbox1/INBOX/i" --regextrans2 "s/Sent Items/Sent/i" --regextrans2 "s/sent-mail/Sent/i" --regextrans2 "s/Deleted Items/Trash/i" --regextrans2 "s/Deleted Messages/Trash/" \ --delete2duplicates --timeout 600 --pidfile "impasync.local.pid" --logfile "imapsync.log" --sep1 "/" --errorsmax "100" Note that this is not a universal version of the script that will work flawlessly in any infrastructure. Much of it may depend on the various features of your infrastructure. That is why, before running a full-scale migration, be sure to test data transfer between systems on a number of non-business-critical accounts.
Let the abundance of parameters with which imapsync runs does not scare you. Most of them are needed precisely so that when transferring folders from the mailbox there is no confusion and all items from MS Exchange are transferred to Zimbra. For example, the --subscribe_all parameter allows imapsync to get all the account folders to work with them in the future. The --f1f2 parameter allows you to change the name of the folder during synchronization. That is, when synchronizing, for example. The Sent items folder will change its name to Sent, which will allow you to perfectly fit its contents into the Zimbra Collaboration Suite folder structure. The --regextrans2 parameter also allows you to change folder names and is based on the regex mechanism of the Perl programming language, so those familiar with this language will easily master this parameter.
In order to explain what the --addheader and --useheader parameters do, it will be necessary to delve into how exactly the synchronization of letters in imapsync occurs. The fact is that to synchronize imapsync, it is necessary to identify letters on both servers in order to decide whether to synchronize them or not. To verify them, this utility uses parameters such as Message-Id: and Received: (identification number and time of receipt), which are not available for such e-mails, such as drafts. In order to synchronize them properly, we need the --addheader command. It allows you to add an identification number to them in order to somehow identify and compare such letters during synchronization. The parameter - useheader allows you to compare letters on only one of the two parameters during synchronization.
After writing the perfect script that will allow us to transfer the perfect transfer of accounts from MS Exchange to Zimbra, a significant problem remains that is connected with the fact that we still have to transfer users between servers one by one. This is not very optimal, especially in those companies where many employees and the number of mailboxes are measured in thousands. However, this process can be significantly optimized by creating a list of accounts, and then writing a small script that automatically transfers mail based on account names. To do this, we will create the imap_users file and add all the necessary data to it in the following form:
domains=( domain1\ domain2) users=( user1@domain1\ user2@domain1\ user3@domain1\ user1@domain2\ user2@domain2\ user3@domain2) After that, let's write the actual script. We did something like this:
. imap_users src_srv = exchange.company.ru dest_srv = zimbra.company.ru for ((i = 0; i < ${#users[@]}; i++ )) do /usr/bin/imapsync --subscribe_all --maxmessagespersecond 4 --addheader --f1f2 "Inbox=INBOX" \ --f1f2 "Sent Items=Sent" --f1f2 "Sent items=Sent" --exclude "^Journal|^Tasks|^Trash|Contacts|Calendar|RSS Feeds" \ --useheader "Message-ID" --buffersize 8192000 --nosyncacls --allowsizemismatch --usecache --syncinternaldates --skipsize --nofoldersizes \ --tls1 --port1 143 --host1 $src_srv --user1 ${users[$i]} --authuser1 "besadmin" --password1 "XXXXXXX" --nosslcheck \ --host2 $dest_srv --user2 ${users[$i]} --authuser2 "XXXXXX" --password2 "XXXX" --tls2 --port2 7143 \ --regextrans2 "s/Inbox1/INBOX/i" --regextrans2 "s/Sent Items/Sent/i" --regextrans2 "s/sent-mail/Sent/i" --regextrans2 "s/Deleted Items/Trash/i" --regextrans2 "s/Deleted Messages/Trash/" \ --delete2duplicates --timeout 600 --pidfile "impasync.local.pid" --logfile "imapsync.log" --sep1 "/" --errorsmax "100" done Since data transfer occurs incrementally, you can first transfer the bulk of data between servers, and then, within a small technical break, transfer the small amount of data that has accumulated since the last synchronization, transfer all incoming mail to Zimbra and make it the main mail system. We remind you that before migrating to a new mail system, you need to notify employees about the upcoming transition, conduct training with them, and also prepare memos about where their usual functions are located in Zimbra. Thanks to this, the migration will go more smoothly and not create problems for the business.
Migration from Lotus / Domino to Zimbra
In many ways, the migration process from the IBM mail system using imapsync repeats the migration process from MS Exchange and any other mail system. However, during the direct migration, many administrators have an error with an IMAP connection failure and, accordingly, termination of synchronization. In this case, no error message appears in the logs. If during the migration from Lotus / Domino you encounter this problem, then the following steps will help to resolve this issue:
- Launch Lotus Notes with administrator privileges and go to the Address book.
- In the Address Book settings, change the default message format to “Preferred MIME”
- In the same Lotus Notes running with administrator privileges, change the format of the user database to Mail (R6), which is taken from the server, and not from the email client
- Shut down the Lotus Domino Router from the Domino admin console using the tell router quit command.
- Pack the user database from the Domino administration console using the “load compact userdb.nsf” command
- Check the database for errors by running the command “fixup userdb.nsf” from the Domino console (not from the administration console)
- From the Domino administration console, run the “convert -e userdb.nsf” command to prepare the database for transfer via imap
- From the Domino administration console, run the command “convert -h userdb.nsf” to add imap headers to all messages.
After completing all these steps, the transfer of mail from Lotus / Domino to Zimbra via imapsync should be done without any problems.
In addition to the utilities for migration, Zimbra clients for desktop computers and mobile devices were left without support. Also, HTML web client was removed from support, which was used to ensure compatibility with older browsers and low-power devices. All of them can be easily replaced by third-party clients, due to the fact that Zimbra supports almost all open standards and protocols.
Recall that the migration between Zimbra servers can be done using the free utility Zextras Migration Tool, which you can download on the official website of Zextras .
Source: https://habr.com/ru/post/458912/
All Articles