Operation “Migration”: if your mail is somewhere there, but you need to be here

For the last two years, my department has been closely involved in projects of migration of corporate mail from foreign services to MS Exchange based on the infrastructure of our data centers. From cloud services like Office 365, Gmail mostly migrates because of the FZ-242, because they want mail to live on the Russian infrastructure. This is not the only reason: after migration, some companies completely transfer the mail system to us for administration under a hard SLA, which is not always possible to obtain in mass services. Those who had a post office on iron and who want to expand without contacting the purchase of new equipment also contact us, a modest Russian cloud provider.

On 27 migration projects and 50 thousand transferred boxes, anything happened. Today I want to tell you what to consider and think about if you are going to move.

What do you want to get out

Before starting the project, we agree with the provider the following points:
')

• the functionality you expect to receive from the new mail system: traffic volume, calendars, UM, shared folders, etc.
• availability at the stage of migration and after. Are you ready for downtime? If so, how much and when? What accessibility do you expect after migration, when your users will work with the new system?
• what business processes are tied to the postal organization : the work of a support service, marketing mailings, etc.
• what solutions to integrate with (1C, Sharepoint, Skype, etc.). There was a case when an IT manager started a mail migration from Lotus to Exchange, and halfway through it turned out that they also have a corporate portal tied to Lotus. They did not plan to move to a new portal, so they curtailed the migration, and the imprudent manager was fired.
• approximate dates of migration . Even if everything is calculated correctly, taking into account the amount of data that needs to be transferred to the new site, unforeseen technical limitations will emerge: the communication channel will be slower than expected, or limits on the export of data from the original mail system will be detected.

We encountered the last one when we were transporting a client from Gmail. In the process it turned out that no more than 1.5 GB of data can be exported per day. Even if the box was only 2 GB, then it had to be exported for two days - 1.5 GB, then we wait for a day and download the remaining 500 MB. Transferring boxes to 20 GB took almost two weeks.

While everything was exported in small portions, life in the company did not stop and a new mail arrived, which also needs to be synchronized with the new site. The problem was solved with the help of the Cloudiway utility: it pulled out the entire contents of the boxes with the daily limit, i.e. we did not have to manually pump out the allowed 1.5 GB once a day. Then she pulled out the delta of letters that fell during the time she pulled the main volume. Over time, the delta became smaller and smaller, and after all the data from the source box was transferred, we switched the user to work with the target mail system.

Another “black hole” where a lot of project time goes is the interaction between the project participants. Especially if there are more than two.

The client migrated from a foreign hosting, and the interaction chain looked like this: we communicated with the customer’s project manager, he forwarded our messages to the IT department, who interacted with the foreign hosting provider Exchange. Somehow we needed to enable MPC-proxy (a service that provides mailbox transfer during migration) at a foreign site, and it took two weeks.

At the start, they always voice very optimistic terms. Be realistic and feel free to multiply your initial estimate by two.

Audit of existing infrastructure

Everything is simple here: you need to analyze the source mail system and understand what it has with the following parameters:

• incoming and outgoing volume of SMTP traffic;
• the number and total volume of mailboxes;
• number of end users;
• customer location: they can connect from inside the network and outside the network, from a domain or non-domain machines;
• client connection protocols and client versions (MacOS client, Outlook Anywhere, POP3, IMAP, ActiveSync);
• current mail system configuration:
  - the number of SMTP domains of the organization;
  - the number of distribution groups;
  - the presence of large boxes (from 20 GB);
  - availability of service boxes (for 1C, scanners, mailings);
  - bulk mailing.

Based on this information, we will think over the sizing and architecture of the new mail system.

In one of the projects, the client forgot to tell about the fact that their marketing sends out millions of letters every day. As a result, the new system was designed without considering this load. When the mailbox from which the mass mailings came, moved to the new system, the infrastructure of the new mail system “lay flat”. I had to do resizing the system on the go. Fortunately, everything worked on virtualization and it was easy to add resources quickly.

Mail system architecture

The choice of MS Exchange architecture will depend on availability requirements. There are two options:

Stand alone. All mailboxes are located on the same server. This option is not about high availability, but about low cost. At the application level, there is no reservation. If there is no backup at other levels (for example, virtualization), then if Exchange fails, it will be unavailable.
This architecture is suitable if you have a few boxes and you are satisfied with the recovery from the backup.

High Availability. All mail system roles are duplicated. MailBox servers (MBX in the diagram) are added to the High Availability Group (Database Availability Group, DAG). If one of the servers fails, the mail databases will move to the backup ones. The end user will not notice anything. This scheme can be implemented within the two data centers. Then the mail system will not be afraid of the failure of the entire data center.

This architecture is for those companies for which simple is not acceptable. Most often these are big companies.


HA schema.

The choice of antispam / antivirus solutions

From my experience I’ll say that large companies choose something like Kaspersky Mail Security. Small projects usually use built-in antivirus and antispam solutions. They may not be as comfortable or functional as third-party solutions, but they also perform their task well. Nobody really complained yet).

Mail system sizing

The question is not, if the new system is a complete clone of the old one and we do not plan to change anything in it. For the remaining cases will have to count resources. MS has a tool (in fact, just an excel-label with a bunch of parameters :)), with which you can calculate the amount of resources for an Exchange installation - the Microsoft Exchange Sizing Calculator . I’ll say right away that the tool is not intuitive and will take time to figure it out, but this is a good option if you don’t have experience with such projects.

When calculating, we do not forget to make allowances for the fact that the mail system will be backed up, which means that the speed of the backup will also depend on the type of disks used. If you plan to use an antivirus that will check the database online, then it is more logical to choose SAS disks, but, of course, the choice of the type of disks should be based on an analysis of the actual load on the system.
Since the migration does not take place simultaneously, especially for large companies, it is necessary to foresee the possibility of scaling the infrastructure for mail (in one project, during the migration, the number of users managed to grow 2 times - it was 1000, it was 2000). If the system uses virtual resources, then it is easier to do.

Migration method

The method of migration is determined by the following points: the size of the postal system, the timing, whether you can afford simple and labor costs, which you are ready to go.

In the case of cutover migration, the entire postal system is transferred to the new site entirely in one iteration. This option can be considered if you have a small mail system (up to 500 GB) and you agree to a simple one for a few hours. The migration itself will be fast: if you successfully plan the technical window, you can move over the weekend or even faster, with minimal inconvenience for the business.

In the coexistence scenario , the mail system is gradually transferred to the new site without downtime. At the same time, the source and target mail systems for some time simultaneously live on two sites: some users already work in the new system, and some - in the old one. Such migration is suitable for any volumes, but the time, labor, cost, as the data is transferred in parts and the number of approvals and other bureaucracies increase. Technically, this is a more difficult option, we need a certain qualification of admins.

Migration plan, instructions, regulations

After we have decided on everything, we fix the order of actions in the migration plan . We register all the technical details of the migration in it:

• architecture of the source and end systems;
• technical windows and downtime;
• proper procedure for migration.

If there are a lot of participants in the project, then we register the procedures for administrative interaction and areas of responsibility . It is not necessary that it be an official paper with a signature and stamp on each page. You can simply agree in a letter about who is responsible for what, how interaction with third parties is built (DNS-hoster, cloud provider), etc.

Do not forget about end users: we warn them about the date of switching to a new mail system, we are preparing an instruction or a whole FAQ on connecting to a new mail system and what to do if something does not work. The same algorithm is used for technical support. No, this is not a formality. When a few hundred or even thousands of employees at the same time will not know what to do, and start attacking technical support, which will also not be up to date, it will not be very fun.

One of the migration projects from Office 365 to Exchange had the following situation. On the original system, users had two accounts — for logging on to the computer (AD account) and for logging in to mail, to Office 365. After the migration, there was only one account left. There were about 600 users, so it was important to clearly convey to them which of the accounts to use in the new system.

That's all. Ask questions in the comments. And all the successful journeys.