How does Yandex.Mail for domains work?

A few years ago, we assumed that our own mail services deployed in companies — especially small ones without their system administrator — are more of a hassle than they do. And over time, more and more people will transfer responsibility for the storage and processing of emails to outsourcing, special services. So we have Mail for domains.



As time has shown, we were not mistaken, and today Mail for Yandex domains is the most popular mail service on the Russian Internet. In total, more than 200 thousand domain names and almost 3.5M mailboxes are connected to the traffic regulations. Every day about 200 domains are connected to the traffic regulations. In this article we will describe how the traffic rules work, and in the comments we will be happy to answer your questions.





')

During its existence, traffic regulations have acquired a lot of additional functions: integration with the address book, jabber server support, mailing lists, mail import from the user's previous servers, and, of course, an external API for automating requests. For example, for a long time we were asked to make sure that a user of a third-party site who is authorized on it immediately got into his box - without re-authorization in Yandex.Mail for domains. And our API allowed us to implement this method of password-free authorization.



All the possibilities offered by traffic regulations require integration with various components of Yandex products and not only. At its core, the SDA is an integration service that brings together disparate components under the control of a domain administrator.



The traffic rules start from the domain: you need to connect it and check that the owner of the domain and the traffic administrator are the same person. It happens that the domains for one reason or another are disconnected from the traffic rules at the DNS level. To confirm ownership and to confirm domain status, we use a check queue system consisting of several check queue repository servers in a relational DBMS and a MongoDB-based distributed lock cluster.



In order not to overload with checks servers hosting domain sites or DNS servers of zones, each subsequent confirmation check occurs with an increasing time interval - from five minutes to 24 hours. Domain status checks occur more frequently — about once every four hours at regular time intervals. This approach can significantly reduce the workload of the check queue workers. As soon as the domain is confirmed, all basic functions become accessible to its administrator and he can create mailboxes.



In order to preserve a universal approach and reduce the number of errors, we have developed an internal mail API for domains, which can be used by other Yandex components to exchange the necessary information. For example, the public mail API for domains is one such component. Public API calls are translated into generic internal API calls. Any actions in the web interface of traffic rules also call the methods of the internal API.



Depending on the required actions, the handler of calls to the internal API may refer to certain components of Yandex. The basic principle in the design of new functionality is to not keep on the side of traffic regulations information that is already stored somewhere else. An exception is made only for queries that are critical at the time of execution - they have to be cached.







For example, to create a new mailbox, you need to create a new user in the Passport, then initialize the store for this user in Mail and set up a new mailbox exactly as the domain administrator chooses.



If a shared address book is enabled for the domain, then each call to the methods for creating or deleting a mailbox is accompanied by additional calls to the address book's API to change the domain book. Thus, information about new recipients of the domain instantly becomes available for search. And if the common contact list is enabled in jabber, then the user must also be added to the contact list on the jabber server side.



With such integration, the response time of the services and the handling of possible errors are of great importance. There are basically two types of errors: network and logical. For completeness, look at the list of services and features of the integration of traffic rules with them.

Passport

In fact, it is the main source of data for traffic rules, no less important than the database, and in some places even more. With the Passport SDA communicates through the internal API via HTTP. Any passport errors are critical. If the Passport returned an error or a failure occurred while communicating with the Passport, we transmit the error to the user and cancel the actions already performed in the database or other services. Thus, the transactional nature of transactions between different integrated services is maintained: for each team there is its opposite - the operation cancellation command.

post office

This is a key service that the domain administrator manages through traffic rules. Creating a mailbox requires a lot of computation, so it happens in the background. A queue of atomic operations needed to complete the registration is formed: start a box, apply the basic settings specified for the domain, etc. Network errors in this case are not so critical, so in case of problems, the operation is quietly postponed for later, in another distributed queue, the handlers of this queue diligently try to tune the box within 24 hours after the failure.

DNS

Approximately every third traffic area domain places the DNS zone on Yandex servers. Errors of working with the DNS backend are considered critical only in the event of network failures and complete inability to write to the database. In all other cases, algorithms are triggered that allow the user to solve the problem.



For example, when delegating a domain, Yandex copies part of the records from the previous DNS server so that the user does not have to do the tedious work of accurately transferring the zone. In this case, a situation may arise when the root domain record is of the CNAME type, which is contrary to the standard. A user with such a CNAME record cannot enable Yandex.Mail on his domain, so such an entry will be skipped during import.

Logging and logging

As with any large farm, there are nowhere without protocols. The traffic regulations keep detailed logs for all interactions with other components of Yandex. Often, these logs can find a problem before it becomes noticeable to users. In order for the service to work quickly and efficiently, follow special software for monitoring system logs. In the case of large accidents or errors, when the request processing time increases significantly or errors appear in the responses, system administrators notice this within a few minutes and start repairing at any time of the day or night. But more about that next time.



We are waiting for your questions!