The other day, we announced the inclusion of a strict DMARC policy on all domains owned by Mail.Ru. On some domains, including bk.ru and mail.ua, the policy p = reject is already included. In this article, we want to clarify some of the technical details of this inclusion and provide recommendations to owners of services, mail servers, and mailing lists.

What is DMARC?

DMARC (Domain Based Message Authentication, Reporting and Conformance, RFC 7489) is a protocol that allows a domain zone administrator to publish a policy for receiving and sending reports for letters that do not have authorization. The authorization protocols are SPF (RFC 7208) and DKIM (RFC 6376).

How does DMARC work?

When you receive a letter that has the example.org domain in the sender’s field (From :), the recipient's server requests the example domain’s DMARC policy, which is published to the DNS as a TXT record, for example
')
_dmarc IN TXT "v=DMARC1; p=none; rua=mailto:a@example.org; ruf=mailto:f@example.org; fo=1;"


SPF and DKIM are checked for writing; in addition, the alignment of the domain that passed the SPF or DKIM validation and the domain used in From: is checked. In order for a domain to pass DMARC authorization, it is necessary that at least one of the SPF or DKIM mechanisms pass authorization for a domain that matches the organization's domain from the From: address. If the letter does not pass authorization or the domain does not match (for example, the DKIM signature of the third-party domain is used or the address in the envelope-from does not match the From :) address, the policy is applied (in this case, none, that is, no special actions are required; reject policies are also possible - do not accept letters and quarantine - put letters into the “Spam” folder). Statistical reports on policy trigger and past emails will be sent to a@example.org. Also, the domain that publishes the policy requests that forensic reports be sent (i.e., reports containing at least the headers of the received letter) for all emails that did not pass SPF or DKIM authorization to f@example.org.

What has changed now?

Mail.Ru was the first in RuNet to support the server part of DMARC more than two years ago. And now DMARC will protect mail.ru domains from forgery. Previously, the strict DMARC policy was enabled only for the Mail.Ru Group service domains (for example, corp.mail.ru), since they are most often forged by phishers. Currently, it is also used for the mail.ua and bk.ru domains, and on May 18 it will be extended to all free mailbox domains (list.ru, inbox.ru and mail.ru).

Why is Mail.Ru?

DMARC addresses the most serious email problem — not verifying the sender’s address. The most common reason for entering DMARC is called phishing emails. In part, this is true - the introduction of strict DMARC policies by just a few large US banks in 2014 led to a decrease in phishing via email by 6% per year in the industry as a whole. In PayPal, the amount of phishing for two years after the introduction of the policy decreased by 70%.

In the case of public mail domains, the main reason is not phishing. First of all, we want to reduce the amount of spam on the Web as a whole: most of it comes from fake addresses, on some days the number of such letters with fake mail.ru addresses is more than 15 times the number of letters sent by Mail.Ru users . Secondly, DMARC allows you to protect users from secondary spam. Very often spam emails from fake addresses generate undeliverable messages (NDR) and auto-responses that go to the mailboxes of legitimate users who have not sent the letter. Recognizing this situation can be quite difficult, as there is no “spam” content in the NDR and auto-answer. In our experience, the implementation of strict DMARC policies reduces the amount of secondary spam by several dozen times.

Wasn't all this in SPF (DKIM, Sender-ID, ADSP)?

SPF (RFC 7208) does not protect the sender address at all, as it only works with the envelope-from address that is invisible to the user. DKIM (RFC 6376) also does not protect the address of the sender at all, since it is a signature algorithm and does not indicate what to do if there is no signature or it is incorrect. Sender ID (RFC 4406) is covered by Microsoft patents and "did not take off." The closest in functionality to DMARC was the ADSP protocol (RFC 5617), but, unlike DMARC, it also “did not take off” and in 2013 was transferred to the status of Historic, since almost all major market players abandoned it in favor of DMARC. Unlike ADSP, which worked only on top of DKIM, DMARC can use different authentication protocols. The basic standard applies both DKIM and SPF. In fact, DMARC combines the mechanisms involved in Sender ID and ADSP, and makes it possible to expand them in the future, which significantly improves the “indirect” mail flow (indirect flows). Already, there is no doubt that the DMARC protocol is working - the server part of the DMARC is supported by all serious postal services, all the major players either have already enabled DMARC on their domains, or have declared such an intention. Among the public postal services that have included DMARC on their domains (and have taken the brunt of criticism from the owners of the mailing lists, as discussed below), are Yahoo and AOL. From their side it was a risky, but very necessary decision - otherwise the fate of Sender ID and ADSP could comprehend the protocol.

What's the catch?

Naturally, any protocol has flaws. DMARC requires sender authentication, which causes problems in several situations:

1. The user sends letters "directly" to bypass the mail server with authentication. For example, a webmaster sends registration letters using a PHP script, using the From: address of the public mail address. Unfortunately, the only solution is not to do so. Register mail for your domain. Mail.Ru provides for this free service . Even if you move to sending from the address of another public mail, most likely, this solution will be temporary, since DMARC will implement all large mail services for its domains.
2. Mailing Lists. Yes, there are again problems with them, and more significant than in the case of SPF, since SPF turned out to be useless precisely because of attempts to maintain compatibility with mailing lists. In order not to violate the DMARC authorization, it is necessary either to “not touch” the headers signed by DKIM (in particular, not to change the subject of the letter), or to overwrite the sender from From :. Fortunately, the main mailing list managers already support correct work with DMARC.
3. Forwards. In some cases, mail servers change their content during transfer, such as the structure of MIME parts or line breaks, which can lead to a violation of the DKIM signature. In such cases, we recommend using reverse mail POP3 / IMAP (reverse POP / reverse IMAP) instead of mail forwarding. Mail.Ru is the first of the postal services that implemented mail collection via IMAP while maintaining the folder structure. In addition, large mail services, including Mail.Ru, support white lists of well-known forwarders (known forwarders) - for mailing lists and other mail services, mail from which is accepted even in the event of a violation of DMARC. If letters from a well-known forwarder or mailing list are blocked by us under the DMARC policy - please report it. In the future, the problem of forwards can be solved by implementing the ARC (Authenticated Received Chain) protocol while this protocol is under discussion.
4. There are other DMARC issues, including those related to security, especially when providing forensic reports. For example, the ability to expose redirects (and thus de-anonymize users) or mailing list subscribers. All this is possible without DMARC, but DMARC makes it much easier. Therefore, we have implemented the anonymizer function - as an alternative to forwarding or one-time / hidden addresses for subscription lists. For security reasons, we do not plan to provide forensic reports with full identification of the recipient to untrusted parties.

What should I do if ...

... I am a regular mail user

Nothing to do. Most likely, you will not notice anything, just avoid situations “someone got spam / phishing from my address, but I didn’t send it” and you will not receive incomprehensible “bounces”.

... I use Business Mail for my domain

Changes affect only Mail.Ru public domains. In order for your domain to be protected by DMARC, you must publish the policy yourself.

... I am sending letters via my ISP's mail server

For addresses served by Mail.Ru, it is necessary to configure sending via mail.ru SMTP servers with authorization according to instructions .

... I use redirects to collect mail from multiple accounts on different services in a shared box

Most likely, this functionality will work, but to eliminate the possibility that some letters will be lost, use the function of collecting mail from other mailboxes.

... I use redirects to place hidden / temporary addresses

Redirects can be detected and disclosed (for this, DMARC support is sufficient for the recipient's server, i.e., virtually any server). If this is critical for you, use the anonymizer function .

... I use the "send as" functionality on Gmail (or similar) to send emails from Mail.Ru

You should check the settings of the sending address in Gmail and make sure that the sending goes through the SMTP mail.ru server (smtp.mail.ru, port 465 using SSL), with the correct login and password. Letters will be sent through the Mail.Ru server with authorization of the sender and will pass SPF / DKIM / DMARC checks.

... I use the return address of the public mail to send letters through scripts on the server

So do not follow. Use " Business Mail " to place the boxes in your own domain (hint: you do not need to have a business, it is enough to have a domain). Use the address of your own domain in all server, automatic or automated mailings.

... I provide clients with email delivery services (ESP)

Do not use the sender’s address from the free mail domains for the From: header. Register a separate domain for customers who do not have their own domain, configure SPF, DKIM and DMARC for it. Allocate to the client an address in this domain for use as the sender's address and write redirections from a dedicated address in your domain to the client's email box. Customers with their own domain, recommend to take the address from this domain as the sender's address. You can continue to use the client's address in the free mail domain for Sender: and Reply-To: headers.

... I administer mailing lists

Update your mailing list software and configure it according to the DMARC compatibility guidelines. Sympa from version 6.2.6, Dada Mail from version 7.0.2, Mailman from version 2.1.16 (better than 2.1.18), GroupServer from version 14.06 work correctly. You may have to enable a feature called “Sending on behalf of” / “Munge the From: header”. If recommendations cannot be fulfilled (for example, outdated or unsupported software is used), try to minimize violations of the DMARC by refusing to change service headers and the content of letters sent to the mailing list — that is, do not change topics, do not add header / footer to letters, so as not to violate the DKIM signature of the letter. If delivery problems persist, contact the support team of the server that does not accept messages.

... I administer the mail domain and I want to publish my DMARC policy

Configure SPF and DKIM for all emails sent, while keeping in mind the specific requirements of the new versions of the standards. The latest version of the SPF standard (RFC 7208) requires an SPF record not only for the main mail domain, but also for names from the HELO / EHLO team, which, as a rule, coincide with the canonical names of mail servers. This is necessary for passing the SPF in the case of an empty sender used in the NDR / MDN. Post a DMARC entry with the policy none. Configure the addresses for collecting reports rua / ruf. Please note that the DMARC standard requires that the domain in which these addresses are located match the domain for which reports are requested, or publish a special policy for receiving reports, therefore, receiving reports to public mail addresses will also not work. Analyze incoming reports, fix problems with SPF, DKIM and misalignment of domains from your network, identify possible sources of legitimate emails outside of it (for example, mailings organized through external mailing services) and tailor SPF and DKIM for all legal emails. You can use the services of specialized services - Agari , proofpoint , Dmarcian . Dmarcian service, besides paid services, offers a convenient free tool for the visual presentation of DMARC XML reports. When all problems are fixed, switch the policy to reject or quarantine.

... I administer the mail server and I want to filter letters by DMARC

Integrate the appropriate filter. DMARC filtering is supported in almost all anti-spam solutions. As separate free DMARC-filters compatible with the main Linux / UNIX MTA, it is possible to recommend OpenDMARC and yenma .

... I administer the mail server / domain and I don’t know this SPF / DKIM / DMARC and don’t want to know

As a strict DMARC policy is implemented by other mail domains, your domain that is not protected by DMARC policy will increasingly be used by spammers as a source of fake sender addresses, which will lead to a constant increase in secondary spam and, as a result, user complaints. Most likely, as a result, you will still be forced to implement SPF / DKIM / DMARC. But even if you remain steadfast, someday domains without a published policy may begin to be perceived as suspicious, including self-learning filters, which will lead to problems with the deliverability of letters.

... I administer the mail server / domain and I have questions

You can ask them here in the comments or on the Toaster , I try to keep track of thematic hubs.

Finally

E-mail is often positioned by critics as an unchanging dinosaur from the past millennium - but this is not so: it is developing, and very active. All current versions of the basic e-mail standards were adopted over the past eight years, DMARC became standard only a year ago, and now standardization work is under way, for example, the Authenticated Received Chain and SMTP Strict Transport Transport Security protocols. But the transition to new standards and protocols is impossible without the support of all parties, and we very much hope that this article will help you to get involved in the process of building a more modern, convenient and secure e-mail ecosystem.