How to make email-mailing and do not mow: practical advice

A developer who first encountered generating emails has little chance of writing an application that will do this correctly. About 40% of letters generated by corporate applications have some kind of violation of standards, and, as a result, problems with delivery and display. There are reasons for this: e-mail is technically much more complicated than the web, mail is governed by several hundred standards and an uncountable number of generally accepted (and not so) practices, and mail clients are diverse and unpredictable. Testing can significantly improve the situation, but there is almost no material on testing mail.



Mail Mail.Ru regularly interacts with its users through e-mails. In our project, all components that are responsible for generating letters, and even single mailings are subject to mandatory testing. In this article we will share our experience (and stuffed cones).

• What are the emails
• Who is involved in the testing and control process
• Mailing and postal transport
• The interface of the postal infrastructure and the boundaries of the application under test
• Determination of test parameters
• Typical Generating Application Structure
• What and when to test
  
  
  1. Delivery infrastructure
  2. Generating Application
  3. Structural and Layout Letter Templates
• Basic requirements for checking infrastructure
• Authorization Requirements
• Verification of the generating application
  
  
  • Requirements for postal addresses
  • Mail Header Requirements
  • Requirements for the structure of the letter
  • URI requirements
  • Letter layout requirements
• Split tests

What are the emails

The application can generate various types of letters. They can be classified into several categories. By the method of selecting recipients: trigger - selective - group. By appointment: transactional - marketing - official. You can set different requirements for different types of letters and apply different testing scenarios.

')

Trigger letters are generated in response to any events, such as user actions or status changes of any system objects. They are generated by the application, and therefore the most interesting in terms of testing. Trigger letters can be both transactional and marketing.



Selective letters are sent to a dynamic selection of users that meet some criteria.



Group emails are sent to a known group of recipients, for example, all users or partners. Selective and group letters are most often marketing, sending such letters is initiated manually or on a schedule.



Transaction letters are generated in the process of the user committing an action. Such letters include, for example, invoices, tickets or delivery status notifications, letters with an access recovery code, etc. Transactional emails are always trigger. Maximum compatibility is important for them, which means that they should be as simple as possible, and they need to be tested on a large number of clients.



Marketing letters encourage the user to take any action, for example, it may be an offer of individual discounts based on previous purchases. In these letters, transactional data can be used; they can be either trigger or mass - periodic or one-time. For these letters, efficiency is more important, usually it is determined by the results of the split test. Some aspects of compatibility can be sacrificed for efficiency.



Group marketing letters, for example, messages about seasonal offers, promotions and sales, are often sent “manually” and are not part of your application, but you can also (and should) apply the general principles of testing.



In addition, there may be service letters generated for employees for automated or automated CRM systems, journaling, auditing, or DWH. Such letters are triggered, which means that they are also part of the application and must be tested.

Who is involved in the testing and control process

1. QA Engineer - participates at all stages of the process.
2. Network Engineer - Responsible for configuring network infrastructure and message delivery infrastructure. Network engineer should be involved in planning infrastructure testing.
3. A deliverability specialist is a person who monitors the deliverability of letters, who also participates in monitoring the technical and administrative parameters of all sent messages and monitors the progress of the mailing process. He is responsible for ensuring that the letters sent reach the maximum possible percentage of users and do not end up in spam, and for this, the specialist must have certain knowledge and contacts. If problems arise with the delivery of letters, it is he who must understand the probable cause and eliminate it: either by removing technical obstacles; or changing something in the content of the letters; or having solved the problem together with the support service of the mail provider, to which the letters do not reach. Such a specialist (if any) should also be involved in drawing up the checklist and testing the infrastructure generating applications and letters. However, the testing process itself must be controlled by the QA service.
4. Email-marketer - determines the effectiveness of marketing newsletters. Under his control, the split-testing for the audience’s marketing newsletters is held. The email marketer also controls the segmentation of the user base, the composition and frequency of emails sent, and the visual “presentation” of the email to the user.

All these roles are not necessarily performed by a dedicated employee: the role of the marketer can be performed by one of the product managers, and the role of the deliverability specialist, for example, a support employee or a network engineer. And in startups, with a high degree of probability, all this will have to be dealt with by one person, and they may be a quality specialist.

Mailing and postal transport

The structure of the mail is similar to a huge iceberg, and there are two levels in it. There are more than a hundred different standards governing mail, but almost all of them belong to one of these two levels:



The underwater part of an iceberg is a network service, the basic protocol of which is the SMTP application protocol defined by RFC 5321. It is responsible for the delivery of letters. For the delivery of the letter, a so-called SMTP envelope is formed, which includes the addresses of the sender and recipient of the SMTP level. Also for the delivery of the letter are responsible other network services, such as DNS. The main component of the network infrastructure is the Message Transfer Agent (Mail Transfer Agent, or more commonly used abbreviation MTA). The MTA is responsible for handling the message delivery queue and the delivery process itself to the recipient servers. MTA examples are Postfix, Sendmail, exim, Microsoft SMTP service.



This underwater part of the iceberg, which includes the MTA, DNS parameters and other network parameters, we will call the mail infrastructure, or message delivery infrastructure.



The tip of the iceberg is the letter itself. The basic structure of a letter is defined by the standard RFC 5322. A letter consists of service headers and one or more data parts. The data may contain plain text text and / or HTML, inline images or attachments of almost any type.

The interface of the postal infrastructure and the boundaries of the application under test

The mail infrastructure, as a rule, has one or several interfaces through which an email is sent (enters the MTA delivery queue). For example, the SMTP Submission service, the mail() function in PHP, data transfer to an external mail or sendmail application, the API of internal or external services (for example, GetResponse, SendPulse, or Amazon SES). We will consider these interfaces as part of the infrastructure. It often happens that application A prepares data for a letter and a list of recipients, and then sends it to application B through its API (for marketing mass mailings, this can be done manually via the user interface - UI), and application B generates a mail message in RFC 5322 and in some way puts it in the MTA delivery queue. From the point of view of application A (and when testing it), application B will be part of the mail infrastructure. The API or UI of Application B will be for the Application A interface of the mail infrastructure. Although from the point of view of application B, the situation will be different, and the mail infrastructure for it will be lower-level network applications or protocols.

Determination of test parameters

When testing each application, it is important to select all the mail infrastructures used by it (there may be several of them), and for each infrastructure to identify the interfaces used (there may also be several of them for each infrastructure). For each interface, the composition and format of the data transmitted to it is determined as accurately as possible, for example: letter text in TEXT / HTML, letter text in TEXT / PLAIN, letter subject, recipient name, recipient address, sender name, sender address (RFC5321.From ), the sender address of the SMTP convector (RFC5322.mailfrom). Next, a set of requirements is developed for each parameter (representations, encodings, boundary values, etc.), methods for monitoring each of the parameters are determined (how you can compare the actual result with the expected one).

Typical Generating Application Structure

For the generation of letters and data in it, as a rule, the same product that we are testing is responsible. This is usually a server (but sometimes client) application. It defines the structure of the letter, part of the service headers, data encapsulation formats, string representations and text encoding. A simple example of such an application is the script that forms the letter and calls the mail() function.



The main elements of the application that must be controlled:

• the code responsible for generating headers and / or letter structure, if the letter structure is dynamically generated, and / or a static letter pattern describing its structure;
• HTML layout of the letter (ideally, it is a separate entity or part of the letter template / layout, but can be embedded in the application code);
• substitution of application data into a letter (or letter pattern);
• integration of the application with the letter delivery infrastructure, the correctness of the parameters passed to the infrastructure interface.

What and when to test

Whether we like it or not, the iceberg should be tested all. There are several main components that require testing:

1. Delivery infrastructure

Emphasis in testing should be done on: letter deliverability; correct DNS records, including PTR / FCrDNS, MX and A records; SMTP protocol parameters (HELO, use TLS); letter authorization (SPF / DKIM / DMARC); SMTP envelope addresses (if they are not managed by the application); correctness of processing the input parameters of the infrastructure interface; tracking, recording and processing of undelivered emails.



It is necessary to test the infrastructure during the initial implementation and every time changes are made to the infrastructure itself (the configuration of the MTA, DNS or network changes) or the interface for sending a letter; using a new domain, network, or API; characteristics of sent letters, such as their language, size or number, change significantly. According to experience, the infrastructure tends to change without declaring war, so basic tests should be carried out periodically, even if there is no information about any changes.



It is possible and necessary to involve the network engineer and the deliverability-specialist in drawing up the plan and checklists for testing the infrastructure.

2. Generating application

The addresses of the SMTP envelope should be monitored (if they are controlled by the application, that is, they are transmitted to the interface — envelope-from, envelope-to), the values ​​of the service headers of the letter (Date, Message-ID, List-Unsubscribe, Auto-Submitted, etc.). p.), letter authorization (DKIM / DMARC), MIME-coding (base64, quoted-printable), general correctness of the letter format, for example, the absence of non-ASCII characters in the headers, the composition of the data to be substituted, the correct triggering of triggers, unsubscribe mechanisms, tracking mechanisms writing and collecting statistics (postmaster headers, for example, Feedback-ID or X-Mailru-Msgtype, as well as tracking pixels) .



It is necessary to test an application during its development, when all its related components that are responsible for generating and storing data change, with significant changes in letter templates, when changing the used infrastructure or interface to it, as well as within the framework of general regressions.

3. Structural and bump letter templates (can be part of a generating application or are developed separately)

The structure of the letter is checked (Content-Type, Content-Disposition, nesting of Multipart-parts of the letter, text encoding, string parameters), the value of the target and displayed headers (From, To, Reply-to, Subject), the letter is displayed in the list of letters and when reading in various interfaces, microformats (for example, that a calendar event is recognized as a calendar event, or an air ticket as an air ticket), branding.



Letter templates should be tested every time you make at least the slightest changes, as well as separately, for example, in a situation where letters get into the application before the server part is ready.



It is recommended that an email marketing specialist and a deliverability specialist be invited to compile a checklist for testing a letter template.

Basic requirements for checking infrastructure

The IP address selected for the mail server should be as close as possible to the IP address of the mail server. It is recommended to check it using the whois utility. In particular, the sender's address should not belong to the network, which can be perceived as dynamic; the selected network must have active contacts to which complaints can be sent; the network must be used (have the status ASSIGNED in RIPE). The IP address must have a correctly configured PTR record. It can be configured independently through the hosting control panel, or with the help of the service provider. A PTR record must point to the real host name and still be meaningful, resolved back to the same IP address (so-called FCrDNS check), not remind the name of the dynamic host, and not include a large group of numbers or characters. A good example is mailserver.example.com.



Each domain used in envelope addresses or letter headers must have a valid MX record pointing to a host A record, which, at a minimum, can handle undeliverable messages. MX is not allowed to directly refer to the IP address.



Control the passage of SPF, DKIM, DMARC. SPF allows the domain owner to specify in the TXT records a list of servers that have the right to send email messages with return addresses in this domain. It is configured for the address used in the envelope-from (SMTP-envelope), in the section of managing DNS zones of the domain. DKIM provides verification of the authorship of a message or of the originator’s address to a specific domain using digital signature technologies that are added to the letter itself (in its DKIM-Signature header). Typically, the DKIM signature is configured at the MTA (infrastructure) level. DMARC sets the policy of checking incoming mail in a specific domain and actions on letters that do not pass SPF or DKIM authentication. When attempting to violate this policy, a structured report comes along with information about the attempt. DMARC, as well as SPF, is published as a TXT record in the domain zone.



Check the deliverability of letters to the main postal services - for Russia Mail.Ru, Yandex, GMail, Microsoft (Hotmail / Outlook.com / Office365), Rambler, nic.ru. In the incoming letters you need to check the correctness of HELO; the presence and passage of checks PTR / FCrDNS, SPF, DKIM and DMARC; the validity of the headers and data in them, in particular, the synchronism of the clock in the dates and the correctness of the time zones.







The formation of some parameters, for example, authorization, deliverability, and spam, are integrally influenced by all components, but for their control there are usually separate operational tools - DMARC and FBL reports, API postmaster services, letter tracking tools, delivery statistics. Testing should take into account the level of implementation of operational monitoring tools in the company - for example, in the absence of operational control of DMARC reports, the authorization of letters should be regularly tested, in the absence of operational control of deliverability - regularly check how and where the letters get, even if there is no development related to sending letters is not conducted.



To test the infrastructure, you can use specialized services, for example, mail-tester.com , mxtoolbox.com . Detailed infrastructure requirements can be found in this article .

Authorization Requirements

Check the passage of SPF, DKIM, DMARC can usually be on the Authentication-Results header on the recipient server.



Check the validity of the SPF record for syntax, limit for DNS queries (for example, using mxtoolbox.com). When forming SPF, all sources of mailings should be taken into account (do not forget CRM systems, all delivery infrastructures used, including those through which one-time marketing campaigns are conducted). It is recommended to specify allowed servers for the domain through the list of networks ('ip4' / 'ip6'). SPF is checked for the sender address from the SMTP envelope. Verify that the SMTP envelope domain (envelope-from) matches the domain from the From header. Typical SPF errors are listed in this article .



Check the DKIM DNS record, the validity and composition of the DKIM signature (DKIM-Signature). Verify that the DKIM key is at least 1024 bits in size. The recommended DKIM signature hash mode is relaxed / relaxed. Make sure that all important headers are signed (From, To, Subject, Date, Message-ID, MIME-Version, Content-Type), the tracking headers are not signed (Received, Delivered-To, Return-Path), and DKIM is validated on main postal services. Set up on one of the postal services forwarding to another, DKIM should not "beat" on the forwarded letters. Ensure that the DKIM signature domain matches the sender's domain from the From header.



Check out the progress of the DMARC on the main postal services. Check for DMARC reports, identify and fix problems with SPF and DKIM for all IP addresses of your infrastructure.



Verify that letters are delivered to external servers using encryption (TLS). It is sometimes possible to check the presence of TLS by the Received header on the recipient's server: for example, specifying the ESMTPS protocol or the presence of type parameters (version = TLS1_2 cipher = ECDHE-RSA-AES128-GCM-SHA256 bits = 128/128); indicates the presence of TLS.

Verification of the generating application

Requirements for postal addresses

Envelope addresses

We will begin verification of the generating application from the addresses in the SMTP envelope.



Envelope addresses are addresses of the level of postal infrastructure. They are not visible to the user, but are important for delivery, because the mail box in which box is determined by the address of the envelope.



The recipient's address in the envelope (envelope-to, aka RCPT TO :) is the address to which the letter will actually be delivered.

• for all letters, except for registration, the address must be legally signed and validated for distribution in accordance with administrative requirements.
• for mailings, the address must be “live”, addresses to which it is regularly impossible to deliver letters should be marked as inactive, mailings should not be made to them. But some categories of letters (for example, restoration of access) may need to be sent to addresses previously marked as inactive.

The sender's address in the SMTP envelope (usually called envelope-from, smtp.mailfrom or Return-Path) - messages on the inability to deliver the letter and auto-replies will be delivered to this address. User this address is invisible. This address is also used to authorize SPF. Check that:

• the address is available;
• It is not an employee’s address and is not redirected to it in order to exclude auto replies, inaccessibility messages, etc .;
• processed by a script that will mark the addresses of inaccessible users as inactive;
• the address can be automatically generated, i.e. unique to each letter;
• a letter to this address should not lead to the generation of any response letter, for example, a message that the box is full.

Email address addresses



These addresses are either directly visible to the user or used when replying to an email.



The sender's address (From :) header is the address and the name of the sender displayed in the list of letters and when reading a letter. Check that:

• This is a “human readable” friendly address that identifies the company.
• Contains not only the e-mail, but also the name of the sender.
• noreply @ is possible, but only if we want to emphasize that we do not expect to receive an answer to the letter and it will not be read. It is better to duplicate this idea in the text of the letter.
• If there are non-ASCII characters (for example, Cyrillic), the sender's name must be encoded according to MIME, the domain if there are non-ASCII characters must be encoded in Punycode.
• Letters of the same category should have the same address; the use of auto-generated addresses is not allowed. This is due to the fact that From: is most often used by people to sort letters into folders using filters.
• The address should be different (better in different subdomains) for transactional, marketing and urgent letters (such as letters from the support service). This is also due to the fact that the user can mark marketing letters as spam or filter them into an unreadable folder.
• The From address and the SMTP envelope address must be in the same domain or in subdomains of the same organizational domain for passing SPF within DMARC.
• The address must be from the organization's domain. It is unacceptable to use the domains of free mail services and other public mails in From, since such mailings will not pass SPF and DKIM authentication within DMARC.

Address Reply-To - the “manual” answers will be sent to this address when the user responds to the letter. It is not obligatory: in its absence, the address from From is used for the response. Check that Reply-To:

• It can be autogenerated, i.e. unique to the letter (allows you to know which letter came to answer).
• It should not fall into the employee's box to avoid auto-responses, and ideally should be “wrapped” in CRM.
• It can generate a standard CRM auto-reply, but should not generate anything superfluous, for example, messages about a box overflow. When generating auto-responses, measures should be taken to avoid looping; they are listed in RFC 3834.
• It may be from any domain that does not necessarily coincide with From, but sometimes it scares users when they answer.
• May be absent, then its function is performed by the From address.
• In addition to the address indicated the name of the sender.

To address:

• Must contain the recipient's e-mail (otherwise it scares the recipient of the letter and the antispam alarming).
• Ideally, it should contain the name of the recipient. But if the name is unknown or doubtful (for example, the address is not confirmed yet), it is better not to indicate it (someone may enter someone else’s address with a bad name, and the recipient might be offended by you).

Mail Header Requirements

The actual encoding of the text must match the one specified in the title. It is advisable to use the same encoding in all headers and parts of the letter. It is recommended to use UTF-8 as widely supported. The encoding is specified in the Content-Type headers and in the <MTA> tag of the HTML part.



The headers From :, Message-ID: and Date: must be formed directly in the script to send the letter (and by standards - along with the text of the letter) and always in the correct format. In the case of their absence or incorrect formation, one of the transit servers can add these headers, which leads to a violation of the integrity of the DKIM signature.



8-bit characters in the headers, including the subject line (Subject) and the names of the attached files (Content-Type and Content-Disposition), should be absent; all non-ASCII characters, including Cyrillic, must be encoded in quoted-printable or base64.

Requirements for the structure of the letter

For the HTML part of the letter, it is desirable to form an alternative - text (plain) part. It is also necessary to check the conformity and readability of the plain text-part of the letter (if available) and the general structure of the letter.



According to RFC 5322, the length of a string in a letter should not exceed 998 8-bit characters. Note that in UTF-8, a character may take several octets. The string terminator in the e-mail is the CRLF pair (<cr> ascii 13, lf> ascii 10), which occupies 2 octets. It is necessary to check the correctness of the string terminator, since letters are often sent using a Unix script, and on Unix, a single character lf is used as a string terminator - for email, this is an error. You should also check whether the terminators break strings of characters in the UTF-8 encoding: you should not allow the presence of a terminator of strings between two octets of one character, for example, Cyrillic. , base64.



— «Content-Disposition: inline», , , «Content-Disposition: attachment», .

, , : , multipart- (mixed, alternative, related), multipart/mixed , multipart/related — , multipart/alternative — plain text- HTML-. , -, :



multipart/alternative

— text/plain

— text/html



, text/plain text/html multipart/related. , HTML-, - — plain-.



- :



multipart/alternative

— text/plain

— multipart/related

—— text/html

—— image/… (-)

—— image/… (-)



- Content-Disposition: inline multipart/related-.



, , , :



multipart/mixed

— multipart/alternative

—— text/plain

—— multipart/related

——— text/html

——— image/png

——— image/png

...

— application/octet-string (content-disposition: attachment)

— application/octet-string (content-disposition: attachment)

...



(multipart/related- multipart/alternative- , multipart/mixed-)

URI

URI ( src, href, ..) ( https://example.com/somepath ). (/somepath) (//example.com/somepath), , .. file://.

• ASCII- ( , ) URI percent encoding.
• , (.. URL, ), <>, . - , . href A , - . «», .
• htt://, htts:// milto:.
• http:// htts://.
• (, example.com :8080/somepath), .. .
• HTML- - (, , ..) , .. - , ; , , , ( - GET-, POST PUT).
• List-Unsubscribe, , - , .. .
• , , , (, ). . , , , , , .
• Since URI , URI data:. URI.
• , . , , .
• - .

?

. — (XSS, Crossite scripting), (interface spoofing), DOM clobbering, / (, IP- ) .., . , . :

• ,
• / ,
• ,
• ( ),
• (.. ) ,
• HTML- ( Microsoft Exchange / Outlook RTF, - Outlook ),
• .

, - URI cid://, . , Mozilla Thunderbird cid:// .

- - .



. , URI, - , - , . : , ( , ).



:

• , , , , .
• .
• , , , . ( ). . , .
• , .
• , , .. -, - , c, , ( , , , , - ). , , .
• .
• , , , . (, - ), , , , .
• .
• :
  
  
  • : « » Mail.Ru, Yandex, Gmail, Rambler Outlook.com;
  • ;
  • , IMAP, , iPhone, Pixel ( Android), Samsung ( Android), MIUI ( Android-);
  • : Chrome, Firefox, Edge, Internet Explorer, Opera .;
  • ( ), Thunderbird, Outlook Apple Mail, The Bat! Opera Mail;
  • - (Exchange, Roundcube, Communigate, Zimbra, SquirrelMail) — B2B-;
  • Retina-, .
• :
  
  
  • , SPF/DKIM/DMARC.
  • : , .
  • : , , , (, «»).
  • : , .., .
  • .
  • .
  • .
  • , . , , - , , .
    
    
    
    
    
    
    
    
• ( ) , . , .
• , , , .. , , , «» .
• . , DKIM- - ( DNS DKIM-, ), - ( , , From, Date Message-ID) - ( , , ). «» , .

-

, , .



, , ( ). . , , .



CTR — . postmaster.mail.ru . (), . CTR < 10% — , . CTR > 30%. CTR («») . ( ). , — . , , : https://help.mail.ru/developers/mailing_rules/technical .



- . CTR . ( ). «» — 10 000 , .



: , , . « » . , .



z3apa3a s4ever . EdT 4Alexander .