The obsession with mailing efficiency: a couple of tests and stories

A year ago, I found out about the site of masquerade costumes that Yandex had banned. The problem was that they received the lion’s share of customers from the search, and not from the street, so an SEO error could cost business owners.

Do you know what they did? They took their customer base and started making a useful newsletter, which allowed them to survive a couple of months without orders. More precisely, I switched the main channel to repeat sales.

And then I started experimenting with mailings. Below are the results.
')

Training

• The first step was mega-transparent and simple unsubscribe. At the time of launch, the mechanic worked like this: it was necessary to go to the personal account on the site and from there turn off the newsletter. Given that a person could forget the password, this turned into a quest. The ideal solution would be a simple link, clicking on which the subscriber could forget about letters from us once and for all. What we did then.
  
  
• The second step - needed the correct layout of mailings. Fortunately, right on Habré was found Dudeonthehorse , who wrote a series of topics about the layout of mailings. I almost accidentally met him in real life and immediately ordered an audit of our code. The result is a template that is guaranteed to be normally displayed in all main mail clients (including Outlook — it is like IE in this area), Mail.ru web interface (it was scary then), and other clients and web services.
  
  
• The third step is to check how many letters reach the recipients. We started dozens of control boxes on different services (they were useful for layout tests), plus we added delivery statistics: the most detailed one is at Mail.ru, it shows, in particular, the number of spam complaints. The delivery level was not very good: some letters were lost, some were delayed for almost a day.
  
  
• The fourth step - you need detailed statistics about the passage of customers on the site, their purchases and behavior in general. Everything was decided here quite simply: the OpenStat tags on each link - and forward to Metric, where the goals and parameters of the online store are set correctly.
  
  
• The fifth step was the outsourcing of the mailing organization. Previously, we sent from our servers - later we moved to Unisender, who provided a very good level of delivery. As a nice bonus, we got good statistics on clicks on specific links in the letter and ready-made split-test mechanics. Before that, I had to share the base with my hands and test it. Plus, they automatically add a link to the unsubscribe in each letter, which solves the issue even more conveniently than our mechanics (because the user can simply unsubscribe in 1 click, and can tell why he did it).
  
  By the way, the best reason for the reply - " I bought you a Monopoly, invited a spectacular girl to visit, but we broke up that day ."
  
  

On the site I will say separately. There are options with a large guaranteed reach, but the difference of 1% is fully compensated by very responsive technical support. The only cant of the site that was not healed is that when working through the API it is very difficult to load large data sets at once (more than 50,000 records), which is why our developers made a bunch of microtransactions. Each transaction meant a letter from your personal account (hundreds of emails per day) - and it did not turn off in the settings. As a result, I had to configure the mailbox filter to land emails from the mailing service directly into spam, which is somewhat contrary to its philosophy :)

Tests

We now turn to the most interesting. The first question is that the graphic should be a “wrapper” for the letter or you need to send plain text, as in a regular mail message. There were two AB-tests: one with our “heavy” graphic layout with a bunch of photos against plain text, one with a light and minimalistic layout from a freelancer in comparison with the same plain text. The text on average beat the graphics by 10% on clicks and sales.


^{Draft of the second version of the test (before reading). It seemed that the drawings were very important in the list of games - but no.}

About the split tests Unisendera should be said separately. The fact is that they offer a logical mechanic: take a base segment, for example, at 20%, send one version to half of this segment, another version to another part, and then compare results after 3 hours. Once I tested the effectiveness of the typeface assignment in a regular text list. After three hours, I had a completely stunning result: it turns out that the designated Tahoma increases conversion by 50%. It was something fantastic. But a day later the statistics leveled off, and quite later it became clear that the variant without a designated font wins by 3-4%.

That is, if I made a standard split with the default settings, I would see only the first part of the post-effect, and I would not know about the second. There was a question: maybe the fact is that the split was sent first to old customers, and then to new ones? But no, I asked the site - it turned out they send it by pseudo-random samples, that is, they mix the base.

Accordingly, the next question I asked, having such results in my hands, is the error of the split test as such. Here are two identical mailings with a difference of one non-breaking space in the code:


^{The difference in conversions is less than 2%; dispatches to spam and formal replies are about 20%}

At the same time, the wrong part of the newsletter was appointed the winner by the results of 3 hours - even if the difference is only a couple of percent, but this is important. It was the same with splits, where the difference was up to 10% after two or three days - the wrong part won. Therefore, be more careful: either send the split for a long time, or, if not, to the entire base.

The next important question is a long or short text. We then went to the screen in size, and in a couple of lines, and in the "medium format". It turned out that the length is completely unimportant. If the text is interesting, then read to the end.

Another question is headlines. After each mailing to us on Habré write, so we began to use large headlines. So, they reduce the conversion contrary to logic. In my opinion, the fact is that each of our letters is a small story, and a personal story cannot contain headlines. Most likely - the question is the same as with the pictures. If the format of your letters is different - yes, most likely, they will help.

Cross mailing

Once we tried to exchange gifts with Zapato: they told their customers about special conditions with us and gave our customers special conditions at their place. In theory, it sounds that everyone is happy: we send our newsletter to one of our base segments, we send them to one of ours. There is a small story there - they wanted to send just a long banner with the inscription "700 rubles to Mosigra's customers as a gift *", but I insisted that the exact conditions from the "asterisk" were necessarily read in large, capital letters. Despite this, the effect was very painful: more formal replies went by, plus many appeals that we were doing something wrong. I was not too lazy to make the conversion of all our partners, counterparties and friends even further - and I found out that with the approach of exchanging the same segments, we give 3-4 times more users because of the significantly higher conversion and higher quality of the audience.

Segmentation

It is clear that the power of mailings is in the segmentation of the base. More precisely, sending letters on the right occasions to the right people. We first messed up when we sent personal offers the day after the general mailing (to people it seemed that we had dramatically increased the frequency of mailings, the outflow of unsubscriptions increased). On the other hand, the conversion was impressive. For example, there were 10% of sales (!) For several thousand people - and this is a normal result without a discount or megabonuses, only due to the exact segment and the correct text of the letter. From the anecdotal - even a complete conversion on a segment of 6 people (although there is a three-year shopping history for everyone, that is, the sample is not quite normal). Now I can rightfully say that I saw a mythical 100% conversion.

Another interesting segment is those managers who order branded games and corporate gifts. There, the specifics are generally separate: people simply don’t have time for it, time is short, everything superfluous is cleared from the mail without going inside. Therefore, we made segments, but did not begin to use mailing services - we simply sent letters by hand, each one separately. At the same time it was possible to add details about this particular client and recommendations only for him. It takes a lot of time for the employees of wholesale, but every second is worth it.

By the way, if you suddenly want to do this, be sure to warn your specialist not to send at all at once. And then I have already received letters to the entire client base of an advertising agency three times with a bunch of large managers, to which I invariably answered: "Hello everyone in this chat, and we have cool board games!" Which is typical - at least two large contracts we got.

Personal relationship

The most important thing for sending is to be trusted. So it works much better - but it only becomes noticeable in six months, for example. Therefore, it was extremely important to take one not very obvious solution for sales - namely, to send a newsletter with simply useful news that does not directly lead to direct transitions. That is, the task was to write like in a personal blog, that is, letters that are interesting to read by themselves.

The second step to high involvement is for the newsletter to be sent by one specific person who is not afraid to sign with his name. With whom you can talk, who can answer and ask for advice. We previously sent mailings to Anya, but once, when she fell ill, another girl, Olga, from the event department sent a letter. In the first lines there was something like “Hello! Usually, Anya writes to you, but now she is ill, therefore I write. ” I think it was a turning point for us: many letters with rays of light came to Anya and a bunch of letters to Ole already on the newsletter plot. Well, the sheet of text there was the size of a screen - and at the same time there were more people reading it than usual. It seems that thanks to this occasion, many have noticed that they are written by a living person, which has become a real event in the world of standard letters from stores. Now the mailing is sent by Marianna, on the first one there came many letters with the request to return Anya back.

PS And yes, we are still children in relation to mailings, so there is still room to grow and grow.