Generate text versions of letters from HTML using lynx

Introduction

Emails sent through Pechkin, for the most part, contain both the html version of the letter and the plain-text version. You can send a newsletter without the HTML version, but by sending it without plain-text, you risk losing a number of subscribers whose email clients cut out the html versions of emails for security reasons without regret. This is especially different corporate accounts, mail administrators who care about protecting their wards from spam and viruses.

Set the task

So, a plain-text version of the letter is necessary, it is a fact. But how difficult it is to force yourself to write it if you just spent a lot of time creating the html version of your newsletter. It is obvious that Pechkin should remove this load and automatically generate a text version of the letters for our clients. But how to do that?

We tried a lot:

• various regular expressions
• a bunch of different libraries for parsing html

')
Everything was crooked and out of place, the class for generation increased in front of our eyes, and the shoals followed the jambs. But we needed a simple and beautiful solution that would allow:

• highlight headings and subheadings
• highlight and neatly display links
• keep track of the number of hyphenations and carefully place blocks with text for their readability
• do it all fast

Lynx solution

And here, unexpectedly for us, a solution emerged with the help of technology from the 90s of the last century, which appeared at the dawn of the emergence of the Internet. Text browser! Well, of course, we feed him our html-code and at the output we get its excellently displayed text version, with all the links, headers and so on. Lynx was taken as a base, it works quickly, and one of our developers used it earlier for their SEO goals.

I had to tune it a little for the perception of the Russian language in our newsletters. Settings in lynx.cfg on Debian (the encoding of all mailings on Pechkin is UTF-8):

LOCALE_CHARSET:FALSE ASSUME_CHARSET:utf-8 CHARACTER_SET:utf-8 MAKE_PSEUDO_ALTS_FOR_INLINES:FALSE 

And, voila, with a simple command:

 lynx -dump 

We get a beautiful text version of the letter.

As a result, instead of a massive class for generation, we implemented the simplest function that Pechkin uses every time you create a mailing list.

This automatic converter is available in Pechkin. Laboratory for everyone, even if you do not have an account on Pechkin.

Link to the demonstration of work: labs.pechkin-mail.ru/html2text