The network has quite a bit of information about such integration. The fact that the problem is solved is indicated by rare comments on the sites, but I have not found a ready-made solution. Perhaps this is due to the fact most use postfix / exim.

Objectives of the article:

1. Describe the problem solving technique to make it easier for those who follow in my footsteps;
2. To divert the attention of the community from the old-good-old postfix and company.

Introduction

In the process of moving the finished web / mail / server infrastructure from the old server to the new server, I was faced with the need to re-deploy the mail infrastructure.

The original mail system was deployed in 2012m year. Ansible then only was born. Configuration systems were not as common as they are now. I then loved automating the work less than now, so everything was configured by hand.

The new system should have been built on the docker (which means the most detailed description of building the system "for the future") and working in the docker cluster.

The configs of the old system are preserved, but I really did not want to go back to the "dnl" sendmail, a bunch of lines describing postfix logic, an incomprehensible process of setting up amavis + spamasassin, a lot of memory that amavis + spamasassin needed. It was decided to look for alternatives.

So that no one says “use some kind of marginal software because you haven’t mastered postfix”, I’ll clarify: mastered. The original system consisted of postfix + dovecot + amavisd-new + spamasassin + dkim-proxy + razor + pyzord + dcc.

Looking ahead: the final system consists of opensmtpd + dbmail + dkim-proxy + rspamd.

Some theory

opensmtpd is a lightweight mail daemon, with an elementary configuration process from the OpenBSD command. The rules for configuring are so simple and intuitive that they are immediately remembered and allow you to describe in ten lines the complete logic of the mail server.

dbmail - mail storage with pop / imap interface. The main feature is mail storage in the database, which allows you to create an arbitrary number of entry points.

rspamd - mail anti-spam system from developers from Rambler.

More alternatives? Why not Zimbra? Why not some kind of ready-made solution like mail-in-a-box , which contains all the necessary components?

Zimbra fell away because of Java - I did not want to spend resources.

mail-in-a-box - again, all the same postfix and company.

Description of the problem

opensmtpd is a great daemon, but it has flaws that make it difficult to integrate it with rspamd.

It:

1. Lack of support for the "milter" protocol. This protocol (extension) would allow integration with rspamd using standard tools.

   
   

2. No filter support. The rspamd developer for integration recommends the use of opensmtpd filters. The development of filters has been going on for several years already, but still not in production. There is even a filter for rspamd, but still in a separate thread on github.

   
   

   After certain improvements in the code, I succeeded in compiling and running this filter, but it gave errors in the process of work and there was no further desire to understand the code.

   
   
3. No method of receiving mail except by mail protocols.

Also, the situation was complicated by the following points:

1. Remoteness of services from each other: demons work in docker containers with different IP. In fact, the container with opensmtpd and the container with rspamd can be thought of as running on different servers.

   
   
2. Software heterogeneity: opensmtpd is in the stable version of Alpine Linux (the linux version, which is often used as the base Docker image), but there is no rspamd there. And vice versa: rspamd is in the unstable version of Alpine, but opensmtpd does not work there.

So, as the original task, we have opensmtpd, which is waiting for incoming mail on port 25 and waiting for mail on port 10029, which rspamd has checked.

Solution to the problem

In opensmtpd there are mda. Probably, it stands for mail delivery agent. Redirecting a letter to mda causes an external program to start. Example of the letter redirection rule:

accept tagged IN from any for domain <domains> userbase <virtual_users> deliver to mda "/usr/local/bin/rspamd.sh %{sender} %{rcpt}" as admin 

Here we call some external script, where we send the sender's address and the recipient's address.

Included with rspamd is rspamc - this is an analog mail mta for rspamd. This program allows you to accept a letter as input, transfer it to rspamd via HTTP API and issue a response.

The integration of mda and rspamd is as follows

 /usr/bin/rspamc --mime -h rspamd.example.ru:11333 

On the STDIN script, we send a letter, specify the rspamd address and say the --mime parameter to add message handling headers (which will be used to determine spam later).

After the letter has been processed by rspamd, it must somehow be sent back to opensmtpd. But the standard mta (complete with opensmtpd) can send only to port 25 (we are waiting for a verified letter on another port).

Solution: to form communication over the mail protocol itself (well, it is simple) and send data to the correct port using netcat.

The final mda script rspamd.sh integration opensmtpd and rspamd

 #!/usr/bin/env sh mail_file=$(mktemp) echo 'HELO localhost' >> $mail_file echo "MAIL FROM: <$1>" >> $mail_file echo "RCPT TO: <$2>" >> $mail_file echo 'DATA' >> $mail_file /usr/bin/rspamc --mime -h rspamd.example.ru:11333 >> $mail_file echo '' >> $mail_file echo '.' >> $mail_file echo 'QUIT' >> $mail_file cut_file=$(mktemp) sed '/Delivered-To/d' $mail_file > $cut_file rm "$mail_file" count=0; IFS=''; while read -r line ; do if [ "$count" -gt "5" ]; then sleep 0.05; else sleep 0.1; fi; echo "$line"; count=$((count+1)); done < "$cut_file" | netcat localhost 10029 rm "$cut_file" 

Features of this script:

1. Sending data via netcat should be delayed after each line of the connection setup phase. To do this, we make delays between the return lines.
2. From the original letter, cut out the "Delivered-To" header, which adds opensmtpd. Otherwise, opensmtpd will consider this letter to be looped.

Solving the problem of combining opensmtpd and rspamd in one container: using the Docker image based on Debian sid, where the rspamd repository is connected and the latest version is installed from it. Installation example:

 wget -O- https://rspamd.com/apt-stable/gpg.key | apt-key add - \ && echo "deb http://rspamd.com/apt-stable/ sid main" > /etc/apt/sources.list.d/rspamd.list \ && apt-get update \ && apt-get install --no-install-recommends --no-install-suggests -y \ rspamd 

Results

As a result of switching from postfix + amavis to opensmtpd + rspamd, I got the simplicity of the service configuration, low memory consumption and less spam in the mail.

The implementation of the final Docker container can be viewed here .