New old method of protection against email spam based on MTA Exim
I want to provide a description of the method of protecting corporate mail from spam, which allows you to use the advantages of individual address filtering tools, while avoiding the drawbacks of the same methods.
It can be emphasized that these techniques can be used on the SMTP proxy, which closes the corporate mail server located in the DMZ.
Often, administrators avoid some effective filtering techniques, due to the shortcomings of this or that approach. For example - DNSBL filters often give false positives to those nodes that fall into it by mistake - for example, as part of the whole block of addresses of an individual provider. A frequently used filtering method based on a simple definition of a PTR record also has a tendency to fail when the A and PTR records do not match, or simply have problems with the DNS service.
In this article, I want to show how to break individual filtering methods into smaller ones and operate with filtering on the totality of the data about the sending node, and not only on the result of one prohibiting rule.
')
This technique has existed for a long time, I met different implementations of this idea by different specialists, and this variation was described more briefly by me 5 years ago in the exim-users@exim.org mailing list (the article can still be found in the mailing list archive ), but despite on the ease of implementation and availability of documentation, they are now rarely used by postal administrators.
Using the example of the Horns'n'Hoofs company mail with the hornsnhoofs.com domain, we will try to consider not imaginary, but quite efficient “in battle” filtering techniques.
The main idea of ​​this implementation is that none of the checks is “critical”, except for the server’s own blacklist stored in the SQL database. In other words, we do not refuse to use DNSBL, nor check for direct and reverse DNS match tests or any other tests for host “spam”, but do not prohibit the host to continue the SMTP session if it has not passed any This is a separate check (for example, it lights up in SpamHaus DSNBL).
Each failed test only adds a certain amount of points to the “spaminess” of the letter, and the decision to accept or reject is made based on the total amount of these points at several “control points” of the configuration. This approach allows you to use a variety of tools for evaluating email senders and, at the same time, reduce the level of filter false positives (so-called false positives).
The article assumes that the reader is able to install and configure Exim as a receiving server. I also rely on the fact that the reader is able to write at least simple lookups for Exim.
Of course, the SMTP protocol should be known to any competent admin, and the principles of his work are perfectly described in RFC 821, 2821 and 5321, whose translations into Russian can be easily found on the network. A theoretical description of many ways to protect against spam can be found in RFC 2505.
So, let's proceed to the description of the mail server configuration:
The total points earned will be stored in the
First, we set its initial value to 0. For example, in acl, which is responsible for checking the argument of theHELO MAIL FROM:
UPD: thanks to the slimlv user who noticed a logical error.
Why initialization does not happen immediately when connected? Very simple - the counter is also reset when the RSET command is sent, which restarts the SMTP session. If this is not done, the number of points would remain the same and new points would be added to it for the same checks made before the RSET team.
This is inconvenient when debugging the work of the MTA, in the combat system it doesn’t matter where the counter will be reset - immediately when the node is connected (in acl_smtp_connect), or after MAIL FROM transfer, as in the example.
Another important variable is
The bulk of the checks are enclosed in the
Start over:
Once we have recorded in the server log, from which email and ip-address the letter was sent, you can disable (drop) the node if it is in the local “black list” of the server (in MySQL database), so as not to generate too much traffic with additional SMTP commands, as well as DNS and DNSBL queries.
The blacklist itself is a table in a database consisting of two fields:
An example of a request to the blacklist is very simple:
Working with the database from the mail server is also implemented elementary. To do this, add a directive to the main configuration section:
So we form a list of one ip-address (or from zero addresses, if the request did not return anything), which we later query as
Of course, you can not forget about the
The filling of the blacklist table with ip-addresses occurs automatically by the MTA itself, which will be shown below.
And now we look at the anti-spam "warhead":
Where trusted_zones is a plain text file in the additional directory in the folder with the configuration of Exim. It contains something like the following:
Regular expressions describe those domain zones, from whose email addresses (DNS has nothing to do with it), correspondence usually comes. The example above contains the necessary minimum, which, if necessary, can and should be edited.
The directive warn indicates to eczema that the letter should neither be accepted nor rejected. It is only necessary to fulfill the condition and process the session further.
From this, it turns out that a node sending a domain zone that is uncharacteristic of incoming mail in the sender's address (not to be confused with the “From:” header) will receive 20 points and move on:
+30 points - for the mismatch of direct (A) and reverse (PTR) DNS records.
Another 50 - for the lack of reverse (PTR) records.
Eighty points are already enough to put the knot in the greylist, as will be shown later.
You can see that the
Its value can be set when the node is connected (in
initialized
But keep in mind that each such lookup will generate a DNS query. With a large flow of mail (spam) it will create unnecessary burdens. With a small flow, you can hardly even feel the difference.
The sending node will receive +70 points if its DNS record points to some dynamic address pool, since such pools are a breeding ground for viruses and, as a result, good soil for botnets.
The dynamic_pools file is similar in structure to the trusted_zones file and contains regular expressions for checking DNS records for modems, adsl users and other nodes with dynamically allocated ip:
You can add your own rules here, this is just the necessary minimum.
The same trusted_zones, which was previously used to verify emails, is now used to filter by real DNS records.
+20 points if the mail comes from the Chinese, Mexican, Korean and other, not listed in the list of domain zones.
+40 points to individual major providers who do not care about filtering outgoing traffic generated by botnets.
The spamvertised_isp file lists many large providers that allow their modems to make outgoing connections to port 25 (calculated by logs):
You can add your notes. Even need.
+60 points to spammers, indicating as an argument HELO our mail server (receiving).
+20 spam points if HELO is not a FQDN.
Another 20 if the HELO argument does not match the primary DNS record (A) for the sending host.
Good old trusted_zones is now used to check the HELO argument.
+20 points to Chinese, Koreans and other Mexican Japanese.
And finally - the correct implementation of the DNSBL system survey:
In no case can you turn off a node that glows only in one such system. There is a high probability that he got there by accident.
Do not completely abandon the use of DNSBL - it is a powerful filtering tool.
Only if you simultaneously hit two or more of these systems, you can declare a node as a spam sender.
+60 points for getting into any of the “black DNS lists”. If you hit the two lists at the same time, the node will receive 120 points, which is enough to stop receiving mail from it, but not enough to automatically add to the local blacklist. If the admin of the sending mail server has time to quickly unsubscribe from at least one DNSBL, then the transfer of mail will suffer minimally.
DNSBL polling in combination with other checks - eliminates spammers very well and it is often these points that become decisive for placing the ip-address in the local blacklist for a week.
This is where callout is made. The server is given a maximum of 3 minutes to check (otherwise there is a risk that the “good” sender will fall off without waiting for the end of the check, it hangs connected), the unavailability of the remote node is considered to be successful passing the check. Those. The node will receive 60 points if it sent mail from an absent (most often - fake) email address.
That is why you should not make legitimate emails from addresses like www@webserver.example.org. Callout is a common filtering method and is supported by many current implementations of MTA.
There is a little trick - nodes with more than 145 points are true candidates for a local blacklist. There is no need to check them more, we transfer them to the next acl and there is a strong ban.
Who has not scored 145 points - passes the torture by delaying the session: the receiving MTA imitates a bad connection and “hangs” for a number of seconds equal to half the number of “spam points”. For example, a node with 60 points will be “suspended” for 30 seconds.
Usually spammers do not have that amount of time waiting for an answer and they fall off in 15-20 seconds.
So we got to the last acl, checking the correctness of the parameters of the SMTP session (it works after RCPT TO :). The “suspension” of the sum of points and the decision of the further fate of the letter occur in it:
+50 points for getting into a spam trap.
The list of trap addresses is in the spamtraps file, by email to the line:
And so on.
Using your own spam traps is a good way to hide from the big mailers, digging internet bots for emails. Then these addresses are transmitted, sold and just distributed as part of databases for mailings. That is - spread by spammers among themselves.
The difficulty arises in the initial distribution of such addresses, so that they would be sent to spammers. For example, for this you can use a mechanism such as wpoison on the company's main site. The main thing is not to forget about the prohibition of their indexing by search engines (via robots.txt or META CONTENT = “NOINDEX, NOFOLLOW”).
Often, old domains already have abandoned or even never existed addresses, which, however, goes one spam. These are ideal candidates for use as spamtraps. The main thing is that they do not use such a box for a long time, and even better so that they are never used.
Paradoxically, but the fact is that many of the trap addresses I use today have never existed anywhere except in spam databases. It is not known how they appeared there, but they regularly send spam. Study the logs of your mailer for a long period, for sure there will be something similar there.
Those who could score 150 points and more are declared champions! Their ip-addresses are placed in the local blacklist and the fame of them does not cease the week.
The nodes that are in it are dropped right after the “MAIL FROM:” command is sent.
As already mentioned, the life of an entry in my blacklist is a week (604800 seconds). Bases are cleaned with a crown every hour:
It's simple. Scored more than 100 points - received "550 Message rejected.". In this case, the log records the number of points scored and reports on checks inundated. The same information goes along with the mail trap on the other side of the wire. Suddenly someone will come in handy for debugging.
It is important to send an bluff without breaking the session (not bringing the letter to the reception in acl and starting processing by routers and transports), otherwise the events may turn out not very good for you: in the worst case, your server can be used as a sender.
Briefly about sending spam as a return: if you write a spam letter to a non-existent email address nosuchaddress@hornsnhoofs.com, pointing to “MAIL FROM:” address vasyapupkin@gmail.com, and the letter will be accepted by the mail server, and after receiving the mail the router determines that the destination address does not exist, then the server will wrap the spam letter in a new envelope, add the lines of its non-response and return it to the “sender” - to the unsuspecting Vasya Pupkin on Gmail.com.
Further, the corporation of dough will quickly add your server to the “black list” and block the reception of mail from its ip. Now you are writing justification letters to tehsapport, and the little spammers giggle.
However, this is a very brief description of the phenomenon and there are other ways to protect against it, but the tactics of transferring an arc pound without stopping the session are the most correct.
You can also add this info to the service headers of the letter, for easier debugging. But it is very optional.
Nodes with less than 70 spam points are considered legitimate, we accept a letter from them if the recipient's address (
I will not describe how this variable gets its value, since this depends on the way the data about the mail users is stored. I will say that the true value (or just 1) from me it gets after checking the email address of the recipient in Active Directory through an LDAP search.This can be done for any database with users, as long as it is supported by eczema.
The most interesting thing happens with nodes that have collected from 70 to 100 points. We cannot refer them to either legitimate hosts or spam mailers. Therefore, we wrap them in a greylist for 29 minutes (the interval is chosen with the expectation of the MTA-sender for the second run of the mail queue):
defer , c 4xx, , .
: SMTP- 451 , « », .
«», : ip- .
, , greylist, :
The tables are also cleared by the crown: the records are stored in the greylist for 24 hours, for whitelist - for the month.
The last filtering rule, according to recommendations of the best dog breeders, is prohibiting His work is somewhat similar to the last rule in a “closed” firewall
Additionally, for each incorrect address in “RCPT TO:” (SMTP protocol allows you to specify several recipients for one letter), the node receives 5 additional spam points, up to and including the unconditional blocking threshold.
Oh yes - torture a delayed session after each “RCPT TO:” also has a place to be.
Using at least these constructions in the mail server configuration will cut off about 90-95% of spam without increasing the number of false positives.
Creative independent use of this technique (counting and weighing spam coefficients) makes it possible to get rid of spam almost completely (except, perhaps, mailings from public mail systems like gmail.com or mail.ru, but there, as a rule, the account is also monitored block) without the use of context filtering systems.
And if the context filter is also planned to be used, this way you can unload the text analyzer due to a strong reduction in the amount of input data.
In the article I didn’t deal with such technologies as SPF and DKIM, but if you wish, you can use them as “whitening functions” (reducing the number of spam points), working in the same way as those that were considered. If the community has an interest in these systems, I will try to consider them separately.
It can be emphasized that these techniques can be used on the SMTP proxy, which closes the corporate mail server located in the DMZ.
Often, administrators avoid some effective filtering techniques, due to the shortcomings of this or that approach. For example - DNSBL filters often give false positives to those nodes that fall into it by mistake - for example, as part of the whole block of addresses of an individual provider. A frequently used filtering method based on a simple definition of a PTR record also has a tendency to fail when the A and PTR records do not match, or simply have problems with the DNS service.
In this article, I want to show how to break individual filtering methods into smaller ones and operate with filtering on the totality of the data about the sending node, and not only on the result of one prohibiting rule.
')
This technique has existed for a long time, I met different implementations of this idea by different specialists, and this variation was described more briefly by me 5 years ago in the exim-users@exim.org mailing list (the article can still be found in the mailing list archive ), but despite on the ease of implementation and availability of documentation, they are now rarely used by postal administrators.
Using the example of the Horns'n'Hoofs company mail with the hornsnhoofs.com domain, we will try to consider not imaginary, but quite efficient “in battle” filtering techniques.
The main idea of ​​this implementation is that none of the checks is “critical”, except for the server’s own blacklist stored in the SQL database. In other words, we do not refuse to use DNSBL, nor check for direct and reverse DNS match tests or any other tests for host “spam”, but do not prohibit the host to continue the SMTP session if it has not passed any This is a separate check (for example, it lights up in SpamHaus DSNBL).
Each failed test only adds a certain amount of points to the “spaminess” of the letter, and the decision to accept or reject is made based on the total amount of these points at several “control points” of the configuration. This approach allows you to use a variety of tools for evaluating email senders and, at the same time, reduce the level of filter false positives (so-called false positives).
The article assumes that the reader is able to install and configure Exim as a receiving server. I also rely on the fact that the reader is able to write at least simple lookups for Exim.
Of course, the SMTP protocol should be known to any competent admin, and the principles of his work are perfectly described in RFC 821, 2821 and 5321, whose translations into Russian can be easily found on the network. A theoretical description of many ways to protect against spam can be found in RFC 2505.
So, let's proceed to the description of the mail server configuration:
The total points earned will be stored in the
$acl_c_spamscore variable. This is the main variable in the configuration program, all the other behavior of the MTA depends on its value.First, we set its initial value to 0. For example, in acl, which is responsible for checking the argument of the
acl_check_sender: warn set acl_c_spamscore = 0 [...] accept UPD: thanks to the slimlv user who noticed a logical error.
Why initialization does not happen immediately when connected? Very simple - the counter is also reset when the RSET command is sent, which restarts the SMTP session. If this is not done, the number of points would remain the same and new points would be added to it for the same checks made before the RSET team.
This is inconvenient when debugging the work of the MTA, in the combat system it doesn’t matter where the counter will be reset - immediately when the node is connected (in acl_smtp_connect), or after MAIL FROM transfer, as in the example.
Another important variable is
$acl_c_bouncemessage , in which messages are added about the results of all checks. It is necessary for high-quality debugging of the MTA. You will immediately see in the server log what checks were not passed and how many points were scored, and admins reading mail failures at the other end of the wire will be able to understand why the SMTP session was broken and correct the error (yeah, dreaming; 95% of them, alas , only scratches turnip). However, there are so few false positives with a properly configured filtering system (I have only one over the last year) that I use this variable only for my own convenience when debugging.The bulk of the checks are enclosed in the
acl_check_sender section (the phase of the SMTP session, coming after the transfer “MAIL FROM: <email@address.any>”):Start over:
acl_check_sender: [...] warn set acl_c_spamscore = 0 drop hosts = +blacklisted_hosts message = Connection closed. IP [$sender_host_address] is listed in Blacklist. [...] accept Once we have recorded in the server log, from which email and ip-address the letter was sent, you can disable (drop) the node if it is in the local “black list” of the server (in MySQL database), so as not to generate too much traffic with additional SMTP commands, as well as DNS and DNSBL queries.
The blacklist itself is a table in a database consisting of two fields:
IP varchar(15) and Timestamp int(11) , where the ip-address and the time it is added to the database are stored in the unix_time format (in this format, it is convenient to carry out operations seconds):An example of a request to the blacklist is very simple:
SELECT IP FROM antispam.blacklist WHERE IP='1.1.1.1' limit 1" .Working with the database from the mail server is also implemented elementary. To do this, add a directive to the main configuration section:
hostlist blacklisted_hosts = ${lookup mysql {SELECT IP FROM antispam.blacklist \ WHERE IP='$sender_host_address' limit 1} \ } So we form a list of one ip-address (or from zero addresses, if the request did not return anything), which we later query as
+blacklisted_hosts in access lists.Of course, you can not forget about the
hide mysql_servers = 127.0.0.1/antispam/mta/mtapass directive hide mysql_servers = 127.0.0.1/antispam/mta/mtapass at the very beginning of the config, which contains the parameters for connecting to the database.The filling of the blacklist table with ip-addresses occurs automatically by the MTA itself, which will be shown below.
And now we look at the anti-spam "warhead":
warn !condition = ${lookup{$sender_address_domain}wildlsearch{/CONFIG_PREFIX/\ additional/trusted_zones}{1}{0}} set acl_c_spamscore = ${eval:$acl_c_spamscore+20} set acl_c_bouncemessage = $acl_c_bouncemessage Suspicious e-mail address; Where trusted_zones is a plain text file in the additional directory in the folder with the configuration of Exim. It contains something like the following:
^.*\\.ru\$ ^.*\\.ua\$ ^.*\\.by\$ ^.*\\.com\$ ^.*\\.org\$ ^.*\\.net\$ ^.*\\.edu\$ Regular expressions describe those domain zones, from whose email addresses (DNS has nothing to do with it), correspondence usually comes. The example above contains the necessary minimum, which, if necessary, can and should be edited.
The directive warn indicates to eczema that the letter should neither be accepted nor rejected. It is only necessary to fulfill the condition and process the session further.
From this, it turns out that a node sending a domain zone that is uncharacteristic of incoming mail in the sender's address (not to be confused with the “From:” header) will receive 20 points and move on:
#-----------------------------DNS Records verify------------------------------------ warn !verify = reverse_host_lookup set acl_c_spamscore = ${eval:$acl_c_spamscore+30} set acl_c_bouncemessage = $acl_c_bouncemessage Reverse host lookup failed; +30 points - for the mismatch of direct (A) and reverse (PTR) DNS records.
warn condition = ${if eq {$acl_c_reverse_zone}{}} set acl_c_spamscore = ${eval:$acl_c_spamscore+50} set acl_c_bouncemessage = $acl_c_bouncemessage No DNS PTR record found; Another 50 - for the lack of reverse (PTR) records.
Eighty points are already enough to put the knot in the greylist, as will be shown later.
#----------------------------------------------------------------------------------- You can see that the
$acl_c_reverse_zone variable appeared here, which contains the result (the DNS value of the PTR of the node) of the following test: set acl_c_reverse_zone = ${escape:${lookup dnsdb{ptr=$sender_host_address}}} Its value can be set when the node is connected (in
acl_smtp_connect , which is more correct) or when checking the HELO argument in acl_check_helo whereinitialized
$acl_c_spamscore . The difference is small. You can generally use the construction everywhere: warn condition = ${if eq {${escape:${lookup dnsdb{ptr=$sender_host_address}}}}\ {}} But keep in mind that each such lookup will generate a DNS query. With a large flow of mail (spam) it will create unnecessary burdens. With a small flow, you can hardly even feel the difference.
#-----------------------------Dynamic IP pools processing--------------------------- warn condition = ${lookup {$acl_c_reverse_zone}wildlsearch{CONFIG_PREFIX/\ additional/dynamic_pools}{1}{0}\ } set acl_c_spamscore = ${eval:$acl_c_spamscore+70} set acl_c_bouncemessage = $acl_c_bouncemessage Suspected PTR DNS record \ points to dynamic IP pool; The sending node will receive +70 points if its DNS record points to some dynamic address pool, since such pools are a breeding ground for viruses and, as a result, good soil for botnets.
#----------------------------------------------------------------------------------- The dynamic_pools file is similar in structure to the trusted_zones file and contains regular expressions for checking DNS records for modems, adsl users and other nodes with dynamically allocated ip:
^.*([0-9]+).([0-9]+).([0-9]+).([0-9]+).* ^.*host.([0-9]+).* ^.*dynamic.* ^.*dial.* ^.*ppp.* ^.*pptp.* ^.*broadband.* ^.*dhcp.* You can add your own rules here, this is just the necessary minimum.
#---------------------------Geographical DNS zone processing------------------------ warn !condition = ${lookup {$sender_host_name}wildlsearch{/CONFIG_PREFIX/\ additional/trusted_zones}{1}{0}} set acl_c_spamscore = ${eval:$acl_c_spamscore+20} set acl_c_bouncemessage = $acl_c_bouncemessage Untrusted domain zone; The same trusted_zones, which was previously used to verify emails, is now used to filter by real DNS records.
+20 points if the mail comes from the Chinese, Mexican, Korean and other, not listed in the list of domain zones.
#------------------------------------------------------------------------------------------ #-------------------Huge DSL & DialUp ISP's DNS zone processing--------------------- warn condition = ${lookup {$sender_host_name}wildlsearch{CONFIG_PREFIX/\ additional/spamvertised_isp}{1}{0}} set acl_c_spamscore = ${eval:$acl_c_spamscore+40} set acl_c_bouncemessage = $acl_c_bouncemessage Spamvertised ISP DNS zone; +40 points to individual major providers who do not care about filtering outgoing traffic generated by botnets.
#----------------------------------------------------------------------------------- The spamvertised_isp file lists many large providers that allow their modems to make outgoing connections to port 25 (calculated by logs):
^.*comcast\\.net ^.*pppoe\\.mtu-net\\.ru ^.*qwerty\\.ru ^.*ono\\.com ^.*virtua\\.com\\.br You can add your notes. Even need.
#----------------------------Handler for impossible HELO's------------------------- warn condition = ${if or {\ {match{$sender_helo_name}{localhost}}\ {match{$sender_helo_name}{mail.hornsnhoofs.com}}\ {match{$sender_helo_name}{^127\\.0\\.0\\.([0-9]+)}}\ }{1}{0}\ } set acl_c_spamscore = ${eval:$acl_c_spamscore+60} set acl_c_bouncemessage = $acl_c_bouncemessage HELO $sender_helo_name is forged; +60 points to spammers, indicating as an argument HELO our mail server (receiving).
#---------------------------------------------------------------------------------- #------------------------------Handler for wrong HELO's---------------------------- warn !condition = ${if or {\ {match{$sender_helo_name}{^.+\\.((?i)[az]+)\$}}\ [...] }{1}{0}\ } set acl_c_spamscore = ${eval:$acl_c_spamscore+20} set acl_c_bouncemessage = $acl_c_bouncemessage HELO name is not \ Fully Qualified Domain Name; +20 spam points if HELO is not a FQDN.
#---------------------------------------------------------------------------------- #--------------------------Handler for forged HELO arguments----------------------- warn !condition = ${if or {\ {eq{$sender_helo_name}{$sender_host_name}}\ }\ } set acl_c_spamscore = ${eval:$acl_c_spamscore+20} set acl_c_bouncemessage = $acl_c_bouncemessage HELO not equals Hostname; Another 20 if the HELO argument does not match the primary DNS record (A) for the sending host.
#---------------------------------------------------------------------------------- #------------------------Handler for suspicious HELO arguments--------------------- warn !condition = ${lookup {$sender_helo_name}wildlsearch{/CONFIG_PREFIX/\ additional/trusted_zones}{1}{0}} set acl_c_spamscore = ${eval:$acl_c_spamscore+20} set acl_c_bouncemessage = $acl_c_bouncemessage Suspicious HELO argument; Good old trusted_zones is now used to check the HELO argument.
+20 points to Chinese, Koreans and other Mexican Japanese.
#---------------------------------------------------------------------------------- And finally - the correct implementation of the DNSBL system survey:
In no case can you turn off a node that glows only in one such system. There is a high probability that he got there by accident.
Do not completely abandon the use of DNSBL - it is a powerful filtering tool.
Only if you simultaneously hit two or more of these systems, you can declare a node as a spam sender.
#-------------------------------DNSBL processing section--------------------------- warn dnslists = sbl.spamhaus.org set acl_c_spamscore = ${eval:$acl_c_spamscore+60} set acl_c_bouncemessage = $acl_c_bouncemessage Listed in DNSBL $dnslist_domain; warn dnslists = bl.spamcop.net set acl_c_spamscore = ${eval:$acl_c_spamscore+60} set acl_c_bouncemessage = $acl_c_bouncemessage Listed in DNSBL $dnslist_domain; warn dnslists = dnsbl.sorbs.net set acl_c_spamscore = ${eval:$acl_c_spamscore+60} set acl_c_bouncemessage = $acl_c_bouncemessage Listed in DNSBL $dnslist_domain; warn dnslists = dul.ru set acl_c_spamscore = ${eval:$acl_c_spamscore+60} set acl_c_bouncemessage = $acl_c_bouncemessage Listed in DNSBL $dnslist_domain; +60 points for getting into any of the “black DNS lists”. If you hit the two lists at the same time, the node will receive 120 points, which is enough to stop receiving mail from it, but not enough to automatically add to the local blacklist. If the admin of the sending mail server has time to quickly unsubscribe from at least one DNSBL, then the transfer of mail will suffer minimally.
DNSBL polling in combination with other checks - eliminates spammers very well and it is often these points that become decisive for placing the ip-address in the local blacklist for a week.
#----------------------------------------------------------------------------------- warn !verify = sender/callout=3m,defer_ok set acl_c_spamscore = ${eval:$acl_c_spamscore+60} set acl_c_bouncemessage = $acl_c_bouncemessage Cannot complete sender verify; This is where callout is made. The server is given a maximum of 3 minutes to check (otherwise there is a risk that the “good” sender will fall off without waiting for the end of the check, it hangs connected), the unavailability of the remote node is considered to be successful passing the check. Those. The node will receive 60 points if it sent mail from an absent (most often - fake) email address.
That is why you should not make legitimate emails from addresses like www@webserver.example.org. Callout is a common filtering method and is supported by many current implementations of MTA.
accept condition = ${if >{$acl_c_spamscore}{145}} There is a little trick - nodes with more than 145 points are true candidates for a local blacklist. There is no need to check them more, we transfer them to the next acl and there is a strong ban.
accept delay = ${eval:$acl_c_spamscore/2}s Who has not scored 145 points - passes the torture by delaying the session: the receiving MTA imitates a bad connection and “hangs” for a number of seconds equal to half the number of “spam points”. For example, a node with 60 points will be “suspended” for 30 seconds.
Usually spammers do not have that amount of time waiting for an answer and they fall off in 15-20 seconds.
So we got to the last acl, checking the correctness of the parameters of the SMTP session (it works after RCPT TO :). The “suspension” of the sum of points and the decision of the further fate of the letter occur in it:
acl_check_rcpt: [...] #------Spamtraps check------- warn condition = ${lookup {$local_part@$domain}lsearch{/CONFIG_PREFIX/\ additional/spamtraps}{1}{0}} set acl_c_spamscore = ${eval:$acl_c_spamscore+50} set acl_c_bouncemessage = $acl_c_bouncemessage Spamtrap hit; +50 points for getting into a spam trap.
#---------------------------- The list of trap addresses is in the spamtraps file, by email to the line:
spamtrap@hornsnhoofs.com honeypot@hornsnhoofs.com And so on.
Using your own spam traps is a good way to hide from the big mailers, digging internet bots for emails. Then these addresses are transmitted, sold and just distributed as part of databases for mailings. That is - spread by spammers among themselves.
The difficulty arises in the initial distribution of such addresses, so that they would be sent to spammers. For example, for this you can use a mechanism such as wpoison on the company's main site. The main thing is not to forget about the prohibition of their indexing by search engines (via robots.txt or META CONTENT = “NOINDEX, NOFOLLOW”).
Often, old domains already have abandoned or even never existed addresses, which, however, goes one spam. These are ideal candidates for use as spamtraps. The main thing is that they do not use such a box for a long time, and even better so that they are never used.
Paradoxically, but the fact is that many of the trap addresses I use today have never existed anywhere except in spam databases. It is not known how they appeared there, but they regularly send spam. Study the logs of your mailer for a long period, for sure there will be something similar there.
#---------------------------Blacklist Processing Section------------------------ drop !senders = : !condition = ${if <{$acl_c_spamscore}{150}} message = Connection closed. Spamscore threshold (150 points) reached. \ Spamscore is $acl_c_spamscore! \ Warning: IP [$sender_host_address] added to Blacklist. \ Details: $acl_c_bouncemessage condition = ${lookup mysql \ {\ insert into antispam.blacklist (IP,Timestamp)\ values ('${sender_host_address}',${tod_epoch});\ }\ } Those who could score 150 points and more are declared champions! Their ip-addresses are placed in the local blacklist and the fame of them does not cease the week.
The nodes that are in it are dropped right after the “MAIL FROM:” command is sent.
#------------------------------------------------------------------------------- As already mentioned, the life of an entry in my blacklist is a week (604800 seconds). Bases are cleaned with a crown every hour:
#!/bin/bash echo "delete from blacklist where Timestamp < `echo "\`date +%s\`-604800" | bc`;" | /usr/local/bin/mysql -u mta -pmtapass antispam echo "optimize table blacklist" | /usr/local/bin/mysql -u mta -pmtapass antispam deny condition = ${if >{$acl_c_spamscore}{100}} condition = ${if ={$acl_c_validrcpt}{1}} message = Message rejected. Spamscore threshold (100 points) reached. \ Spamscore is $acl_c_spamscore! Details: acl_c_bouncemessage It's simple. Scored more than 100 points - received "550 Message rejected.". In this case, the log records the number of points scored and reports on checks inundated. The same information goes along with the mail trap on the other side of the wire. Suddenly someone will come in handy for debugging.
It is important to send an bluff without breaking the session (not bringing the letter to the reception in acl and starting processing by routers and transports), otherwise the events may turn out not very good for you: in the worst case, your server can be used as a sender.
Briefly about sending spam as a return: if you write a spam letter to a non-existent email address nosuchaddress@hornsnhoofs.com, pointing to “MAIL FROM:” address vasyapupkin@gmail.com, and the letter will be accepted by the mail server, and after receiving the mail the router determines that the destination address does not exist, then the server will wrap the spam letter in a new envelope, add the lines of its non-response and return it to the “sender” - to the unsuspecting Vasya Pupkin on Gmail.com.
Further, the corporation of dough will quickly add your server to the “black list” and block the reception of mail from its ip. Now you are writing justification letters to tehsapport, and the little spammers giggle.
However, this is a very brief description of the phenomenon and there are other ways to protect against it, but the tactics of transferring an arc pound without stopping the session are the most correct.
You can also add this info to the service headers of the letter, for easier debugging. But it is very optional.
accept condition = ${if <{$acl_c_spamscore}{70}} condition = ${if ={$acl_c_validrcpt}{1}} Nodes with less than 70 spam points are considered legitimate, we accept a letter from them if the recipient's address (
$acl_c_validrcpt ) exists.I will not describe how this variable gets its value, since this depends on the way the data about the mail users is stored. I will say that the true value (or just 1) from me it gets after checking the email address of the recipient in Active Directory through an LDAP search.This can be done for any database with users, as long as it is supported by eczema.
The most interesting thing happens with nodes that have collected from 70 to 100 points. We cannot refer them to either legitimate hosts or spam mailers. Therefore, we wrap them in a greylist for 29 minutes (the interval is chosen with the expectation of the MTA-sender for the second run of the mail queue):
#--------------------------Greylist Processing Section-------------------------- defer condition = ${if ={$acl_c_validrcpt}{1}} condition = ${lookup mysql \ {\ select Source from antispam.greylist where \ Source='$sender_host_address' \ and Timestamp > ${eval:$tod_epoch-1740} limit 1\ }{1}{0}\ } message = Message deferred. Try again later. You was been already greylisted. accept condition = ${lookup {$acl_c_reverse_zone}wildlsearch{CONFIG_PREFIX/\ additional/dynamic_pools}{1}{0}} condition = ${if ={$acl_c_validrcpt}{1}} condition = ${lookup mysql \ {\ select Source from antispam.greylist where \ Source='$sender_host_address' \ and grey_hash = '${md5:${lc:$sender_address\ $local_part@$domain}}' \ and Timestamp < ${eval:$tod_epoch-1740} limit 1 \ }{1}{0}\ } condition = ${lookup mysql \ {\ delete from antispam.greylist where \ Source='$sender_host_address' \ and grey_hash = '${md5:${lc:$sender_address\ $local_part@$domain}}' \ }\ } accept !condition = ${lookup {$acl_c_reverse_zone}wildlsearch{CONFIG_PREFIX/\ additional/dynamic_pools}{1}{0}} condition = ${if ={$acl_c_validrcpt}{1}} condition = ${lookup mysql \ {\ select Source from antispam.greylist where \ Source='$sender_host_address' \ and grey_hash = '${md5:${lc:$sender_address\ $local_part@$domain}}' \ and Timestamp < ${eval:$tod_epoch-1740} limit 1 \ }{1}{0}\ } condition = ${lookup mysql \ {\ insert into antispam.whitelist (IP, Timestamp) \ values ('$sender_host_address',$tod_epoch) \ }{1}{1}\ } condition = ${lookup mysql \ {\ delete from antispam.greylist where \ Source='$sender_host_address' \ and grey_hash = '${md5:${lc:$sender_address\ $local_part@$domain}}' \ }\ } defer condition = ${if ={$acl_c_validrcpt}{1}} condition = ${if >={$acl_c_spamscore}{70}} condition = ${lookup mysql \ {\ insert into antispam.greylist (Source,grey_hash,Timestamp) \ values ('$sender_host_address',\ '${md5:${lc:$sender_address$local_part@$domain}}',\ ${tod_epoch}); \ }\ } message = Message deferred. Spamscore is $acl_c_spamscore! Try again later. \ Greylisting in progress. Details: $acl_c_bouncemessage #----------------------------------------------------------------------------- defer , c 4xx, , .
: SMTP- 451 , « », .
«», : ip- .
, , greylist, :
Source varchar(15) , grey_hash varchar(32) , Timestamp int(11). The Source field contains the sender's ip address, grey_hash the md5 hash from the sender and recipient addresses, and Timestamp the time to add the record in unix_time format. In addition to it, greylisting also requires a whitelist table, which allows you to add the addresses of nodes that have passed greylisting there. The whitelist table format completely repeats that for the blacklist table described above.The tables are also cleared by the crown: the records are stored in the greylist for 24 hours, for whitelist - for the month.
deny message = Message rejected. No such user here. Relaying denied. $acl_c_support set acl_c_spamscore = ${eval:$acl_c_spamscore+5} set acl_c_bouncemessage = $acl_c_bouncemessage RCPT Fail; delay = ${eval:$acl_c_spamscore/2}s The last filtering rule, according to recommendations of the best dog breeders, is prohibiting His work is somewhat similar to the last rule in a “closed” firewall
deny ip from any to any, it only affects the SMTP session.Additionally, for each incorrect address in “RCPT TO:” (SMTP protocol allows you to specify several recipients for one letter), the node receives 5 additional spam points, up to and including the unconditional blocking threshold.
Oh yes - torture a delayed session after each “RCPT TO:” also has a place to be.
Using at least these constructions in the mail server configuration will cut off about 90-95% of spam without increasing the number of false positives.
Creative independent use of this technique (counting and weighing spam coefficients) makes it possible to get rid of spam almost completely (except, perhaps, mailings from public mail systems like gmail.com or mail.ru, but there, as a rule, the account is also monitored block) without the use of context filtering systems.
And if the context filter is also planned to be used, this way you can unload the text analyzer due to a strong reduction in the amount of input data.
In the article I didn’t deal with such technologies as SPF and DKIM, but if you wish, you can use them as “whitening functions” (reducing the number of spam points), working in the same way as those that were considered. If the community has an interest in these systems, I will try to consider them separately.
Source: https://habr.com/ru/post/133215/
All Articles