What is SPF
I think no one needs to explain what the problem is spam in our time. The fight against this evil is not a simple matter, and if you want to approach the ideal, requiring a combination of several elements. One of these elements is the SPF protocol. Being published in April 2006 in RFC 2006, the protocol currently has the status of “experimental”, and quite good prevalence.
SPF is adopted by such giants as Google, Yandex, Mail.Ru, Microsoft, Rambler. Yahoo does not support SPF, but tries to promote its DKIM development, by the way, not too successfully.
So, how does SPF work?
The basic idea is that the host of the address indicates from which addresses his mail should be expected, and how to interpret the error in case of non-compliance. It is important to understand that this data is only the recommendation of the sender for verification. Yes, yes, of course, the standard describes how the receiving side should handle the result, but for all reasons the receiving side can actually do whatever it wants with them - fully comply, interpret it in its own way, or ignore the recommendations altogether.
')
The TXT field in DNS is used for data placement. So IANA has assigned a DNS field code 99 for SPF. The format of the SPF field is identical to TXT. In general, using the TXT field is not optimal, but the problem is that not every DNS server and client understands the new SPF record type. Therefore, sharing TXT and SPF is considered a good approach for compatibility and future development.
The standard recommends that domains that meet SPF requirements have both types of records. However, at least one entry must be present necessarily. If there are two entries, they must be identical. For example:
Mail exchange using SPF occurs according to the following algorithm.
The mx.example.com mail server sends an email to user@example.net. The example.com DNS records contain the following strings:
So, mx.example.com connects to SMTP example.net and gets from it, something like
The SPF checking module currently does the following things. A query is made to the DNS. SPF and TXT fields are requested. If there is an SPF rule in the received response, then it is parsed and analysis for compliance occurs. In our example, this rule is “v = spf1 + mx -all”. According to the rule, MX records are checked, and the IP addresses specified in the records obtained from the DNS name and IP address of the connected sender are checked. In this case, everything is the same, control is returned to the mailer, and he starts receiving the message.
If, suddenly, the actual IP of the connecting sender was different, the analyzer would tell the mailer that the incoming message is not valid, and it would be better not to receive it at all, well, or in extreme cases, mark it separately.
An SPF record looks like this:
This entry contains the following information:
SPF version - 1 (by the way, while the only one used)
the letter can have a sender with an IP address, the corresponding MX records for the example.com domain
the letter may have a sender with an IP address corresponding to one of the addresses on the subnet colo.example.com/28
In all other cases, when the address does not meet the listed conditions, consider the sender unreliable.
In general, the condition has 2 parts - the determinant and the mechanism.
Identifiers can be: "+" Pass, "-" Fail, "~" SoftFail, "?" Neutral
Mechanisms: all, include, A, MX, PTR, IP4, IP6, exists
The results of a condition check can be the following specific results:
Take, for example, some real domain. Let's say google.com. TXT request returns
it says that you need to include rules from the _netblocks.google.com entry. Interestingly, _netblocks.google.com does not have an A-record, but only a TXT-record. Here she is:
Here are the subnets that Google's mail servers can be located on. The last all-in mechanism with the Neutral determinant informs the analyzer that the sender's address may be out of the allowed range. Letters from foreign address spaces are recommended to be checked additionally, and not rejected unconditionally. For such a large-scale structure, like Google, this is the right decision, because in the process of work the addresses may change, for example, in case of temporary failure and switching to the reserve. And besides, the address may vary shipments.
More tricky SPF entry from Rambler:
there are 2 subnets from which mail is allowed to be received, and 2 sets of sources, mail from which will have the result of the Fail check.
Imagine the following scheme - the user sends an email from vasily@xyz.com to pupkin@mylo.ru, and there is an automatic forwarding to pupkin@gmail.com. And the xyz.com domain is spelled SPF "v = spf1 + mx-all". As a result, the final recipient of gmail.com will make a check, and will receive a mismatch between the address of the actual sender and the specified one, and according to the rules SPF will not accept the letter. To solve this problem, there is the SRS: Sender Rewriting Scheme. In a nutshell, the forwarder rewrites the return-path header.
In general, I believe that this moment is very controversial. Using an intermediate mailbox to redirect traffic to another mailbox is very similar to a spam attack. Here, for example, I register a box, put a candle on it wherever possible, subscribe to a million mailings, and put a redirect on a certain box that I want to fill up with spam. If the senders have SPF and there is no SRS on the forwarding mailer, the target will reject these streams as spam, but when the SRS is running, it will receive completely “legitimate” mail flows.
SPF is a simple and fairly effective way to assess the legitimacy of the transmitted mail. Mail server administrators are minimally required to add SPF records to the DNS. Ease of implementation and support of the main popular MTAs make SPF spreading wider and wider, which benefits all email users, mail servers reduce traffic and generally makes the world a better place :)
SPF is adopted by such giants as Google, Yandex, Mail.Ru, Microsoft, Rambler. Yahoo does not support SPF, but tries to promote its DKIM development, by the way, not too successfully.
So, how does SPF work?
General principles of work
The basic idea is that the host of the address indicates from which addresses his mail should be expected, and how to interpret the error in case of non-compliance. It is important to understand that this data is only the recommendation of the sender for verification. Yes, yes, of course, the standard describes how the receiving side should handle the result, but for all reasons the receiving side can actually do whatever it wants with them - fully comply, interpret it in its own way, or ignore the recommendations altogether.
')
Technical details
The TXT field in DNS is used for data placement. So IANA has assigned a DNS field code 99 for SPF. The format of the SPF field is identical to TXT. In general, using the TXT field is not optimal, but the problem is that not every DNS server and client understands the new SPF record type. Therefore, sharing TXT and SPF is considered a good approach for compatibility and future development.
The standard recommends that domains that meet SPF requirements have both types of records. However, at least one entry must be present necessarily. If there are two entries, they must be identical. For example:
example.com. IN TXT "v = spf1 + mx a: colo.example.com/28 -all"If the SPF record is set, then TXT records should be ignored.
example.com. IN SPF "v = spf1 + mx a: colo.example.com/28 -all"
Interaction mechanism
Mail exchange using SPF occurs according to the following algorithm.
The mx.example.com mail server sends an email to user@example.net. The example.com DNS records contain the following strings:
mx.example.com IN A 208.77.188.166
example.com. IN MX 10 mx.example.com.
example.com. IN TXT "v = spf1 + mx-all"
So, mx.example.com connects to SMTP example.net and gets from it, something like
>> 220 example.net ESMTP Service (Mailer v1.0) ready at 07/30/2009 12:28:21 UTCthen mx.example.com is submitted via HELO / EHLO.
<< HELO mx.example.comDepending on the settings of the receiving party, after this submission there may already be a check for compliance of the submitted name with the SPF rules, but in this place it is not necessary
>> 250 example.net Hello mx.example.com., Pleased to meet youAnd here in this place, after the sender issued the MAIL FROM, a mandatory check, interpretation of the result and reaction to it should follow.
<< MAIL From: <user@example.com>
The SPF checking module currently does the following things. A query is made to the DNS. SPF and TXT fields are requested. If there is an SPF rule in the received response, then it is parsed and analysis for compliance occurs. In our example, this rule is “v = spf1 + mx -all”. According to the rule, MX records are checked, and the IP addresses specified in the records obtained from the DNS name and IP address of the connected sender are checked. In this case, everything is the same, control is returned to the mailer, and he starts receiving the message.
If, suddenly, the actual IP of the connecting sender was different, the analyzer would tell the mailer that the incoming message is not valid, and it would be better not to receive it at all, well, or in extreme cases, mark it separately.
Recording format
An SPF record looks like this:
example.com. IN SPF "v = spf1 + mx a: colo.example.com/28 -all"
This entry contains the following information:
SPF version - 1 (by the way, while the only one used)
the letter can have a sender with an IP address, the corresponding MX records for the example.com domain
the letter may have a sender with an IP address corresponding to one of the addresses on the subnet colo.example.com/28
In all other cases, when the address does not meet the listed conditions, consider the sender unreliable.
In general, the condition has 2 parts - the determinant and the mechanism.
Identifiers can be: "+" Pass, "-" Fail, "~" SoftFail, "?" Neutral
Mechanisms: all, include, A, MX, PTR, IP4, IP6, exists
The results of a condition check can be the following specific results:
* None - means that either there are no records in the domain, or the domain cannot be checked at all. In general, no intelligible answer was received.
* Neutral - arises in a situation where the domain owner does not want or can not tell whether IP is allowed. This result should be processed in the same way as None.
* Pass - means that everything is OK and the recipient can accept the letter.
* Fail - clearly indicates that the letter should not be accepted. The relying party can mark the letter or reject.
* SoftFail - located somewhere between Fail and Neutral. The receiving party should not discard the letter on the basis of only this result.
* TempError - the result that occurs if the client at the time of verification receives a temporary error. The message can be accepted or temporarily rejected.
* PermError - an error that occurs when it is impossible to correctly interpret the DNS records of the sender.
Take, for example, some real domain. Let's say google.com. TXT request returns
"V = spf1 include: _netblocks.google.com ~ all"
it says that you need to include rules from the _netblocks.google.com entry. Interestingly, _netblocks.google.com does not have an A-record, but only a TXT-record. Here she is:
"V = spf1 ip4: 216.239.32.0/19 ip4: 64.233.160.0/19 ip4: 66.249.80.0/20 ip4: 72.14.192.0/18 ip4: 209.85.128.0/17 ip4: 66.102.0.0/20 ip4: 74.125. 0.0 / 16 ip4: 64.18.0.0/20 ip4: 207.126.144.0/20? All »
Here are the subnets that Google's mail servers can be located on. The last all-in mechanism with the Neutral determinant informs the analyzer that the sender's address may be out of the allowed range. Letters from foreign address spaces are recommended to be checked additionally, and not rejected unconditionally. For such a large-scale structure, like Google, this is the right decision, because in the process of work the addresses may change, for example, in case of temporary failure and switching to the reserve. And besides, the address may vary shipments.
More tricky SPF entry from Rambler:
"V = spf1 ip4: 81.19.66.0/23 ip4: 81.19.88.0/24 -exists:% {ir} .spf.rambler.ru -exists:% {l} .u.spf.rambler.ru ~ all"
there are 2 subnets from which mail is allowed to be received, and 2 sets of sources, mail from which will have the result of the Fail check.
Shipping Problem
Imagine the following scheme - the user sends an email from vasily@xyz.com to pupkin@mylo.ru, and there is an automatic forwarding to pupkin@gmail.com. And the xyz.com domain is spelled SPF "v = spf1 + mx-all". As a result, the final recipient of gmail.com will make a check, and will receive a mismatch between the address of the actual sender and the specified one, and according to the rules SPF will not accept the letter. To solve this problem, there is the SRS: Sender Rewriting Scheme. In a nutshell, the forwarder rewrites the return-path header.
In general, I believe that this moment is very controversial. Using an intermediate mailbox to redirect traffic to another mailbox is very similar to a spam attack. Here, for example, I register a box, put a candle on it wherever possible, subscribe to a million mailings, and put a redirect on a certain box that I want to fill up with spam. If the senders have SPF and there is no SRS on the forwarding mailer, the target will reject these streams as spam, but when the SRS is running, it will receive completely “legitimate” mail flows.
Conclusion
SPF is a simple and fairly effective way to assess the legitimacy of the transmitted mail. Mail server administrators are minimally required to add SPF records to the DNS. Ease of implementation and support of the main popular MTAs make SPF spreading wider and wider, which benefits all email users, mail servers reduce traffic and generally makes the world a better place :)
Source: https://habr.com/ru/post/63768/
All Articles