Yandex now supports encryption of outgoing and incoming mail
Last week, we included in Yandex.Mail mail encryption for cross-server SMTP connections using STARTTLS - both for receiving and sending emails. Now all emails from our mail system to users of other services that support such encryption (for example, Gmail) are transmitted in encrypted form, and no one will be able to read them on the way. On this occasion, I will talk a little bit about the protocols that are used when sending email in encrypted form.
Despite the fact that the encryption of the connection between the user and the mail interface is already quite common among mass services, the correspondence between the mail systems often went unprotected. And the security of correspondence is a very important thing for us (although it is not very noticeable to the average user).
')
Researchers have yet to find the cause of the true love of Internet technologists for abbreviations. Since the days of ARPANET, all networks, protocols, standards, etc. prefer to call letter abbreviations. This simple and clear way of word formation leads to the appearance of sentences like: "IETF published RFC6594 by CZ.NIC on the use of SHA-256 with RSA, DSA and ECDSA in SSHFP". As you can see, there are especially many such cuts in cryptography.
In the nineties of the last century, one of the “incubators” where all interesting ideas from the academic environment were translated into reality was the company Netscape . At the beginning of the previous year, its employees realized the ability to encrypt connections, published a protocol and named it Secure Sockets Layer (SSL). Its first public version was SSL v2, in which a number of vulnerabilities were quickly found . It was followed by the last currently SSL v3. The original description on the Netscape website sank into oblivion at the time of the acquisition of Netscape by AOL, but in 2011 it was nevertheless published for history as RFC6101 .
Then the first free implementation of SSL appeared. Enthusiast Eric Young began publishing the SSLeay library (this time the combined abbreviation: SSL + Eric A. Young) under a BSD-like license . A few years later SSLeay will turn into the OpenSSL library known to all specialists.
What is important to know about SSL v2 and v3? First, these protocols are designed to work on top of the transport protocol with reliable delivery and connections (for example, TCP). Secondly, SSL v2 can no longer be used - it is officially considered to be too vulnerable and in the current environment can only give the illusion of security.
Based on SSL v3, a team of scientists and engineers within the IETF has created a Transport Layer Security ( TLS) protocol. In fact, TLS v1.0 is a small (but incompatible) reworking of SSL v3, which includes all of its features and adds some details.
The end of the nineties and the beginning of zero were marked by the widespread use of HTTPS (and this, I remind you, just HTTP over SSL / TLS - HTTP with encryption), and therefore more careful attention to these protocols by computer security specialists in hats of both colors . As a result, a whole class of vulnerabilities was discovered when using SSL v3 or TLS 1.0 with block encryption algorithms in CBC mode (and other modes of block algorithms were not used in SSL). In 2006, as early as 7 years later, an updated version of TLS 1.1 was released, eliminating these vulnerabilities. However, engineers were not in a hurry to implement TLS 1.1 right up to 2011, when there was a loud attack on browsers using TLS 1.0, called BEAST , all over the Internet. Several crutches were immediately invented (for example, switching to a stream cipher), and priorities for upgrading everything and everyone to TLS 1.1+ were also raised.
In 2008, the most recent TLS version 1.2 was released. It appeared a lot of innovations. First, it became mandatory to use the AES encryption algorithm, and a new encryption mode appeared in addition to CBC - GCM . Secondly, support for obsolete DES and IDEA algorithms was excluded. Third, the protocol abandoned the use of HMAC based on the MD5 hashing algorithm and switched to more robust SHA256 and SHA384. Fourthly, the TLS extensions extension mechanism has appeared, allowing to enable new features without a complete processing of the protocol. One of these extensions is SNI , which is actively used in Yandex services. Finally, so that all innovations do not turn out to be useless, in the protocol the possibility of a downgrade on SSLv3 was declared obsolete and not recommended.
So, summing up:
We can confidently say that the main user-defined protocol for sending email is HTTP. This may sound unusual, but the vast majority of users read and send their emails in 2013 in this way.
The mail programs, though relegated to the background, are still widely used and work on three protocols: the old POP3, a slightly newer IMAP and universal SMTP. About the history of POP3 and IMAP, I told a little bit earlier , and now I’ll add a little bit about SMTP.
E-mail is rightly called the first killerpp of computer networks. The roots of modern SMTP (Simple Mail Transfer Protocol) goes very deep into ARPANET, in the pre-TCP / IP era of transferring files between computers of American universities with the money of the US Department of Defense. These were naive beautiful times of widespread mutual trust, when not only encryption was required, but even the simplest authentication. Than to us it was backfired years later, by the way, you can see it in your own “Spam” folder.
See, the first version of SMTP for the Internet (with a capital letter!) Is specified in RFC 788 at the end of 1981. Already then it was the result of more than a decade of e-mail development in ARPANET. And only almost 20 years later, in 1999, the official standard for authentication appeared , that is, logins and passwords in SMTP. All this time, sending mail via SMTP was possible immediately after connecting to anyone, anyone. This mode, of course, was suitable for inter-server communication on a network with a large number of hops ─ in this case, the hosts can serve as relays for someone else's mail. But the initial sending, the so-called submission, without authentication led to the appearance of spam, and then to the invention of SMTP AUTH. By the way, old-timers can still remember the “POP3 before SMTP” mode, which used POP3 as an authenticating crutch to SMTP.
As soon as passwords appear in the protocol, they immediately begin to think about encryption. In 2002, a standard was released to support the upgrade of an open SMTP session to TLS mode ─ RFC3207 . It was an attempt to fix and improve the defacto situation ─ SMTP encryption on a separate port 465 had already been used for several years.
The STARTTLS operation scheme is of separate interest. This is a universal extension for any text protocol. In reality, it gained distribution for SMTP, IMAP / POP3 and a little bit for FTP.
At the moment when the client and the server in the already established open connection realize that there is an opportunity and desire on both sides to encrypt the data transfer, the client gives the STARTTLS command and immediately starts the TLS handshake.
The only thing that depends on the application protocol is to ensure that TLS is mutually supporting before encryption starts. In the case of SMTP, this happens through the ESMTP extension mechanism ─ there are more details about this.
SMTP encryption via STARTTLS on standard ports (recall, 25 and second 587, which is often forgotten) or port 465 immediately began to gain popularity. TLS allows a lot: both certificate authentication and server authentication ─ all of these are interesting, necessary, and hitherto inaccessible features. Now, it seems, there are no mail systems that do not accept mail from users over an encrypted connection. Another thing is interserver communication.
SMTP is not just an email workhorse, it is also a two-headed horse.
The entire exchange of letters between servers of different mail systems also takes place via SMTP. Of course, there is no longer password authentication ─ instead, verification of the address of the recipient of the letter at the application level is used. If it doesn’t exist, it turns out Open Relay, for which your IP address will necessarily be punished with a negative reputation by all the anti-spam systems in the world. Usually there is no encryption. There are several reasons for this. First, it is believed that the communication channels between the servers of the postal systems themselves are quite safe. And indeed, during transmission between servers, letters usually do not descend into the “last mile”, where small, “casual” traffic interception usually occurs. Secondly, this transfer is asynchronous and non-interactive, due to which there is no possibility to clarify with the person what to do in case of inconsistency or overdueness of the certificate of the receiving side of the connection.
All these arguments, however, prove to be untenable. The receiving server may not be in the rack of a large secure data center, but anywhere. Government organizations around the world are launching wiretapping programs on all channels of communication. Verification of certificates, although useful, is not at all necessary in order to secure the transfer of information. Therefore, more and more, we find emails that are sent in encrypted form all the way ─ from the user's browser via HTTPS to the mail system server, then to the recipient's server via SMTP over TLS, and then the encrypted IMAP to the phone of the person to whom message. And this is good.
In 2009, simultaneously with the launch of IMAP, we added SSL / TLS support to our POP3 interface. At about the same time, encryption in SMTP for receiving emails from mail programs appeared. Today, more than 4 years later, our servers still allow you to open unprotected connections from mail programs, although we do not recommend using them and even, if possible, do not document them.
Yandex.Mail mobile applications always use SSL / TLS connections, and inside they use the XMPP protocol.
In 2011, we enabled HTTPS in the web interface mail.yandex.ru and covered in this way all the basic ways to transfer user information to the cloud. This year, we made HTTPS mandatory when accessing the Mail, and this week we turned off any opportunities to connect to it via an unprotected protocol. We also did a great job of protecting user passwords in the authentication system - Yandex.Passport - and be sure to tell about it.
This is what the ratio of encrypted and regular traffic on our SMTP communication servers with the outside world looks like now.
The end of 2013 can be confidently called the year of high-profile scandals associated with mass tapping of data by government organizations, and the first year of widespread cloud encryption. We urge all of our industry colleagues to join the trend and set a course for one hundred percent encryption of all user data on the network. So it goes.
PS: Edward, hello! :)
PPS: Edward Snowden does not work and never got a job at Yandex.
Despite the fact that the encryption of the connection between the user and the mail interface is already quite common among mass services, the correspondence between the mail systems often went unprotected. And the security of correspondence is a very important thing for us (although it is not very noticeable to the average user).
')
Researchers have yet to find the cause of the true love of Internet technologists for abbreviations. Since the days of ARPANET, all networks, protocols, standards, etc. prefer to call letter abbreviations. This simple and clear way of word formation leads to the appearance of sentences like: "IETF published RFC6594 by CZ.NIC on the use of SHA-256 with RSA, DSA and ECDSA in SSHFP". As you can see, there are especially many such cuts in cryptography.
What is SSL and TLS?
In the nineties of the last century, one of the “incubators” where all interesting ideas from the academic environment were translated into reality was the company Netscape . At the beginning of the previous year, its employees realized the ability to encrypt connections, published a protocol and named it Secure Sockets Layer (SSL). Its first public version was SSL v2, in which a number of vulnerabilities were quickly found . It was followed by the last currently SSL v3. The original description on the Netscape website sank into oblivion at the time of the acquisition of Netscape by AOL, but in 2011 it was nevertheless published for history as RFC6101 .
Then the first free implementation of SSL appeared. Enthusiast Eric Young began publishing the SSLeay library (this time the combined abbreviation: SSL + Eric A. Young) under a BSD-like license . A few years later SSLeay will turn into the OpenSSL library known to all specialists.
What is important to know about SSL v2 and v3? First, these protocols are designed to work on top of the transport protocol with reliable delivery and connections (for example, TCP). Secondly, SSL v2 can no longer be used - it is officially considered to be too vulnerable and in the current environment can only give the illusion of security.
Based on SSL v3, a team of scientists and engineers within the IETF has created a Transport Layer Security ( TLS) protocol. In fact, TLS v1.0 is a small (but incompatible) reworking of SSL v3, which includes all of its features and adds some details.
The end of the nineties and the beginning of zero were marked by the widespread use of HTTPS (and this, I remind you, just HTTP over SSL / TLS - HTTP with encryption), and therefore more careful attention to these protocols by computer security specialists in hats of both colors . As a result, a whole class of vulnerabilities was discovered when using SSL v3 or TLS 1.0 with block encryption algorithms in CBC mode (and other modes of block algorithms were not used in SSL). In 2006, as early as 7 years later, an updated version of TLS 1.1 was released, eliminating these vulnerabilities. However, engineers were not in a hurry to implement TLS 1.1 right up to 2011, when there was a loud attack on browsers using TLS 1.0, called BEAST , all over the Internet. Several crutches were immediately invented (for example, switching to a stream cipher), and priorities for upgrading everything and everyone to TLS 1.1+ were also raised.
In 2008, the most recent TLS version 1.2 was released. It appeared a lot of innovations. First, it became mandatory to use the AES encryption algorithm, and a new encryption mode appeared in addition to CBC - GCM . Secondly, support for obsolete DES and IDEA algorithms was excluded. Third, the protocol abandoned the use of HMAC based on the MD5 hashing algorithm and switched to more robust SHA256 and SHA384. Fourthly, the TLS extensions extension mechanism has appeared, allowing to enable new features without a complete processing of the protocol. One of these extensions is SNI , which is actively used in Yandex services. Finally, so that all innovations do not turn out to be useless, in the protocol the possibility of a downgrade on SSLv3 was declared obsolete and not recommended.
So, summing up:
- SSL is a secure connection protocol over TCP. New versions of SSL does not happen. TLS ─ this is the new SSL, with new version numbering.
- There is an old SSL v2. It is useless - if it is supported, it must be disabled. There is SSL v3, which is everywhere supported for many years. But it found flaws that may adversely affect security. There is TLS v1.0, SSL v3 vulnerabilities are preserved in it, and it would be better to use it only with the streaming RC4 encryption algorithm (which is separately considered theoretically vulnerable). Therefore, it is best to use TLS v1.1 or v1.2 and disable the RC4 cipher.
Email Protocols
We can confidently say that the main user-defined protocol for sending email is HTTP. This may sound unusual, but the vast majority of users read and send their emails in 2013 in this way.
The mail programs, though relegated to the background, are still widely used and work on three protocols: the old POP3, a slightly newer IMAP and universal SMTP. About the history of POP3 and IMAP, I told a little bit earlier , and now I’ll add a little bit about SMTP.
E-mail is rightly called the first killerpp of computer networks. The roots of modern SMTP (Simple Mail Transfer Protocol) goes very deep into ARPANET, in the pre-TCP / IP era of transferring files between computers of American universities with the money of the US Department of Defense. These were naive beautiful times of widespread mutual trust, when not only encryption was required, but even the simplest authentication. Than to us it was backfired years later, by the way, you can see it in your own “Spam” folder.
See, the first version of SMTP for the Internet (with a capital letter!) Is specified in RFC 788 at the end of 1981. Already then it was the result of more than a decade of e-mail development in ARPANET. And only almost 20 years later, in 1999, the official standard for authentication appeared , that is, logins and passwords in SMTP. All this time, sending mail via SMTP was possible immediately after connecting to anyone, anyone. This mode, of course, was suitable for inter-server communication on a network with a large number of hops ─ in this case, the hosts can serve as relays for someone else's mail. But the initial sending, the so-called submission, without authentication led to the appearance of spam, and then to the invention of SMTP AUTH. By the way, old-timers can still remember the “POP3 before SMTP” mode, which used POP3 as an authenticating crutch to SMTP.
As soon as passwords appear in the protocol, they immediately begin to think about encryption. In 2002, a standard was released to support the upgrade of an open SMTP session to TLS mode ─ RFC3207 . It was an attempt to fix and improve the defacto situation ─ SMTP encryption on a separate port 465 had already been used for several years.
The STARTTLS operation scheme is of separate interest. This is a universal extension for any text protocol. In reality, it gained distribution for SMTP, IMAP / POP3 and a little bit for FTP.
At the moment when the client and the server in the already established open connection realize that there is an opportunity and desire on both sides to encrypt the data transfer, the client gives the STARTTLS command and immediately starts the TLS handshake.
The only thing that depends on the application protocol is to ensure that TLS is mutually supporting before encryption starts. In the case of SMTP, this happens through the ESMTP extension mechanism ─ there are more details about this.
SMTP encryption via STARTTLS on standard ports (recall, 25 and second 587, which is often forgotten) or port 465 immediately began to gain popularity. TLS allows a lot: both certificate authentication and server authentication ─ all of these are interesting, necessary, and hitherto inaccessible features. Now, it seems, there are no mail systems that do not accept mail from users over an encrypted connection. Another thing is interserver communication.
SMTP is not just an email workhorse, it is also a two-headed horse.
The entire exchange of letters between servers of different mail systems also takes place via SMTP. Of course, there is no longer password authentication ─ instead, verification of the address of the recipient of the letter at the application level is used. If it doesn’t exist, it turns out Open Relay, for which your IP address will necessarily be punished with a negative reputation by all the anti-spam systems in the world. Usually there is no encryption. There are several reasons for this. First, it is believed that the communication channels between the servers of the postal systems themselves are quite safe. And indeed, during transmission between servers, letters usually do not descend into the “last mile”, where small, “casual” traffic interception usually occurs. Secondly, this transfer is asynchronous and non-interactive, due to which there is no possibility to clarify with the person what to do in case of inconsistency or overdueness of the certificate of the receiving side of the connection.
All these arguments, however, prove to be untenable. The receiving server may not be in the rack of a large secure data center, but anywhere. Government organizations around the world are launching wiretapping programs on all channels of communication. Verification of certificates, although useful, is not at all necessary in order to secure the transfer of information. Therefore, more and more, we find emails that are sent in encrypted form all the way ─ from the user's browser via HTTPS to the mail system server, then to the recipient's server via SMTP over TLS, and then the encrypted IMAP to the phone of the person to whom message. And this is good.
Encryption support in Yandex Mail protocols
In 2009, simultaneously with the launch of IMAP, we added SSL / TLS support to our POP3 interface. At about the same time, encryption in SMTP for receiving emails from mail programs appeared. Today, more than 4 years later, our servers still allow you to open unprotected connections from mail programs, although we do not recommend using them and even, if possible, do not document them.
Yandex.Mail mobile applications always use SSL / TLS connections, and inside they use the XMPP protocol.
In 2011, we enabled HTTPS in the web interface mail.yandex.ru and covered in this way all the basic ways to transfer user information to the cloud. This year, we made HTTPS mandatory when accessing the Mail, and this week we turned off any opportunities to connect to it via an unprotected protocol. We also did a great job of protecting user passwords in the authentication system - Yandex.Passport - and be sure to tell about it.
This is what the ratio of encrypted and regular traffic on our SMTP communication servers with the outside world looks like now.
The end of 2013 can be confidently called the year of high-profile scandals associated with mass tapping of data by government organizations, and the first year of widespread cloud encryption. We urge all of our industry colleagues to join the trend and set a course for one hundred percent encryption of all user data on the network. So it goes.
PS: Edward, hello! :)
PPS: Edward Snowden does not work and never got a job at Yandex.
Source: https://habr.com/ru/post/203882/
All Articles