Own implementation of https using crypto ++ for I2P bootstrapping
Each new I2P node should first get an initial list of nodes from where it first starts. To do this, there are special servers (reseed), the addresses of which are strictly written in the code. Previously, the download was carried out on http, but more recently, reseed-s began to move to https. For successful work "purple" I2P also required to make the appropriate changes. The crypto ++ cryptographic library used there does not support ssl. Instead of using an additional library like openssl, which actually duplicates cryptography, the option described below was chosen.
Boot is the only place in I2P where https is used.
On the other hand, the article will be interesting to those who are interested in understanding how ssl works and try it yourself.
Our goal is to get an i2pseeds.su3 file of about 100K in size from one of the I2P reseed nodes. This file is signed by a separate certificate independent of the site certificate, so certificate verification can be excluded. The relatively short length of the received data allows us not to implement the mechanisms of compression and restoration of broken connections.
Only TLS 1.2 and the TLS_RSA_WITH_AES_256_CBC_SHA256 cipher suite will be used. In other words, AES256 in CBC mode is used to encrypt data, and RSA is used for key negotiation.
This choice is due to the fact that AES256-CBC is the most used encryption in I2P, and RSA to simplify the implementation of the protocol by reducing the number of messages required for key negotiation. In addition to RSA and AES, the following cryptographic functions from crypto ++ are also required:
')
The RSA implementation is based on PKCS v1.5. The key length can be any and is determined by the certificate.
Absolutely all transmitted messages begin with a 5-byte header, the first byte of which contains the message type, the next 2 bytes — the protocol version number (0x03, 0x03 for TSL 1.2) and then the length of the rest (content) of the message — 2 bytes in Big Endian, most defining message boundaries.
Thus, when receiving new data, you should first read 5 bytes of the header, and then how many bytes are in the length field.
There are 4 types of messages:
In our implementation, the encrypted data is as follows:
16 bytes IV for CBC, in TSL 1.2 for each message own IV;
data length up to 64K is the length of the headers;
32 MAC bytes, calculated for a 13-byte header and data, the header consists of an 8-byte sequence number starting with zero, message type (0x17 or 0x16), version and data length. All in BigEndian. The key for HMAC is also calculated during the connection setup process;
a placeholder, so that the length of the encrypted data is a multiple of 16 bytes, the last byte contains the number of bytes of the placeholder, not taking into account itself. If the message length is a multiple of 16 bytes, then another 16 bytes will be added for this last byte with a length.
During the installation process, we must solve two problems:
In our case, the sequence of messages is as follows:
ClientHello -> (0x01)
<- ServerHello (0x02)
<- Certificate (0x0B)
<- ServerHelloDone (0x0E)
-> ClientKeyExchange (0x10)
-> ChangeCipherSpec
-> Finished (0x14)
<- ChangeCipherSpec
<- Finished (0x14)
where "->" means sending a message, and "<-" means receiving.
All messages, except for ChangeChiperSpec, are a message of the type 0x16 - connection establishment. The content of this type of message begins with its own 4-byte header, the first byte of which is the type of the connection setup message, as indicated above, and 3 bytes of the length of the remaining message, the highest byte of which in our case is always zero.
Consider these messages in detail.
The first message that we send to the server after a successful connection. Since we use one particular set of ciphers, in our case it will be permanent. Here it is:
This message tells the server that we support TLS 1.2, this new connection (the length of the session identifier is zero) and support a single cipher suite - RSA with AES256. We also send a set of 32 "random" bytes for key generation. If these bytes are really random, then they should be remembered somewhere, because they will be needed later.
The “twin brother” is ClientHello, except that the message type is 0x02 instead of 0x01, and a non-empty session identifier. From this message, we need only 32 random bytes.
It may contain several certificates, first comes the length of the entire group of certificates, then each certificate has its own length. We are only interested in the first certificate and read the length should be 2 times. The certificate itself is X.509 in DER encoding. From it, we need the RSA public key.
It does not contain anything useful, but is taken into account when calculating a hash for Finished.
At this point, we have enough information to generate and reconcile the keys that occur in 3 stages: the generation of a random secret key, the master key calculation, the master key extension to obtain the encryption keys and the checksum.
The random secret key is 48 bytes, the first 2 of which are the version number (0x03, 0x03), and the remaining 46 are randomly generated. Further, these 48 bytes are encrypted with the RSA public key, and together with the length of the encrypted block are sent to the server. It should be noted that the length of the encrypted block will be equal to the key length, and not 48 bytes. For example, for certificates with a 2048-bit key, this length will be 256, and the length of the transmitted data will be 258.
Dispatched immediately after ClientKeyExchange. Always the same:
This message is of type 0x14 and hash calculation for Finished is not involved.
To further calculate the keys, we need a pseudo-random function that accepts 4 parameters at the input: the secret key just sent to the server, a label in the form of a text string, a block of initial data and the desired length of the result.
In TLS 1.2, it is defined as follows:
PRF (secret, label, seed) = P_SHA256 (secret, label + seed);
P_SHA256 (secret, seed) = HMAC_SHA256 (secret, A (1) + seed) +
HMAC_SHA256 (secret, A (2) + seed) +
HMAC_SH256 (secret, A (3) + seed) + ...
where A is determined by induction
A (0) = seed,
A (i) = HMAC_SHA256 (secret, A (i -1)).
That is, at each step we recalculate the checksum from the previous step, and then we calculate the checksum from combining the result with the text string and the original data, repeating this until the desired length is obtained.
Now the master key is calculated by the formula
PRF (secret, "master secret", clientRandom + serverRadom, 48);
where clientRandom is 32 random bytes from ClientHello, and serverRandom is from ServerHello.
Then it should be expanded to a 128-byte block, containing 4 32-byte keys in the following sequence: a MAC key for sending, a MAC key for receiving, an encryption key for sending, a decryption key for receiving.
MAC key for receiving is not used.
Key expansion is made according to the formula
PRF (masterSecret, "key expansion", serverRandom + clientRadom, 128)
clientRadom and serverRadom change places here.
At this point, we have everything we need to start the data exchange, but, unfortunately, we have to send a Finished message containing the correct data, otherwise the server will break the connection.
If all the previous posts were rather trivial, then Finshed is more complicated. Firstly, it is of type 0x16, but its contents are completely encrypted, while 0x16 also appears in the calculation of the checksum, and not 0x17 as for other encrypted messages.
The message itself contains the first 12 bytes from
PRT (masterSecret, "client finished", hash, 12)
where hash is SHA256 from the following sequence of messages:
ClientHello, ServerHello, Certficate, ServerHelloDone, ClientKeyExchange. All messages are counted without a 5 byte header.
If the message is generated correctly, the server will respond with ChangeCipherSpec and Finished, otherwise with an error message.
After that, we server is ready for data exchange and we send our HTTP request and receive a response.
The approach considered in the article allows you to work effectively with https for applications that do not require its full implementation. Instead of third-party implementations of ssl, which pull their own cryptography, you can use those already present in the project, as shown in the example of crypto ++, which reduces the number of dependencies, improves support and portability.
Implemented and used in almost i2pd - C ++ I2P implementation
Boot is the only place in I2P where https is used.
On the other hand, the article will be interesting to those who are interested in understanding how ssl works and try it yourself.
Inventing a bicycle
Our goal is to get an i2pseeds.su3 file of about 100K in size from one of the I2P reseed nodes. This file is signed by a separate certificate independent of the site certificate, so certificate verification can be excluded. The relatively short length of the received data allows us not to implement the mechanisms of compression and restoration of broken connections.
Only TLS 1.2 and the TLS_RSA_WITH_AES_256_CBC_SHA256 cipher suite will be used. In other words, AES256 in CBC mode is used to encrypt data, and RSA is used for key negotiation.
This choice is due to the fact that AES256-CBC is the most used encryption in I2P, and RSA to simplify the implementation of the protocol by reducing the number of messages required for key negotiation. In addition to RSA and AES, the following cryptographic functions from crypto ++ are also required:
')
- HMAC for computing checksums of encrypted messages and pseudo-random functions. Please note that the standard implementation of HMAC is used, not from I2P
- SHA256 hash for use with HMAC and for calculating the checksum of all messages involved in setting up a connection
- Functions for working with descriptions in the ASN.1 language in DER encoding. Required to extract a public key from an X.509 certificate
The RSA implementation is based on PKCS v1.5. The key length can be any and is determined by the certificate.
SSL messaging
Absolutely all transmitted messages begin with a 5-byte header, the first byte of which contains the message type, the next 2 bytes — the protocol version number (0x03, 0x03 for TSL 1.2) and then the length of the rest (content) of the message — 2 bytes in Big Endian, most defining message boundaries.
Thus, when receiving new data, you should first read 5 bytes of the header, and then how many bytes are in the length field.
There are 4 types of messages:
- 0x17 - data. The content is encrypted HTTP messages, and in our case using AES256, the key of which is calculated during the connection setup process. Data size must be a multiple of 16 bytes.
- 0x16 - set the connection. Multiple types defined by the corresponding field inside the content. Unencrypted, with the exception of the message 'finished', sent last.
- 0x15 is a warning. The message that "something went wrong." Close the connection. Contains codes of what exactly went wrong, you can use for debugging.
- 0x14 - change cipher. It is sent immediately after the key is negotiated. Content is 1 byte, always containing 0x01. In fact, it is part of the connection setup process.
In our implementation, the encrypted data is as follows:
16 bytes IV for CBC, in TSL 1.2 for each message own IV;
data length up to 64K is the length of the headers;
32 MAC bytes, calculated for a 13-byte header and data, the header consists of an 8-byte sequence number starting with zero, message type (0x17 or 0x16), version and data length. All in BigEndian. The key for HMAC is also calculated during the connection setup process;
a placeholder, so that the length of the encrypted data is a multiple of 16 bytes, the last byte contains the number of bytes of the placeholder, not taking into account itself. If the message length is a multiple of 16 bytes, then another 16 bytes will be added for this last byte with a length.
Connection setup
During the installation process, we must solve two problems:
- Coordinate and calculate encryption keys and HMAC
- Send the correct sequence of messages so that the other side does not close the connection, but goes into the data exchange mode.
In our case, the sequence of messages is as follows:
ClientHello -> (0x01)
<- ServerHello (0x02)
<- Certificate (0x0B)
<- ServerHelloDone (0x0E)
-> ClientKeyExchange (0x10)
-> ChangeCipherSpec
-> Finished (0x14)
<- ChangeCipherSpec
<- Finished (0x14)
where "->" means sending a message, and "<-" means receiving.
All messages, except for ChangeChiperSpec, are a message of the type 0x16 - connection establishment. The content of this type of message begins with its own 4-byte header, the first byte of which is the type of the connection setup message, as indicated above, and 3 bytes of the length of the remaining message, the highest byte of which in our case is always zero.
Consider these messages in detail.
Clienthello
The first message that we send to the server after a successful connection. Since we use one particular set of ciphers, in our case it will be permanent. Here it is:
static uint8_t clientHello[] = { 0x16, // handshake 0x03, 0x03, // version (TLS 1.2) 0x00, 0x2F, // length of handshake // handshake 0x01, // handshake type (client hello) 0x00, 0x00, 0x2B, // length of handshake payload // client hello 0x03, 0x03, // highest version supported (TLS 1.2) 0x45, 0xFA, 0x01, 0x19, 0x74, 0x55, 0x18, 0x36, 0x42, 0x05, 0xC1, 0xDD, 0x4A, 0x21, 0x80, 0x80, 0xEC, 0x37, 0x11, 0x93, 0x16, 0xF4, 0x66, 0x00, 0x12, 0x67, 0xAB, 0xBA, 0xFF, 0x29, 0x13, 0x9E, // 32 random bytes 0x00, // session id length 0x00, 0x02, // chiper suites length 0x00, 0x3D, // RSA_WITH_AES_256_CBC_SHA256 0x01, // compression methods length 0x00, // no compression 0x00, 0x00 // extensions length }; This message tells the server that we support TLS 1.2, this new connection (the length of the session identifier is zero) and support a single cipher suite - RSA with AES256. We also send a set of 32 "random" bytes for key generation. If these bytes are really random, then they should be remembered somewhere, because they will be needed later.
Serverhello
The “twin brother” is ClientHello, except that the message type is 0x02 instead of 0x01, and a non-empty session identifier. From this message, we need only 32 random bytes.
Certificate
It may contain several certificates, first comes the length of the entire group of certificates, then each certificate has its own length. We are only interested in the first certificate and read the length should be 2 times. The certificate itself is X.509 in DER encoding. From it, we need the RSA public key.
ServerHelloDone
It does not contain anything useful, but is taken into account when calculating a hash for Finished.
ClientKeyExchange
At this point, we have enough information to generate and reconcile the keys that occur in 3 stages: the generation of a random secret key, the master key calculation, the master key extension to obtain the encryption keys and the checksum.
The random secret key is 48 bytes, the first 2 of which are the version number (0x03, 0x03), and the remaining 46 are randomly generated. Further, these 48 bytes are encrypted with the RSA public key, and together with the length of the encrypted block are sent to the server. It should be noted that the length of the encrypted block will be equal to the key length, and not 48 bytes. For example, for certificates with a 2048-bit key, this length will be 256, and the length of the transmitted data will be 258.
ChangeCipherSpec
Dispatched immediately after ClientKeyExchange. Always the same:
static uint8_t changeCipherSpecs[] = { 0x14, // change cipher specs 0x03, 0x03, // version (TLS 1.2) 0x00, 0x01, // length 0x01 // type }; This message is of type 0x14 and hash calculation for Finished is not involved.
Pseudo-random function (PRF)
To further calculate the keys, we need a pseudo-random function that accepts 4 parameters at the input: the secret key just sent to the server, a label in the form of a text string, a block of initial data and the desired length of the result.
In TLS 1.2, it is defined as follows:
PRF (secret, label, seed) = P_SHA256 (secret, label + seed);
P_SHA256 (secret, seed) = HMAC_SHA256 (secret, A (1) + seed) +
HMAC_SHA256 (secret, A (2) + seed) +
HMAC_SH256 (secret, A (3) + seed) + ...
where A is determined by induction
A (0) = seed,
A (i) = HMAC_SHA256 (secret, A (i -1)).
That is, at each step we recalculate the checksum from the previous step, and then we calculate the checksum from combining the result with the text string and the original data, repeating this until the desired length is obtained.
Now the master key is calculated by the formula
PRF (secret, "master secret", clientRandom + serverRadom, 48);
where clientRandom is 32 random bytes from ClientHello, and serverRandom is from ServerHello.
Then it should be expanded to a 128-byte block, containing 4 32-byte keys in the following sequence: a MAC key for sending, a MAC key for receiving, an encryption key for sending, a decryption key for receiving.
MAC key for receiving is not used.
Key expansion is made according to the formula
PRF (masterSecret, "key expansion", serverRandom + clientRadom, 128)
clientRadom and serverRadom change places here.
Finished
At this point, we have everything we need to start the data exchange, but, unfortunately, we have to send a Finished message containing the correct data, otherwise the server will break the connection.
If all the previous posts were rather trivial, then Finshed is more complicated. Firstly, it is of type 0x16, but its contents are completely encrypted, while 0x16 also appears in the calculation of the checksum, and not 0x17 as for other encrypted messages.
The message itself contains the first 12 bytes from
PRT (masterSecret, "client finished", hash, 12)
where hash is SHA256 from the following sequence of messages:
ClientHello, ServerHello, Certficate, ServerHelloDone, ClientKeyExchange. All messages are counted without a 5 byte header.
If the message is generated correctly, the server will respond with ChangeCipherSpec and Finished, otherwise with an error message.
After that, we server is ready for data exchange and we send our HTTP request and receive a response.
findings
The approach considered in the article allows you to work effectively with https for applications that do not require its full implementation. Instead of third-party implementations of ssl, which pull their own cryptography, you can use those already present in the project, as shown in the example of crypto ++, which reduces the number of dependencies, improves support and portability.
Implemented and used in almost i2pd - C ++ I2P implementation
Source: https://habr.com/ru/post/251143/
All Articles