Once I ran into an incomprehensible mistake in establishing an OpenVPN connection and felt a lack of understanding of how it works inside. Under the cut a story about how the cryptographic part of the protocol is arranged, how it all looks in reality (ie in Wireshark) and how to look inside the VPN, i.e. decipher traffic hands (with the keys, of course ;-)). In this part, we consider only the mode with static keys.

Modes of operation

OpenVPN itself without plug-ins supports 2 ways to establish a connection:

• PSK (PreSharedKey or static key mode ) - with a symmetric key that is generated and written to all machines during the VPN setup process. The first part of the article deals only with this mode.
• TLS - based on public key infrastructure (PKI) . At the beginning, a TLS connection is established with the authentication of both parties using certificates, through which key material is exchanged. Symmetric keys are generated from this key material and the traffic inside the VPN is already encrypted with these keys in the same way as in the PSK mode. There is no user traffic channel inside the TLS channel - only the service information of OpenVPN itself. I plan to write the second part about the preparation of OpenVPN in this mode.

More about PSK mode

PSK mode is enabled by the secret option, the first parameter of which is the name of the file with static keys that are generated during the VPN configuration step. The file contains 2 pairs of 512 bit random keys (in each pair, the key for encryption and the key for HMAC), written down one by one (without headers and the rest) and encoded in HEX. The same key file must be on all computers connected by VPN. If the second optional parameter, direction , is not specified for the secret option, then only the first pair of keys will be used, that is, in both directions, the traffic will be signed and encrypted with the same keys.

In static key mode, only CBC and No-crypto data packet formats ( data channel crypto format ) can be used. Further we consider the CBC format, since No-crypto is not so interesting.

By default, BlowFish is used for encryption in CBC mode with a 128-bit key, and SHA1 and a 160-bit key are used for HMAC . The first 128 and 160 bits from the 512 bit keys from the file with the static key are used as keys, respectively.

Since all key information and settings are set in advance, if the address of the remote host (the remote option) is specified, the encapsulated traffic can be transmitted immediately after the launch of OpenVPN even before the exchange of control packets (they will be discussed below).

Test stand

The following key was used for tests (generated by the openvpn --genkey --secret psk.key command ):

# # 2048 bit OpenVPN static key # -----BEGIN OpenVPN Static key V1----- 5234f60f846bb1d5f059c70e75434be6 <--      89f41113ef56e2bf69253fad6a30ab5f a449204f52f64a0265fd5744a9489f41 cefd95a1d642830e9cf9cdce13c55245 270cf7d367ddc6b56eb1ba749be40e42 <--      HMAC 80cdf7cfbad178348a0e057f1fdc87f0 e5ba84717475b868a7fd617fee8c561c b4575d983534ef71dd8aaa48a53ed469 0ae85faf4522f7defd89ba373a0d22ee <--      f9356e487fdfa0796b0b1fc393fd3ab5 52db7a1ed691ddd50eeb7be2cc32d912 52df93987f4878ec42a12e3a7cda5a40 b1aa850a05821b300359a796313cbcec <--      HMAC 4076483ec7692708c32d323ed080beba 60c58d9281fb5d27c688ac271d3b6d15 1695093291fb788dbbfefb8b7c7f9bef -----END OpenVPN Static key V1----- 

In fact, every 4 lines with HEX is a 512 bit key. 1st and 3rd - keys for encryption, from each in the example only the first 128 bits will be used (the remaining 384 bits are not used at all), 2nd and 4th - keys for the HMAC , from each the first 160 bits will be used . In other words:

 key_encrypt_0 = unhexlify("5234f60f846bb1d5f059c70e75434be689f41113ef56e2bf69253fad6a30ab5fa449204f52f64a0265fd5744a9489f41cefd95a1d642830e9cf9cdce13c55245")[:16] key_hmac_0 = unhexlify("270cf7d367ddc6b56eb1ba749be40e4280cdf7cfbad178348a0e057f1fdc87f0e5ba84717475b868a7fd617fee8c561cb4575d983534ef71dd8aaa48a53ed469")[:20] key_encrypt_1 = unhexlify("0ae85faf4522f7defd89ba373a0d22eef9356e487fdfa0796b0b1fc393fd3ab552db7a1ed691ddd50eeb7be2cc32d91252df93987f4878ec42a12e3a7cda5a40")[:16] key_hmac_1 = unhexlify("b1aa850a05821b300359a796313cbcec4076483ec7692708c32d323ed080beba60c58d9281fb5d27c688ac271d3b6d151695093291fb788dbbfefb8b7c7f9bef")[:20] 

Settings files when working through UDP in static key mode using both pairs of keys (each direction is encrypted by its own pair):

 #   dev tun ifconfig 192.168.1.1 192.168.1.2 remote 192.168.0.2 secret psk.key 0 

 #   dev tun ifconfig 192.168.1.2 192.168.1.1 remote 192.168.0.1 secret psk.key 1 

Where 192.168.0.1 and 192.168.0.2 are the addresses of the physical interfaces of the machines, and 192.168.1.1 and 192.168.1.2 are the corresponding addresses inside the tunnel.

The remote option is sufficient to specify only one of the configuration files. In this case, the host to which the remote is not specified will simply wait for the incoming connection (in this case, the UDP packet).

We prepare

The format of the transmitted packet is:

 [Len (2 )] | HMAC (20 ) | IV (8 ) | packet ID (4 ) | timestamp (4 ) | packet payload 

Len - the length of the entire packet, except for the first two bytes; used only when working via TCP. When working via UDP, the OpenVPN packet length is calculated from the UDP packet length, and the Len field in the OpenVPN packet does not exist.
HMAC - HMAC-SHA1 from everything that comes after HMAC (IV and encrypted data)
IV - initialization vector for CBC mode.
Everything that comes after IV is transmitted encrypted.
Packet ID - package number. It is included here to protect against repetition, and is not used to organize guaranteed data delivery.
timestamp - also used to protect against repetitions.
packet payload - data that carries the packet. This is either the OpenVPN service information or the encapsulated traffic itself.

Decrypt service packages

Now promised wireshark (contrary to the statement on the dissector page , he does not understand OpenVPN in static key mode):

Take the first packet and try to decrypt it. Since VPN is configured to work via UDP, then the Len field is not present, the package starts immediately with HMAC. Let's try to calculate what the HMAC from this packet should be equal to without the HMAC field:

 packet = unhexlify("7c0a45b025b3225c9a60051699fc87bae44dec9026e792a7ff3842aabd7cc77b439564b42002e5e327cba04c620cd80ca92c60139e2345a2c5d99b0f") print("HMAC = %s" % HMAC(key_hmac_0, packet[20:], 'sha1').hexdigest()) 

 HMAC = 7c0a45b025b3225c9a60051699fc87bae44dec90 

We see that the HMAC calculated by us exactly coincides with the HMAC inserted into the packet. So packet authentication is passed, and we are on the right track! Now it's time to decipher it.

 iv = packet[20:28] encrypted_part = packet[28:] decrypted = Blowfish.new(key_encrypt_0, mode=Blowfish.MODE_CBC, IV=iv).decrypt(encrypted_part) print(" : %s" % hexlify(decrypted).decode()) 

  : 00000001 58069a45 287f346bd4ef7a812d56b8d3afc5459c00 07070707070707 

The first 4 bytes are the packet number. That's right, we took the first package. The second 4 bytes are a timestamp. Seven bytes 0x07 at the end is padding (the bytes added during encryption so that the message length is a multiple of the block length), it is correct, discarded.

 packet_id = decrypted[:4] timestamp = decrypted[4:8] decrypted_data = decrypted[8:-decrypted[-1]] #  packet_id  timestamp       print("ID : %s" % hexlify(packet_id).decode()) print(" : %s" % hexlify(timestamp).decode()) print(" : %s" % hexlify(decrypted_data).decode()) 

 ID : 00000001  : 58069a45  : 287f346bd4ef7a812d56b8d3afc5459c00 

At first glance, it looks like a 16-byte random string and a zero byte are being transmitted. Since OpenVPN is an open source project, we go into the code and in the occ.c file we find the answer, partially confirming the hypothesis.

287f346bd4ef7a812d56b8d3afc5459c - this is the occ_magic constant - a sign that the package is the OpenVPN control package. It is followed by a type sign of the control packet, in this case it is byte 0x00 == OCC_REQUEST . In the packet with OCC_REQUEST nothing more should be transmitted.

Now decipher the second package:

 packet = unhexlify("91861a4479c376d3013cdcd7f3e657ac093048cdb3a40775fc5804f68842ef06288a9343acb298562aaa4c92c10aae2213441619c3df0d8b89a0f13f5dd69d1eaa0e22667938b68e38a1238e39b76b889b7b6e5dbadd8f3a264f6fc9bb9c225de387d5d17914b2e2853d7fbb706da429f596721ab394febe7e52875183f0e9e10ec16042398c4116c2069ef18f06da45b55581a9997d1f54def143b053226727934404be44379a0172d1b618bbf1b676b60f57889e734af6b54291d07d9c7e13573cc81e") #        'secret'    'direction',         . key_hmac = key_hmac_1 key_encrypt = key_encrypt_1 print("HMAC = %s" % HMAC(key_hmac, packet[20:], 'sha1').hexdigest()) iv = packet[20:28] encrypted_part = packet[28:] decrypted = Blowfish.new(key_encrypt, mode=Blowfish.MODE_CBC, IV=iv).decrypt(encrypted_part) print(" : %s" % hexlify(decrypted).decode()) packet_id = decrypted[:4] timestamp = decrypted[4:8] decrypted_data = decrypted[8:-decrypted[-1]] #  packet_id  timestamp       print("ID : %s" % hexlify(packet_id).decode()) print(" : %s" % hexlify(timestamp).decode()) print("  HEX: %s" % hexlify(decrypted_data).decode()) print(" : %s" % decrypted_data) 

 HMAC = 91861a4479c376d3013cdcd7f3e657ac093048cd  : 0000000158069a45287f346bd4ef7a812d56b8d3afc5459c0156342c6465762d747970652074756e2c6c696e6b2d6d747520313534342c74756e2d6d747520313530302c70726f746f2055445076342c6966636f6e666967203139322e3136382e312e31203139322e3136382e312e322c6b657964697220312c6369706865722042462d4342432c6175746820534841312c6b657973697a65203132382c73656372657400030303 ID : 00000001  : 58069a45   HEX: 287f346bd4ef7a812d56b8d3afc5459c0156342c6465762d747970652074756e2c6c696e6b2d6d747520313534342c74756e2d6d747520313530302c70726f746f2055445076342c6966636f6e666967203139322e3136382e312e31203139322e3136382e312e322c6b657964697220312c6369706865722042462d4342432c6175746820534841312c6b657973697a65203132382c73656372657400    ASCII: b'(\x7f4k\xd4\xefz\x81-V\xb8\xd3\xaf\xc5E\x9c\x01V4,dev-type tun,link-mtu 1544,tun-mtu 1500,proto UDPv4,ifconfig 192.168.1.1 192.168.1.2,keydir 1,cipher BF-CBC,auth SHA1,keysize 128,secret\x00' 

We see that the first 16 bytes of this packet are occ_magic , and the next byte (type of control packet) is 0x01 == OCC_REPLY . In the OCC_REPLY package, after the type of the package, a C-string with parameters is passed. In this case, the following text is transmitted: V4, dev-type tun, link-mtu 1544, tun-mtu 1500, proto UDPv4, ifconfig 192.168.1.1 192.168.1.2, keydir 1, cipher BF-CBC, auth SHA1, keysize 128, secret . The same line can be found in the OpenVPN log.

The next two packages are OCC_REQUEST from the second host to the first and OCC_REPLY to it. We decipher them in the same way:

 HMAC = fc52f6981ccb4d7c2a3c951bd1ce8e882f9155da ID : 00000002  : 58069a45   HEX: 287f346bd4ef7a812d56b8d3afc5459c00  : b'(\x7f4k\xd4\xefz\x81-V\xb8\xd3\xaf\xc5E\x9c\x00' 

 HMAC = e4987efec6d4625e89ba26b69b0d8f54ed391c3c ID : 00000002  : 58069a45   HEX: 287f346bd4ef7a812d56b8d3afc5459c0156342c6465762d747970652074756e2c6c696e6b2d6d747520313534342c74756e2d6d747520313530302c70726f746f2055445076342c6966636f6e666967203139322e3136382e312e32203139322e3136382e312e312c6b657964697220302c6369706865722042462d4342432c6175746820534841312c6b657973697a65203132382c73656372657400  : b'(\x7f4k\xd4\xefz\x81-V\xb8\xd3\xaf\xc5E\x9c\x01V4,dev-type tun,link-mtu 1544,tun-mtu 1500,proto UDPv4,ifconfig 192.168.1.2 192.168.1.1,keydir 0,cipher BF-CBC,auth SHA1,keysize 128,secret\x00' 

After receiving the settings of the remote host, the local host checks the compliance of its own and other people's settings and issues a warning if there are inconsistencies. In this case, we see that the VPN settings on both sides match up to the direction:

V4, dev-type tun, link-mtu 1544, tun-mtu 1500, proto UDPv4, ifconfig 192.168.1.1 192.168.1.2, keydir 1, cipher BF-CBC, auth SHA1, keysize 128, secret
V4, dev-type tun, link-mtu 1544, tun-mtu 1500, proto UDPv4, ifconfig 192.168.1.2 192.168.1.1, keydir 0, cipher BF-CBC, auth SHA1, keysize 128, secret

Decrypt Encapsulated Traffic

The next packet in this traffic dump carries user data. We decipher them in the same way as before:

 HMAC = efe4850d41f0e4a7535f13310685d74c2722a0da ID : 00000003  : 58069a45   HEX: 4500005420874000400196cec0a80102c0a80101080066ed1a1300014d9a0658000000005a390a0000000000101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f3031323334353637 

There is neither occ_magic, nor ping_magic (another magic constant equal to 2a187bf3641eb4cb07ed2d0a981fc748 ), so this is a data packet. In fact, 0x45 is the first byte of an IPv4 packet without optional header fields. We see that this is exactly the same IPv4 packet with ping-request, which came out of the tun interface (in hex in the picture, IPv4 starts from the 2nd line):

Return package:

 HMAC = a0a0108cd173e8829b91e8b34bd5cac2ab8a9c0b ID : 00000003  : 58069a45   HEX: 450000547f00000040017855c0a80101c0a8010200006eed1a1300014d9a0658000000005a390a0000000000101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f3031323334353637 

It can be seen that this is exactly the same IPv4 with the ping-reply that was sent to the tunnel:

Remarks

OpenVPN Ping

There are also ping packages - service packages that OpenVPN sends to confirm the connection if too much time has passed since the last package was sent to another host (configured with keepalive, inactive and ping * options). Unlike the usual icmp ping, OpenVPN does not respond to the received packet, but simply remembers that the connection is alive.

nmap vs OpenVPN

The UDP port, on which OpenVPN listens in PSK mode (as well as in TLS mode with the TLS-AUTH option enabled), for anyone who does not know the secret key, will look as if all the packets on it are dropping. nmap will show the status of this port as "open | filtered". In the OpenVPN logs (if the verb option is high enough), when scanning the port, the entries "Authenticate / Decrypt packet error: packet HMAC authentication failed" will be displayed.

OpenVPN traffic detection

In PSK mode, when working through UDP, the entire contents of the OpenVPN packets for an outside observer (who does not have secret keys) will look random, i.e. there are no signatures that could be used to attribute these packages to OpenVPN. However, the length of the packets will be out of the random data image - it depends on the hash function used and varies with a step equal to the block size of the cipher used, as well as the presence of service packets of a fixed length (OCC_REQUEST and OpenVPN PING packets).

When working via TCP, besides, at the beginning of each OpenVPN packet there will be an unencrypted 2-byte packet length field.

Conclusion

It can be seen that in the PSK mode, everything is very simple. There is not even a client / server separation here.

In TLS mode, establishing a connection is much more complicated, but it allows using the public key infrastructure , provides regular session key changes and implements Perfect forward secrecy . But about TLS mode in the next part.