Geekly Articles each Day
⬆️ ⬇️

GOSTIM: P2P F2F E2EE IM in one evening with GOST-cryptography

As a developer of the PyGOST library (GOST cryptographic primitives in pure Python), I often get questions about how to implement simplest secure messaging on the knee. Many people consider application cryptography to be a fairly simple thing, and the .encrypt () call for a block cipher will be enough to send it safely over the communication channel. Others believe that applied cryptography is the lot of the few, and it is acceptable that rich companies like Telegram with mathematicians cannot implement a secure protocol.



All this prompted me to write this article to show that the implementation of cryptographic protocols and secure IM is not such a difficult task. However, inventing your own authentication and key agreement protocols is not worth it.



Hearing


The article will write a peer-to-peer , friend-to-friend , end-to-end encrypted instant messenger with SIGMA-I key authentication and key agreement (based on which IPsec IKE is implemented), using only PyGOST GOST-standard cryptographic algorithms and ASN.1 encoding of messages by the PyDERASN library (about which I already wrote earlier ). Prerequisite: it must be so simple that it can be written from scratch in one evening (or working day), otherwise it is no longer a simple program. It certainly has errors, unnecessary complexity, shortcomings, plus this is my first program using the asyncio library.



IM design



First you need to understand how our IM will look like. For simplicity, let it be a peer-to-peer network, without any detection of participants. We will personally indicate to which address: the port to connect to communicate with the interlocutor.

')

I understand that at the moment, the assumption of the availability of a direct connection between two arbitrary computers is a significant limitation of the applicability of IM in practice. But the more developers will implement all sorts of NAT-traversal crutches, the longer we will remain on the IPv4 Internet, with a depressing likelihood of communication between arbitrary computers. How much can you tolerate the lack of IPv6 at home and at work?



We will have a friend-to-friend network: all possible interlocutors should be known in advance. First, it simplifies everything a lot: they introduced themselves, found or didn’t find the name / key, disconnected or continue working, knowing the interlocutor. Secondly, in the general case, it is safe and eliminates many attacks.



The IM interface will be close to the classic solutions of suckless projects , which I like very much with its minimalism and Unix-way philosophy. An IM program for each interlocutor creates a directory with three Unix domain sockets:





In addition, a conn socket is created, by writing to which host port, we initiate a connection to a remote peer. 



 | - alice
 |  | - in
 |  | - out
 |  `- state
 | - bob
 |  | - in
 |  | - out
 |  `- state
 `- conn


This approach allows you to make independent implementations of IM transport and user interface, because there is no friend to the taste and color, you cannot please everyone. Using tmux and / or multitail , you can get a multi-window interface with syntax highlighting. And using rlwrap you can get a GNU Readline-compatible line for entering messages.



In fact, suckless projects use FIFO files. Personally, I could not understand how in asyncio to work with files competitively without a handwritten substrate from selected threads (for such things I have been using the Go language for a long time). Therefore, I decided to do without Unix domain sockets. Unfortunately, this makes it impossible to do echo 2001: 470: dead :: babe 6666> conn. I solved this problem using socat : echo 2001: 470: dead :: babe 6666 | socat - UNIX-CONNECT: conn, socat READLINE UNIX-CONNECT: alice / in.



Initial insecure protocol



TCP is used as a transport: it guarantees delivery and its order. UDP does not guarantee either one or the other (which would be useful when cryptography is used), and there is no SCTP support in Python out of the box.



Unfortunately, there is no message concept in TCP, only a stream of bytes. Therefore, it is necessary to come up with a format for messages so that they can be divided among themselves in this stream. We can agree to use the newline character. For a start, however, when we begin to encrypt our messages, this symbol may appear anywhere in the ciphertext. In networks, therefore, protocols are popular, sending first the length of the message in bytes. For example, in Python, there is xdrlib out of the box, which allows you to work with a similar XDR format.



We will not work properly and efficiently with TCP reading - simplify the code. We read data from the socket in an infinite loop until we decode the complete message. JSON with XML can also be used as a format for this approach. But when cryptography is added, the data will have to be signed and authenticated - and this will require byte-by-byte identical representation of objects, which is not provided by JSON / XML (dumps result may differ).



XDR is suitable for this task, but I choose ASN.1 with DER encoding and the PyDERASN library, since we will have high-level objects on hand, which are often nicer and more convenient to work with. Unlike schemaless bencode , MessagePack or CBOR , ASN.1 automatically checks data against a hard-coded scheme. 



# Msg ::= CHOICE { # text MsgText, # handshake [0] EXPLICIT MsgHandshake } class Msg(Choice): schema = (( ("text", MsgText()), ("handshake", MsgHandshake(expl=tag_ctxc(0))), )) # MsgText ::= SEQUENCE { # text UTF8String (SIZE(1..MaxTextLen))} class MsgText(Sequence): schema = (( ("text", UTF8String(bounds=(1, MaxTextLen))), )) # MsgHandshake ::= SEQUENCE { # peerName UTF8String (SIZE(1..256)) } class MsgHandshake(Sequence): schema = (( ("peerName", UTF8String(bounds=(1, 256))), ))


The received message will be Msg: either the text MsgText (for now with one text field), or the message of the handshake MsgHandshake (in which the name of the interlocutor is transmitted). Now it looks over complicated, but this is a reserve for the future. 



      ─────┐ ┌─────┐
      │PeerA│ │PeerB│
      ──┬──┘ └──┬──┘
         │MsgHandshake (IdA) │
         ─────────────────>>
         │ │
         │MsgHandshake (IdB) │
         │ <─────────────────│
         │ │
         │ MsgText () │
         ─────────────────>>
         │ │
         │ MsgText () │
         │ <─────────────────│
         │ │




IM without cryptography



As I said, for all operations with sockets will be used asyncio library. Let's announce what we expect at the time of launch: 



 parser = argparse.ArgumentParser(description="GOSTIM") parser.add_argument( "--our-name", required=True, help="Our peer name", ) parser.add_argument( "--their-names", required=True, help="Their peer names, comma-separated", ) parser.add_argument( "--bind", default="::1", help="Address to listen on", ) parser.add_argument( "--port", type=int, default=6666, help="Port to listen on", ) args = parser.parse_args() OUR_NAME = UTF8String(args.our_name) THEIR_NAMES = set(args.their_names.split(","))


Set your own name (--our-name alice). A comma is used to list all the peers expected (--their-names bob, eve). For each of the interlocutors, a directory with Unix sockets is created, as well as a corortine for each in, out, state: 



 for peer_name in THEIR_NAMES: makedirs(peer_name, mode=0o700, exist_ok=True) out_queue = asyncio.Queue() OUT_QUEUES[peer_name] = out_queue asyncio.ensure_future(asyncio.start_unix_server( partial(unixsock_out_processor, out_queue=out_queue), path.join(peer_name, "out"), )) in_queue = asyncio.Queue() IN_QUEUES[peer_name] = in_queue asyncio.ensure_future(asyncio.start_unix_server( partial(unixsock_in_processor, in_queue=in_queue), path.join(peer_name, "in"), )) asyncio.ensure_future(asyncio.start_unix_server( partial(unixsock_state_processor, peer_name=peer_name), path.join(peer_name, "state"), )) asyncio.ensure_future(asyncio.start_unix_server(unixsock_conn_processor, "conn"))


Messages coming from the user from the in socket are sent to the IN_QUEUES queue: 



 async def unixsock_in_processor(reader, writer, in_queue: asyncio.Queue) -> None: while True: text = await reader.read(MaxTextLen) if text == b"": break await in_queue.put(text.decode("utf-8"))


Messages coming from interlocutors send queues to OUT_QUEUES, from which data is written to the out socket: 



 async def unixsock_out_processor(reader, writer, out_queue: asyncio.Queue) -> None: while True: text = await out_queue.get() writer.write(("[%s] %s" % (datetime.now(), text)).encode("utf-8")) await writer.drain()


When reading from a state socket, the program searches the PEER_ALIVE dictionary for the address of the interlocutor. If there is still no connection to the other party, then an empty string is written. 



 async def unixsock_state_processor(reader, writer, peer_name: str) -> None: peer_writer = PEER_ALIVES.get(peer_name) writer.write( b"" if peer_writer is None else (" ".join([ str(i) for i in peer_writer.get_extra_info("peername")[:2] ]).encode("utf-8") + b"\n") ) await writer.drain() writer.close()


When the address is written to the conn socket, the connection “initiator” function is launched: 



 async def unixsock_conn_processor(reader, writer) -> None: data = await reader.read(256) writer.close() host, port = data.decode("utf-8").split(" ") await initiator(host=host, port=int(port))


Consider the initiator. First, he obviously opens the connection to the specified host / port and sends a handshake message with his name: 



  130 async def initiator(host, port): 131 _id = repr((host, port)) 132 logging.info("%s: dialing", _id) 133 reader, writer = await asyncio.open_connection(host, port) 134 # Handshake message {{{ 135 writer.write(Msg(("handshake", MsgHandshake(( 136 ("peerName", OUR_NAME), 137 )))).encode()) 138 # }}} 139 await writer.drain()


Then waiting for a response from the remote side. Attempts to decode the received answer according to the Msg ASN.1 scheme. We assume that the entire message will be sent by one TCP segment and we will receive it atomically when we call .read (). We check that we received a handshake message. 



  141 # Wait for Handshake message {{{ 142 data = await reader.read(256) 143 if data == b"": 144 logging.warning("%s: no answer, disconnecting", _id) 145 writer.close() 146 return 147 try: 148 msg, _ = Msg().decode(data) 149 except ASN1Error: 150 logging.warning("%s: undecodable answer, disconnecting", _id) 151 writer.close() 152 return 153 logging.info("%s: got %s message", _id, msg.choice) 154 if msg.choice != "handshake": 155 logging.warning("%s: unexpected message, disconnecting", _id) 156 writer.close() 157 return 158 # }}}


We check that the incoming name of the interlocutor is known to us. If not, then break the connection. We check if we have already established a connection with him (the interlocutor again gave the command to connect to us) and close it. The IN_QUEUES queue contains Python strings with the message text, but there is a special value of None, signaling msg_sender to stop the work so that it will forget about its writer associated with the obsolete TCP connection. 



  159 msg_handshake = msg.value 160 peer_name = str(msg_handshake["peerName"]) 161 if peer_name not in THEIR_NAMES: 162 logging.warning("unknown peer name: %s", peer_name) 163 writer.close() 164 return 165 logging.info("%s: session established: %s", _id, peer_name) 166 # Run text message sender, initialize transport decoder {{{ 167 peer_alive = PEER_ALIVES.pop(peer_name, None) 168 if peer_alive is not None: 169 peer_alive.close() 170 await IN_QUEUES[peer_name].put(None) 171 PEER_ALIVES[peer_name] = writer 172 asyncio.ensure_future(msg_sender(peer_name, writer)) 173 # }}}


msg_sender accepts outgoing messages (attached to the queue from the in socket), serializes them to the MsgText message and sends it via a TCP connection. It can break at any time - we obviously intercept it. 



 async def msg_sender(peer_name: str, writer) -> None: in_queue = IN_QUEUES[peer_name] while True: text = await in_queue.get() if text is None: break writer.write(Msg(("text", MsgText(( ("text", UTF8String(text)), )))).encode()) try: await writer.drain() except ConnectionResetError: del PEER_ALIVES[peer_name] return logging.info("%s: sent %d characters message", peer_name, len(text))


At the end, the initiator enters an infinite loop reading messages from the socket. Checks whether these are text messages, and puts the queue into OUT_QUEUES, from which they will be sent out to the socket of the corresponding interlocutor. Why not just make .read () and decode the message? Because it is not excluded that several messages from the user will be aggregated in the operating system buffer and sent by one TCP segment. We will be able to decode the first, and then a part of the subsequent one may remain in the buffer. For any abnormal situation, we close the TCP connection and stop the msg_sender corute (by sending None to the OUT_QUEUES queue). 



  174 buf = b"" 175 # Wait for test messages {{{ 176 while True: 177 data = await reader.read(MaxMsgLen) 178 if data == b"": 179 break 180 buf += data 181 if len(buf) > MaxMsgLen: 182 logging.warning("%s: max buffer size exceeded", _id) 183 break 184 try: 185 msg, tail = Msg().decode(buf) 186 except ASN1Error: 187 continue 188 buf = tail 189 if msg.choice != "text": 190 logging.warning("%s: unexpected %s message", _id, msg.choice) 191 break 192 try: 193 await msg_receiver(msg.value, peer_name) 194 except ValueError as err: 195 logging.warning("%s: %s", err) 196 break 197 # }}} 198 logging.info("%s: disconnecting: %s", _id, peer_name) 199 IN_QUEUES[peer_name].put(None) 200 writer.close() 66 async def msg_receiver(msg_text: MsgText, peer_name: str) -> None: 67 text = str(msg_text["text"]) 68 logging.info("%s: received %d characters message", peer_name, len(text)) 69 await OUT_QUEUES[peer_name].put(text)


Let's go back to the main code. After creating all the korutin at the time of launching the program, we start the TCP server. For each established connection, he creates a responder (responder) coruntine. 



 logging.basicConfig( level=logging.INFO, format="%(levelname)s %(asctime)s: %(funcName)s: %(message)s", ) loop = asyncio.get_event_loop() server = loop.run_until_complete(asyncio.start_server(responder, args.bind, args.port)) logging.info("Listening on: %s", server.sockets[0].getsockname()) loop.run_forever()


The responder is similar to initiator and mirrors all the same actions, but the endless message reading loop starts right away for simplicity. Now the handshake protocol sends one message from each side, but in the future, there will be two from the initiator of the connection, after which it is immediately possible to send text messages. 



  72 async def responder(reader, writer): 73 _id = writer.get_extra_info("peername") 74 logging.info("%s: connected", _id) 75 buf = b"" 76 msg_expected = "handshake" 77 peer_name = None 78 while True: 79 # Read until we get Msg message {{{ 80 data = await reader.read(MaxMsgLen) 81 if data == b"": 82 logging.info("%s: closed connection", _id) 83 break 84 buf += data 85 if len(buf) > MaxMsgLen: 86 logging.warning("%s: max buffer size exceeded", _id) 87 break 88 try: 89 msg, tail = Msg().decode(buf) 90 except ASN1Error: 91 continue 92 buf = tail 93 # }}} 94 if msg.choice != msg_expected: 95 logging.warning("%s: unexpected %s message", _id, msg.choice) 96 break 97 if msg_expected == "text": 98 try: 99 await msg_receiver(msg.value, peer_name) 100 except ValueError as err: 101 logging.warning("%s: %s", err) 102 break 103 # Process Handshake message {{{ 104 elif msg_expected == "handshake": 105 logging.info("%s: got %s message", _id, msg_expected) 106 msg_handshake = msg.value 107 peer_name = str(msg_handshake["peerName"]) 108 if peer_name not in THEIR_NAMES: 109 logging.warning("unknown peer name: %s", peer_name) 110 break 111 writer.write(Msg(("handshake", MsgHandshake(( 112 ("peerName", OUR_NAME), 113 )))).encode()) 114 await writer.drain() 115 logging.info("%s: session established: %s", _id, peer_name) 116 peer_alive = PEER_ALIVES.pop(peer_name, None) 117 if peer_alive is not None: 118 peer_alive.close() 119 await IN_QUEUES[peer_name].put(None) 120 PEER_ALIVES[peer_name] = writer 121 asyncio.ensure_future(msg_sender(peer_name, writer)) 122 msg_expected = "text" 123 # }}} 124 logging.info("%s: disconnecting", _id) 125 if msg_expected == "text": 126 IN_QUEUES[peer_name].put(None) 127 writer.close()


Secure protocol



It's time to secure our communication. What do we mean by security and what we want:





Surprisingly, almost everyone wants to have this minimum in any protocol of a handshake, and very few of the above are performed for homegrown protocols. And now we will not invent a new one. I would definitely recommend using the Noise framework for building protocols, but we’ll choose something simpler.



Two protocols are most popular:





How is SIGMA, as the last link in the development of STS / ISO protocols, good? It satisfies all our requirements (including “hiding” identifiers of interlocutors), has no known cryptographic problems. It is minimalistic - removing at least one element from the protocol message will lead to its insecurity.



Let's go from the simplest homebrew protocol to SIGMA. The most basic operation we are interested in is the key agreement : a function at the output of which both participants will receive the same value that can be used as a symmetric key. Without going into details: each of the parties generates an ephemeral (used only within one session) key pair (public and private keys), exchange public keys, call the reconciliation function, to the input of which they transmit their private key and the interlocutor's public key. 



 ─────┐ ┌─────┐
 │PeerA│ │PeerB│
 ──┬──┘ └──┬──┘
    │ IdA, PubA │ ╔═════════════════════╗
    ───────────────> │ PRVA, PubA = DHgen ()
    │ ╚════════════════════╝
    │ IdB, PubB │ ╔═════════════════════╗
    │ <───────────────│ ║PrvB, PubB = DHgen ()
    │ ╚════════════════════╝
    ────┐ ╔════════════════════╗
        │ ║Key = DH (PrvA, PubB) ║
    <───┘ ╚════════════════════╝
    │ │
    │ │




Anyone can enter-in the middle and replace the public keys with their own - this protocol does not authenticate interlocutors. Add a signature with long-lived keys. 



 ─────┐ ┌─────┐
 │PeerA│ │PeerB│
 ──┬──┘ └──┬──┘
    │IdA, PubA, sign (SignPrvA, (PubA)) │ ╔═══════════════════════════╗
    ─────────────────────────────────> ║ ║SignPrvA, SignPubA = load ()
    │ │ PrvA, PubA = DHgen ()
    │ ╚════════════════════════════╝
    │IdB, PubB, sign (SignPrvB, (PubB)) │ ╔═══════════════════════════╗
    <─────────────────────────────────│ SignPrvB, SignPubB = load ()
    │ │ PrvB, PubB = DHgen ()
    │ ╚════════════════════════════╝
    ────┐ ╔═════════════════════╗ │
        │ ║verify (SignPubB, ...) ║ │
    <───┘ ║Key = DH (PrvA, PubB) ║ │
    ╚═════════════════════╝ │
    │ │




Such a signature will not work, since it is not tied to a specific session. Such messages "fit" for sessions with other participants. Subscribe must be the entire context. It also forces you to add another message from A.



In addition, it is critical to add your own identifier for the signature, because otherwise we can replace IdXXX and re-sign the message with the key of another well-known interlocutor. To prevent the reflection of attacks , it is necessary that the elements under the signature are in clearly defined places in their meaning: if A signs (PubA, PubB), then B must sign (PubB, PubA). This also shows the importance of choosing the structure and format of the serialized data. For example, the sets in ASN.1 DER coding are sorted: SET OF (PubA, PubB) will be identical to SET OF (PubB, PubA). 



 ─────┐ ┌─────┐
 │PeerA│ │PeerB│
 ──┬──┘ └──┬──┘
    │ IdA, PubA │ ╔════════════════════════════╗
    ────────────────────────────────────────────> ║SignPrvA, SignPubA = load () ║
    │ │ PrvA, PubA = DHgen ()
    │ ╚════════════════════════════╝
    │IdB, PubB, sign (SignPrvB, (IdB, PubA, PubB)) │ ═════════════════════════════╗
    │ <───────────────────────────────────────────│ ║SignPrvB, SignPubB = load () ║
    │ │ PrvB, PubB = DHgen ()
    │ ╚════════════════════════════╝
    │ sign (SignPrvA, (IdA, PubB, PubA)) │ ╔═════════════════════╗
    ────────────────────────────────────────────> ║ rifverify ( SignPubB, ...) ║
    │ │ ║Key = DH (PrvA, PubB) ║
    │ ╚═════════════════════╝
    │ │




However, we still have not “proved” that we have developed the same common key for this session. In principle, you can do without this step - the very first transport message will be invalid, but we want that when the handshake is completed, we would be sure that everything is really coordinated. We currently have the ISO / IEC IS 9798-3 protocol in our hands.



We could sign the key itself. This is dangerous, since it is possible that there may be leaks in the signature algorithm used (let bits for signature, but still leaks). It is possible to sign a hash of a key that has been generated, but even a hash of a key that has been generated can have a value when a brute-force attack on a function of production has been leaked. SIGMA uses a MAC function that authenticates the sender ID. 



 ─────┐ ┌─────┐
 │PeerA│ │PeerB│
 ──┬──┘ └──┬──┘
    │ IdA, PubA │ ╔════════════════════════════╗
    ────────────────────────────────────────────────── > │ ║SignPrvA, SignPubA = load () ║
    │ │ PrvA, PubA = DHgen ()
    │ ╚════════════════════════════╝
    │IdB, PubB, sign (SignPrvB, (PubA, PubB)), MAC (IdB) │ ╔════════════════════════════╗
    │ <───────────────────────────────────────────────── ─│ ║SignPrvB, SignPubB = load () ║
    │ │ PrvB, PubB = DHgen ()
    │ ╚════════════════════════════╝
    │ ╔═════════════════════╗
    │ sign (SignPrvA, (PubB, PubA)), MAC (IdA) │ ║Key = DH (PrvA, PubB)
    ────────────────────────────────────────────────── > │ ║verify (Key, IdB) ║
    │ │ ║verify (SignPubB, ...) ║
    │ ╚═════════════════════╝
    │ │




As an optimization, some may want to reuse their ephemeral keys (which, of course, is pitiable for PFS). For example, we generated a key pair, tried to connect, but TCP was not available or broke off somewhere in the middle of the protocol. It's a pity to spend the spent entropy and processor resources on a new pair. Therefore, we introduce a so-called cookie - a pseudo-random value that will protect against possible accidental replay attacks when reusing ephemeral public keys. Because of the binding between the cookie and the ephemeral public key, the public key of the opposite participant can be removed from the signature as unnecessary. 



 ─────┐ ┌─────┐
 │PeerA│ │PeerB│
 ──┬──┘ └──┬──┘
    │ IdA, PubA, CookieA │ ╔════════════════════════════╗
    ────────────────────────────────────────────────── ─────────────────────> ─ ║SignPrvA, SignPubA = load ()
    │ │ PrvA, PubA = DHgen ()
    │ ╚════════════════════════════╝
    │IdB, PubB, CookieB, sign (SignPrvB, (CookieA, CookieB, PubB)), MAC (IdB) │ ╔════════════════════════ ═══╗
    │ <───────────────────────────────────────────────── ──────────────────────│ ║SignPrvB, SignPubB = load ()
    │ │ PrvB, PubB = DHgen ()
    │ ╚════════════════════════════╝
    │ ╔═════════════════════╗
    │ sign (SignPrvA, (CookieB, CookieA, PubA)), MAC (IdA) │ ║Key = DH (PrvA, PubB)
    ────────────────────────────────────────────────── ──────────────────────> ║ rifverify (Key, IdB)
    │ │ ║verify (SignPubB, ...) ║
    │ ╚═════════════════════╝
    │ │




Finally, we want to get the privacy of our interlocutor identifiers from a passive observer. To do this, SIGMA proposes first exchanging ephemeral keys, working out a common key on which to encrypt authenticating and identifying messages. SIGMA describes two options:



Source: https://habr.com/ru/post/452200/

More articles:



All Articles