Distributed nature of Tox Messenger
While the owners are going to block the centralized Telegram , the community of users of the distributed Tox messenger is growing. Today, according to www.toxstats.com site statistics , Russia ranks second after the United States in terms of the number of users by just 30-50 nodes.
In this publication, I would like to tell you about the distributed nature of this messenger, the general principles of operation of the Tox DHT network, as well as how to " catch up and overtake America " by the number of nodes.
')
Each Tox node is identified by its IP address, port number, and 256-bit public key. There are two conditional types of nodes:
The public key of the node is used to encrypt the packets sent to this node. Packets are decrypted on the side of the node using a 256-bit private key. To transmit DHT packets, the UDP protocol is used.
To “enter” a Tox network, it is enough to have connectivity with any Tox node that is already on the network. Usually, a list of known bootstrap nodes on the Internet is used for this. Additionally, the libtoxcore library uses sending packets to broadcast addresses, which allows you to connect to the Tox network without having access to the Internet (provided that you have the necessary node in your network segment). And even without going online, two or more Tox nodes form an isolated Tox network, allowing local clients to interact.
Since each Tox node has a unique identifier in the form of a public key, the relationship between any two nodes can be expressed by an artificial metric based on these keys. This metric is the distance between the keys. The distance is calculated as modulo 2 addition (bitwise XOR) between the two keys and interpreted as an unsigned integer.
The properties of the XOR operation imply that the zero distance can only be between the same key (the node key), and half of the entire key space with respect to the key of the node will have the maximum possible distance - with a certain convention the node with such keys can be considered “unreachable "And interaction with such nodes will be a rare event.
The process of logging into the Tox network begins with sending the " Nodes request " packet to a known bootstrap node (one or more), where the contents of the packet are the wanted public key. The node that received the Nodes request package searches for the public keys known to it (except for its own and requested) keys with the smallest distance from the wanted one and responds with the Nodes response package containing from 1 to 4 found keys and the corresponding nodes (IP / port). Iteratively repeating “Nodes request” requests to the nodes from the client-node response can find another node with a minimum distance from the desired key (in parallel, receiving information about existing intermediate nodes).
If the client-node will specify its own key as the required key, then the found nodes with the minimum distance between the keys will be its “neighbors”. If the client-node will specify the key of the requested node as the required key, but the found nodes will be neighbors of the node being polled - this is how the network statistics work.
The liveliness of each known node is checked by periodically sending the " Ping request " packet, to which the recipient must respond with the " Ping response " packet. Additionally, at some intervals, the client node sends a random (from known) node a Nodes Request packet to receive information about changes in the DHT network. Non-responding nodes are removed from the list of known ones after the timeout expires.
The need to frequently send the “Ping request” packet for a large list of known nodes leads to an unnecessary load on the network. At the same time, saving information only about nearest neighbors will lead to an increase in the search time for an unknown node. To achieve a balance, the concept of a key index is introduced in the libtoxcore library - this is the index of the first distinct bit of the key relative to the node key. For each new key, its index is calculated, and for each index, the core stores up to 8 keys, displacing the keys with the greatest distance. Additionally, the kernel stores 8 keys of the nearest neighbors with the same crowding algorithm.
In the current implementation of libtoxcore, the index is limited to 128 “baskets”, which with a certain luck allows each node to store information about 1024 nodes (until some time in the past, while the network was very, very small, 32 baskets and 190 nodes were used, respectively). With a minimum packet size of 82 bytes (“Ping request”) and polling each node every 60 seconds, we get traffic at ~ 22Kbit with the maximum index load.
It also follows from the rule for calculating the index of the basket that the smaller the distance between the two keys, the larger the index will have the key and the less likely it will be to meet such a key. In the libtoxcore library implementation, keys with an index of more than 127 either become “neighbors” or fall into the 128th basket, depending on the distance.
Thus, each node of the Tox network maintains effective connectivity not only with its neighbors, but also with nodes on the “far frontiers” while maintaining a balance between the load on the network and the search time of any other node.
Unlike a DHT node, all information about which is known or can be obtained by any client of a Tox DHT network, client applications are hidden from an outside observer - the simple knowledge of the ToxID of a contact (containing its public key) is not enough to establish the location of this contact. To connect two Tox applications, onion routing is used.
The mechanism for establishing communication between two clients can be represented as follows. Two clients (A and Z) announce their public key on the nearest (for their public key) nodes through three random intermediate DHT-nodes, each of which only knows its packet routing neighbors, but cannot read the contents of the packet.
The second client (Z), who wants to connect to the first (A), also through three random DHT nodes makes a request to establish a connection to the node closest to the desired key (A), which knows about the announcement of the first client and the route that needs to be sent request from the second client.
The first client (A), in case of accepting the request to establish a connection, performs the reverse operation to the nearest DHT node of the second client (Z).
After exchanging information about each other's location and temporary keys, clients can connect directly.
If customers do not want to share their location information even with their contacts, they can use nodes that support TCP-relay through a Tor- like proxy.
A special feature of TCP relays is that they try to use “well-known” (“well-known”) tools: 443 (HTTPS) and 3389 (RDP), which makes them difficult to track and identify.
To protect a distributed network from most known threats, you can use a simple rule of thumb - the more trustworthy nodes in a network, the more resistant the network is to possible attacks. Moreover, for some types of attacks neutralization of each trustworthy node requires a dozen or two attackers.
Based on the description above, if you use any Tox client, then you are already a network node (while your node is in no way connected to your ToxID, except that both are on the same host). If you are not yet using Tox, but want to help the project, you can install the bootstrap node on the servers and computers that you control - it does not consume much traffic or computing resources (unlike the good old days of Folding @ home ).
A detailed description of the assembly and installation process of the node can be found under the link " We are joining the tox-community or installing the node in 5 minutes ". However, I tried to simplify this process as much as possible by assembling the tox-easy-bootstrap package for most popular Linux distributions.
To install the tox-easy-bootstrap package, click on the link to the project repository, select your distribution package and follow the simple instructions on how to add a repository and install the package on your system.
The package installer will automatically get the current list of public bootstrap nodes, will create the /etc/tox-bootstrapd.conf configuration file and launch the tox-bootstrapd daemon under a separate system user. Once a week by cron, a special script will update the list of public bootstrap nodes in the configuration file; therefore, you will not need to worry about its relevance if the server is restarted - “set and forget”.
For cases when the server uses a “normally closed” firewall, you may need to enter allowing rules for incoming traffic on UDP: 33445 and TCP: 3389,33445 (TCP port: 443 is not used, because the process is started under an unprivileged user) - In order to avoid potential “sabotage” I did not automate this part:
The configuration file /etc/tox-easy-bootstrap.conf allows you to change the default settings (which are suitable for most cases) and, for example, “linking” some of your private nodes in case all public ones are unavailable - as described above, connectivity with any one node in the network to access the network of the second node.
The question of whether to publish data about your node in the list of public nodes is left to your personal discretion - from a technical point of view, a private node is no different from a public one.
The Tox protocol allows you to not only write bots , exchange messages, files or make audio-video calls, but also use it as a basic protocol for other network tasks.
For example, the tuntox and toxvpn projects use the Tox protocol to organize P2P host connections for NAT by adding to the list of Full-Mesh VPN solutions .
There are experimental toxmail projects for organizing a mail message, toxscreen for gaining access to a remote desktop, toxshare for file sharing between the trusted circle of persons.
For experiments on writing new applications, you can use wrappers (supported and not so) for other languages ​​and platforms: python , rust , go , node.js - almost unlimited space for new ideas and projects.
In this publication, I would like to tell you about the distributed nature of this messenger, the general principles of operation of the Tox DHT network, as well as how to " catch up and overtake America " by the number of nodes.
')
Introduction
Each Tox node is identified by its IP address, port number, and 256-bit public key. There are two conditional types of nodes:
- Bootstrap-node - a node that serves the network, but does not interact directly with the user.
- The Tox client is a node that, in addition to servicing the network, performs any additional work (bot, messenger). In this case, the client and node use different pairs of keys.
The public key of the node is used to encrypt the packets sent to this node. Packets are decrypted on the side of the node using a 256-bit private key. To transmit DHT packets, the UDP protocol is used.
To “enter” a Tox network, it is enough to have connectivity with any Tox node that is already on the network. Usually, a list of known bootstrap nodes on the Internet is used for this. Additionally, the libtoxcore library uses sending packets to broadcast addresses, which allows you to connect to the Tox network without having access to the Internet (provided that you have the necessary node in your network segment). And even without going online, two or more Tox nodes form an isolated Tox network, allowing local clients to interact.
DHT network self organization
Since each Tox node has a unique identifier in the form of a public key, the relationship between any two nodes can be expressed by an artificial metric based on these keys. This metric is the distance between the keys. The distance is calculated as modulo 2 addition (bitwise XOR) between the two keys and interpreted as an unsigned integer.
The properties of the XOR operation imply that the zero distance can only be between the same key (the node key), and half of the entire key space with respect to the key of the node will have the maximum possible distance - with a certain convention the node with such keys can be considered “unreachable "And interaction with such nodes will be a rare event.
The process of logging into the Tox network begins with sending the " Nodes request " packet to a known bootstrap node (one or more), where the contents of the packet are the wanted public key. The node that received the Nodes request package searches for the public keys known to it (except for its own and requested) keys with the smallest distance from the wanted one and responds with the Nodes response package containing from 1 to 4 found keys and the corresponding nodes (IP / port). Iteratively repeating “Nodes request” requests to the nodes from the client-node response can find another node with a minimum distance from the desired key (in parallel, receiving information about existing intermediate nodes).
If the client-node will specify its own key as the required key, then the found nodes with the minimum distance between the keys will be its “neighbors”. If the client-node will specify the key of the requested node as the required key, but the found nodes will be neighbors of the node being polled - this is how the network statistics work.
The liveliness of each known node is checked by periodically sending the " Ping request " packet, to which the recipient must respond with the " Ping response " packet. Additionally, at some intervals, the client node sends a random (from known) node a Nodes Request packet to receive information about changes in the DHT network. Non-responding nodes are removed from the list of known ones after the timeout expires.
The need to frequently send the “Ping request” packet for a large list of known nodes leads to an unnecessary load on the network. At the same time, saving information only about nearest neighbors will lead to an increase in the search time for an unknown node. To achieve a balance, the concept of a key index is introduced in the libtoxcore library - this is the index of the first distinct bit of the key relative to the node key. For each new key, its index is calculated, and for each index, the core stores up to 8 keys, displacing the keys with the greatest distance. Additionally, the kernel stores 8 keys of the nearest neighbors with the same crowding algorithm.
In the current implementation of libtoxcore, the index is limited to 128 “baskets”, which with a certain luck allows each node to store information about 1024 nodes (until some time in the past, while the network was very, very small, 32 baskets and 190 nodes were used, respectively). With a minimum packet size of 82 bytes (“Ping request”) and polling each node every 60 seconds, we get traffic at ~ 22Kbit with the maximum index load.
It also follows from the rule for calculating the index of the basket that the smaller the distance between the two keys, the larger the index will have the key and the less likely it will be to meet such a key. In the libtoxcore library implementation, keys with an index of more than 127 either become “neighbors” or fall into the 128th basket, depending on the distance.
Thus, each node of the Tox network maintains effective connectivity not only with its neighbors, but also with nodes on the “far frontiers” while maintaining a balance between the load on the network and the search time of any other node.
DHT network clients
Unlike a DHT node, all information about which is known or can be obtained by any client of a Tox DHT network, client applications are hidden from an outside observer - the simple knowledge of the ToxID of a contact (containing its public key) is not enough to establish the location of this contact. To connect two Tox applications, onion routing is used.
The mechanism for establishing communication between two clients can be represented as follows. Two clients (A and Z) announce their public key on the nearest (for their public key) nodes through three random intermediate DHT-nodes, each of which only knows its packet routing neighbors, but cannot read the contents of the packet.
The second client (Z), who wants to connect to the first (A), also through three random DHT nodes makes a request to establish a connection to the node closest to the desired key (A), which knows about the announcement of the first client and the route that needs to be sent request from the second client.
The first client (A), in case of accepting the request to establish a connection, performs the reverse operation to the nearest DHT node of the second client (Z).
After exchanging information about each other's location and temporary keys, clients can connect directly.
If customers do not want to share their location information even with their contacts, they can use nodes that support TCP-relay through a Tor- like proxy.
A special feature of TCP relays is that they try to use “well-known” (“well-known”) tools: 443 (HTTPS) and 3389 (RDP), which makes them difficult to track and identify.
"Catch up and overtake America"
To protect a distributed network from most known threats, you can use a simple rule of thumb - the more trustworthy nodes in a network, the more resistant the network is to possible attacks. Moreover, for some types of attacks neutralization of each trustworthy node requires a dozen or two attackers.
Based on the description above, if you use any Tox client, then you are already a network node (while your node is in no way connected to your ToxID, except that both are on the same host). If you are not yet using Tox, but want to help the project, you can install the bootstrap node on the servers and computers that you control - it does not consume much traffic or computing resources (unlike the good old days of Folding @ home ).
A detailed description of the assembly and installation process of the node can be found under the link " We are joining the tox-community or installing the node in 5 minutes ". However, I tried to simplify this process as much as possible by assembling the tox-easy-bootstrap package for most popular Linux distributions.
To install the tox-easy-bootstrap package, click on the link to the project repository, select your distribution package and follow the simple instructions on how to add a repository and install the package on your system.
The package installer will automatically get the current list of public bootstrap nodes, will create the /etc/tox-bootstrapd.conf configuration file and launch the tox-bootstrapd daemon under a separate system user. Once a week by cron, a special script will update the list of public bootstrap nodes in the configuration file; therefore, you will not need to worry about its relevance if the server is restarted - “set and forget”.
For cases when the server uses a “normally closed” firewall, you may need to enter allowing rules for incoming traffic on UDP: 33445 and TCP: 3389,33445 (TCP port: 443 is not used, because the process is started under an unprivileged user) - In order to avoid potential “sabotage” I did not automate this part:
-A INPUT -p tcp --dport 3389 -j ACCEPT
-A INPUT -p tcp --dport 33445 -j ACCEPT
-A INPUT -p udp --dport 33445 -j ACCEPT
The configuration file /etc/tox-easy-bootstrap.conf allows you to change the default settings (which are suitable for most cases) and, for example, “linking” some of your private nodes in case all public ones are unavailable - as described above, connectivity with any one node in the network to access the network of the second node.
The question of whether to publish data about your node in the list of public nodes is left to your personal discretion - from a technical point of view, a private node is no different from a public one.
Conclusion
The Tox protocol allows you to not only write bots , exchange messages, files or make audio-video calls, but also use it as a basic protocol for other network tasks.
For example, the tuntox and toxvpn projects use the Tox protocol to organize P2P host connections for NAT by adding to the list of Full-Mesh VPN solutions .
There are experimental toxmail projects for organizing a mail message, toxscreen for gaining access to a remote desktop, toxshare for file sharing between the trusted circle of persons.
For experiments on writing new applications, you can use wrappers (supported and not so) for other languages ​​and platforms: python , rust , go , node.js - almost unlimited space for new ideas and projects.
Links
- toxstats.com , tox.viktorstanchev.com , stats.letstox.com - Tox network statistics.
- Draft protocol specification tox - toktok.imtqy.com .
- The project for building binary versions of clients and libraries is tox.pkg on github .
- tox.chat is the official site of the Tox project.
Source: https://habr.com/ru/post/280208/
All Articles