Writing a bot for Tox messenger
Against the background of the general enthusiasm for creating bots for Telegram, I would like to tell you about the API of the not very well-known messenger Tox and show with the example of a simple echo-bot how you can easily and quickly create your own.
The Tox kernel ( toxcore on github ) consists of three parts:
')
The contact unit is ToxID (aka “address” in terms of API) - a hexadecimal string 76 characters long that encodes:
The general cycle of working with Tox API can be presented sequentially in the form:
Since The main way to get knowledge of how Tox API works is to read the source code (written in C), to simplify the following, I will use the python wrapper ( pytoxcore on github ). For those who do not wish to engage in self-build libraries from source, there are also links to ready-made binary packages for common distributions.
When using python wrappers, you can get library help in the following way:
Below we will focus on each step of working with the API in a little more detail.
To initialize the kernel, the Tox_Options structure is used as a parameter. In python it can be a dictionary with the same fields. The default values ​​can be obtained by calling the tox_options_default method:
Here:
In most cases, all parameters except the savedata_data can be left as default:
When you first start the kernel will generate public and private keys, which, if necessary, you can save:
Since the data is always kept in memory, to protect against accidental failures, I recommend periodically saving the data to a temporary file and (if successful, and atomically) replacing the main data file with it.
The current values ​​of the address to transfer to users, public and private keys and the value of "nospam" can be obtained by calling:
After initializing the kernel, the bot can at any time set a nickname and signature by calling tox_self_set_name and tox_self_set_status_message . The name must not exceed the TOX_MAX_NAME_LENGTH value, the length of the signature must not exceed the TOX_MAX_STATUS_MESSAGE_LENGTH value (sizes in bytes). Installation of the avatar will be discussed below, because technically is sending a file to a contact.
In the python wrapper, the connection to the supported callback functions is performed automatically. The handlers themselves can be methods of the successor to ToxCore and have the suffix * _cb :
The specific handlers and their arguments will be discussed below.
DHT-node is determined by IP address, port number and public key. The initial list of DHT nodes can be found on the project wiki .
According to the Tox Client Guidelines , the client every 5 seconds should try to connect to at least four random nodes until the kernel reports a successful connection (see tox_self_connection_status_cb ). In the case of loading from the state file, the client should not attempt to connect within 10 seconds after the first call tox_iterate and, in the case of no connection, repeat the aggressive connection strategy above.
For the bot, who plans to be always in touch, these recommendations look a bit over-complicated. You can try to reduce the required number of DHT nodes for connection by raising your own local DHT node. An additional plus of having a local node, in addition to constant communication with it, is the “gain” of the Tox network itself.
To raise the local node, you need to install and configure the tox-bootstrapd daemon. It can be assembled with the toxcore library, as well as obtained in binary form from the developers repository .
The daemon configuration is set in the /etc/tox-bootstrapd.conf file and is well documented . Additional information on the daemon launch can be obtained in the corresponding README , and for the deb distributions of the tox.pkg project, the installation of the tox-bootstrapd package is self- sufficient. The public key of the local DHT node can be found in the system log after starting the daemon.
Thus, a simplified version of the connection with the DHT node and the duty cycle can be represented as:
In some cases, a client can work without calling tox_bootstrap at all — for this, it is necessary that another client or a DHT node be running within the same broadcast domain of the network. This feature makes it possible to communicate within the local network without the need to access the Internet and communicate with the outside world, if at least one client has access to the network and is a relay.
As it was written earlier, to connect an event handler in python, it is enough to add the required method with the necessary arguments to the inheriting class, and as an example, the tox_self_connection_status_cb connection state handler, which is called when the bot is connected and disconnected from the DHT network, was given.
In order for the bot to interact with other members of the network, they must be added to the bot's contact list. In API terms, a contact is called “friend” and is denoted by an integer (" friend_number "), unique in the lifetime of a ToxCore instance.
When another client makes a request to add a bot to the contact list, the tox_friend_request_cb (public_key, message) handler is called on the side of the bot, where:
Inside the handler, you can either add a person as a friend by calling tox_friend_add_norequest , or just ignore the request. All further event handlers will use friend_number as a friend identifier, which can be obtained from the public key by calling tox_friend_by_public_key .
After adding a contact as a friend on the bot side, the following events may occur:
In the case of the echo-bot when you receive a message from a friend, you just need to send it back:
A message is sent to a friend by calling the tox_friend_send_message method, which returns the message_id message id — a monotonically increasing number that is unique to each friend. The method takes as parameters the friend's identifier, message type, and the message text itself. The following restrictions apply to the message text:
If the processing of a message from a friend takes some time, the bot's behavior can be varied by sending randomly “message set” events ( tox_self_set_typing ).
By the value of a friend_number of a friend, at any moment of work you can get the following information about him:
Additional operations with friends:
Here everything seems simple and intuitive. It will be a little more difficult.
When a friend sends a file to the bot, a tox_file_recv_cb event (friend_number, file_number, kind, file_size, filename) occurs , where:
Here you should pay special attention to the filename parameter. Although the specification requires all data to be transferred to UTF-8 and the file name should not contain parts of the path, in real life anything can flow up to unreadable binary data containing newline characters and zeros.
When this event occurs, the following bot action should be the call to the controlling method tox_file_control (friend_number, file_number, control) , where:
For echo-bot reception of files is not required, so he can always cancel the operation:
In the case of the transfer of control via TOX_FILE_CONTROL_RESUME , the tox_file_recv_chunk_cb event starts to be triggered (friend_number, file_number, position, data) , where:
Here you should pay attention to the fact that position does not have to monotonously increase - in general, chunks can come in any sequence and of any length.
To start the file transfer procedure, you need to call the tox_file_send method (friend_number, kind, file_size, file_id, filename) , where:
Here the special parameter is file_id . In the case of automatic generation, it can later be obtained by calling tox_file_get_file_id , however, when sending an avatar, it is recommended to set its value to the result of a tox_hash call from the avatar file data, which allows the receiving party to cancel the transfer of previously loaded avatars to save traffic.
It should also be noted that the transfer of files is only possible to friends who are connected to the network. Disconnecting a friend from the network stops file transfer.
After calling tox_file_send, the kernel waits for the decision from the receiving side. The solution is processed by the tox_file_recv_control_cb event (friend_number, file_number, control) , where:
Processing this event allows you to free up resources in case of failure of the client to accept the file.
Echo bot only needs to send an avatar. Avatar transfer is recommended to be done every time a friend appears on the network. If a friend’s tox client has previously uploaded an avatar with the given file_id , then he can cancel the retransmission of the avatar.
In addition to receiving and transmitting messages and files, the kernel provides the possibility of packet transmission (lossy or lossless). To do this, use the tox_friend_send_lossy_packet and tox_friend_send_lossless_packet methods , as well as the tox_friend_lossy_packet_cb and tox_friend_lossless_packet_cb events .
Introduction
The Tox kernel ( toxcore on github ) consists of three parts:
')
- ToxCore - in fact, the kernel itself, which allows you to manage contacts, messages, files, avatars, statuses and group chats.
- ToxAV - voice and video call subsystem.
- ToxDNS is a subsystem for obtaining ToxID from a human-readable address (approximate equivalent of email or JabberID) via a DNS query.
The contact unit is ToxID (aka “address” in terms of API) - a hexadecimal string 76 characters long that encodes:
- Public key (32 bytes or 64 characters).
- Anti-spam protection "nospam" (4 bytes or 8 characters) is a random data set, changing which allows you to continue to support previously authorized contacts, but ignore requests from new ones.
- Checksum (2 bytes or 4 characters) - XOR operation on the public key and the value of "nospam", serves to quickly check the correctness of ToxID.
The general cycle of working with Tox API can be presented sequentially in the form:
- Initialization of the kernel ( tox_new ) - installation of protocols (IPv4 / IPv6, UDP / TCP), proxy settings (HTTP / SOCKS5), ranges of ports used and (if available) loading of the previously saved state with the list of contacts.
- Setting callback functions for event handling ( tox_callback_ * ) - event handlers are called from the main loop (4) and they usually contain the main logic of the application.
- Connect to one or more DHT nodes ( tox_bootstrap ).
- The main work cycle ( tox_iterate ) and event handling.
- Pause tox_iteration_interval and return to the previous step.
Since The main way to get knowledge of how Tox API works is to read the source code (written in C), to simplify the following, I will use the python wrapper ( pytoxcore on github ). For those who do not wish to engage in self-build libraries from source, there are also links to ready-made binary packages for common distributions.
When using python wrappers, you can get library help in the following way:
$ python >>> from pytoxcore import ToxCore >>> help(ToxCore) class ToxCore(object) | ToxCore object ... | tox_add_tcp_relay(...) | tox_add_tcp_relay(address, port, public_key) | Adds additional host:port pair as TCP relay. | This function can be used to initiate TCP connections to different ports on the same bootstrap node, or to add TCP relays without using them as bootstrap nodes. | | tox_bootstrap(...) | tox_bootstrap(address, port, public_key) | Sends a "get nodes" request to the given bootstrap node with IP, port, and public key to setup connections. | This function will attempt to connect to the node using UDP. You must use this function even if Tox_Options.udp_enabled was set to false. ... Below we will focus on each step of working with the API in a little more detail.
Kernel initialization
To initialize the kernel, the Tox_Options structure is used as a parameter. In python it can be a dictionary with the same fields. The default values ​​can be obtained by calling the tox_options_default method:
$ python >>> from pytoxcore import ToxCore >>> ToxCore.tox_options_default() {'start_port': 0L, 'proxy_host': None, 'tcp_port': 0L, 'end_port': 0L, 'udp_enabled': True, 'savedata_data': None, 'proxy_port': 0L, 'ipv6_enabled': True, 'proxy_type': 0L} Here:
- ipv6_enabled - True or False depending on whether you are the proud owner of IPv6.
- udp_enabled - except when working through a proxy, it is recommended to set it to True, since UDP is the native protocol for Tox.
- proxy_type is a proxy type, can be:
- TOX_PROXY_TYPE_NONE - do not use proxy.
- TOX_PROXY_TYPE_HTTP - HTTP proxy.
- TOX_PROXY_TYPE_SOCKS5 - SOCKS5 proxy (for example, Tor).
- proxy_host - proxy server host or IP address.
- proxy_port - proxy port (ignored for TOX_PROXY_TYPE_NONE).
- start_port - starting port from the range of allowed ports.
- end_port - the end port from the range of allowed ports. If the start and end ports are 0, then the range [33445, 33545] is used. If only one of the ports is zero, then a single non-zero port is used. In the case of start_port> end_port, they will be swapped.
- tcp_port - port for raising the TCP server (relay), if set to 0, the server will be disabled. TCP relay allows other users to use your instance as an intermediate node (super-node concept).
- savedata_data - data to load or None in case of their absence.
In most cases, all parameters except the savedata_data can be left as default:
#!/usr/bin/env python # -*- coding: utf-8 -*- import os from pytoxcore import ToxCore class EchoBot(ToxCore): def __init__(self): tox_options = ToxCore.tox_options_default() if os.path.isfile("tox.save"): with open("tox.save", "rb") as f: tox_options["savedata_data"] = f.read() super(EchoBot, self).__init__(tox_options) When you first start the kernel will generate public and private keys, which, if necessary, you can save:
class EchoBot(ToxCore): ... def save_data(self): with open("tox.save", "wb") as f: f.write(self.tox_get_savedata()) Since the data is always kept in memory, to protect against accidental failures, I recommend periodically saving the data to a temporary file and (if successful, and atomically) replacing the main data file with it.
The current values ​​of the address to transfer to users, public and private keys and the value of "nospam" can be obtained by calling:
- tox_self_get_address - current address (ToxID).
- tox_self_get_public_key - public key.
- tox_self_get_secret_key - private key.
- tox_self_get_nospam / tox_self_set_nospam — getting and setting the nospam value.
$ python >>> from pytoxcore import ToxCore >>> core = ToxCore(ToxCore.tox_options_default()) >>> core.tox_self_get_address() '366EA3B25BA31E3ADC4C476098A8686E4EAE87B04E4E4A3A3A0B865CBB9725704189FEDAEB26' >>> core.tox_self_get_public_key() '366EA3B25BA31E3ADC4C476098A8686E4EAE87B04E4E4A3A3A0B865CBB972570' >>> core.tox_self_get_secret_key() '86003764B4C99395E164024A17DCD0ECB80363C5976FF43ECE11637FA0B683F9' >>> core.tox_self_get_nospam() '4189FEDA' After initializing the kernel, the bot can at any time set a nickname and signature by calling tox_self_set_name and tox_self_set_status_message . The name must not exceed the TOX_MAX_NAME_LENGTH value, the length of the signature must not exceed the TOX_MAX_STATUS_MESSAGE_LENGTH value (sizes in bytes). Installation of the avatar will be discussed below, because technically is sending a file to a contact.
Setting callback functions
In the python wrapper, the connection to the supported callback functions is performed automatically. The handlers themselves can be methods of the successor to ToxCore and have the suffix * _cb :
class EchoBot(ToxCore): ... def tox_self_connection_status_cb(self, connection_status): if connection_status == ToxCore.TOX_CONNECTION_NONE: print("Disconnected from DHT") elif connection_status == ToxCore.TOX_CONNECTION_TCP: print("Connected to DHT via TCP") elif connection_status == ToxCore.TOX_CONNECTION_UDP: print("Connected to DHT via UDP") else: raise NotImplementedError("Unknown connection_status: {0}".format(connection_status)) The specific handlers and their arguments will be discussed below.
Connect to DHT node
DHT-node is determined by IP address, port number and public key. The initial list of DHT nodes can be found on the project wiki .
According to the Tox Client Guidelines , the client every 5 seconds should try to connect to at least four random nodes until the kernel reports a successful connection (see tox_self_connection_status_cb ). In the case of loading from the state file, the client should not attempt to connect within 10 seconds after the first call tox_iterate and, in the case of no connection, repeat the aggressive connection strategy above.
For the bot, who plans to be always in touch, these recommendations look a bit over-complicated. You can try to reduce the required number of DHT nodes for connection by raising your own local DHT node. An additional plus of having a local node, in addition to constant communication with it, is the “gain” of the Tox network itself.
To raise the local node, you need to install and configure the tox-bootstrapd daemon. It can be assembled with the toxcore library, as well as obtained in binary form from the developers repository .
The daemon configuration is set in the /etc/tox-bootstrapd.conf file and is well documented . Additional information on the daemon launch can be obtained in the corresponding README , and for the deb distributions of the tox.pkg project, the installation of the tox-bootstrapd package is self- sufficient. The public key of the local DHT node can be found in the system log after starting the daemon.
Thus, a simplified version of the connection with the DHT node and the duty cycle can be represented as:
class EchoBot(ToxCore): ... def run(self): checked = False self.tox_bootstrap("127.0.0.1", 33445, "366EA...72570") while True: status = self.tox_self_get_connection_status() if not checked and status != ToxCore.TOX_CONNECTION_NONE: checked = True if checked and status == ToxCore.TOX_CONNECTION_NONE: self.tox_bootstrap("127.0.0.1", 33445, "366EA...72570") checked = False self.tox_iterate() interval = self.tox_iteration_interval() time.sleep(interval / 1000.0) In some cases, a client can work without calling tox_bootstrap at all — for this, it is necessary that another client or a DHT node be running within the same broadcast domain of the network. This feature makes it possible to communicate within the local network without the need to access the Internet and communicate with the outside world, if at least one client has access to the network and is a relay.
Event handling
As it was written earlier, to connect an event handler in python, it is enough to add the required method with the necessary arguments to the inheriting class, and as an example, the tox_self_connection_status_cb connection state handler, which is called when the bot is connected and disconnected from the DHT network, was given.
Work with contacts
In order for the bot to interact with other members of the network, they must be added to the bot's contact list. In API terms, a contact is called “friend” and is denoted by an integer (" friend_number "), unique in the lifetime of a ToxCore instance.
When another client makes a request to add a bot to the contact list, the tox_friend_request_cb (public_key, message) handler is called on the side of the bot, where:
- public_key - friend's public key.
- message - a message from a friend like “Hi, this is abbat! Add me as a friend?
Inside the handler, you can either add a person as a friend by calling tox_friend_add_norequest , or just ignore the request. All further event handlers will use friend_number as a friend identifier, which can be obtained from the public key by calling tox_friend_by_public_key .
After adding a contact as a friend on the bot side, the following events may occur:
- tox_friend_connection_status_cb (friend_number, connection_status) - change the friend’s connection status, where connection_status can be:
- TOX_CONNECTION_NONE is a friend offline.
- TOX_CONNECTION_TCP is a friend online and connected via TCP.
- TOX_CONNECTION_UDP is a friend online and connected via UDP.
- tox_friend_name_cb (friend_number, name) - change friend's nickname.
- tox_friend_status_message_cb (friend_number, message) - change the signature.
- tox_friend_status_cb (friend_number, status) - a change in state, where status can be:
- TOX_USER_STATUS_NONE - available ("online").
- TOX_USER_STATUS_AWAY - departed.
- TOX_USER_STATUS_BUSY - busy.
- tox_friend_message_cb (friend_number, message) - message from a friend.
- tox_friend_read_receipt_cb (friend_number, message_id) - receipt of receipt of another message sent by the tox_friend_send_message call (see below).
In the case of the echo-bot when you receive a message from a friend, you just need to send it back:
class EchoBot(ToxCore): ... def tox_friend_request_cb(self, public_key, message): self.tox_friend_add_norequest(public_key) def tox_friend_message_cb(self, friend_number, message): message_id = self.tox_friend_send_message(friend_number, ToxCore.TOX_MESSAGE_TYPE_NORMAL, message) A message is sent to a friend by calling the tox_friend_send_message method, which returns the message_id message id — a monotonically increasing number that is unique to each friend. The method takes as parameters the friend's identifier, message type, and the message text itself. The following restrictions apply to the message text:
- The message can not be empty.
- The message can not be more than TOX_MAX_MESSAGE_LENGTH (byte), long messages must be divided into parts.
If the processing of a message from a friend takes some time, the bot's behavior can be varied by sending randomly “message set” events ( tox_self_set_typing ).
By the value of a friend_number of a friend, at any moment of work you can get the following information about him:
- tox_friend_get_connection_status - friend’s current network status (last value from tox_friend_connection_status_cb ).
- tox_friend_get_name - friend's current nickname (last value from tox_friend_name_cb ).
- tox_friend_get_status_message - friend’s current signature (last tox_friend_status_message_cb ).
- tox_friend_get_status - friend’s current status (last value from tox_friend_status_cb ).
- tox_friend_get_last_online - the date of the last appearance in online (unixtime).
Additional operations with friends:
- tox_self_get_friend_list_size - get the number of friends.
- tox_self_get_friend_list — Get a friend_number of friends list.
- tox_friend_delete - delete a friend from the contact list.
Here everything seems simple and intuitive. It will be a little more difficult.
Work with files
Receiving files
When a friend sends a file to the bot, a tox_file_recv_cb event (friend_number, file_number, kind, file_size, filename) occurs , where:
- friend_number — friend’s number (see “Working with Contacts”).
- file_number is the file number, a unique number within the current list of received and transmitted files from this friend.
- kind - file type:
- TOX_FILE_KIND_DATA - the transmitted file is a simple file.
- TOX_FILE_KIND_AVATAR - the transmitted file is a friend's avatar.
- file_size - file size. For TOX_FILE_KIND_AVATAR, a file size of 0 means that the friend has no avatar installed. File size equal to UINT64_MAX indicates an unknown file size (streaming).
- filename is the name of the file.
Here you should pay special attention to the filename parameter. Although the specification requires all data to be transferred to UTF-8 and the file name should not contain parts of the path, in real life anything can flow up to unreadable binary data containing newline characters and zeros.
When this event occurs, the following bot action should be the call to the controlling method tox_file_control (friend_number, file_number, control) , where:
- friend_number - friend number.
- file_number is the file number.
- control - file control command:
- TOX_FILE_CONTROL_CANCEL - cancel receiving the file.
- TOX_FILE_CONTROL_PAUSE - put the file transfer on pause (not supported by all clients).
- TOX_FILE_CONTROL_RESUME - continue the file transfer.
For echo-bot reception of files is not required, so he can always cancel the operation:
class EchoBot(ToxCore): ... def tox_file_recv_cb(self, friend_number, file_number, kind, file_size, filename): self.tox_file_control(friend_number, file_number, ToxCore.TOX_FILE_CONTROL_CANCEL) In the case of the transfer of control via TOX_FILE_CONTROL_RESUME , the tox_file_recv_chunk_cb event starts to be triggered (friend_number, file_number, position, data) , where:
- friend_number - friend number.
- file_number is the file number.
- position - the current position in the file.
- data - data chunk or None for the end of the transfer.
Here you should pay attention to the fact that position does not have to monotonously increase - in general, chunks can come in any sequence and of any length.
File transfer
To start the file transfer procedure, you need to call the tox_file_send method (friend_number, kind, file_size, file_id, filename) , where:
- friend_number - friend number.
- kind is the value of TOX_FILE_KIND_DATA or TOX_FILE_KIND_AVATAR .
- file_size is the file size (special values ​​0 and UINT64_MAX are discussed above).
- file_id is a unique identifier for a file of length TOX_FILE_ID_LENGTH , which allows you to continue the transfer after a kernel restart or None for automatic generation.
- filename is the name of the file.
Here the special parameter is file_id . In the case of automatic generation, it can later be obtained by calling tox_file_get_file_id , however, when sending an avatar, it is recommended to set its value to the result of a tox_hash call from the avatar file data, which allows the receiving party to cancel the transfer of previously loaded avatars to save traffic.
It should also be noted that the transfer of files is only possible to friends who are connected to the network. Disconnecting a friend from the network stops file transfer.
After calling tox_file_send, the kernel waits for the decision from the receiving side. The solution is processed by the tox_file_recv_control_cb event (friend_number, file_number, control) , where:
- friend_number - friend number.
- file_number is the file number.
- control - file control command ( TOX_FILE_CONTROL_CANCEL , TOX_FILE_CONTROL_PAUSE or TOX_FILE_CONTROL_RESUME discussed earlier).
Processing this event allows you to free up resources in case of failure of the client to accept the file.
Echo bot only needs to send an avatar. Avatar transfer is recommended to be done every time a friend appears on the network. If a friend’s tox client has previously uploaded an avatar with the given file_id , then he can cancel the retransmission of the avatar.
class EchoBot(ToxCore): ... def __init__(self): ... self.avatar_name = "avatar.png" self.avatar_size = os.path.getsize(avatar_name) self.avatar_fd = open(avatar_name, "rb") data = self.avatar_fd.read() self.avatar_id = ToxCore.tox_hash(data) def tox_friend_connection_status_cb(self, friend_number, connection_status): if connection_status != ToxCore.TOX_CONNECTION_NONE: send_avatar(friend_number) def send_avatar(self, friend_number): file_number = self.tox_file_send(friend_number, ToxCore.TOX_FILE_KIND_AVATAR, self.avatar_size, self.avatar_id, self.avatar_name) def tox_file_recv_control_cb(self, friend_number, file_number, control): pass def tox_file_chunk_request_cb(self, friend_number, file_number, position, length): if length == 0: return self.avatar_fd.seek(position, 0) data = self.avatar_fd.read(length) self.tox_file_send_chunk(friend_number, file_number, position, data) In addition to receiving and transmitting messages and files, the kernel provides the possibility of packet transmission (lossy or lossless). To do this, use the tox_friend_send_lossy_packet and tox_friend_send_lossless_packet methods , as well as the tox_friend_lossy_packet_cb and tox_friend_lossless_packet_cb events .
Links
- tox.chat is the current official site of the Tox project (tox.im instead).
- Kernel toxcore project on github .
- The pytoxcore python wrapper project on github .
- A project to build binary versions of clients and tox.pkg libraries on github .
- The project of the "official" Python wrapper PyTox on github (at the time of this writing, the library was not compiled due to the use of the "old" api).
Source: https://habr.com/ru/post/264179/
All Articles