About µTP in new versions of µTorrent: what is it, how, why?
Traditionally, most P2P applications used TCP to exchange data. About the fact that µTorrent begins to use a new protocol based on UDP, it was already mentioned at Habré ( one , two ). In this post, the new µTP protocol is described in more detail, including its tuning and the ability to turn off. Details are described in such a way that it is understandable to people far from network protocols.
Update : Official documentation on the protocol: www.bittorrent.org/beps/bep_0029.html
A couple of words about TCP and UDP. The first is decrypted as Transmission Control Protocol, or flow control protocol. It is convenient in that it gives the program using it a guarantee that the data will reach the addressee in its entirety and in the order in which they were sent. Its use requires pre-installing the connection and closing it at the end. This is the creation of a "pipe" through which data will be exchanged. UDP does not provide any guarantees that the data will reach, or that will reach in the correct order. It only allows you to send a small block of data (datagram) from one address to another. All work on verifying delivery, and if necessary - re-sending, falls on the program itself.
')
Since the torrent client is already doing this, this is not a big problem. The fact is that the data sent via the TCP pipe is broken into pieces (“packets”) in the process, each of which is sent independently. In this case, one packet can go one route, another - another, the last piece can come first, the first one can get lost on the way. Therefore, each participant of the "pipe" (from the operating system to routers) is forced to keep a buffer in which it collects individual packets, checking integrity and order, and requiring re-sending if part of the packets have not reached.
At the same time, if the sender is sitting on a wide channel, and the receiver is on a modem, the first one immediately sends a large block of data that can quickly reach the provider of the second one and slowly seep into the modem. At this time, the first one, without receiving confirmation of receipt, will forward some of the pieces again. Once again, and again, as a result, the provider's backbone is crammed with this unnecessary re-sending. One of the main goals of uTP is to eliminate this extra burden on providers from P2P traffic.
Latent uTP appeared in µTorrent version 1.8, but was able to accept only incoming uTP connections, it wasn’t able to initiate them. For the first time, the alpha version 1.9 learned this, then it became possible to include it in the new versions 1.8 with the bt.transp_disposition key. Its value from version to version has changed, but now it has settled on the following bit flags:
1 - allow initiating outgoing TCP connections,
2 - allow initiating outgoing uTP connections,
4 - allow incoming TCP connections,
8 - allow incoming uTP connections
Thus, 13 (1 + 4 + 8), the default value in recent versions of 1.8, means that you can accept all types of connections, but you can only install TCP yourself. 15 (the default value is 2.0) allows all types of both outgoing and incoming connections. To ban uTP in general (if it causes any problems) you need to put 5 (1 + 4). Is it worth it to put 15 in 1.8 - a moot point, they write on the official forum that uTP support in version 2.0 is much better, so the speed of 1.8 may be worse than via TCP.
Version 2.0 also adds support for UDP trackers. For a tracker in general, TCP is very redundant and requires a lot of extra resources for setting up / closing connections, so for open UDP trackers it’s good. For example, the last time anirena tracker is very buggy over TCP and far from the first time it responds, DHT is often prohibited there, and the UDP tracker works fine. But for closed trackers, it is not suitable, because It does not allow to send a passkey to identify the pumper in this way.
Even in 2.0, a method of bypassing some NATs ( STUN ) will appear, which will help connect more NAT sufferers, although it will not work in all cases.
Personally, of all the innovations 2.0, I was most pleased with TCP Rate Control. It allows you to adjust the speed and TCP connections so that they do not interfere with other applications and minimize the extra forwarding of packets, which is written above. Previously, this was achieved by installing a cFos driver, in which you could set low priority to a torrent, high - to the browser, and with version 2.0 this is no longer required. This is controlled by the bt.tcp_rate_control option, if it is more important for you that the torrent does not interfere with other Internet applications - it should be turned on, if the maximum speed of the torrent is important - it makes sense to turn it off, in the official forum they write that this sometimes increases.
I note that in uTP this is always done, even in version 1.8, and the speed of TCP traffic is adjusted to load the channel only in 2.0. UTP constantly measures the response time from peers with which data is exchanged. As soon as this “ping” starts to increase, long before the start of packet loss, µTorrent slows down.
Due to this, as long as the channel is free - it is used to the full. as soon as, for example, another application (browser) starts loading the channel, the response time in µTorrent begins to increase, and it automatically releases the channel for the browser. As soon as the ping is back (the channel is free again), µTorrent increases the channel load. At the same time, the extra packet forwarding is much less than if it happened at the TCP level
However, there is an interesting point here: with it or via uTP, the outgoing channel can be clogged up by 95%, and without it only with TCP, by 100%, but this difference of 5% can turn out to be the very excessive re-sending of the same packets that the channel is scoring, but it does not bring any real benefit, so imho it’s not always when the channel is a bit underused, which means that the new version is worse.
Still in the discussions, the key net.calc_overhead often flashes. If it is enabled, then when adjusting the speed, service traffic is also taken into account - block requests, confirmations, notifications which units have, etc. If disabled, only the data blocks themselves are counted. Therefore, before it was advised to limit the outgoing channel in the torrent to 80-90% of the real one, otherwise it was completely blocked by the sent blocks, and notifications about receiving blocks and new requests did not have time to go through, therefore the download speed seriously suffered. Again, on the wide channels, some people write that when you turn off this option, the speed is slightly higher, but maybe it's a beta version bug and everything will be fine in the release.
There is still a moment in which I am not 100% sure, but it seems that there is more official traffic in uTP. For in each small datagram it is necessary to transmit what kind of block it is, from which torrent, and over TCP you can send large blocks without signing each slice. However, there will be TCP service traffic itself, so it’s not a fact that it is very “more economical”.
One can not but mention the phenomenon of "shaping" or "cutting" P2P traffic by some providers. For Russia, this (for now?) Is not relevant, but on open trackers, where there are many peers from around the world, the uTP speed is much higher.
On the other hand, not all network hardware — modems, marchers — and not all software was calculated on that amount of UDP traffic, which is why some users have bugs with it. For example, with a speed - when periodically it suddenly drops to zero, then recovers again, or floats from zero to maximum. Apparently somewhere in the network environment, a certain UDP buffer is full, hell knows. Again, on the official forum, you can search for compatibility with different software and hardware in case of problems.
Other "advanced" options that you can pay attention to:
bt.connect_speed - how many maximum new connections can be established per second (it should be increased, I have 80),
net.max_halfopen - much has been written about it, and it’s worth changing along with the tcpip.sys patch, although with the uTP protocol this is no longer important.
net.utp_target_delay is a certain target “ping” when adjusting connections, in some cases when it increases somewhere up to 400-500, the speed becomes better.
peer.disconnect_inactive_interval - after how many seconds the connection to the peer is closed, with which there is no data exchange, it is more important for open trackers with more people and “bad” peers, or in case of network glitches - in order to quickly determine the disconnection and reset it. in some cases it makes sense to lower it to 90-120.
Version 2.0, although beta is quite stable, can be downloaded here: forum.utorrent.com/viewtopic.php?id=60602
According to personal feelings, in the old alphas 1.9, when downloading from open trackers, if the incoming channel was loaded at about 1/3 before, from uTP it began to load at about 2/3. When downloading from closed trackers before the channel was loaded so that the browser began to slow down, and in 2.0 everything flies.
Update : Official documentation on the protocol: www.bittorrent.org/beps/bep_0029.html
A couple of words about TCP and UDP. The first is decrypted as Transmission Control Protocol, or flow control protocol. It is convenient in that it gives the program using it a guarantee that the data will reach the addressee in its entirety and in the order in which they were sent. Its use requires pre-installing the connection and closing it at the end. This is the creation of a "pipe" through which data will be exchanged. UDP does not provide any guarantees that the data will reach, or that will reach in the correct order. It only allows you to send a small block of data (datagram) from one address to another. All work on verifying delivery, and if necessary - re-sending, falls on the program itself.
')
Since the torrent client is already doing this, this is not a big problem. The fact is that the data sent via the TCP pipe is broken into pieces (“packets”) in the process, each of which is sent independently. In this case, one packet can go one route, another - another, the last piece can come first, the first one can get lost on the way. Therefore, each participant of the "pipe" (from the operating system to routers) is forced to keep a buffer in which it collects individual packets, checking integrity and order, and requiring re-sending if part of the packets have not reached.
At the same time, if the sender is sitting on a wide channel, and the receiver is on a modem, the first one immediately sends a large block of data that can quickly reach the provider of the second one and slowly seep into the modem. At this time, the first one, without receiving confirmation of receipt, will forward some of the pieces again. Once again, and again, as a result, the provider's backbone is crammed with this unnecessary re-sending. One of the main goals of uTP is to eliminate this extra burden on providers from P2P traffic.
Latent uTP appeared in µTorrent version 1.8, but was able to accept only incoming uTP connections, it wasn’t able to initiate them. For the first time, the alpha version 1.9 learned this, then it became possible to include it in the new versions 1.8 with the bt.transp_disposition key. Its value from version to version has changed, but now it has settled on the following bit flags:
1 - allow initiating outgoing TCP connections,
2 - allow initiating outgoing uTP connections,
4 - allow incoming TCP connections,
8 - allow incoming uTP connections
Thus, 13 (1 + 4 + 8), the default value in recent versions of 1.8, means that you can accept all types of connections, but you can only install TCP yourself. 15 (the default value is 2.0) allows all types of both outgoing and incoming connections. To ban uTP in general (if it causes any problems) you need to put 5 (1 + 4). Is it worth it to put 15 in 1.8 - a moot point, they write on the official forum that uTP support in version 2.0 is much better, so the speed of 1.8 may be worse than via TCP.
Version 2.0 also adds support for UDP trackers. For a tracker in general, TCP is very redundant and requires a lot of extra resources for setting up / closing connections, so for open UDP trackers it’s good. For example, the last time anirena tracker is very buggy over TCP and far from the first time it responds, DHT is often prohibited there, and the UDP tracker works fine. But for closed trackers, it is not suitable, because It does not allow to send a passkey to identify the pumper in this way.
Even in 2.0, a method of bypassing some NATs ( STUN ) will appear, which will help connect more NAT sufferers, although it will not work in all cases.
Personally, of all the innovations 2.0, I was most pleased with TCP Rate Control. It allows you to adjust the speed and TCP connections so that they do not interfere with other applications and minimize the extra forwarding of packets, which is written above. Previously, this was achieved by installing a cFos driver, in which you could set low priority to a torrent, high - to the browser, and with version 2.0 this is no longer required. This is controlled by the bt.tcp_rate_control option, if it is more important for you that the torrent does not interfere with other Internet applications - it should be turned on, if the maximum speed of the torrent is important - it makes sense to turn it off, in the official forum they write that this sometimes increases.
I note that in uTP this is always done, even in version 1.8, and the speed of TCP traffic is adjusted to load the channel only in 2.0. UTP constantly measures the response time from peers with which data is exchanged. As soon as this “ping” starts to increase, long before the start of packet loss, µTorrent slows down.
Due to this, as long as the channel is free - it is used to the full. as soon as, for example, another application (browser) starts loading the channel, the response time in µTorrent begins to increase, and it automatically releases the channel for the browser. As soon as the ping is back (the channel is free again), µTorrent increases the channel load. At the same time, the extra packet forwarding is much less than if it happened at the TCP level
However, there is an interesting point here: with it or via uTP, the outgoing channel can be clogged up by 95%, and without it only with TCP, by 100%, but this difference of 5% can turn out to be the very excessive re-sending of the same packets that the channel is scoring, but it does not bring any real benefit, so imho it’s not always when the channel is a bit underused, which means that the new version is worse.
Still in the discussions, the key net.calc_overhead often flashes. If it is enabled, then when adjusting the speed, service traffic is also taken into account - block requests, confirmations, notifications which units have, etc. If disabled, only the data blocks themselves are counted. Therefore, before it was advised to limit the outgoing channel in the torrent to 80-90% of the real one, otherwise it was completely blocked by the sent blocks, and notifications about receiving blocks and new requests did not have time to go through, therefore the download speed seriously suffered. Again, on the wide channels, some people write that when you turn off this option, the speed is slightly higher, but maybe it's a beta version bug and everything will be fine in the release.
There is still a moment in which I am not 100% sure, but it seems that there is more official traffic in uTP. For in each small datagram it is necessary to transmit what kind of block it is, from which torrent, and over TCP you can send large blocks without signing each slice. However, there will be TCP service traffic itself, so it’s not a fact that it is very “more economical”.
One can not but mention the phenomenon of "shaping" or "cutting" P2P traffic by some providers. For Russia, this (for now?) Is not relevant, but on open trackers, where there are many peers from around the world, the uTP speed is much higher.
On the other hand, not all network hardware — modems, marchers — and not all software was calculated on that amount of UDP traffic, which is why some users have bugs with it. For example, with a speed - when periodically it suddenly drops to zero, then recovers again, or floats from zero to maximum. Apparently somewhere in the network environment, a certain UDP buffer is full, hell knows. Again, on the official forum, you can search for compatibility with different software and hardware in case of problems.
Other "advanced" options that you can pay attention to:
bt.connect_speed - how many maximum new connections can be established per second (it should be increased, I have 80),
net.max_halfopen - much has been written about it, and it’s worth changing along with the tcpip.sys patch, although with the uTP protocol this is no longer important.
net.utp_target_delay is a certain target “ping” when adjusting connections, in some cases when it increases somewhere up to 400-500, the speed becomes better.
peer.disconnect_inactive_interval - after how many seconds the connection to the peer is closed, with which there is no data exchange, it is more important for open trackers with more people and “bad” peers, or in case of network glitches - in order to quickly determine the disconnection and reset it. in some cases it makes sense to lower it to 90-120.
Version 2.0, although beta is quite stable, can be downloaded here: forum.utorrent.com/viewtopic.php?id=60602
According to personal feelings, in the old alphas 1.9, when downloading from open trackers, if the incoming channel was loaded at about 1/3 before, from uTP it began to load at about 2/3. When downloading from closed trackers before the channel was loaded so that the browser began to slow down, and in 2.0 everything flies.
Source: https://habr.com/ru/post/68332/
All Articles