Make bittorrent better
Recently, users of file-sharing networks are increasingly calling for a transition to a subspace: anonymous networks like i2p tor, etc.
Undoubtedly, this idea has a lot of positive aspects, however, in essence, this means breaking down the established traffic exchange systems, some of which have been formed for more than 10 years and starting to build anew, albeit with old mistakes, but also trying not to make new ones. For the time being, the Bittorrent network is dozens, and probably even hundreds of millions of well-established ties, “breaking off” which will be extremely difficult to restore everything in full volume.
Let's still look at our old man, and ponder whether everything is so bad, and whether it is possible to correct the shortcomings that he has.
The first reason for which bittorrent is usually criticized is its centralization in the form of a tracker, which, as a rule, is a vulnerable spot for attacks by copywriters, DDoSers, and authorities of all stripes (sorry for the pun).
')
Here you can also add the presence of the torrent file itself, which is the initiator of the distribution, and which the responsible persons are often confused with the content of the distribution itself, with all the ensuing consequences.
However, bittorrent already has solutions to these problems that work effectively. These are DHT and peer-exchange as substitutes for trackers, and operations with hash sums - as substitutes for torrent and magnet links.
About DHT and PE, I will discuss a little more below, but for now let's talk about hash amounts.
In fact, this is the main and necessary component of the magnet links, and it is necessary and almost always sufficient to obtain the distribution described by it.
(For the time being) the law does not prohibit counting hash amounts and publishing them on the Internet. Just as it does not prohibit the counting of oprichniki in the crowd or the number of frames in the main erotic scene of the film, which, by the way, are always unique and can serve as its unique identifier.
From the point of view of your client - he almost doesn’t care, you slipped the torrent to him, or the magnet link or hash is the amount. “Almost” - in the event that, he has an initial cloud of peers, which will allow him to interrogate them for the presence of the corresponding hash sum of content.
A simple conclusion follows from this - the more you download / distribute - the more your cloud of peers, and the faster your client will be able to find those peers by the hash sum that will give you its contents. So a lot of hands at the client is not only pleasant, but also useful.
And most importantly:
Well, as a modification of technology - an algorithm that allows you to make an arbitrary number of digital "aliases" of the hash. This will significantly entertain copywriters and censors, repeatedly expanding and complicating the volume of search and control over the distribution of hashes, in the event that the hunt for them unfolds in full scale.
In addition, hash sums are easily converted into QR codes (barcode as an option), which opens up new opportunities for outdoor and banner advertising, as well as for equipping each policeman with a QR code reader connected to the Internet.
The last one at this point is that all of the above is fully compatible with the technology that we have now, and is its extension, or, more correctly, the logical continuation.
The second problem of file-sharing systems, and indeed of the entire Internet, is the problem of the integrity of the data involved in file sharing. The file hosting service can close the FBI or Roskomnadzor, the data will disappear irreversibly, the same thing can happen to the tracker, but the data will remain, although the distribution clouds will collapse (the clouds will disappear, the clouds will be the same!) .
There is another way to combat file sharing: how to scare users. Then, like a zombie, they erase everything from their discs and make the TV louder.
But still, the main reason for the loss of information in peer-to-peer networks is the users themselves ... They simply leave distributions, and distributions in this way “die” (or are deleted from file hosting services for uselessness). If you downloaded a torrent, and there are no siders in the distribution, the maximum time you wait is a month, sometimes two. Then you stop waiting, it means the distribution is dead, and only the human factor is to blame here: one of the last siders spilled coffee on the laptop, the other drowned the portable disk in the toilet. And the third got married, and Lars Von Trier's films no longer interest him.
The number of dead for this reason, the content is very large, and significantly exceeds all the efforts of the copywriters in this field. Of course, one can argue that the missing information is of much lower value than that maintained by users, and even more so than that which is being closed by the right holders, but this does not change the essence of the matter. Firstly, because the notion of “value” of information is relative, and secondly, once having solved the problem of “indecipability” we will solve others at the same time.
(Here I will deliberately omit the discussion about the other consequences of the “indestructibility” of information, since it is quite obvious that there is content that should not be distributed throughout the network - even from the point of view of The PirateBay).
Oddly enough, to implement this mechanism is not so difficult.
It is only necessary to make an add-on over the protocol discussed here, allowing any client to give a part of their traffic and disk space for caching, storing and distributing random blocks of random hands.
Without forming a single whole, such data blocks will have no applied value for the computer on which they are cached.
The cached distribution will never be downloaded entirely on this computer and the user will never know that he has cached his computer.
With the current speed of Internet access and the cost of disk space there is no inconvenience to donate a few gigabytes on the screw and a few percent of the width of the channel to the Internet,especially as windows update does it without demand .
As a result of this, any distribution other than its immediate user-sided siders will be block-by-block downloaded and distributed by random nodes, the probability of its “death” will decrease. It will become an extremely difficult task to find and clean out parts of it from those places where it is cached, if such a need arises.
If there is an algorithm for estimating the prevalence of this or that content on the network, the client can cache only rare distributions, because artificial support for popular distributions does not require a rule. As soon as the "prevalence index" of distribution falls, the self-supporting mechanism turns on, and customers cache its parts. By the same principle, blocks with growing popularity will be replaced in the cache, the distribution blocks of which are falling in popularity.
A side effect of this innovation will be the additional protection of the releasers - by making the distribution, you can wait until it disperses through the peer cloud before publishing the hash amount in a crowded place. It is practically impossible to establish the source of the release after this (the followers of Assange and Snowden will certainly appreciate it).
This add-on is also easy to make backward compatible with existing technology, as an additional feature in any bit torrent client (preferably not disabled)
If we want to decentralize and, thus, protect the network, we need to move from trackers to DHT and Peer-Exchange networks as the main method of exchanging peers among bittorrent clients, using trackers only to initialize hands, and maybe just to issue an initial list of peers DHT network, whatever the specific distribution.
This would seem an obvious conclusion, however, many trackers in the face of their administrations act exactly the opposite - artificially narrowing and limiting the clouds of their peers!
I mean closed trackers, entry to which is possible only by invites, and which conduct their own internal policies in the form of private distributions, keys, etc., making it difficult, if not restricting the exchange of data on DHT.
No, I have nothing against closed communities with their own rules and internal rules of communication, but what's the point of creating such artificial reservations for robots, that is, bit torrent clients?
Here we are talking not only about private keys in torrents, but also the rules on “repacking” releases for a specific tracker (with a vanity holiday in the form of its own * .nfo file in the distribution), due to which the distribution hash changes, and as a result a clone appears content, but with a great hash sum, which is “incompatible” for peers with the source.
Your bittorrent client is no more pleasant to download from IP addresses registered on your tracker than from any other received via DHT. He absolutely does not care! Give freedom on the Internet at least robots!
If we summarize a little, then we should probably talk about the introduction of such a code ofpiracy honor for trackers, a set of standard rules, among which there will be a ban on private torrents and keys, a ban on “prohibiting DHT” in torrent files, a ban on “repacking” releases under needs or rules of a specific tracker or group release.
As a conclusion, I would like to note that, despite loud statements, there are no technologies yet for filtering bittorrent traffic. Some providers, using DPI, can only limit the speed or prohibit it entirely. Unlike the system of blocking sites by URL, where filtering is already underway.
Undoubtedly, this idea has a lot of positive aspects, however, in essence, this means breaking down the established traffic exchange systems, some of which have been formed for more than 10 years and starting to build anew, albeit with old mistakes, but also trying not to make new ones. For the time being, the Bittorrent network is dozens, and probably even hundreds of millions of well-established ties, “breaking off” which will be extremely difficult to restore everything in full volume.
Let's still look at our old man, and ponder whether everything is so bad, and whether it is possible to correct the shortcomings that he has.
one.
The first reason for which bittorrent is usually criticized is its centralization in the form of a tracker, which, as a rule, is a vulnerable spot for attacks by copywriters, DDoSers, and authorities of all stripes (sorry for the pun).
')
Here you can also add the presence of the torrent file itself, which is the initiator of the distribution, and which the responsible persons are often confused with the content of the distribution itself, with all the ensuing consequences.
However, bittorrent already has solutions to these problems that work effectively. These are DHT and peer-exchange as substitutes for trackers, and operations with hash sums - as substitutes for torrent and magnet links.
About DHT and PE, I will discuss a little more below, but for now let's talk about hash amounts.
In fact, this is the main and necessary component of the magnet links, and it is necessary and almost always sufficient to obtain the distribution described by it.
- The hash is not a torrent or magnet file, it does not need to be downloaded and saved to disk, it is much easier to publish it in the open form, it is not a link, it is very difficult to fit the definition of “a means of disseminating data” as it is written in the last law of the Russian Federation "on locks".
(For the time being) the law does not prohibit counting hash amounts and publishing them on the Internet. Just as it does not prohibit the counting of oprichniki in the crowd or the number of frames in the main erotic scene of the film, which, by the way, are always unique and can serve as its unique identifier.
From the point of view of your client - he almost doesn’t care, you slipped the torrent to him, or the magnet link or hash is the amount. “Almost” - in the event that, he has an initial cloud of peers, which will allow him to interrogate them for the presence of the corresponding hash sum of content.
A simple conclusion follows from this - the more you download / distribute - the more your cloud of peers, and the faster your client will be able to find those peers by the hash sum that will give you its contents. So a lot of hands at the client is not only pleasant, but also useful.
And most importantly:
- searching and initiating distributions by hash sums is a long time ago to embed clients into ALL bittorrent. In addition to working with magnet links, it goes without saying that for some reason it is not everywhere either. It is clear that making a link magnet from a hash sum is one movement in a text editor, but still for an ordinary user and this can be difficult.
Browsers can (in fact - should!) Learn to recognize hash sums, highlight them like links, opening them by clicking in the bit torrent client.
Well, as a modification of technology - an algorithm that allows you to make an arbitrary number of digital "aliases" of the hash. This will significantly entertain copywriters and censors, repeatedly expanding and complicating the volume of search and control over the distribution of hashes, in the event that the hunt for them unfolds in full scale.
In addition, hash sums are easily converted into QR codes (barcode as an option), which opens up new opportunities for outdoor and banner advertising, as well as for equipping each policeman with a QR code reader connected to the Internet.
The last one at this point is that all of the above is fully compatible with the technology that we have now, and is its extension, or, more correctly, the logical continuation.
2
The second problem of file-sharing systems, and indeed of the entire Internet, is the problem of the integrity of the data involved in file sharing. The file hosting service can close the FBI or Roskomnadzor, the data will disappear irreversibly, the same thing can happen to the tracker, but the data will remain, although the distribution clouds will collapse (the clouds will disappear, the clouds will be the same!) .
There is another way to combat file sharing: how to scare users. Then, like a zombie, they erase everything from their discs and make the TV louder.
But still, the main reason for the loss of information in peer-to-peer networks is the users themselves ... They simply leave distributions, and distributions in this way “die” (or are deleted from file hosting services for uselessness). If you downloaded a torrent, and there are no siders in the distribution, the maximum time you wait is a month, sometimes two. Then you stop waiting, it means the distribution is dead, and only the human factor is to blame here: one of the last siders spilled coffee on the laptop, the other drowned the portable disk in the toilet. And the third got married, and Lars Von Trier's films no longer interest him.
The number of dead for this reason, the content is very large, and significantly exceeds all the efforts of the copywriters in this field. Of course, one can argue that the missing information is of much lower value than that maintained by users, and even more so than that which is being closed by the right holders, but this does not change the essence of the matter. Firstly, because the notion of “value” of information is relative, and secondly, once having solved the problem of “indecipability” we will solve others at the same time.
- We can come to a situation where any information laid out in the p2p network will remain in it forever. Perhaps it will have limited availability in the form of a small number of sources, due to low demand, however, it will be available and there will be no way to remove it from the network ...
(Here I will deliberately omit the discussion about the other consequences of the “indestructibility” of information, since it is quite obvious that there is content that should not be distributed throughout the network - even from the point of view of The PirateBay).
Oddly enough, to implement this mechanism is not so difficult.
It is only necessary to make an add-on over the protocol discussed here, allowing any client to give a part of their traffic and disk space for caching, storing and distributing random blocks of random hands.
Without forming a single whole, such data blocks will have no applied value for the computer on which they are cached.
The cached distribution will never be downloaded entirely on this computer and the user will never know that he has cached his computer.
With the current speed of Internet access and the cost of disk space there is no inconvenience to donate a few gigabytes on the screw and a few percent of the width of the channel to the Internet,
As a result of this, any distribution other than its immediate user-sided siders will be block-by-block downloaded and distributed by random nodes, the probability of its “death” will decrease. It will become an extremely difficult task to find and clean out parts of it from those places where it is cached, if such a need arises.
If there is an algorithm for estimating the prevalence of this or that content on the network, the client can cache only rare distributions, because artificial support for popular distributions does not require a rule. As soon as the "prevalence index" of distribution falls, the self-supporting mechanism turns on, and customers cache its parts. By the same principle, blocks with growing popularity will be replaced in the cache, the distribution blocks of which are falling in popularity.
A side effect of this innovation will be the additional protection of the releasers - by making the distribution, you can wait until it disperses through the peer cloud before publishing the hash amount in a crowded place. It is practically impossible to establish the source of the release after this (the followers of Assange and Snowden will certainly appreciate it).
This add-on is also easy to make backward compatible with existing technology, as an additional feature in any bit torrent client (preferably not disabled)
3
If we want to decentralize and, thus, protect the network, we need to move from trackers to DHT and Peer-Exchange networks as the main method of exchanging peers among bittorrent clients, using trackers only to initialize hands, and maybe just to issue an initial list of peers DHT network, whatever the specific distribution.
This would seem an obvious conclusion, however, many trackers in the face of their administrations act exactly the opposite - artificially narrowing and limiting the clouds of their peers!
I mean closed trackers, entry to which is possible only by invites, and which conduct their own internal policies in the form of private distributions, keys, etc., making it difficult, if not restricting the exchange of data on DHT.
No, I have nothing against closed communities with their own rules and internal rules of communication, but what's the point of creating such artificial reservations for robots, that is, bit torrent clients?
Here we are talking not only about private keys in torrents, but also the rules on “repacking” releases for a specific tracker (with a vanity holiday in the form of its own * .nfo file in the distribution), due to which the distribution hash changes, and as a result a clone appears content, but with a great hash sum, which is “incompatible” for peers with the source.
Your bittorrent client is no more pleasant to download from IP addresses registered on your tracker than from any other received via DHT. He absolutely does not care! Give freedom on the Internet at least robots!
If we summarize a little, then we should probably talk about the introduction of such a code of
- We (tracker operators in this case) should strive to ensure that the distribution of the same content on different trackers has the same hash, and therefore - a common cloud of peers. Then the “vitality” of such a distribution will grow in proportion to the number of trackers where it is published, and the trackers themselves will only benefit from it.
As a conclusion, I would like to note that, despite loud statements, there are no technologies yet for filtering bittorrent traffic. Some providers, using DPI, can only limit the speed or prohibit it entirely. Unlike the system of blocking sites by URL, where filtering is already underway.
Source: https://habr.com/ru/post/234975/
All Articles