Combining browsers in unsinkable tracker

Peer-to-peer networks have become a powerful force in the modern Internet. Client programs of p2p networks unite computers into giant network “organisms”, and the end users themselves create, maintain and control these networks. Today, every computer is almost always open browser. Combining browsers in a p2p-network we get a distributed system, which is always at hand and which can be useful for solving many pressing problems.

Recently, I talked about the extension of Firefox, which integrates browsers into the p2p search network Allisk. Now this network has mastered additional functions: publishing / searching torrents and a tracker for BitTorrent. Before proceeding to the detailed description, I will briefly explain the formulation of the problem.

The most important nodes of the BitTorrent p2p network are torrent sites. Through them, users find the distribution, through them the coordination of current distributions is carried out. However, torrent sites are not only the most important, but also the weakest points of BitTorrent. The vulnerability lies primarily in the fact that the corresponding server can be blocked, banned, selected, etc.
')
If the dispatch functions are distributed according to the p2p network principle, this will at once make the coordination service not subject to either technical or administrative threats. I decided to create a p2p service that will implement the usual functions of the torrent site (torrent search and distribution manager), and, when interacting with the client program, the service will look like an ordinary tracker.

In this article, I will first briefly tell the distinguished meeting how all this works technically, then I will show in more detail in the pictures how to use the service.

How does it work

technical section, you can skip
In the case of a classic torrent tracker, distribution is organized as follows: when creating a torrent file, one or several tracker URLs are added to it. For example, let it be http: // mytracker.com/announce. Then the user receives the torrent file and opens it in the client program, which addresses the specified address 'http: //mytracker.com/announce' to announce the participation in the distribution and get a list of other participants from the tracker.

In contrast to the classic version, in my decision the client program does not apply to the centralized tracker, but to the local node of the Allisk network. To do this, when creating a torrent file, the URL is added to it http: // localhost: 7583 / anns. For a BitTorrent client, everything looks like in the classic case - there is a URL with which a dialogue is being conducted using a standard protocol, just a call to the tracker is sent to the local port 7583. This port listens to the Allisk extension.

Next, the following happens:
1) BitTorrent client sent request to port 7583
2) the Allisk node received a request and found in a p2p network a node (group of nodes) responsible for a specific distribution
3) the Allisk node requested from the responsible node a list of the distribution participants
4) The Allisk node returned to the BitTorrent client a list of distribution participants

As a result, the client program received the necessary information not from a specific site, but from a p2p network. It turns out distributed p2p-tracker that works with any client programs.



Immediately you need to clarify that now there are quite a few distributed trackers. In fact, if you meet the abbreviation DHT in the context of some client BitTorrent program or when they talk about “trackless distribution”, then we are talking about the distributed tracker. However, existing trackers are internal modules of each client program separately, which means that in general they do not work for any client programs, but only for a specific program . The solution I proposed does not depend on the client program, so it allows you to coordinate the distribution between any clients globally.

The next important point: other distributed solutions implement only the dispatcher function, there is no search by name. In most clients, the ability to search is reduced to accessing external search sites. In the proposed solution, a torrent search is performed in the same p2p network, Allisk, while publishing a torrent file is reduced to copying the file to a folder selected by the user.

Thus, the built system actually works as a standard torrent site (search + tracker), but at the same time is deprived of some drawbacks of such sites, for example, a distributed p2p system cannot be closed / blocked / turned off. There are no restrictions on published materials, but measures will be taken to limit the distribution of criminal materials. In such a self-organizing system, users themselves should be able to limit the spread of any nastiness.

How torrents fall into Allisk

Option one: create a torrent file yourself
Suppose the user made a small film “Cheerfulness on a winter day” and decides to share the corresponding file with everyone. For this you need to create a torrent file using any BitTorent client. The pictures show how to do this in uTorrent and Vuze clients. Pay attention to the comment - this field must be filled so that this torrent file can be found in the Allisk on the relevant phrase. It is important that in this field was the exact title of the work, without unnecessary information. The word order is not important. The second important point is the address of the tracker. As you can see, this address points to localhost, and the port number may be different if you changed the corresponding settings parameter of the Allisk extension (the 'settings' page).



So, a torrent file was created for the distributed tracker, now you can publish the file to the Allisk network. To do this, the user simply places the torrent file in a special folder (you can customize the folder for publishing on the 'settings' page in the 'Torrent folder' field). Further, the very extension Allisk will analyze and publish data about the torrent. At this step, other users can already find and receive the torrent file, now you need to start the distribution using a BitTorrent client. To do this, simply open the torrent file in the client program and, at the same time, specify the location of the source file with the movie.

Option Two: change the existing torrent file
Consider a situation where we already have a torrent file, but it was created to work with another tracker. Copy the torrent file into the folder and open the 'torrents' page.



In the general list of torrents, you can immediately see the files marked with an exclamation mark, which the Allisk cannot accept. If you hover the mouse over the triangle icon at the beginning of the line, an explanation will appear about the problem with this file. To edit a file simply click on the name and open the edit form.



Fill in the field "Title" and save the file. Now it remains only to open it in the client program.
Note: torrent files marked as “private distribution” cannot be edited, but you can make a copy by adding the name of the new file in the “file name” field .

Option Three: distribution without publication
If the user wants to distribute the file only to his friends / acquaintances and not publish data about the file in the search network, then it is enough just not to place the torrent file in the folder for publishing. The rest of the distribution will work as usual, even if the torrent file is not registered. Of course, in this embodiment, the user himself must take care of the transfer of the torrent file to the recipients.

Search and retrieve torrent file

To start the search, select the type of search "torrents" and write the query:



Get the list of available torrents:



Select a file and start receiving it. Now the extension will try to download the torrent file from other users.



When the torrent file is received and you go back to the "torrents" page, the file will appear in the general list of torrents. From here, you can click it to open it in the client program to connect to the distribution.

Settings, restrictions

Currently, to work with a distributed tracker, you need to open external access to the AllExport expansion port. This requirement is not so much technical as organizational. It seems to me logical if the user wants to receive something, then you need to give something in return. If the port is open, other network nodes will be able to contact each other directly to save information and search. In this case, the load on the network will be really even. If only some nodes are open, then the load on them will obviously be greater.

In addition, there are technical reasons. Unfortunately, so far Firefox does not support a UDP socket, so simple NAT passage technologies cannot be applied. And more sophisticated technologies for TCP are simply not available at the level of high-level network functions available in Firefox.

Of course, for many users, this restriction will be deterrent, and I myself cannot say that I like this solution very much. Therefore, I will continue to look for other ways. But so far - so.

You can set the network port for the extension on the Settings page. I will not tell you in detail how to open a port on a machine for access from the outside, since much has been written on this topic ( example ), and the readers of this site probably know how to do it.
Explanation: in order to use only Internet search, it is not necessary to open the port, this restriction applies only to the tracker function.

Changes, additions

I want to thank the participants in the discussion of the first article for constructive criticism and interesting ideas. Immediately I made it possible to add links to Allisk without making a bookmark (this was an offer from rednaxi ), now on the bookmark editing page there is a corresponding check mark. I started making changes to automatically analyze the pages that were opened, as well as the statistics of visiting pages with an eye on (semi?) Automatic addition of interesting pages to the Allisk network.

Conclusion

At the first stage, the Allisk network was able to keep links to pages on the Internet and search for these links by their respective keywords. Now the network has mastered additional functions: publishing / searching for torrents and the BitTorrent tracker. In fact, the Allisk network in this case acts as a dispatch service for another p2p network - BitTorrent.

Version 0.8 beta has been released. You can download it here . If you are interested in this project, I will be very grateful to readers who will try to use the extension and share their comments, thoughts, errors found.