How to tie a BitTorrent tracker to a Python site: integration with XBT Tracker

Suppose you have a certain site written in Python and you want to attach a BitTorrent tracker to it, like rutracker.org.

Task separation

The task can be divided into two big functionalities:

1. The directory of torrent distributions on the site (historically usually implemented as a forum),
2. Tracker itself, directly involved in the process of distribution.

The tracker is an http-application, according to the BitTorrent protocol specification , informing the client about all the distribution participants on request. Since clients send requests constantly periodically, the Tracker should be productive: the response time should be minimal.

In the PHP world, the Directory and Tracker are often not divided into two dedicated applications. For example, the popular TBDev Tracker exists as an application combining Directory-Forum and Tracker. (The author seems so tired of the popularity of his application that the site has not been updated for a long time and it’s impossible to download an application from it. However, many clones can be found on the Web.)
')
Some tracker implementations were originally written in Python, but then rewritten in C ++ for performance reasons. So nowadays there are no Python trackers (at least I could not find it). Therefore, the only thing that remains is to install a separate Tracker application and integrate it with the Python-Directory.

Control and protection

If you do not need any control, that is:

• You do not care what distributions your tracker will support - an anonymous user can add his distribution to your Tracker,
• and you consider in the order of things that an anonymous user can access any existing distribution,

then you just need to install one of the programs , choosing the easiest and most productive among them.
Further, you simply enable your users to create directory entries and upload and download .torrent files created by themselves. Preliminarily informing them of the correct announce-address of your tracker, which they will record in the .torrent file when creating the distribution.

If you want to establish at least some control, and even more so, as in my case, you want to limit the access of some user groups to some distributions, then for this you will need a so-called “private tracker” .

In theory, this is done as follows:

• Prevention of adding anonymous distributions: the tracker should support only those distributions that your authorized users have added to the directory. When creating a Directory entry and attaching a .torrent file to it, the code of your site should add a distribution to the list of allowed distributions for the tracker.
• Preventing an .torrent file from being received by an anonymous user is implemented by means of the Directory
• Restriction of user access to distribution (as well as tracking users participating in the distribution) is performed according to the principle: if someone can see the directory entry and download the attached .torrent file, then participation in the distribution is allowed. In this case, the tracker must identify the user. And this is done by flashing the unique user code in the announce tracker URL - on the fly, when the user downloads a .torrent file. That is, the Directory code must do this.
• Prohibition of broadcast sharing protocols: Distribute Hash Table (DHT), Peer EXchange (PEX), Local Service Discovery (LSD), which allow you to work without Tracker at all, and therefore without access control. Implemented by setting the flag “private = 1” in the .torrent file.

Implementation

Considering the above, my choice fell on XBT Tracker , as the only implementation of a private Tracker designed for integration with any Directory site.
XBT Tracker is written in C ++ and is installed by building from source. On Ubuntu Server 12.04, 64-bit is built the first time.

Dependencies

Interacting with XBT Tracker is supposed through the MySQL database Tracker, so our Python-Directory will need to be able to write-read the MySQL database. For this, I used the pymysql package.
To parse and modify .torrent files, you also need the BitTorrent-bencode package.

 import bencode, pymysql 

Functions

Adding an authorized user, if there is no such yet. Here, the torrent_pass field of the torrent_pass table xbt_users used to bundle your site's user ID and XBT Tracker user ID. Previously, torrent_pass was used to authorize a user by the key specified in the announce URL, but now XBT Tracker uses a different algorithm - see below. The uid field is auto-increment.

  c = db.cursor() c.execute("SELECT uid, torrent_pass, torrent_pass_version, downloaded, uploaded FROM xbt_users WHERE torrent_pass = %s", (request.user.id,)) rec = c.fetchone() if not rec: # insert a new user c.execute("INSERT INTO xbt_users(torrent_pass) VALUES(%s)", (request.user.id)) c.execute("SELECT uid, torrent_pass, torrent_pass_version, downloaded, uploaded FROM xbt_users WHERE torrent_pass = %s", (request.user.id,)) #rowcount =0 rec = c.fetchone() db.commit() uid, torrent_pass, torrent_pass_version, downloaded, uploaded = rec 

Reading and parsing a .torrent file:

  with open(fpath, 'rb') as f: buf = f.read() bt = bencode.bdecode(buf) 

Forcibly set a private flag and calculate info_hash (private flag is included in the info section and affects its hash):

  if xbt_force_private: bt['info']['private'] = 1 info_hash_obj = hashlib.sha1(bencode.bencode(bt['info'])) 

Registration of distribution in the xbt_files table if not already present.

  sha = info_hash_obj.hexdigest() c = db.cursor() c.execute("SELECT flags FROM xbt_files WHERE info_hash=UNHEX(%s)", (sha,)) flag = c.fetchone() if flag is None: c.execute("INSERT INTO xbt_files(info_hash, mtime, ctime) VALUES(UNHEX(%s), UNIX_TIMESTAMP(), UNIX_TIMESTAMP())", (sha,)) db.commit() elif flag[0] % 2 : error_msg(pagename, request, _("Torrent is marked for deletion!")) return 

Calculate authorization key for announce URL

XBT Tracker uses a very clever key calculation algorithm. A set of values ​​is taken:

• User ID: xbt_users.uid ,
• user key version: xbt_users.torrent_pass_version ,
• server's secret key (generated automatically when XBT Tracker is first xbt_config ): xbt_config table, xbt_config value,
• The value of the info_hash of this distribution: xbt_files.info_hash .

All this is magically mixed by the principle “I twist-twist, I want to confuse”:

  c.execute("select value from xbt_config where name = 'torrent_pass_private_key'") torrent_pass_private_key = c.fetchone()[0] s = "%s %d %d %s" % (torrent_pass_private_key, torrent_pass_version, uid, info_hash_obj.digest()) sha = hashlib.sha1(s).hexdigest()[0:24] pwd = "%08x%s" % (uid, sha) bt['announce'] = 'http://xbt.host:port/%s/announce' % pwd # remove other trackers if bt.has_key('announce-list'): del bt['announce-list'] 

From here it becomes clear the purpose of xbt_users.torrent_pass_version : by changing this value, you can make all previously downloaded .torrent files invalid. That is, it is - some kind of analog password reset.
And finally, we encode the .torrent file back into the string, which we will send to the client:

  buf = bencode.bencode(bt) 

Deleting a hand

When deleting an attached file, we must remove the distribution from the list of registered distributions ( xbt_files tables).
There is a polite deletion method in which we mark the distribution as deleted, but in reality it is deleted by the tracker when it is downloaded completely by the last lich.

 c.execute("update xbt_files set flags = 1 where info_hash = UNHEX(%s)", (info_hash, )) 

Well that's all. Further spinning of the site: displaying the number and the list of distributors, counting the rating, displaying the list of files in the distribution, I refer to decorations. They are implemented quite obviously: all the necessary information, including statistics, is available in the tracker database, - and I, after Pierre Fermat, feel sorry for spending space on them.

The solution outlined is embodied as a plug-in to the popular wiki engine: MoinMoin

It should be noted that the proposed solution has a significant drawback: virtual hosting with Python support is generally relatively rare, and I haven’t met with the opportunity to build a C ++ application in nature. The only alternative, as I see it, is to take some TBDev Tracker clone and bite out the code that directly implements the Tracker functions.

I hope my experience will be useful to someone.