Parsing BEncode in javascript. View torrent files in Firefox

I. Why

There are several ways to view torrent files: in the torrent client, in the BEncode Editor , in file managers with plug-ins, perhaps in network services (but this is a bit dumb).

But it is not always convenient to call an external program from the browser. Not always this program gives full information. Not always in a convenient way. Not always searchable. Therefore, I would like to have in the browser an easy way to view the torrent file, for example:

- find out the contents of the distribution;
- find out the number of files in the distribution;
- find out information about files (some trackers are very lenient to the incompleteness of descriptions, and more information about files appears in torrent files - for example, resolution, video and audio codecs, duration of movies, etc.);
- find out information about the torrent file itself (creation time, trackers, privacy flag, etc.);
- be able to text search all the information.

Ii. Strategy

For our task, you can write an extension to the browser, but this is fraught with a number of additional difficulties. Therefore, we use the simplified method.
')
Extension Custom Buttons allows you to create buttons with arbitrary code. Even better, this code runs in the context of the browser, has access to the same components and interfaces as extensions, and can even create GUI elements of arbitrary complexity. Therefore, we simply create a new button and fill it with code (for all we need two hundred lines) All the following code needs to be inserted into the initialization tab of the newly created button so that it runs every time the browser is started, once and for the entire session defining the desired behavior of the button. Or you can not insert it: the extension adds the custombutton: // protocol to the browser, and at the end of the article I’ll give you a link just by clicking on which you can create a ready-made button with code (you just have to move it from the tool palette to a convenient place).

Iii. Tactics

1. User Interface

var btn = this; var imgMain = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAMAAAAoLQ9TAAAAdVBMVEX////////////////////////////////XzeWjisHx7vaKaq5nNpV8VqSDYaqtmMiTdbS2pM/MwN6YfLjBstd7VqRuQZnk3u7y7/eKa6+SdbR1S56CX6jXzuabgLv49/uhh7+JaK39/P3e1ury7vbLv97g2es6Rn8YAAAAB3RSTlMAYMAg0PDzqTbVzAAAAAFiS0dEAIgFHUgAAAAJcEhZcwAAAEgAAABIAEbJaz4AAACcSURBVBjTXY/bAoIgEEQRsWERUykDMu1m/f8ntqC+dJ6WYfYyQjCFBCMLsVIqaGI0VJnflaltpjZVUpRp7EZjFPcj/R/bLntQCKm56IE2e7QUIGsJ7tSeaUhVEi5wgw/OR2uvWegQvR9zy+ogWDi7CzyUMO4CD+W1EdQj7mvTYT7cJkx0nO9qPf2B8JxdwOtQbuHeC5bPdwv3F/8HCk4KcI8+awQAAAAldEVYdGRhdGU6Y3JlYXRlADIwMTItMDYtMjZUMjE6MDk6MDQrMDQ6MDD1mOHZAAAAJXRFWHRkYXRlOm1vZGlmeQAyMDEyLTA2LTI2VDIxOjA5OjA0KzA0OjAwhMVZZQAAAABJRU5ErkJggg=="; var imgThrobber = "data:image/gif;base64,R0lGODlhEAAQAOMIAAAAABoaGjMzM0xMTGZmZoCAgJmZmbKysv///////////////////////////////yH/C05FVFNDQVBFMi4wAwEAAAAh+QQBCgAIACwAAAAAEAAQAAAESBDJiQCgmFqbZwjVhhwH9n3hSJbeSa1sm5GUIHSTYSC2jeu63q0D3PlwCB1lMMgUChgmk/J8LqUIAgFRhV6z2q0VF94iJ9pOBAAh+QQBCgAPACwAAAAAEAAQAAAESPDJ+UKgmFqbpxDV9gAA9n3hSJbeSa1sm5HUMHTTcTy2jeu63q0D3PlwDx2FQMgYDBgmk/J8LqWPQuFRhV6z2q0VF94iJ9pOBAAh+QQBCgAPACwAAAAAEAAQAAAESPDJ+YSgmFqb5xjV9gQB9n3hSJbeSa1sm5EUQXQTADy2jeu63q0D3PlwDx2lUMgcDhgmk/J8LqUPg+FRhV6z2q0VF94iJ9pOBAAh+QQBCgAPACwAAAAAEAAQAAAESPDJ+cagmFqbJyHV9ggC9n3hSJbeSa1sm5FUUXRTEDy2jeu63q0D3PlwDx3FYMgAABgmk/J8LqWPw+FRhV6z2q0VF94iJ9pOBAAh+QQBCgAPACwAAAAAEAAQAAAESPDJ+QihmFqbZynV9gwD9n3hSJbeSa1sm5GUYXSTIDy2jeu63q0D3PlwDx3lcMgEAhgmk/J8LqUPAOBRhV6z2q0VF94iJ9pOBAAh+QQBCgAPACwAAAAAEAAQAAAESPDJ+UqhmFqbpzHV9hAE9n3hSJbeSa1sm5HUcXTTMDy2jeu63q0D3PlwDx0FAMgIBBgmk/J8LqWPQOBRhV6z2q0VF94iJ9pOBAAh+QQBCgAPACwAAAAAEAAQAAAESPDJ+YyhmFqb5znV9hQF9n3hSJbeSa1sm5EUAHQTQTy2jeu63q0D3PlwDx0lEMgMBhgmk/J8LqUPgeBRhV6z2q0VF94iJ9pOBAAh+QQBCgAPACwAAAAAEAAQAAAESPDJ+c6hmFqbJwDV9hgG9n3hSJbeSa1sm5FUEHRTUTy2jeu63q0D3PlwDx1FIMgQCBgmk/J8LqWPweBRhV6z2q0VF94iJ9pOBAA7"; function clickBtn(event) { if (event.button == 0) { event.preventDefault(); var tFileURL = prompt("Torrent File URL:"); if (tFileURL) { getTFile(tFileURL); } } } function checkDrag(event) { if (event.dataTransfer.types.contains("text/uri-list")) { event.preventDefault(); } } function onDrop(event) { var tFileURL = event.dataTransfer.getData("URL"); if (tFileURL) { getTFile(tFileURL); } event.preventDefault(); } btn.addEventListener("click", clickBtn, true); btn.addEventListener("dragenter", checkDrag, true); btn.addEventListener("dragover", checkDrag, true); btn.addEventListener("drop", onDrop, true); btn.onDestroy = function() { btn.removeEventListener("click", clickBtn, true); btn.removeEventListener("dragenter", checkDrag, true); btn.removeEventListener("dragover", checkDrag, true); btn.removeEventListener("drop", onDrop, true); } 

In this piece we get the button object, set two images (one is the main thing, the other is the standard file loading indicator, they will alternate), define event handlers and attach them to the button.

We get two ways to give the program the address of the torrent file: if there is a link, we simply drag it to the button ( description of the mechanisms ). If there is a line with the address in the buffer, we click on the button and paste the address into the appearing field.

At the end, we set the destructors for the attached handlers. In Custom Buttons, there is an unpleasant bug: if you do not explicitly specify a decoupling, handlers will overlap with each other with each opening and closing of the tool palette (even if you tune something else with its help).

2. Getting a torrent file

 function getTFile(tFileURL) { btn.image = imgThrobber; var xhr = new XMLHttpRequest(); xhr.mozBackgroundRequest = true; var sendData; if (tFileURL.indexOf("http://dl.rutracker.org/forum/dl.php") > -1) { xhr.open("POST", tFileURL, true); sendData = tFileURL.replace(/^.+\b(t=\d+).*$/, "$1"); xhr.setRequestHeader("Content-Type", "application/x-www-form-urlencoded"); xhr.setRequestHeader("Referer", "http://rutracker.org/forum/viewtopic.php?t=" + tFileURL.replace(/^.+\bt=(\d+).*$/, "$1")); } else { xhr.open("GET", tFileURL, true); sendData = null; } xhr.timeout = 10000; if(!/^file:/.test(tFileURL)) { xhr.channel.loadFlags |= Components.interfaces.nsIRequest.LOAD_BYPASS_CACHE; xhr.channel.QueryInterface(Components.interfaces.nsIHttpChannelInternal) .forceAllowThirdPartyCookie = true; } xhr.responseType = "arraybuffer"; xhr.onload = function() { btn.image = imgMain; processTFile(this.response); } xhr.ontimeout = function() { btn.image = imgMain; alert("Timeout"); } xhr.onerror = function() { btn.image = imgMain; alert("HTTP error"); } xhr.send(sendData); } 

Note, for a start, that torrent files are written in BEncode ( language description and torrent file format ). It is a bit like JSON in its capabilities. But there is one quotation in it: tags, numbers and strings in it are encoded, as a rule, on UTF-8, but some strings contain binary data (a simple sequence of bytes not obeying the UTF-8 rules). Therefore, it is impossible to simply process the entire string as UTF-8, the decoder will start and give an error message about the wrong UTF-8. When parsing a file you need to keep this precaution in mind.

Now a few notes about the request itself and getting the file.

We run the Pulsator image and set the background request flag to make the browser easier to use. Then we check the address and reassure the protection of the most well-known domestic torrent tracker, prohibiting downloading torrent files from non-site pages (this method, which allows extensions to work with torrent tracker files, was once described by the site developers themselves when protection was introduced).

To force Firefox to send cookies with XHR, even when the user has prohibited cookies from third-party sites, set the flag to force sending (without this, cookies are not accepted or sent). Of course, the flags of forced sending of cookies and cache traversal are needed only for network protocols, so in the case of the protocol of local files we do not install them.

Previously, in order to get a binary file from XMLHttpRequest, you had to resort to some kind of witchcraft . With the introduction of new types of response in XHR, everything is simplified . Therefore, we will use the type of arraybuffer and work in the future with typed arrays .

In the end, we set the handlers for different outcomes of our request (each time not forgetting to change the pulsator to a regular picture). If successful, we proceed to parse the resulting file.

3. Processing and output of the torrent file

 function processTFile(tFile) { var byteArray = new Uint8Array(tFile); var torrentObject = bdecode(byteArray); if (torrentObject) { if (torrentObject['creation date']) { torrentObject['creation date'] = (new Date(torrentObject['creation date']*1000)).toLocaleString(); } if (torrentObject.info) { var files = torrentObject.info.files; if (files && files instanceof Array) { for (var i = 0, file; file = files[i]; i++) { if (file.length) { file.length = Number((file.length / 1024).toFixed(2)).toLocaleString() + " KB"; } if (file['path.utf-8']) { file['path.utf-8'] = file['path.utf-8'].join("/"); } if (file.path) { file.path = file.path.join("/"); } } if (files[0]['path.utf-8']) { files = files.sort( function(o1, o2) { if (o1['path.utf-8'] > o2['path.utf-8']) {return 1;} else if (o1['path.utf-8'] < o2['path.utf-8']) {return -1;} else {return 0;} } ); } else if (files[0].path) { files = files.sort( function(o1, o2) { if (o1['path'] > o2['path']) {return 1;} else if (o1['path'] < o2['path']) {return -1;} else {return 0;} } ); } files.unshift(files.length); } else { if (torrentObject.info.length) { torrentObject.info.length = Number((torrentObject.info.length / 1024).toFixed(2)).toLocaleString() + " KB"; } } } if (gBrowser.selectedBrowser.currentURI.spec == "about:blank" && !gBrowser.selectedBrowser.webProgress.isLoadingDocument) { gBrowser.selectedBrowser.loadURI( "data:application/json;charset=utf-8," + encodeURIComponent(JSON.stringify(torrentObject, null, '\t')) ); } else { gBrowser.selectedTab = gBrowser.addTab( "data:application/json;charset=utf-8," + encodeURIComponent(JSON.stringify(torrentObject, null, '\t')) ); } torrentObject = null; } else { alert("Parsing error"); } } 

Having created an array of bytes, we pass it to the BEncode decoder (about it a little later) and receive from it a regular object (associated array, hash), which copies the structure of the torrent file (and if we don’t receive it, we give the parsing error message). Then we present some data in a more readable form (creation date, file sizes and paths to it), sort the file objects by the path property and insert the total number of files at the beginning of the file list. After that, we check if we have a blank page in the current tab and if something is loaded into it. If it is open and does not load, we will display it in the current tab, if not, open a new one and display it in it.

The output will be implemented in JSON for simplicity and versatility. The output is slightly formatted. But it is best to install some extension that highlights JSON and allows you to treat it as a tree structure, folding and expanding nodes (for example, JSONView ). After the withdrawal, just in case, we reset a huge object (if it is not paranoia).

4. BEncode parser

 function bdecode(byteArray, byteIndex, isRawBytes) { if (byteIndex === undefined) { byteIndex = [0]; } var item = String.fromCharCode(byteArray[byteIndex[0]++]); if(item == 'd') { var dic = {}; item = String.fromCharCode(byteArray[byteIndex[0]++]); while(item != 'e') { byteIndex[0]--; var key = bdecode(byteArray, byteIndex); if (key == "pieces") { dic[key] = bdecode(byteArray, byteIndex, true); } else { dic[key] = bdecode(byteArray, byteIndex); } item = String.fromCharCode(byteArray[byteIndex[0]++]); } return dic; } if(item == 'l') { var list = []; item = String.fromCharCode(byteArray[byteIndex[0]++]); while(item != 'e') { byteIndex[0]--; list.push(bdecode(byteArray, byteIndex)); item = String.fromCharCode(byteArray[byteIndex[0]++]); } return list; } if(item == 'i') { var num = ''; item = String.fromCharCode(byteArray[byteIndex[0]++]); while(item != 'e') { num += item; item = String.fromCharCode(byteArray[byteIndex[0]++]); } return Number(num); } if(/\d/.test(item)) { var num = ''; while(/\d/.test(item)) { num += item; item = String.fromCharCode(byteArray[byteIndex[0]++]); } num = Number(num); var line = ''; if (isRawBytes) { byteIndex[0] += num; return "[" + num + "]"; } else { while(num--) { line += escape(String.fromCharCode(byteArray[byteIndex[0]++])); } try { return decodeURIComponent(line); } catch(e) { return unescape(line) + " (?!)"; } } } return null; } 

I took Perl's parser as a basis, tempted by its brevity and simplicity. At first I tried to turn the typed array into a regular byte array so that it was possible to work with shift , but this implementation worked very slowly (probably, due to the constant reworking of a large array). Therefore, we had to introduce an ever-increasing access index, wrapping it into an array (so that it could be passed by reference to recursion).

The main difference from the original sample is in the block parsing lines. First, we remove from the output a huge string with bytes containing segment hashes (it has a clear location, therefore, having reached the necessary key in parsing the associated array, we temporarily set the flag to disable encoding in a call to parse the value of this key). Secondly, we perform some manipulations on the conversion of bytes into UTF-8 for the remaining rows. There are dangers lurking here: sometimes the lines are not UTF-8 (for example, the popular tracker tracker.0day.kiev.ua for some reason inserts the word “Tracker” in the “source” key in Windows-1251 encoding) and the decodeURIComponent crashes with an error . Therefore, for such cases, we return the string raw view, marking it slightly. It would be possible to delete such lines altogether, but sometimes only a piece of them causes the problem, but the main part is quite readable due to the coincidence of ASCII and the onset of UTF-8.

Iv. Perspectives

On the basis of parsing and obtaining the described information, you can implement more complex tasks. For example:

- track the update of the torrent file to the identical address (checking the content or creation time) and notify about the reloading; examples of such checks (regarding web pages, but everything can be easily altered) can be found here ;

- if the file is updated, it can be automatically saved from the browser to a folder on the disk, from where it will be picked up by the torrent client (here we may be interested in the interfaces nsIFile , nsILocalFile , nsIFilePicker , nsIFileOutputStream , nsIBinaryOutputStream and sample code ).

- Since the latest XHR implementations support the file: // protocol, you can also view local torrent files and even client databases (like settings.dat or resume.dat) using the button, however, in the latter case there will be a lot of binary strings with crochets. To do this, open the folder with the files in the browser and drag them to the button.

---

Promised link to install the button (if someone does not want to copy the code in parts to the initialization tab): since habraprser rewrites the link using the http protocol, you need to go to this page and click on the link “View torrent files” (of course, after installing Custom Buttons).

I apologize if I upset someone with the incompetence of handicrafts or incorrect formulations. I hope someone from this experience will come in handy.