Another way to update torrents
On one tracker, I am an active sider. But when it comes time to update the distribution, horror begins for me: some distributions have different names in the torrent client and on the tracker, there are a lot of distributions with the same name on the tracker, and it is very difficult to look for any particular distribution. In addition, I do not have so much time to engage in such a routine matter. Therefore, I needed a small script that would update the distributions in the client, if they were updated on the tracker.
I was faced with the task: to find some ready-made solution, or try to write the necessary script myself. On Habré met ways, in some way performing my task, but the method either did not suit me, or I didn’t quite like it. At the same time, I never wrote programs or even scripts, so I liked the version of my own handwritten script even more. First I had to choose a tool, a language that would be easy to learn and dive into programming, and my attention was drawn to python.
I immediately liked Python. It seems that it gives a certain "ease" in writing code. As the first python reading matter, I chose Mark Lutz’s book, Learning Python (4th edition). Well, there is a tool, some kind of help in the form of a book, let's go!
')
So, first you need to determine that the torrent file in our client (in this case we mean uTorrent 2.2) is outdated and you need to download a new one. The first thing I could think of was page parsing and comparison with data in a torrent file. This method worked, but it had a huge minus in speed: a hundred pages parsing, and this is the limit of the distribution on the tracker, took about three minutes. In addition, it was necessary to compare all the distribution options with the result of the page parsing, and this also took a lot of time. This method worked without failures, but I didn’t particularly like it, so I continued to search for all sorts of solutions to the problem.
Soon, after much deliberation and searching, I learned about such a thing as scrape. Scrape, as Wikipedia says, is an additional client request protocol to the tracker, in which the tracker tells the client the total number of seeds and peers in the hand. With a scrape request, you can easily find out if a distribution exists or not. Also scrape-request is sent by clients more often than announce. But you need to know whether a particular tracker supports this protocol or not. To my luck, my tracker supports it. A scrape request is sent using the GET method with a header, and this is how the address to which the request goes:
The hash is unique for each distribution, it includes 20 characters and can be obtained from the resume.dat file. But before getting information, you need to know that this file, like files with the extension .torrent and settings.dat, is presented in bencode format. If you need to decrypt the file quickly and without recesses in the encoding method, then you should download a special package for python here .
Let's proceed to decrypt the file:
Now we have a dictionary with names and hashes of distributions. Now we can only send scrape-requests with a substituted and modified hash and check whether there is a distribution with such a hash on the tracker or it is no longer there. Also, do not forget that you need to make such a request on behalf of the client, otherwise the tracker will refuse access.
The usual response to a query is:
If the distribution does not exist, the response to the request takes the form:
The response to the request is also presented in bencode format, but we do not need to decrypt it, you can simply compare the resulting string with the string returned if there is no distribution on the tracker with such a hash.
Next you need to download our file from the tracker, put it in the client's startup folder and, if possible, delete the record about the outdated torrent in the client itself.
From the tracker just download the file does not work: need authorization. The function itself is described above under the heading “authentication”. And then we log in, download the file, put it in the startup folder and delete the old .torrent file from the folder with the torrents.
So that a non-existent .torrent file does not confuse us with its record in the client, it should be removed from the client. But uTorrent is designed in such a way that editing resume.dat, namely, information about all the torrents is stored, when the client is running, it will not give the result: uTorrent will restore resume.dat the way it remembers at startup. Therefore, for such a case, you need to constantly turn off uTorrent, edit resume.dat, turn on uTorrent. Such a method would be suitable for one changed distribution per day, and what if the distribution changes in batches, i.e. several at once? At first, being far from programming as a whole, I thought that I would have to work with processes directly, which is very difficult for me. But then I learned about the existence of uTorrent WebUI. WebUI has an API, the documentation for which is on the official site . Thanks to the capabilities of the WebUI API, you can delete the record, and not only delete the torrent from the client. First we need to get a cookie with a special password and token. The second we need if the parameter webui.token_auth in the client is activated.
In uTorrent, the authorization in the web-interface is not implemented in the same way as on the site, so simple data sending will not work. Then we get the token and together with it we perform some function in the client. Of course, it would be possible to allocate a class for actions in the client, but I considered that the usual function would be enough for that.
(Note: Unfortunately, my knowledge at the moment was not enough to properly log in to the web interface, so I used the method described on the Internet.)
As a result, I received a script that satisfies my need, a little knowledge and a lot of pleasure: it is very fun to sit on the code until the morning, and then, when you lie down to sleep, understand what the snag was.
I hope this method will be able to help someone.
UPD: I wildly apologize for my carelessness: I brought the code into a more readable form before publication, as a result of which I myself got confused and confused you.
The code is uploaded to Github . I work with him for the first time, so if I did something wrong, contact me.
What to do?
I was faced with the task: to find some ready-made solution, or try to write the necessary script myself. On Habré met ways, in some way performing my task, but the method either did not suit me, or I didn’t quite like it. At the same time, I never wrote programs or even scripts, so I liked the version of my own handwritten script even more. First I had to choose a tool, a language that would be easy to learn and dive into programming, and my attention was drawn to python.
I immediately liked Python. It seems that it gives a certain "ease" in writing code. As the first python reading matter, I chose Mark Lutz’s book, Learning Python (4th edition). Well, there is a tool, some kind of help in the form of a book, let's go!
')
Problem statement and its solution
So, first you need to determine that the torrent file in our client (in this case we mean uTorrent 2.2) is outdated and you need to download a new one. The first thing I could think of was page parsing and comparison with data in a torrent file. This method worked, but it had a huge minus in speed: a hundred pages parsing, and this is the limit of the distribution on the tracker, took about three minutes. In addition, it was necessary to compare all the distribution options with the result of the page parsing, and this also took a lot of time. This method worked without failures, but I didn’t particularly like it, so I continued to search for all sorts of solutions to the problem.
Soon, after much deliberation and searching, I learned about such a thing as scrape. Scrape, as Wikipedia says, is an additional client request protocol to the tracker, in which the tracker tells the client the total number of seeds and peers in the hand. With a scrape request, you can easily find out if a distribution exists or not. Also scrape-request is sent by clients more often than announce. But you need to know whether a particular tracker supports this protocol or not. To my luck, my tracker supports it. A scrape request is sent using the GET method with a header, and this is how the address to which the request goes:
htt://example.com/scrape.php?info_hash=aaaaaaaaaaaaaaaaaaaaThe hash is unique for each distribution, it includes 20 characters and can be obtained from the resume.dat file. But before getting information, you need to know that this file, like files with the extension .torrent and settings.dat, is presented in bencode format. If you need to decrypt the file quickly and without recesses in the encoding method, then you should download a special package for python here .
Let's proceed to decrypt the file:
# -*- coding: utf-8 -*- import urllib2 from urllib import urlencode from binascii import b2a_hex as bta, a2b_hex as atb from os import remove from shutil import move from lxml.html import document_fromstring as doc from bencode import bdecode, bencode from httplib2 Http http = Http() username = 'username' password = 'password' ut_port = '12345' # web- uTorrent'. ut_username = 'utusername' ut_password = 'utpassword' site = 'http://example.com/' scrape_body = site + 'scrape.php?info_hash=' # URL scrape-. login_url = site + 'takelogin.php' torrent_body = site + 'download.php?id={0}&name={0}.torrent' announce = site + 'announce.php?' # URL . webui_url = 'http://127.0.0.1:{0}/gui/'.format(ut_port) webui_token = webui_url + 'token.html' # .torrent . settings.dat, dir_torrent_files. torrent_path = 'c:/utorrent/torrent/' # . autoload_path = 'c:/utorrent/autoload/' # uTorrent'a ( resume.dat) sys_torrent_path = 'c:/users/myname/appdata/utorrent/' def authentication(username, password): data = {'username': username, 'password': password} headers = {'Content-type': 'application/x-www-form-urlencoded', 'User-agent':'Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.6'} resp, login = http.request(login_url, 'POST', headers=headers, body=urlencode(data)) # , cookiekeys = ['uid', 'pass', 'PHPSESSID', 'pass_hash', 'session_id'] split_resp = resp['set-cookie'].split(' ') lst = [] # . for split_res in split_resp: if split_res.split('=')[0] in cookiekeys: lst.append(split_res) cookie = ' '.join(lst) return {'Cookie': cookie} def torrentDict(torr_path): #torr_path - resume.dat . Dict = {} with open(u'{0}resume.dat'.format(torr_path), 'rb') as resume: t = bdecode(resume.read()) for name in t: if name != '.fileguard' and name != 'rec': for tracker in t[name]['trackers']: if isinstance(tracker, str) and tracker.startswith(announce): Dict[name.split('\\')[-1]] = bta(t[name]['info']) return Dict Now we have a dictionary with names and hashes of distributions. Now we can only send scrape-requests with a substituted and modified hash and check whether there is a distribution with such a hash on the tracker or it is no longer there. Also, do not forget that you need to make such a request on behalf of the client, otherwise the tracker will refuse access.
uthead = {'User-Agent':'uTorrent/2210(21304)'} # uTorrent'. main_dict = torrentDict(sys_torrent_path) for key in main_dict: lst = [] for i in range(0, len(main_dict[key]), 2): lst.append('%{0}'.format(main_dict[key][i:i+2].upper())) scrp_str = ''.join(lst) # , . resp, scrp = http.request('{0}{1}'.format(scrape_body, scrp_str), 'GET', headers=uthead) The usual response to a query is:
d5:filesd20: aaaaaaaaaaaaaaaaaaaa d8:completei 5 e10:downloadedi 50 e10:incompletei 10 eeeeIf the distribution does not exist, the response to the request takes the form:
d5:filesdeeThe response to the request is also presented in bencode format, but we do not need to decrypt it, you can simply compare the resulting string with the string returned if there is no distribution on the tracker with such a hash.
Next you need to download our file from the tracker, put it in the client's startup folder and, if possible, delete the record about the outdated torrent in the client itself.
From the tracker just download the file does not work: need authorization. The function itself is described above under the heading “authentication”. And then we log in, download the file, put it in the startup folder and delete the old .torrent file from the folder with the torrents.
# "for key in Dict:". with open('{0}{1}'.format(torrent_path, key), 'rb') as torrent_file: torrent = bdecode(torrent_file.read()) t_id = torrent['comment'][36:] # . brhead = authentication(username, password) resp, torrent = http.request(torrent_body.format(t_id), headers=brhead) with open('{0}.torrent'.format(t_id),'wb') as torrent_file: torrent_file.write(torrent) # .torrent . remove('{0}{1}'.format(torrent_path, key)) move('{0}.torrent'.format(t_id), '{0}{1}.torrent'.format(autoload_path, t_id)) # . . authkey, token = uTWebUI(ut_username, ut_password) webuiActions(main_dict[key], 'remove', authkey, token) So that a non-existent .torrent file does not confuse us with its record in the client, it should be removed from the client. But uTorrent is designed in such a way that editing resume.dat, namely, information about all the torrents is stored, when the client is running, it will not give the result: uTorrent will restore resume.dat the way it remembers at startup. Therefore, for such a case, you need to constantly turn off uTorrent, edit resume.dat, turn on uTorrent. Such a method would be suitable for one changed distribution per day, and what if the distribution changes in batches, i.e. several at once? At first, being far from programming as a whole, I thought that I would have to work with processes directly, which is very difficult for me. But then I learned about the existence of uTorrent WebUI. WebUI has an API, the documentation for which is on the official site . Thanks to the capabilities of the WebUI API, you can delete the record, and not only delete the torrent from the client. First we need to get a cookie with a special password and token. The second we need if the parameter webui.token_auth in the client is activated.
def uTWebUI(ut_name, ut_passw): # cookie token. passmgr = urllib2.HTTPPasswordMgrWithDefaultRealm() passmgr.add_password(None, webui_token, ut_name, ut_passw) authhandler = urllib2.HTTPBasicAuthHandler(passmgr) opener = urllib2.build_opener(authhandler) urllib2.install_opener(opener) req = urllib2.Request(webui_token) tkp = urllib2.urlopen(req) page = tkp.read() token = doc(page).xpath('//text()')[0] passw = req.unredirected_hdrs['Authorization'] return passw, token def webuiActions(torrent_hash, action, password, token): head = {'Authorization': password} if action == 'remove': # . action_req = '?token={0}&action=remove&hash={1}'.format(token, torrent_hash) r, act = http.request(webui_url+action_req, headers=head) In uTorrent, the authorization in the web-interface is not implemented in the same way as on the site, so simple data sending will not work. Then we get the token and together with it we perform some function in the client. Of course, it would be possible to allocate a class for actions in the client, but I considered that the usual function would be enough for that.
(Note: Unfortunately, my knowledge at the moment was not enough to properly log in to the web interface, so I used the method described on the Internet.)
What is the result
As a result, I received a script that satisfies my need, a little knowledge and a lot of pleasure: it is very fun to sit on the code until the morning, and then, when you lie down to sleep, understand what the snag was.
I hope this method will be able to help someone.
UPD: I wildly apologize for my carelessness: I brought the code into a more readable form before publication, as a result of which I myself got confused and confused you.
The code is uploaded to Github . I work with him for the first time, so if I did something wrong, contact me.
Source: https://habr.com/ru/post/135874/
All Articles