We add informativeness to RSS feeds from RuTracker.org

Introduction

In my first topic, I wrote how to organize downloading of selected hands from a beloved torrent tracker, without being distracted from reading the news. In this topic, overkill will be described , a method that allows us to further simplify the process of obtaining information about each particular distribution that came to us in the form of an RSS feed. As you know, RuTracker.org itself publishes in the RSS feed only links to the relevant forum threads. No description of the distribution element of the tape does not contain, but I would like otherwise.

Anticipating objections in the spirit of “Stop reinventing the wheel, better get ready-made RSS. For example, from xpoft.ru ”(thanks to egor_bulychev for the tip; without sarcasm, really, thanks), I would say this: the tool is interesting, smart, the tape with the desired content is formed in a few mouse clicks. But, unfortunately, the RSS feeds generated by him are not encouraging by the constancy of the content of the elements: then comprehensive information about the distribution will come, and then only the title, and the poster. Moreover, the dependence of the contents of the tape on the actual description of the distribution on RuTracker.org was not identified. This layout does not suit me: I want to read the description, and view the media-info with screenshots, without even going to the original description page.

Immediate task

Get a description of the distribution from the post body on the torrent tracker and add it to the corresponding element of the RSS feed. The resulting tape to give in the format of RSS. At the entrance we have a rather dull and uninformative tape:
')

Instruments

The composition of the tools will be almost the same as in the previous article, with the exception of the torrent client.

• Yahoo Pipes for RSS manipulation;
• Google Reader account (any other RSS reader, however);
• Of course, time.

Decision

In total, we will need three pipes to solve the problem:

• URL converter to the name of a post on the forum of the torrent tracker;
• "Pipe-Ripper" for the extraction of useful information from the topics;
• The main pipe processing the source RSS feed.

You can create them manually and well pump the skill of using Yahoo Pipes, you can quickly and painlessly clone mine and finish them for yourself, you can (for the most lazy) use the pipes I created without any changes. Everyone, except those who like the first option, can safely skip the following three points and read the instructions for use and links immediately. For those who dislike simple ways, I will talk about everything in order.

Pipe # 1: URL Converter to Post Name

It would have been possible to do without this pipe if it were not for one feature of the work of Yahoo Pipes with Cyrillic character sets. I will not speak for everything, but the RSS feed from RuTracker.org in the process of processing turns into a completely unreadable abracadabra. To get a normal distribution name, we need this pipe. Directly in Yahoo Pipes there is no tool for working with encodings, so you have to work around the workaround.



Everything is simple here: we request using the URL Input block (surprise!) The URL of the topic, using the Fetch Page block we pull out the distribution name from the tag

  <title> </ title> 

and remove all unnecessary from it using the Regex block and the template

  <title> (. +) ::. + 

Link to the finished pipe - http://pipes.yahoo.com/ivopupkin/url2nameconv

Pipe # 2: Pipe The Ripper

It does the main job of extracting useful information from the topic-description of the distribution.



The most difficult thing is to choose the right boundaries of useful information and discard too much. Otherwise, everything is just as simple: a block for entering the URL of the topic, passing it to the Fetch Page block. The boundaries of useful information for myself, I identified the following:

  <div class = "post_body" id = " 

and

  <legend> Download </ legend> 

respectively, the beginning and the end. They almost match the boundaries of the first post in the forum thread

On this would go to the next stage, but, because The topic markup is quite specific, in this form it is of little value: the browser will show us only the text. For the normal display of pictures and serves the last Regex-block, replacing the markup RuTracker.org

  <var class = "postImg" title = "([^"] +) "> 

on a clear browser without additional dancing with a tambourine

  <img src = "$ 1"> 

for the bulk of the images and with

  <var class = "postImg. +" title = "([^"] +) "> 

on

  <img src = "$ 1"> 

for the title image.

Link to the finished pipe - http://pipes.yahoo.com/ivopupkin/pipheripper

Pipe №3: Basic

He will do all the dirty work for us. Namely: request the URL of the source tape, get a list of its elements, execute child pipes, finally form the names of the elements and create the resulting tape.


With the first two blocks, everything is clear and without explanation: the URL Input asks the user for the URL of the RSS feed, and the Fetch Feed loads the feed itself.


The next Rename block is used to preliminarily form the “Updated” attribute for distribution and a separate field with distribution size. This is needed to bypass the problems with the encoding of the original RSS feed described above.


Then, using the Regex block, we perform several actions at once (pay attention to the fields in which replacements take place):

• From the “cropped” link to the topic with the distribution, we get a full URL by replacing the text using the pattern (.+) With the string http://rutracker.org/forum/$1 http://rutracker.org/forum/$1 ;
• Delete unnecessary information from the item.isupdated field, replacing the abracadabra ^(\[Îáíîâëåíî\])*.+ template $1 ;
• Let it be crooked, but we will overcome the problem of transcoding characters by performing a replacement following the pattern \[Îáíîâëåíî\] on the text \[\] ;
• We obtain the size of the distribution in a separate field by replacing .+\[(.+)\]$ With $1 .

The Loop blocks and the corresponding child pipes (as an input parameter, telling them the value of item.link ) update the header and content for the whole feed. The result is stored in the item.title and item.description fields, respectively.


In the next Loop + String Builder block, we finally create the tape item header, which contains information about the distribution update and its size. Save it in some field. For example, in item.title.content .


Finally, from this pile of data that we have accumulated by this moment, we are forming a slender RSS feed using the Create RSS block, the output of which is linked to the Pipe Output.

Link to the finished pipe - http://pipes.yahoo.com/ivopupkin/rutrackerrssexpander

Instructions for use

It's as simple as two and two:

• Open the main pipe;
• We feed him a link to the tape with RuTracker.org;
• Run the pipe for execution;
• Using the Get as RSS button, we get a link to the RSS feed with the desired distribution information.

The content, of course, is a little disheveled, but still suitable for evaluating hands. Reading, as you may have guessed, is possible in any RSS reader.
If you wish and have enough skill, you can pack several RSS feeds from RuTracker.org into one resulting tape. To do this, it is enough to create your own pair of URL Input and Fetch Feed blocks for each incoming tape, the result of which should be combined with the Union block. Further changes will not be required.

Instead of conclusion

I showed how using simple manipulations to transform an interesting, but something that does not suit us RSS-feed. Similarly, you can both edit existing RSS feeds and create your own based on data from arbitrary pages. Moreover, with the help of Yahoo Pipes, it is possible to turn elements that are on an arbitrary page into RSS-feed elements (for example, this way you can track the appearance of new topics on the forum or their updates). It is noteworthy that for this there is no need to have serious programming skills, as well as to configure and maintain the necessary infrastructure.

Links

• Yahoo pipes
• Main pipe
• URL Converter - the name of the post on the torrent tracker
• Pipe the ripper
• Tape Generator for RuTracker.org from xpoft.ru