We transfer content distribution to BitTorrent

Under the cat, an example of transferring file servers to BitTorrent is described.

My friend has a small local area network, there are 200-300 subscribers, actually a regular local area network. Several game servers, several file servers, and a string of uncontrolled switches. Typical Pioneer.Net start-mid zero. Everything works, everyone is happy. But with the increase in the number of subscribers, file servers had problems with content delivery. After much deliberation and other experiments, it was decided to switch to bittorrent. And now, for almost a year and a half, there have been no problems with servers, and my friend does not cease to be happy about it.
What exactly should be the server, which system to choose, how many disks to install, whether to use raid, lvm, which file system to prefer - each administrator has his own work. Personally, I prefer raid5 of 5-6 hard drives and xfs, the performance obtained from such a bundle completely suits me.
Wikipedia Overview: BitTórrent is a peering (P2P) network protocol for cooperative file sharing over the Internet. Files are transferred in parts, each torrent-client, receiving (downloading) these parts, at the same time gives (downloads) them to other clients, which reduces the load and dependence on each source client and ensures data redundancy. Before starting the download, the client connects to the tracker at the address specified in the torrent file, informs it of the address and the hash sum of the torrent file, to which the client receives the addresses of other clients downloading or distributing the same file.
Let's start with the tracker, I chose opentracker. Easy to install and configure, undemanding to resources.

fs:~# apt-get install gcc make cvs
fs:~# cvs -d :pserver:cvs@cvs.fefe.de:/cvs -z9 co libowfat
fs:~# cd libowfat/
fs:~/libowfat# make
fs:~/libowfat# cd ..
fs:~# cvs -d:pserver:anoncvs@cvs.erdgeist.org:/home/cvsroot co opentracker
fs:~# cd opentracker/
fs:~/opentracker# make
fs:~/opentracker# cp opentracker /usr/local/bin/
fs:~# cat /etc/rc.local |grep opentracker
start-stop-daemon --start --quiet -m -b --pidfile /var/run/opentracker.pid --exec /usr/local/bin/opentracker

In the tracker settings, I allowed the registration of any torrent files, so I wanted, and so it is easier.
I chose btpb as a client. Again, easy to install and configure, very undemanding to resources and without any patches normally works with a large number of files.

fs:~# wget www.murmeldjur.se/btpd/btpd-0.15.tar.gz
fs:~# tar -xf btpd-0.15.tar.gz
fs:~# cd btpd-0.15
fs:~/btpd-0.15# chmod +x configure
fs:~/btpd-0.15# ./configure
fs:~/btpd-0.15# make
fs:~/btpd-0.15# make install
fs:~# cat /etc/rc.local |grep btpd
/usr/local/bin/btpd -d /root/.btpd

Generate torrent files will be through ctorrent
fs:~# apt-get install ctorrent

Actually now we have a torrent tracker, a torrent client and a bunch of files that need to be distributed to users.
Suppose, in addition to a heap of files, there is also a directory with a beautiful interface and a database where all the information we need is stored.
In my case, there was a file label in which the file path and a unique file id were stored.
To control the generation of torrent files, I created an additional table torrents.
')
mysql> desc torrents;

+--------------+---------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+---------------+------+-----+---------+----------------+
| id | int (16) | NO | PRI | NULL | auto_increment |
| file_id | int (16) | YES | | NULL | |
| is_torrent | enum( '1' , '0' ) | YES | | 0 | |
+------------+---------------+------+-----+---------+----------------+
3 rows in set (0.01 sec


and 3 triggers

mysql> show triggers;

+-----------+--------+-------+------ --
| Trigger | Event | Table | Statement | Timing |
+-----------+--------+-------+------ --
| on_insert| INSERT |files| insert into torrents (file_id) values (LAST_INSERT_ID())| AFTER
| on_update| UPDATE |files| update torrents set is_torrent= '0' where file_id= NEW .id | AFTER
| on_delete| DELETE |files| delete torrents.* from torrents where torrents.file_id= old .id | AFTER
+-----------+--------+-------+--------


The table stores data from torrent files, and thanks to the triggers, the torrent file is created, regenerated, or deleted.

Now the script itself controls the status of torrent files.
fs:~# cat add_torrent.sh


#!/bin/bash
#
tracker= "http://tracker_domain:6969/announce"
#
video_home= "/home/video" ;
#
torrent_dir= " /var/www/torrents" ;

ctorrent= "/usr/bin/ctorrent" ;
btcli= "/usr/local/bin/btcli" ;
mysql= "/usr/bin/mysql -pdpass -u duser -D video" ;

# id
for i in `echo "select files.id from files,torrents where torrents.file_id=files.id and torrents.is_torrent='0' limit 1;" |${mysql}|sed 1d`
do
id=${i};
#
path=`echo "select files.path from files where files.id=" ${id} " limit 1;" |${mysql}|sed 1d`
#
torrent_file=${torrent_dir}/${id}.torrent
#
if [ -e ${torrent_file} ]
then
[ `${btcli} list|awk '{print $1}' |grep -w ${id}|wc -l` -ne 0 ] && ${btcli} del ${torrent_file}
rm ${torrent_file}
fi
cd ${video_home}
path1=`dirname "${path}" `
#
${ctorrent} -t -u ${tracker} -s ${torrent_file} "${path}" && echo "update torrents set is_torrent='1' where file_id=" ${id}|${mysql}
#
[ -e ${torrent_file} ] && ${btcli} add -d "${path1}" -n ${id} --topdir ${torrent_file}
done

#
ls ${torrent_dir}|sed 's/.torrent//' |sort -n > /tmp/tor_list.files
${btcli} list|awk '{print $1}' |sed 1d|sort -n > /tmp/tor_list.load

diff_list=(`diff /tmp/tor_list.files /tmp/tor_list.load|grep "<" |awk '{print $2}' `)
diff_num=$((${#diff_list[*]}-1))
for i in `seq 0 $diff_num`
do
id=${diff_list[${i}]};
path=`echo "select files.Path from files where files.id=" ${id} ";" |${mysql}|sed 1d`
torrent_file=${torrent_dir}/${id}.torrent
${btcli} del ${torrent_file}
if [ -z ${path[0]} ]
then
rm ${torrent_file};
else
path=`dirname "$path"`
${btcli} add -d ${path} -n ${id} --topdir ${torrent_file}
fi
done


As can be seen from the listing, the script generates torrent files, if the torrent already existed will be regenerated if necessary, it also ensures that all generated torrents are added to the client.
For normal operation of the system, it is enough to run this script once for 2-3 minutes.
The only thing that is not rational is the removal of torrent files and content when deleting records about it from the database.
I will not be afraid to repeat this system has been working for a year and a half and there are no problems, the only thing I recommend in the names of files and folders is not to use Cyrillic, anything can happen.