Bulk recording from cameras on elections
Attention. Habr - not for politics. Please refrain from discussing this in the comments.
In anticipation of the first elections in Russia, during which all the sites were equipped with webcams, many expressed a desire to record for themselves video from the camera. A variety of solutions were proposed for this, from FRAPS recording, to using ffmpeg, and so on. The most successful, in my opinion, was the utility Qwertovsky , laid out here
In this small topic, I would like to offer my solution and briefly recall how the whole system works at all, since tomorrow there will be parliamentary elections in fraternal Ukraine, the progress of which anyone can watch on the site vybory2012.gov.ua .
So, what you should know:
For each camera, every 15 seconds, a fresh playlist is generated, which contains direct links to the last 4 pieces of video, the duration of each piece is 15 seconds. Accordingly, once a minute the playlist is fully updated, and chunks continue to be available for some time.
The playlist is duplicated on several servers, available from the link of the form http: // server /variant.m3u8?cid= uid & var = orig and it looks like this:
Link type /segment.ts?cid= f0ffd596-aaa6-4601-9432-70d717dd666a & var = orig & ts = 1351335728.24-1351335741.20 in this case refers to a piece of video f0ffd596-aaa6-4601-9432-70d717dd666a chamber and with an interval of 1351335728.24 several 1351335741.20 complicated unix time format.
Periodically, once a minute, playing out the playlist and downloading all the available pieces will result in maximum information from the camera, except for those moments when the Internet access disappears from the camera side. For example, something like this:
We run this script, indicating to it the folder for storing chunks and, optionally, the address of the server of the system, where all current information will be poured.
However, we ignored one thing - where to get the source data for playlists, namely the server and the camera id? It is worth making a small digression. This system was successfully used in elections in Russia, in particular, I was using the above-mentioned python script to record all the cameras in my city. However, since then there have been some changes in the engine. Previously, all the necessary data could be obtained without even logging in. Now, you first need to add the camera to your favorites, and then at / account / channels? Station_id = cid (where cid is the camera id), the file with camera hashes and server ip addresses will be available. Somewhere around two o'clock in the afternoon I decided to put together a full database of hashes and put it into public, but the servers already now, even before the main load, periodically throw out 502 errors, which complicates the process =) Of the 32,183 plots, the hashes are available from about 5000, and a little this figure is increasing. Current data
In principle, the current data can be obtained by opening the desired camera and executing CorePlayer.instances.core_player_1.source.origin in Firebug or its equivalents, but only the current server will turn out (it will fall off, judging by the experience of the Russian elections).
')
At the moment, stubs are shown on the cameras, apparently the Ukrainian colleagues took into account the Russian experience and decided not to show the everyday life of schools, libraries and dormitories.
UPD 7:50 MSK: Collected data on 99% of cameras. I went to the picture from the cameras, the script successfully works.
In anticipation of the first elections in Russia, during which all the sites were equipped with webcams, many expressed a desire to record for themselves video from the camera. A variety of solutions were proposed for this, from FRAPS recording, to using ffmpeg, and so on. The most successful, in my opinion, was the utility Qwertovsky , laid out here
In this small topic, I would like to offer my solution and briefly recall how the whole system works at all, since tomorrow there will be parliamentary elections in fraternal Ukraine, the progress of which anyone can watch on the site vybory2012.gov.ua .
So, what you should know:
For each camera, every 15 seconds, a fresh playlist is generated, which contains direct links to the last 4 pieces of video, the duration of each piece is 15 seconds. Accordingly, once a minute the playlist is fully updated, and chunks continue to be available for some time.
The playlist is duplicated on several servers, available from the link of the form http: // server /variant.m3u8?cid= uid & var = orig and it looks like this:
#EXTM3U #EXT-X-MEDIA-SEQUENCE:6885 #EXT-X-TARGETDURATION:15 #EXT-X-ALLOW-CACHE:NO #EXT-X-PROGRAM-DATE-TIME:2012-10-27T11:02:08Z #EXTINF:13, /segment.ts?cid=f0ffd596-aaa6-4601-9432-70d717dd666a&var=orig&ts=1351335728.24-1351335741.20 #EXTINF:11, /segment.ts?cid=f0ffd596-aaa6-4601-9432-70d717dd666a&var=orig&ts=1351335741.20-1351335752.28 #EXTINF:11, /segment.ts?cid=f0ffd596-aaa6-4601-9432-70d717dd666a&var=orig&ts=1351335752.28-1351335763.40 #EXTINF:14, /segment.ts?cid=f0ffd596-aaa6-4601-9432-70d717dd666a&var=orig&ts=1351335763.40-1351335776.92 #EXTINF:11, /segment.ts?cid=f0ffd596-aaa6-4601-9432-70d717dd666a&var=orig&ts=1351335776.92-1351335788.36 Link type /segment.ts?cid= f0ffd596-aaa6-4601-9432-70d717dd666a & var = orig & ts = 1351335728.24-1351335741.20 in this case refers to a piece of video f0ffd596-aaa6-4601-9432-70d717dd666a chamber and with an interval of 1351335728.24 several 1351335741.20 complicated unix time format.
Periodically, once a minute, playing out the playlist and downloading all the available pieces will result in maximum information from the camera, except for those moments when the Internet access disappears from the camera side. For example, something like this:
# -*- coding: utf-8 -*- # vybory2012 (Proof of concept), yegorov-p import urllib import os from time import strftime, localtime, sleep import socket import threading # syslog, syslog_server='127.0.0.1' syslog_port=514 # , directory='dumps' # id , , cams=[ ['563-1', "f0ffd596-aaa6-4601-9432-70d717dd666a",["82.207.0.3","82.207.0.3","82.207.0.3"]] ] # LEVEL = { 'emerg': 0, 'alert':1, 'crit': 2, 'err': 3, 'warning': 4, 'notice': 5, 'info': 6, 'debug': 7 } #, . , =( def syslog(message, level=LEVEL['notice'], host=syslog_server, port=syslog_port): sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) data = '<%d>%s' % (level + 24, message) sock.sendto(data, (host, port)) sock.close() # def cam_rip(num,hash,servers): syslog('Recording cam %s at %s'%(hash, num), level=LEVEL['info']) try: os.mkdir('./%s/%s'%(directory,num)) except: pass # while 1: try: server=servers[0] # page = urllib.urlopen("http://%s/variant.m3u8?cid=%s&var=orig"%(server,hash)).read() # , if '/segment' in page: for i in page.split('\n'): if '/segment' in i: filename=strftime("%d-%b-%H-%M-%S", localtime(int(i[-13:-3]))) f=open('./%s/%s/%s.ts'%(directory,num,filename),'wb') #syslog('Chunk %s saved'%(filename), level=LEVEL['notice']) f.write(urllib.urlopen("http://%s%s"%(server,i)).read()) f.close() else: # syslog('No signal!Rotating server on cam %s at %s'%(hash,num), level=LEVEL['err']) servers.append(servers[0]) del servers[0] sleep(60) except Exception,e: syslog('Error on cam %s: %s'%(hash,e), level=LEVEL['err']) servers.append(servers[0]) del servers[0] try: os.mkdir(directory) except: pass # for i in cams: threading.Thread(target=cam_rip, kwargs={"num": i[0],"hash": i[1],"servers": i[2]}).start() sleep(1) syslog('System started.', level=LEVEL['notice']) We run this script, indicating to it the folder for storing chunks and, optionally, the address of the server of the system, where all current information will be poured.
However, we ignored one thing - where to get the source data for playlists, namely the server and the camera id? It is worth making a small digression. This system was successfully used in elections in Russia, in particular, I was using the above-mentioned python script to record all the cameras in my city. However, since then there have been some changes in the engine. Previously, all the necessary data could be obtained without even logging in. Now, you first need to add the camera to your favorites, and then at / account / channels? Station_id = cid (where cid is the camera id), the file with camera hashes and server ip addresses will be available. Somewhere around two o'clock in the afternoon I decided to put together a full database of hashes and put it into public, but the servers already now, even before the main load, periodically throw out 502 errors, which complicates the process =) Of the 32,183 plots, the hashes are available from about 5000, and a little this figure is increasing. Current data
In principle, the current data can be obtained by opening the desired camera and executing CorePlayer.instances.core_player_1.source.origin in Firebug or its equivalents, but only the current server will turn out (it will fall off, judging by the experience of the Russian elections).
')
At the moment, stubs are shown on the cameras, apparently the Ukrainian colleagues took into account the Russian experience and decided not to show the everyday life of schools, libraries and dormitories.
UPD 7:50 MSK: Collected data on 99% of cameras. I went to the picture from the cameras, the script successfully works.
Source: https://habr.com/ru/post/156417/
All Articles