📜 ⬆️ ⬇️

Video surveillance from idea to ... idea

I have a main job, there are friends who always want to "start a new business." They have their own, but always want to expand. And every time they want it, they come to me for advice. The idea lies on the surface. "We are engaged in the Internet, and let's do something else on our topic." So we began to engage in telephony, the sale of network equipment, hosting, kolokeyshinom. Problem statement always pleased me. "Let's do telephony," "Let's do hosting." Well, the last thing was "Let's do video surveillance."
Here "Let's do video surveillance" this was the cherished TK.
What came out of it and experience I will describe in this article.

It so happened historically that I work in organizations where I am surrounded by salespeople, and I am a technical specialist, Dartanyan , as it turned out (over time) with a broad profile.
During my professional career I built not one service-technical department, but I did not reach the level of educated (programmers) people. By this, I am always in sight, all questions on IT to me. I have been advising them (acquaintances) for many years in the field of IT, being a systems and business analyst who is free :). Now to the topic.

One day people came to me and said, "we want video surveillance." I was busy and there was no desire to develop in this direction; then, even without this, there was enough care. They found another “specialist”, worked for 6 months, it ended with nothing. The man turned out to be “beautiful”, he sang about POWER of zoneminder and motion, then everything will be cloudy. For 6 months, not a single solution, not a single client.
They are tired, they came to me. Since I have been working with them for about 10 years, then why not take part in the adventure again. The offer, as always, was captivating with its originality and novelty:
- We want to do cloud surveillance. Investment 0, buy equipment, for the development of 0, divide 50/50 money.
By the way, it was always like this, and money was always 50/50, or 30/30/30, depending on the number of people involved.
“Why not,” I thought. Experience will not be superfluous. (although I am confused by this experience, if you are a generalist and change this profile once a year :)

The idea is voiced: "I want cloud video surveillance."


People who want to sell something, they will always be selling something.
It remains to come up with this "Something" and arrange.
')
Cloud. 0 money, the load is large. So you need to come up with a prototype that will solve these problems.
At first glance fell on Openstack. You can expand computing power on the fly, but it takes several machines to start. Having played with different versions of Openstack, I had to find a more suitable solution. Glance fell on Proxmox.
A wonderful solution that fit our needs at the start. And always those VMs can be ported to hardware or another cloud.
I did not have time to take part in the purchase of iron, before my appearance the workhorse was acquired: Intel Core i5-3570K @ 3.40GHz
And here on this gland it was necessary to be engaged in cloudy video surveillance. Absurd situation :) But we do not give up.
It comes up, some kind of sales system. Video surveillance coupled with security systems and all sorts of alerts with a tie to the chopy (there is an exit at the state level).
This is stuck in the future
1) permanent video recording
2) motion recording
3) timelapse recording
4) SMS notification
5) a call from the security center for motion detection. The operator checks the record, calls the chop, notifies the client.
PSC market in Moscow will expand from year to year. Mi Police every year reduces the amount of security business. So all this is gradually creeping on private traders.

The overall system looked like this:

image

Cameras come on any piece of hardware, it can be tied to security. This piece of hardware can be either a software client or an OpenWrt hardware solution. RPC is not just drawn. The idea was to create 2 modules. The first is completely console. This would allow it to be implanted into iron, the second graphic module that controls the console. This would allow to control a piece of iron from any place. From the user's computer, from the support computer, etc. Also, the first module is needed to standardize streams and freeze frames from cameras. Such glands we already run in. For the prototype fit, the experience was. No one forbade the logic of this piece of iron to be transferred to us on the server.

Now there are dances with a tambourine about how to drive the h264 zoo to disk, and attach detectors and similar crap to the web +, and fit it all into one i5.
ZoneMinder can be a good system, but after adding 4 cameras, it loaded all 4 cores of the system by more than 70%. From him immediately abandoned.

For the prototype took VLC. It has been developed for a long time, run in by many, we used it many times for some projects.
There are at least 2 tasks to solve.
1) issue to the network
2) record stream

The zoo of cameras consists of rtsp and http streams. VLC can do everything. We decided to use the VLM. Has control over tlenet and http.
The input feed is fed into the output.
"#{$transcode}std{access=http{mime={$this->mime}},mux=ts{use-key-frames},dst=*:{$this->getPort()}/{$this->path}}";
We get HTTP stream from any incoming url.
HTTP is convenient to us than? You can drive in hginx, distribute the load, encrypt. send to https.
1 client - 1 vlc process.
They decided to script the prototype in PHP, since the developer is one, time is short, funding 0.
Startup Team. (I will give the ready code, it is not difficult to get a command out of it if necessary)
 $vlc_ifs = "-I http --http-host=0.0.0.0 --http-port {$this->getHttpPort()} -I telnet --telnet-port {$this->getTelnetPort()} --telnet-password ".TLPWD; if(VLC_USE_LOG) $vlc_logs = "--extraintf=http:logger --file-logging --log-verbose 0 --logfile {$this->getLogFile()}"; else $vlc_logs = '--extraintf=http'; //$vlc_shell = VLCBIN." --rtsp-tcp --ffmpeg-hw --http-reconnect --http-continuous --sout-keep ".VLCD." $vlc_ifs --repeat --loop --network-caching ".VLCNETCACHE." --sout-mux-caching ".VLCSOUTCACHE." $vlc_vlm --pidfile {$this->getPidFile()} $vlc_logs \n"; $vlc_shell = VLCBIN." --rtsp-tcp ".VLCD." $vlc_ifs --repeat --loop --live-caching ".VLC_LIVE_CACHE." --network-caching ".VLCNETCACHE." --sout-mux-caching ".VLCSOUTCACHE." --sout-ts-dts-delay 400 $vlc_vlm --pidfile {$this->getPidFile()} $vlc_logs "; if($this->isValgrind()){ $vlc_shell = "valgrind -v --trace-children=yes --log-file={$this->getValgrindFile()} --error-limit=no --leak-check=full $vlc_shell"; } 


Since we already have a live stream, we can write from it.
return "#std{access=file{append},mux=ts{use-key-frames},dst=$filePath.avi}";

VLM is created simply
3 streams can be created on the camera
1) live stream
2) rec stream
3) motion stream
$this->execute("new {$this->cam} broadcast enabled loop");
after you can play, stop doing
$command = "control {$this->cam} $command";
for rec it is enough, for example, to stop every 10 seconds, change $ filePath, do play
We get a set of files with records. For motion play and stop.
VLC can also be run both "at us" and "at them".
Motion question.
There is a motion tool. He computes motion by Jpeg. Cameras give a shotframe in Jpeg. We take a freeze frame, we feed in motion, at the output we get what we need.

We have a zoo, we need to make a system out of it.
So we do. We call everything System and try to make a loosely coupled system so that in future we can replace the elements of the system without losses.
We have cameras, Vlc, Motion, recording to disk, online broadcasting.

Vlc and motion are unix processes. Let's call them demons. (abstract class Daemon)
It turns out that we need 2 streams from the camera. jpeg and h264.
In the system we create entities
user-dvr-cam-stream
user - user
dvr system that is responsible for daemons and cameras associated with demons
cam - camera entities, stores physical camera settings and virtual streams
stream - any abstract stream.

Straem can be started, stopped, disengaged, “pulled” at the time of the “update” of the system, for these events he himself decides what to do.

We get the System-common entity, which consists only of abstract declarations.

We receive the general factory of construction of system.
 abstract class AbstractFactory { private static $instance = null; /** * @var array */ private $commands; /** * @return AbstractFactory */ public static function getInstance(){ if(self::$instance == null) self::$instance = new static; return self::$instance; } /** * permanentCommand * @param ICommand $command */ protected function addPermanentCommand(ICommand $command){ $this->commands[] = $command; } /** * @param ISystem $system */ protected function addCommands(ISystem $system){ foreach($this->commands as $command) $system->addPermanentCommand($command); } //todo buildSystem method /** * @return ISystem */ public function createSystem(){ return System::getInstance(); } /** * @return array of Users */ public function createUsers(){ return array(AbstractFactory::getInstance()->createUser(1)); } /** * @param int $id * @return IUser */ public function createUser($id) { return new User($id); } /** * @param IUser $user * @return IDVR */ public function createDvr(IUser $user) { $dvr = new Dvr($user); $cams = $this->createCams($dvr); //new+add foreach($cams as $cam){ /** @var $cam Cam */ $dvr->addCam($cam); } $daemons = $this->createDaemons($dvr); foreach($daemons as $daemon){ /** @var $daemon Daemon */ $dvr->addDaemon($daemon); } return $dvr; } /** * @param DVR $dvr * @return array of Cams */ abstract protected function createCams(DVR $dvr); /** * @param DVR $dvr * @return array of Daemons */ abstract protected function createDaemons(DVR $dvr); /** * @param IDVR $dvr * @param ICamSettings $cs * @return ICam */ public function createCam(IDVR $dvr, ICamSettings $cs) { return new Cam($dvr, $cs); } /** * @param $from * @param $to * @return MoveToNfsCommand */ public function createMoveToNfsCommand($from, $to){ return new MoveToNfsCommand($from, $to); } /** * @param ICam $cam * @return ICamStream */ abstract public function createStream(ICam $cam); } 



This allows us to hang on the camera absolutely any streams and do with them what comes to our mind. And allows at least somehow control our zoo. (he is a prototype)

Since our workhorse is VLC, it begins to customize VLC for our system.
Create an abstract class VlcStream extends Stream, class LiveVlcStream extends VlcStream.
VlcStream is “friendly” with Vlm ($ this-> vlm = new HttpVlm ($ this-> getVlcName (), 'localhost', HTSTART + $ this-> cam-> getDVR () -> getID ());)
And allows you to handle start, stop
Further
abstract class VlcReStream extends VlcStream
VlcReStream - LiveVlcStream takes as input
class RecVlcStream extends VlcReStream - records, “moves” video from Ram to Nfs

And the class Vlc extends Daemon is created, it will be contained in the DVR entity.

DVR processes start and stop
 public function start() { $this->log(__FUNCTION__); $this->startDaemons(); $this->startCams(); } 

Starting the camera starts all stream related to the camera.
Those. We run unix processes Vlc, motion, do VLM via HTTP.
The stop occurs in the reverse order.
Instead of VLC, you can use any thing you like.

But we have video surveillance in the cloud. Means Sql and similar data storage systems are necessary.
The working system turned out like this:

Factory
 <?php /** * Created by PhpStorm. * User: calc * Date: 16.06.14 * Time: 10:56 */ namespace system2; /** * Class BBFactory * @package system2 */ class BBFactory extends AbstractFactory { /** * @return ISystem */ public function createSystem() { $system = parent::createSystem(); $e = new BBLogMotionEvent(Motion::EVENT_MOTION_START); $system->addEventHandler($e); $e = new BBLogMotionEvent(Motion::EVENT_MOTION_STOP); $system->addEventHandler($e); $e = new BBLogMotionEvent(Motion::EVENT_MOTION_DETECTED); $system->addEventHandler($e); $e = new BBLogMotionEvent(Motion::EVENT_CAMERA_LOSS); $system->addEventHandler($e); $recMotionEvent = new BBRecMotionEvent(Motion::EVENT_MOTION_START); $system->addEventHandler($recMotionEvent); $recMotionEvent = new BBRecMotionEvent(Motion::EVENT_MOTION_STOP); $system->addEventHandler($recMotionEvent); //   30    update //$system->addPermanentCommand(new RotateRecCommand()); $this->addPermanentCommand(new RotateRecCommand()); $this->addCommands($system); return $system; } /** * @param DVR $dvr * @return array of Daemons */ protected function createDaemons(DVR $dvr) { $vlc = new Vlc($dvr); $this->addPermanentCommand(new BBDaemonWatchdog($vlc)); //     ,    mtn $cams = $dvr->getCamIDs(); $ids = array(); foreach($cams as $id){ $cam = $dvr->getCam($id); $cs = $cam->getSettings(); /** @var $cs BBCamSettings */ if($cs->mtn) $ids[] = $id; } $motion = new Motion($dvr, $ids); if(count($cams)) $this->addPermanentCommand(new BBDaemonWatchdog($motion)); return array($vlc, $motion); } /** * @return array */ public function createUsers() { $users = array(); $db = \Database::getInstance(); $q = "select id from users where banned=0"; $r = $db->query($q); while(($row = $r->fetch_row())){ try{ $users[] = AbstractFactory::getInstance()->createUser($row[0]); } catch(\Exception $e){ Log::getInstance()->put($e->getCode().' '.$e->getMessage()."\n".$e->getTraceAsString()."\n", __CLASS__, Log::ERROR); } } return $users; } /** * @param DVR $dvr * @return array */ protected function createCams(DVR $dvr){ $db = \Database::getInstance(); $q = mysql::getQuery(mysql::CAM_SETTINGS, array('{dvr_id}' => $dvr->getID())); $r = $db->query($q); $cams = array(); while(($row = $r->fetch_object('system2\BBCamSettings')) != null){ /** @var BBCamSettings $row */ //$dvr->addCam(new BBCam($this, $row)); $cam = $this->createCam($dvr, $row); $cams[] = $cam; //    Motion     ,    $timelapse = new BBArchiveTimelapseCommand($cam); $this->addPermanentCommand($timelapse); } return $cams; } /** * @param ICam $cam * @return ICamStream */ public function createStream(ICam $cam) { $stream = new Streams($cam); $cs = $cam->getSettings(); /** @var $cs BBCamSettings */ $motion = new MotionStream($cam, $cs); $motion->setEnabled($cs->live && $cs->mtn); $stream->addStream($motion); $live = new BBLiveStream($cam); $live->setTestInputCommand(new BBTestInputFailSaveCommand($cam, $live)); $live->setEnabled($cs->live); $stream->addStream($live); $hls = new HLSVlcStream($cam, $live); $hls->setEnabled($cs->live); $stream->addStream($hls); //$this->streams[] = new FlvVlcReStream($this, $live); //nginx rtmp stream //$this->streams[] = new RtmpVlcReStream($this, $live); $rec = new BBRecStream($cam, $live); $rec->setEnabled($cs->live && $cs->rec); $rec->setTestInputCommand(new BBTestInputFailSaveCommand($cam, $rec)); $stream->addStream($rec); $mtn = new BBRecStream($cam, $live, TIME_LOCK_RECORD, Path::MOTION); $mtn->setEnabled($cs->live && $cs->mtn && BBRecMotionEvent::isMotion($cam)); $mtn->setTestInputCommand(new BBTestInputFailSaveCommand($cam, $mtn)); $stream->addStream($mtn, Path::MOTION); //motion flv stream $flv = new UrlFlvVlcStream($cam, "http://localhost:".(MOTION_STREAM_PORT + $cam->getID())); $flv->setEnabled($cs->live); $stream->addStream($flv); return $stream; } } 



createSystem, add events of interest to us. The System class is responsible for their processing.
EVENT_MOTION_START, EVENT_MOTION_STOP, EVENT_MOTION_DETECTED, EVENT_CAMERA_LOSS
this is just what is called from the moton config
thread
 # Command to be executed when an event starts. (default: none) # An event starts at first motion detected after a period of no motion defined by event_gap on_event_start php ~/dvr/bin/system2/main.php event {user_id} {cam_id} motion_start 0 "%Y-%m-%d %T" # Command to be executed when an event ends after a period of no motion # (default: none). The period of no motion is defined by option event_gap. on_event_end php ~/dvr/bin/system2/main.php event {user_id} {cam_id} motion_stop 0 "%Y-%m-%d %T" # Command to be executed when a picture (.ppm|.jpg) is saved (default: none) # To give the filename as an argument to a command append it with %f ; on_picture_save value # Command to be executed when a motion frame is detected (default: none) ; on_motion_detected php ~/dvr/bin/system2/main.php event {user_id} {cam_id} motion_detected 0 "%Y-%m-%d %T" # Command to be executed when motion in a predefined area is detected # Check option 'area_detect'. (default: none) ; on_area_detected value # Command to be executed when a movie file (.mpg|.avi) is created. (default: none) # To give the filename as an argument to a command append it with %f ; on_movie_start value # Command to be executed when a movie file (.mpg|.avi) is closed. (default: none) # To give the filename as an argument to a command append it with %f ; on_movie_end value # Command to be executed when a camera can't be opened or if it is lost # NOTE: There is situations when motion don't detect a lost camera! # It depends on the driver, some drivers dosn't detect a lost camera at all # Some hangs the motion thread. Some even hangs the PC! (default: none) on_camera_lost php ~/dvr/bin/system2/main.php event {user_id} {cam_id} motion_camera_lost 0 "%Y-%m-%d %T" 


there is a call to main.php event uid cid event_name parameters
The system class handles this.
By cron, once every 10 minutes (every time any amount of mine), System-update is called.
By update, update of all subsystems is performed and
 $this->executeCommands(); $this->executePermanentCommands(); 

Commands are what the sub-systems need, and permanent commands, such as RotateRecCommand, are responsible for deleting old entries.

When creating demons, an addition is being made, for example, BBDaemonWatchdog, it monitors whether the demon has fallen.

Users are already created from the database.
Then the camera from the database. A permanent command is added to create a time-lapse. Which, in turn, works only once in TIME_LOCK_TIMELAPSE (8 hours) and puts information into Mysql

The last is the most interesting, it is streams.
creates a stream of motion, live, hls (HTTP Live Stream), rec, mtn, flv.
live - usual http
rec, mtn - streams for recording
hls and flv for browser output.

Github: github.com/Calc86/dvr/tree/master/bin/system2

The logic is there. Need a "physics".
There are many cameras, many streams, one processor.
The entire entry is made in ramdisk, hls in ramdisk, the vlc setting goes this way so that there is no transcoding. The net flow was taken away, the net flow was saved. Moving from ram disk to storage (in the prototype was NFS) went through ffmpeg codec copy
This allowed to display in the browser through html5 records.
3 types of records. Full recording with cutting every 10 minutes, recording on the movement and timelaps recording for 8 hours, where per second is displayed 3 minutes.

The construction of the system was planned by the independent placement of the DVR (recording machines), even geographically with minimal code changes.

What was implemented
1) all records we want
2) output to the web, hls, flash, records in html5, because of the zoo h264 live codecs in h264 until you went to the web, output to mjpeg (allows you to look at old online video)
3) the web interface of this terrible thing with https, nginx security url and the like.
We stopped on adding cameras via Onvif

Why the project has stalled.
1) poor sales (lack of sales and sales system)
2) lack of funding (at least a admin is needed for such a system)
3) lack of free time
4) completely free development
5) no desire to purchase cameras for the test
6) there was no desire to put free cameras on the test, although there was an agreement with the district council.
7) I change my permanent job. It will not be possible to do this.
8) Purchase of design and purchase of sliced, but in the future, cut on their own.
9) The absence of a person who would constantly monitor the system (found bugs in both motion and vlc, but they are solved by administering the system)
10) Salesmen for the year did not come up with a single system of sales and pricing.
11) I did a lot of things in parallel with the main work, the creation of video surveillance and personal life.
12) Everyone scored a bolt on their agreed duties :)

But we had 3 clients, they liked it. They did not take money for the system, they took money only for the work on installing cameras.

What it looked like :)
image

If the topic goes, I will describe in more detail about VLC and pitfalls. And about the "mythical" piece of iron and experiments with it too.

Source: https://habr.com/ru/post/234139/


All Articles