Enhance your video surveillance system using OpenCV and Telegram bot

How it all began

It all started with the fact that I wanted to install a “smart” video surveillance system on Raspberry.

I want to separately note that for this I used several articles on Habré. Thanks to the authors for their posts. They really helped.

As a result, I installed a Logitech USB camera on a purchased Raspberry Pi3, mounted Yandex.Disk and, at intervals of 30 seconds, took pictures, which I then copied to a folder on Yandex.Disk.
Having played with the further archiving of files, mounting video from individual snapshots, I abandoned a new “toy” for several months.

Continuation of a story

While Raspberry was taken out of production, I wondered how to get rid of a large number of images that would be accumulated on Yandex.Disk when the solution was working. I wanted to optimize the implemented solution ...
')

Idea

A little googling worked out such an improvement: copy only those images on Yandex.Disk, in which the presence of a person will be revealed or the image in the image will be changed, for example, first the door is closed on the image and then opened. At the same time, send a notification when such events are detected using Telegram. Telegram was chosen instead of mail, because provides delivery of messages in real-time mode, plus, it is fashionable.
I decided to develop it in Python and bash.

Implementation

To detect the presence of a person, the OpenCV library was chosen, which among other things, is able to determine the presence of an object in a picture, for example, a person's face. Also, this library offers the implementation of a large number of Machine Learning algorithms.

Sample Python code below. I used one of the ready-made xml to determine the face (haarcascade_frontalface_default.xml).

import cv2 import sys imagePath = sys.argv[1] cascPath = sys.argv[2] faceCascade = cv2.CascadeClassifier(cascPath) image = cv2.imread(imagePath) gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) faces = faceCascade.detectMultiScale( gray, scaleFactor=1.3, minNeighbors=5, minSize=(30, 30), flags = cv2.cv.CV_HAAR_SCALE_IMAGE ) if(len(faces)>0): print("Found {0} faces!".format(len(faces))) else: print("Not found") 

Identifying changes in the snapshot turned out to be not a completely trivial task, besides on Raspberry, I want to note that I have the latest version, Pi3, it takes time to identify the difference (changes) for the two snapshots. Rummaged on the Internet, found several different methods.

At once I will make a reservation that I used ready algorithms in Python, I modified them only a little, if necessary. Therefore, the lead codes did not.

1. I started with the subtract function from the OpenCV library.

 diff=cv2.subtract(img2,img1) 

2. I tried to determine the Euclidean and Manhattan distance using scipy.
3. Played with the PIL library.

Having tried these options, I stopped at determining the difference in file size between two images as a percentage. Implemented this “approach” in a shell script. This method was the most productive of the tested, which is not surprising. Of course, there is no need to talk about high precision yet, but since my cron scheduler is set to take periodic surveys, which is performed every 30 seconds, I found myself to spend an extra 7-8 seconds to detect the difference, considered it too wasteful. My face definition already “eats up” for about 5 seconds (taking into account the low resolution of the shooting).

A small piece of shell script below.

 f1=`stat -c%s "$prev_file"` f2=`stat -c%s "$new_file"` if [[ $f1 -le $f2 ]];then top=$f1 base=$f2 else top=$f2 base=$f1 fi percentage=$(( (100-(100*top / base)) )) echo "Difference is $percentage%" if(($percentage >= $hurdle));then changed=0 echo "Big change" else echo "Small change" fi 

Telegram integration

Integration with Telegram is already well described. I completed the necessary steps, registered a bot, and received a token.

Then I wrote a small Python script that sends me a message from the telegram bot created. A message is one of the parameters of the script that I specify when calling it.

 import sys import telepot bot = telepot.Bot('###############################') #bot.getMe() #from pprint import pprint #response = bot.getUpdates() #pprint(response) #bot.sendMessage(########, 'Alarm from Telegram bot!') bot.sendMessage(########, sys.argv[1]) 

Algorithm of the decision

The solution algorithm is implemented on a shell script that sequentially performs a survey, then analyzes whether there are changes in the received images, detects the face in the picture and sends a message from the created Telegram bot with a specific message if a change is detected or a face is detected. If a person or change is identified, the file is copied to Yandex.Disk.

Below is only a part of the script that reflects the calls of python scripts.

 #! /bin/bash #... face=$(python face_detect.py "$new_file" haarcascade_frontalface_default.xml 2>&1) echo "$face" if [[ $face != "Not found" ]];then ffound=0 echo "Faces found" else echo "Faces does not found" fi echo "----------" echo "Changes: $changed" echo "Faces found: $ffound" echo "----------" #=====================Processing========================================= now=$(date +"%Y-%m-%d_%H%M") # 1. Copy if [ $changed == 0 ] || [ $ffound == 0 ];then cp "$new_file" "/home/pi/webcam/$now---$new_file" \ && echo "File copied." else echo "All Ok." fi # 2. Rename cd /home/pi \ && mv "$new_file" previous.jpg \ && echo "File renamed." # 3. Send telegram nessage if [ $changed == 0 ] && [ $ffound == 0 ];then python send_telegram.py 'Alarm: Changes & Faces detected!' \ && echo "Alarm: Changes & Faces detected!" elif [ $changed == 1 ] && [ $ffound == 0 ];then python send_telegram.py 'Alarm: Faces detected!' \ && echo "Alarm: Faces detected!" elif [ $changed == 0 ] && [ $ffound == 1 ];then python send_telegram.py 'Alarm: Changes detected!' \ && echo "Alarm: Changes detected!" else echo "Nothing to send." fi #... exit 0 

results

Below I present some results of the solution.

1. The results of the identification of persons in the photo. I did not apply my photos.




2. Messages from Telegram bot.


3. Photos on Yandex.Disk

Conclusions and Next Steps

Testing a new solution revealed a number of problems:

Face detection and change control are things that require fine tuning. I still have to tinker with this, so that, for example, my bot does not spam me with messages about the discovery of faces that a normal person would never think of as a person.

In general, it remains quite a bit to run the idea in production.

1. install an infrared camera;
2. choose reliable methods of change control;
3. and of course, adjust the settings for detecting faces in the pictures.

Further research / testing solutions will be continued ...