Transition to cloud video analytics: problems and solutions

Probably, if Habra's spherical reader hears the word "cloud", he will not look into the sky. Today we decided to speculate about the future of video analytics. Make predictions "on the clouds," so to speak. If you are waiting for an article with technical descriptions and examples of algorithms - this is a little different. We share developer experience.

The future is in smartphones and mobile Internet, manufacturers understand this in every area. On the other hand, the use of video surveillance has also climbed into almost all areas of our life. In the public sector, these are safe cities, in large business, infrastructure monitoring, in retail chains and in the financial sector, means of sales management, in the private sector, property protection. But intellectual surveillance - video analytics - even if it is developing steadily, but still not massive. At best, in the cloud service, a motion detector is implemented, and then without the possibility of selecting protected areas or setting the schedule. What caused this lag? What prevents the adaptation of the classic video analytics system to the cloud service? And most importantly: what to do with it?

Lack of means of dynamic system scaling as the number of video streams or video analytics modules increases.
Conventional systems designed for a dedicated server do not have the functionality to balance the computational load within the cluster. In such systems, video streams are physically and logically tied to one specific server. To reconnect one video stream to another, the administrator is forced to re-configure both the parameters of the connection with the camera and the video analytics. In addition, in case of failure of one of the servers, the computational load is not distributed among the other servers in the cluster.
')

Lack of support for virtualization.

Video analytics software modules use HASP hardware security keys (especially for recognizing faces and license plates) and specialized graphics accelerators. This significantly limits their use in a virtualized environment, abstracted from the hardware implementation. Any software binding to physical hardware makes it difficult to scale the system in the cloud.

Lack of cross-platform client and media player.

Today, almost all video analytics systems work only with a fat client for the Windows operating system. At best, the system provides access through the Internet Explorer browser with the addition of non-standard components that use, for example, ActiveX technology, Java or SilverLight. This significantly limits the range of users of the system and creates a lot of problems for technical support. The ideal cross-platform client should be implemented based on HTML5 technology. In practice, all cloud-based video surveillance services rely on flash. At the same time, existing flash-players lack the ability to display metadata of video analytics in the form of graphic annotation.

The lack of implementation of a web-based architecture (WOA).

WOA implies that any meaningful data (video analytics event, some point in time in the video archive, etc.) has a unique hyperlink - like a video on YouTube. Most classic and even cloud-based video surveillance systems, unfortunately, do not allow creating such hyperlinks, which significantly complicates the exchange of data between users. Difficulties resemble a bureaucratic system: instead of simply sending a link to the selected moment by video via email, the user must first remove the archive, and then upload it to the file sharing service. There is also a shortage of web-based programming interfaces for easy integration of cloud-based video analytics into business analytics and other information systems that were already in the enterprise before the introduction of video surveillance.

Now a little more. Here are a few tasks that web-based video analytics solves in video surveillance systems:

Quality control of the video stream

This is the basic function of any video surveillance system - continuous recording and protection from interference with the system. Why it is so important, we think, it is not worth explaining. A web interface for monitoring video quality control and equipment status does not miss anything important, see an example:

Green is normal, gray is offline, yellow is an event detected, and red is a camera malfunction. In this case, a record of the state change is logged and transferred to the service.

Search the archive for incident investigation

It would seem that such a decision suggests itself. But in practice, in the event of an emergency, video surveillance is often useless: watching a few hours of video may not give any results.

Intellectual search tools save not only time, but also nerves. Data in the video archive can be found, for example, by type of object (person, person, vehicle, etc.) or type of movement of an object (appearance / stop in a given area) and, of course, by time. It is extremely important here that the ability to search does not require a priori adjustment of the rules, that is, the user can set the search rules after the event has occurred. For example, you can select a parking space and instantly find moments related to it (the time of theft, arrival, the fact of people approaching it).

Search results are given in the form of a photo album of enlarged images of objects by analogy with the search for pictures on the Internet. Alternative ways to display the results - in the form of points on the map with trajectories on the map or in the form of a compact list. Clicking starts video playback.

If the view angle and the technical capabilities of the camera allows, then the objects can be accurately identified: a person by face, and auto - by registration plate.

Dispatch Center

Until now, the management of many large (and not very) organizations imagines the dispatch center as a spacious room with hundreds of monitors, behind which watchful guards sit, day and night watching the cameras, enjoying the work of powerful communication channels. they don’t even imagine how expensive and resource intensive it is! And, unfortunately, not always effective.
The web-based architecture of the video surveillance system allows you to build a truly economical solution with a distributed control center, where operators can work, including in different time zones: an observer can connect from any computer at home. On the exclusion of the human factor and optimization, too, can not be extended once again - everyone understands it, at least a little familiar with the video analytics.



Operator's workplace and media player for web-based analytics. Events are displayed on the timeline in different colors depending on the type and priority. Close-up images of objects pop up above the timeline.

Import and export business process data

WOA allows you to integrate cloud-based video analytics with systems already installed in the enterprise. Such quality is especially valuable for warehouses or storages - each object can be tracked by a bar code and see all its movements thanks to the work with the archive. This way nothing is lost. Another example is that video is marked by benchmarks of an industrial process — oil production in our case.


Combining the results of video monitoring and data on the business process (production process) of the organization. Employees receive an “augmented reality”.

In conclusion, I would like to emphasize that the adaptation of the traditional video analytics system to the “cloud” does not provide the desired level of system scaling. A radical redesign of the architecture, user interfaces and programming interfaces is required. But the costs are worth it, we have long been convinced of this.