Video analysis in perimeter protection systems
Today, perimeter protection is the main application of professional video analytics (if you do not include the task of recognizing car license plates in this concept). Unlike video analysis systems used in public places, perimeter video analytics solve a more specific and simple task - the primary detection of a person or vehicle in a sterile area. In our first publication in 2009, we looked at the general problems of perimeter video analytics and assessing its accuracy.
The main difference between perimeter video analytics and ordinary motion detection is the need for stable detection of an object of interest (target) against a dynamic background, the changes of which are caused by the environment. Video analytics should not respond to changes in lighting, shade, movement of plants, animals, insects, birds, precipitation, camera shake from the wind, but it should maintain high sensitivity to potential perimeter violators.
A trained intruder may look completely unpredictable for a video analyst developer, and “sharpening” the detector to reduce the frequency of false alarms, for example, under a walking person will not provide adequate triggering of video analytics if the violator crawls or moves in a group of people.
')
The integral characteristic of the video analytics accuracy for the perimeter is the F1 indicator used in i-LIDS tests, which depends on the error rates of types I and II, as well as on the response time of the system. Disruption of the tracking of the target leads to repeated triggers, which is considered a type I error. Therefore, tracking is an important component of perimeter video analytics (as opposed to an ordinary motion detector).
This article complements the above publication with current industry trends and describes in more detail about the popular analytics functions in perimeter protection systems.
The main incentive for the use of sensors operating in different spectral ranges is to ensure all-weather operation and / or an increase in the range of the camera. On the perimeters, fixed cameras are used near infrared, medium thermal and far thermal imaging regions of the spectrum. As shown in the figures fig. 1-3, the sensors form an image of different informativeness and require the adaptation of video analytics to the specific features of observation in each range of the spectrum. Here, the most difficult tasks are: target detection with an unfavorable signal-to-noise ratio, tracking low-contrast targets at a long range (with a significant amplitude of image jitter). There is also a complex industry specificity: for example, when monitoring the perimeter of a railroad bed, video analytics should not react to trains and interferences (shadows, whirlwinds of snow, strong camera vibrations).
Figure 1 Observation in the near infrared region of the spectrum: a cyclist on rough terrain, a boat on the water, a man on a bridge
Figure 2 Observation in the zone of the average thermal imaging region of the spectrum: a cyclist on rough terrain, a boat on the water, a man on a bridge
Figure 3 Observation in the zone of a far-infrared spectral region: a crawler on rough terrain, a boat on the water, a rowing boat under a bridge
Trend 2. Managing video transmission on video analytics events
When building a centralized video surveillance system for geographically distributed objects, for example, for a fuel and energy complex or a railway, it becomes necessary to transmit video and audio data over communication channels with limited bandwidth. The growth in the number of cameras, including high-resolution cameras, is significantly ahead of the capabilities of telecom operators and data centers.
A good video analytics allows you to select the most important video fragments for transmitting through narrow communication channels not by reducing the image quality and frame rate, but by removing video fragments that are not of interest to the user.
The transition to megapixel cameras is a significant trend of the video surveillance industry as a whole, but is not noticeable in the tasks of perimeter security. On the one hand, a camera with a 1-2 megapixel sensor and good optics allows you to increase the threat detection range up to 2 times compared to a standard definition sensor (0.4 megapixel). On the other hand, this advantage only manifests itself in ideal observation conditions (good uniform illumination, lack of precipitation and camera shake). Under adverse conditions, megapixel cameras can work worse than standard ones due to the smaller effective area of ​​photocells, but they can also create a significant amount of data with a low signal-to-noise ratio for network transmission and storage.
Figure 4 Comparison of a standard-definition frame (0.4 megapixels) with a high-definition frame (2 megapixels) below. A person climbs a fence at a distance of 16 from the beginning of the observation zone.
From the point of view of video analytics, the transition from standard to high resolution is not trivial. Most video analytics systems on the market today do not provide a noticeable advantage in recognition accuracy due to a set of problems.
First, the complexity of video analytic algorithms that are not optimized for high resolution (HD) increases in a non-linear manner with respect to the number of frame pixels due to the resource-intensive segmentation operations. For example, on one core of a modern processor, a typical video analytics processes 8 channels of CIF (frame 352 x 288 pixels, a total of about 100 thousand pixels) at a speed of 8-12 frames per second. If this analytics is applied on a 720p stream (frame 1280 x 720, a total of about 921 thousand pixels), then the resources of one processor core are most likely not even enough to process 1 channel, due to the increase in data volume (number of pixels) in more than 9 times, as well as increasing the cost of processing each pixel by 2 times (the total increase in complexity is 18 times). Non-linear growth of complexity is observed in the segmentation of large objects in the foreground, if the multiscale representation is NOT used.
Secondly, the use of video analytics on megapixel cameras suggests a more accurate spatial calibration of the depth of the scene, taking into account the nonlinear distortion of optics. When using approximate calibration methods applicable on standard-resolution cameras, the video analyst may be mistaken in determining the scale and coordinates of the target, which will have a negative effect on the overall accuracy indicators.
Due to the problems discussed, megapixel cameras today are not being displaced by guided PTZ cameras on the perimeters, as is the case when observing inside a room or in an urban environment.
The most common detector is the signal line (tripwire), which allows you to automatically detect the fact of crossing the perimeter border in the camera's field of view (Fig. 5). The advantage of video analytics in comparison with classical perimeter security devices (for example, radio wave and vibration detectors) is the possibility of earlier detection of a target at a long turn and assignment of different priorities to targets, depending on the distance. For example, the intersection of the signal line along the fence TW-1 has a higher priority than the appearance in the adjacent zone Z-1.
Figure 5. Signal line (TW-1) and early human detection zone (Z-1)
In the case of video analytics in the “semi-physical” zone where a limited number of people are allowed to appear, elements of behavior classification are used, for example, a person’s stopping or “idleness” detector, as well as target classifiers (person, group of people, vehicle) in the zone . five).
The tracking system in the field of view of a fixed overview camera allows you to compare multiple presentations of one target to a single trajectory and to ensure strictly one response for each target. Disruption tracking in the perimeter system leads to the resubmission of the alarm message. This leads to an increase in the total number of false positives and the load on the operator, both at the stage of processing operational data, and at the stage of retrospective analysis of the archive. If the operator receives a “stack” of alarm messages relating to one person, then he may not notice the appearance of a new person among the redundant data.
In the tasks of protecting extended objects with rare events, the card is the main automated workplace of the operator. The map allows you to quickly assess the global state of the protected object and switch to the desired video channel in case of an emergency situation. In such a system, the primary detection task is completely shifted to the technical means of video analytics and / or perimeter detectors, and the operator makes decisions on each triggering of the detector.
Unlike traditional perimeter detectors, video analytics allow you to project on the map not only the approximate location of the intruder, but also the trajectory of its movement. Video analytics can effectively track multiple targets at once and highlight images of each target for compact display on the map immediately. This allows you to fix the infiltration of the offender, in the case when "your" and "alien" appear next.
Figure 6 Layout of a graphical user interface with an interactive map. The map displays: a) camera observation areas in the form of trapezoids; b) the active camera is highlighted in red; c) two objects on the approaches to the protected line.
The use of an interactive map creates the prerequisites for the introduction of multi-camera video analytics , which implements the tracking of people between cameras. A multi-camera video analytics allows you to reduce the number of false positives caused by the appearance of people in the observation areas of several cameras at once and to get a complete trajectory of movement of people around the object.
When using dome PTZ cameras (PTZ cameras), interesting problems arise at the intersection of video analytics and video management systems (VMS): patrolling, target selection, assignment of target priority (if there are several targets), pointing PTZ cameras and tracking the target with PTZ cameras. The most fault-tolerant tracking scheme is to use a standalone tracking algorithm built into a PTZ camera with periodic correction for this survey camera (for example, a thermal imager). In more detail, the issues of intellectualization of PTZ-cameras are covered in a separate habpost .
The diagram in fig. 7. illustrates the architecture of a geographically distributed perimeter protection system with video analytics. The example uses the dedicated IP video server MagicBox and MagicBox HD analytics , as well as the MagicBox Thermal thermal imaging IP video server to ensure guaranteed accuracy F1 = 0.995 regardless of the central server load by the operators. The use of analog cameras (close observation) is shown; thermal imagers and PTZ cameras (long-range monitoring) and IP cameras (near and medium monitoring).
Figure 7 Diagram of a geographically distributed perimeter protection system with video analytics
The IP video server also implements additional functions: compression, transmission of video, audio and / or alarm frames; integration of perimeter detectors through physical interfaces "dry contacts", RS232, RS485; integration of the calling panel and / or loudspeaker; and control of actuators via relay output (lighting, siren).
The IP video server optionally performs local video and audio recording on embedded or external (via USB interface) to work with communication channels of limited bandwidth (from 64 Kbps). If events rarely occur (less than 1% of the total time), then the quality of the event video can be high due to batch (delayed) transmission. The key role in the implementation of such an architecture is the quality of video analytics, which can significantly reduce the amount of information transmitted via the communication channel.
For event recording (including pre- and post-event recording), you can use a RAM disk, which eliminates the deterioration of non-volatile memory such as NAND FLASH or a hard disk.
Perimeter video analytics are configured using the free ONVIF Device Manager application .
The main difference between perimeter video analytics and ordinary motion detection is the need for stable detection of an object of interest (target) against a dynamic background, the changes of which are caused by the environment. Video analytics should not respond to changes in lighting, shade, movement of plants, animals, insects, birds, precipitation, camera shake from the wind, but it should maintain high sensitivity to potential perimeter violators.
A trained intruder may look completely unpredictable for a video analyst developer, and “sharpening” the detector to reduce the frequency of false alarms, for example, under a walking person will not provide adequate triggering of video analytics if the violator crawls or moves in a group of people.
')
The integral characteristic of the video analytics accuracy for the perimeter is the F1 indicator used in i-LIDS tests, which depends on the error rates of types I and II, as well as on the response time of the system. Disruption of the tracking of the target leads to repeated triggers, which is considered a type I error. Therefore, tracking is an important component of perimeter video analytics (as opposed to an ordinary motion detector).
This article complements the above publication with current industry trends and describes in more detail about the popular analytics functions in perimeter protection systems.
Trend 1. Different observation spectra
The main incentive for the use of sensors operating in different spectral ranges is to ensure all-weather operation and / or an increase in the range of the camera. On the perimeters, fixed cameras are used near infrared, medium thermal and far thermal imaging regions of the spectrum. As shown in the figures fig. 1-3, the sensors form an image of different informativeness and require the adaptation of video analytics to the specific features of observation in each range of the spectrum. Here, the most difficult tasks are: target detection with an unfavorable signal-to-noise ratio, tracking low-contrast targets at a long range (with a significant amplitude of image jitter). There is also a complex industry specificity: for example, when monitoring the perimeter of a railroad bed, video analytics should not react to trains and interferences (shadows, whirlwinds of snow, strong camera vibrations).
Figure 1 Observation in the near infrared region of the spectrum: a cyclist on rough terrain, a boat on the water, a man on a bridge
Figure 2 Observation in the zone of the average thermal imaging region of the spectrum: a cyclist on rough terrain, a boat on the water, a man on a bridge
Figure 3 Observation in the zone of a far-infrared spectral region: a crawler on rough terrain, a boat on the water, a rowing boat under a bridge
Trend 2. Managing video transmission on video analytics events
When building a centralized video surveillance system for geographically distributed objects, for example, for a fuel and energy complex or a railway, it becomes necessary to transmit video and audio data over communication channels with limited bandwidth. The growth in the number of cameras, including high-resolution cameras, is significantly ahead of the capabilities of telecom operators and data centers.
A good video analytics allows you to select the most important video fragments for transmitting through narrow communication channels not by reducing the image quality and frame rate, but by removing video fragments that are not of interest to the user.
Trend 3. Megapixel cameras
The transition to megapixel cameras is a significant trend of the video surveillance industry as a whole, but is not noticeable in the tasks of perimeter security. On the one hand, a camera with a 1-2 megapixel sensor and good optics allows you to increase the threat detection range up to 2 times compared to a standard definition sensor (0.4 megapixel). On the other hand, this advantage only manifests itself in ideal observation conditions (good uniform illumination, lack of precipitation and camera shake). Under adverse conditions, megapixel cameras can work worse than standard ones due to the smaller effective area of ​​photocells, but they can also create a significant amount of data with a low signal-to-noise ratio for network transmission and storage.
Figure 4 Comparison of a standard-definition frame (0.4 megapixels) with a high-definition frame (2 megapixels) below. A person climbs a fence at a distance of 16 from the beginning of the observation zone.
From the point of view of video analytics, the transition from standard to high resolution is not trivial. Most video analytics systems on the market today do not provide a noticeable advantage in recognition accuracy due to a set of problems.
First, the complexity of video analytic algorithms that are not optimized for high resolution (HD) increases in a non-linear manner with respect to the number of frame pixels due to the resource-intensive segmentation operations. For example, on one core of a modern processor, a typical video analytics processes 8 channels of CIF (frame 352 x 288 pixels, a total of about 100 thousand pixels) at a speed of 8-12 frames per second. If this analytics is applied on a 720p stream (frame 1280 x 720, a total of about 921 thousand pixels), then the resources of one processor core are most likely not even enough to process 1 channel, due to the increase in data volume (number of pixels) in more than 9 times, as well as increasing the cost of processing each pixel by 2 times (the total increase in complexity is 18 times). Non-linear growth of complexity is observed in the segmentation of large objects in the foreground, if the multiscale representation is NOT used.
Secondly, the use of video analytics on megapixel cameras suggests a more accurate spatial calibration of the depth of the scene, taking into account the nonlinear distortion of optics. When using approximate calibration methods applicable on standard-resolution cameras, the video analyst may be mistaken in determining the scale and coordinates of the target, which will have a negative effect on the overall accuracy indicators.
Due to the problems discussed, megapixel cameras today are not being displaced by guided PTZ cameras on the perimeters, as is the case when observing inside a room or in an urban environment.
Perimeter Video Analytics Detectors
The most common detector is the signal line (tripwire), which allows you to automatically detect the fact of crossing the perimeter border in the camera's field of view (Fig. 5). The advantage of video analytics in comparison with classical perimeter security devices (for example, radio wave and vibration detectors) is the possibility of earlier detection of a target at a long turn and assignment of different priorities to targets, depending on the distance. For example, the intersection of the signal line along the fence TW-1 has a higher priority than the appearance in the adjacent zone Z-1.
Figure 5. Signal line (TW-1) and early human detection zone (Z-1)
In the case of video analytics in the “semi-physical” zone where a limited number of people are allowed to appear, elements of behavior classification are used, for example, a person’s stopping or “idleness” detector, as well as target classifiers (person, group of people, vehicle) in the zone . five).
Target tracking
The tracking system in the field of view of a fixed overview camera allows you to compare multiple presentations of one target to a single trajectory and to ensure strictly one response for each target. Disruption tracking in the perimeter system leads to the resubmission of the alarm message. This leads to an increase in the total number of false positives and the load on the operator, both at the stage of processing operational data, and at the stage of retrospective analysis of the archive. If the operator receives a “stack” of alarm messages relating to one person, then he may not notice the appearance of a new person among the redundant data.
Interactive Map
In the tasks of protecting extended objects with rare events, the card is the main automated workplace of the operator. The map allows you to quickly assess the global state of the protected object and switch to the desired video channel in case of an emergency situation. In such a system, the primary detection task is completely shifted to the technical means of video analytics and / or perimeter detectors, and the operator makes decisions on each triggering of the detector.
Unlike traditional perimeter detectors, video analytics allow you to project on the map not only the approximate location of the intruder, but also the trajectory of its movement. Video analytics can effectively track multiple targets at once and highlight images of each target for compact display on the map immediately. This allows you to fix the infiltration of the offender, in the case when "your" and "alien" appear next.
Figure 6 Layout of a graphical user interface with an interactive map. The map displays: a) camera observation areas in the form of trapezoids; b) the active camera is highlighted in red; c) two objects on the approaches to the protected line.
Multi-camera video analytics
The use of an interactive map creates the prerequisites for the introduction of multi-camera video analytics , which implements the tracking of people between cameras. A multi-camera video analytics allows you to reduce the number of false positives caused by the appearance of people in the observation areas of several cameras at once and to get a complete trajectory of movement of people around the object.
PTZ Automation
When using dome PTZ cameras (PTZ cameras), interesting problems arise at the intersection of video analytics and video management systems (VMS): patrolling, target selection, assignment of target priority (if there are several targets), pointing PTZ cameras and tracking the target with PTZ cameras. The most fault-tolerant tracking scheme is to use a standalone tracking algorithm built into a PTZ camera with periodic correction for this survey camera (for example, a thermal imager). In more detail, the issues of intellectualization of PTZ-cameras are covered in a separate habpost .
Distributed Video Analytics Architecture
The diagram in fig. 7. illustrates the architecture of a geographically distributed perimeter protection system with video analytics. The example uses the dedicated IP video server MagicBox and MagicBox HD analytics , as well as the MagicBox Thermal thermal imaging IP video server to ensure guaranteed accuracy F1 = 0.995 regardless of the central server load by the operators. The use of analog cameras (close observation) is shown; thermal imagers and PTZ cameras (long-range monitoring) and IP cameras (near and medium monitoring).
Figure 7 Diagram of a geographically distributed perimeter protection system with video analytics
The IP video server also implements additional functions: compression, transmission of video, audio and / or alarm frames; integration of perimeter detectors through physical interfaces "dry contacts", RS232, RS485; integration of the calling panel and / or loudspeaker; and control of actuators via relay output (lighting, siren).
The IP video server optionally performs local video and audio recording on embedded or external (via USB interface) to work with communication channels of limited bandwidth (from 64 Kbps). If events rarely occur (less than 1% of the total time), then the quality of the event video can be high due to batch (delayed) transmission. The key role in the implementation of such an architecture is the quality of video analytics, which can significantly reduce the amount of information transmitted via the communication channel.
For event recording (including pre- and post-event recording), you can use a RAM disk, which eliminates the deterioration of non-volatile memory such as NAND FLASH or a hard disk.
Perimeter video analytics are configured using the free ONVIF Device Manager application .
Source: https://habr.com/ru/post/137006/
All Articles