Video Analytics System Research Articles

Presentation video analytics systems data in the form of heatmaps

Describes the method of presentation video analytics data collected for continues time in the form of heatmaps for analysis of the movement of human flows in accumulation sites. The scene is divided into zones where objects are counted. The counting results are stored in the data store with a specified periodicity. The heat map is based on matrix of counters of objects in specific areas of the scene. Over the matrix, the normalization operation is performed and, depending on the value of the elements, the color of the heat map areas is assigned based on the linear color gradient.

The method to annotate buildings with images from CCTV cameras with an intersecting field of view

Video surveillance and tracking systems help to increase the safety of human activity in a given area and improve detecting the geo-location of an observer (a swarm of drones, autopilot) in modern cities. This article proposes the method for determining the full or partial address (annotation) of buildings by using images from several CCTV cameras with an intersecting field of view. As input parameters, the method takes several images from surveillance cameras, their geographical coordinates and directions of the optical axes and the vector map of the area where the cameras are located. The outputs are annotations of buildings for the initial images. The proposed method allows solving the problem of georeferencing the video analytics system and improving the work of the surveillance and object tracking systems. The resulting annotations of buildings provide an opportunity to determine more accurately the situations with overlapping of the tracked objects with buildings and predict the probable paths of the objects. A lost object on one camera can be found by using annotations from other cameras, which field of view is not overlapped by this building. Besides, the proposed ap0070roach is sustainable to full or partial overlapping of buildings and allows solving to solve the problem in the absence of a detailed height map of the terrain and buildings. Also, the article researches the restrictions of the method. It is necessary to find an angle between two cameras, at which the best result can be achieved. That is, the method should return the correct addresses of all buildings visible in the image.

Robust Multi-modality Anchor Graph-based Label Prediction for RGB-Infrared Tracking

Given massive video data generated from different applications such as security monitoring and traffic management, to save cost and human labour, developing an industrial intelligent video analytic system, which can automatically extract and analyze the meaningful content of videos, is essential. For achieving the objective of motion perception in video analytic system, a key problem is how to perform effective tracking of object of interest so that the location and the status of the tracked object can be inferred accurately. To solve this problem, with the popularity of RGB-infrared dual camera systems, this paper proposes a new RGB-infrared tracking framework which aims to exploit information from both RGB and infrared modalities to enhance the tracking robustness. In particular, within the tracking framework, a robust multi-modality anchor graph-based label prediction model is developed, which is able to 1) construct a scalable graph representation of the relationship of the samples based on local anchor approximation; 2) defuse a limited amount of known labels to large amount of unlabeled sample efficiently based on transductive learning strategy; and 3) adaptive incorporate importance weights for measuring modality discriminability. Efficient optimization algorithms are derived to solve the prediction model. Experimental results on various multi- modality videos demonstrate the effectiveness of the proposed method.

An Adaptive Multi-Projection Metric Learning for Person Re-Identification Across Non-Overlapping Cameras

Person re-identification is one of the most important and challenging problems in video analytics systems; it aims to match people across non-overlapping camera views. For person re-identification, metric learning is introduced to improve the performance by providing a metric adapted for cross-view matching. The essence of metric learning is to search for an optimal projection matrix to project the original features into a new feature space. However, most existing metric learning methods overlook the inconsistency of feature distributions in multiple cameras. In this paper, we propose a multi-projection metric learning (MPML) method to overcome the inconsistency among multiple cameras in person re-identification. Our solution is to jointly learn multiple projection matrices using paired samples from different cameras to project features from different cameras into a common feature space. To make our method adaptive to newly added cameras without affecting the learned projection matrices, we further propose an adaptive MPML method, which can learn new camera projection matrices without having to update any of the obtained projection matrices. The proposed methods are evaluated on four major person re-identification data sets, with comprehensive experiments showing the effectiveness of the proposed methods and notable improvements over the state-of-the–art approaches.

Control and preprocessing of graphic data for effective dynamic object recognition

The features of detecting static objects are considered. The methods of Sparse Feature Propagation and Dense Feature Aggregation based on the neural network approach of multi-frame end-to-end learning are described. The methods are designed to increase the efficiency of dynamic object recognition. The general structure of the video analytics system is presented. The method of increasing the accuracy of the obtained recognition results and increasing the probability of detecting high-speed objects by the built-in motion detection module is proposed.

Analysis of Customer Emotion from Video based Feedback of a Product

Due to the high levels of competition in a global market, companies have put more effort on building strong customer relationships and increasing customer satisfaction levels. Now-a-days due to technological improvements in information and communication technologies gives a highly anticipated key contributor to improve the customer experience and satisfaction in service episodes is through the application of video analytics, such as to evaluate the customer’s emotions over the complete service cycle. Currently, emotion recognition from video could be a difficult analysis space. One of the foremost effective solutions to deal with this challenge is to utilize each audio and visual part as two sources contained within the video knowledge to form an overall assessment of the emotion. The combined use of audio and visual knowledge sources presents further challenges, such as determining the optimal data fusion technique prior to classification. In this paper, we propose an audio–visual emotion recognition system to detect the universal six emotions (happy, angry, sad, disgust, surprise, and fear) from video data. The detected customer emotions are then mapped and translated to provide client satisfaction scores. The projected client satisfaction video analytics system will operate over video conferencing or video chat. The effectiveness of our proposal is verified through numerical results.

Neural Networks in Video-Based Age and Gender Recognition on Mobile Platforms

The paper considers the use of convolutional neural networks for the concurrent recognition of the gender and age of a person by video records of his face. The emphasis is on the incorporation of the approach into mobile video analytics systems. We have investigated the fusion of decisions obtained during the processing of each video frame, including the use of the classifier committee based on Dempster-Shafer theory. We propose the novel age prediction method using the evaluation of the expectation of the most probable ages. We have compared existing neural-net models with a specially trained modification of the MobileNet convolution network with two outputs. The experimental results are given for such data collections as Kinect, IJB-A, Indian Movie and EmotiW. As compared with other conventional methods, our approach makes it possible to increase the age and gender recognition accuracy by 2–5% and 5–10% respectively.

Cloud‐based video analytics using convolutional neural networks

SummaryObject classification is a vital part of any video analytics system, which could aid in complex applications such as object monitoring and management. Traditional video analytics systems work on shallow networks and are unable to harness the power of distributed processing for training and inference. We propose a cloud‐based video analytics system based on an optimally tuned convolutional neural network to classify objects from video streams. The tuning of convolutional neural network is empowered by in‐memory distributed computing. The object classification is performed by comparing the target object with the prestored trained patterns, generating a set of matching scores. The matching scores greater than an empirically determined threshold reveal the classification of the target object. The proposed system proved to be robust to classification errors with an accuracy and precision of 97% and 96%, respectively, and can be used as a general‐purpose video analytics system.

Video Analytics for Customer Emotion and Satisfaction at Contact Centers

Due to the high levels of competition in a global market, companies have put more emphasis on building strong customer relationships and increasing customer satisfaction levels. With technological improvements in information and communication technologies, a highly anticipated key contributor to improve the customer experience and satisfaction in service episodes is through the application of video analytics, such as to evaluate the customer's emotions over the full service cycle. Currently, emotion recognition from video is a challenging research area. One of the most effective solutions to address this challenge is to utilize both the audio and visual components as two sources contained in the video data to make an overall assessment of the emotion. The combined use of audio and visual data sources presents additional challenges, such as determining the optimal data fusion technique prior to classification. In this paper, we propose an audio-visual emotion recognition system to detect the universal six emotions (happy, angry, sad, disgust, surprise, and fear) from video data. The detected customer emotions are then mapped and translated to give customer satisfaction scores. The proposed customer satisfaction video analytics system can operate over video conferencing or video chat. The effectiveness of our proposal is verified through numerical results.

Mortise terrorism on the main pipelines

The research aim of the work is to analyze the effectiveness of the methods of physical protection of main pipelines proposed in the article from the "mortise terrorism" A mathematical model has been developed that made it possible to predict the dynamics of "mortise terrorism" in the short term. An analysis of the effectiveness of physical protection methods proposed in the article to prevent unauthorized impacts on the objects under investigation is given. A variant of a video analytics system has been developed that allows detecting violators with recognition of the types of work they perform at a distance of 150 meters in conditions of complex natural backgrounds and precipitation. Probability of detection is 0.959.

USING THE SYSTEM OF BIOMETRIC VIDEO ANALYTICS TO DETERMINE THE PSYCHOPHYSIOLOGICAL STATE MAN

In the presented material, a description of the hardware-software complex for assessing a person's psychophysiological state is given. Its main technical characteristics are indicated, including the ability for the user to fill the classifier with target templates that are to be identified in real time. The subject areas are indicated, in which the use of biometric video analytics systems is expedient for increasing the labor productivity of the corresponding specialist (bibliography: 15 refs).

A Single-Pass Connected Component Labeler without Label Merging Period

In this paper, for real-time multi-channel video analytics, an architecture of a single-pass connected component labeler without label merging period is proposed; this structure can handle multiple video channels from incoming video signals or from memory. During the line scan period, after determining an event based on the 8-connectivity check of incoming pixels, an 8-connectivity checker notifies all the label registers inside a label shift register of the event and the control signals for label handling. Each label register takes a label input either normally from the shifted label in the label shift register, or from the merging label without additional cycles for the label merging event. A label stack is designed for efficient label allocation with label reuse. An information extractor calculates bounding boxes and centers of gravity for labeled objects using the information from the 8-connectivity checker and the pixel counter. The proposed architecture is finally implemented on a field programmable gate array (FPGA) device, integrated into a video analytics system, and verified as to its functions and performance. More than 28 % of the memory usage by conventional architectures is reduced in the proposed architecture for the same maximum number of labels in video surveillance environments.

Making sense of video analytics: Lessons learned from clickstream interactions, attitudes, and learning outcome in a video-assisted course

<p>Online video lectures have been considered an instructional media for various pedagogic approaches, such as the flipped classroom and open online courses. In comparison to other instructional media, online video affords the opportunity for recording student clickstream patterns within a video lecture. Video analytics within lecture videos may provide insights into student learning performance and inform the improvement of video-assisted teaching tactics. Nevertheless, video analytics are not accessible to learning stakeholders, such as researchers and educators, mainly because online video platforms do not broadly share the interactions of the users with their systems. For this purpose, we have designed an open-access video analytics system for use in a video-assisted course. In this paper, we present a longitudinal study, which provides valuable insights through the lens of the collected video analytics. In particular, we found that there is a relationship between video navigation (repeated views) and the level of cognition/thinking required for a specific video segment. Our results indicated that learning performance progress was slightly improved and stabilized after the third week of the video-assisted course. We also found that attitudes regarding easiness, usability, usefulness, and acceptance of this type of course remained at the same levels throughout the course. Finally, we triangulate analytics from diverse sources, discuss them, and provide the lessons learned for further development and refinement of video-assisted courses and practices.</p>

Correlations of Melt Pool Geometry and Process Parameters During Laser Metal Deposition by Coaxial Process Monitoring

Abstract One major demand of today's laser metal deposition (LMD) processes is to achieve a fail-save build-up regarding changing conditions like heat accumulations. Especially for the repair of thin parts like turbine blades is the knowledge about the correlations between melt pool behavior and process parameters like laser power, feed rate and powder mass stream indispensable. The paper will show the process layout with the camera based coaxial monitoring system and the quantitative influence of the process parameters on the melt pool geometry. Therefore the diameter, length and area of the melt pool are measured by a video analytic system at various parameters and compared with the track wide in cross-sections and the laser spot diameter. The influence of changing process conditions on the melt pool is also investigated. On the base of these results an enhanced process of the build-up of a multilayer one track fillet geometry will be presented.

Real-time multi-camera video analytics system on GPU

In this article, parallel implementation of a real-time intelligent video surveillance system on Graphics Processing Unit (GPU) is described. The system is based on background subtraction and composed of motion detection, camera sabotage detection (moved camera, out-of-focus camera and covered camera detection), abandoned object detection, and object-tracking algorithms. As the algorithms have different characteristics, their GPU implementations have different speed-up rates. Test results show that when all the algorithms run concurrently, parallelization in GPU makes the system up to 21.88 times faster than the central processing unit counterpart, enabling real-time analysis of higher number of cameras.

Quantifying Limb Movements in Epileptic Seizures Through Color-Based Video Analysis

This paper proposes a color-based video analytic system for quantifying limb movements in epileptic seizure monitoring. The system utilizes colored pyjamas to facilitate limb segmentation and tracking. Thus, it is unobtrusive and requires no sensor/marker attached to patient's body. We employ Gaussian mixture models in background/foreground modeling and detect limbs through a coarse-to-fine paradigm with graph-cut-based segmentation. Next, we estimate limb parameters with domain knowledge guidance and extract displacement and oscillation features from movement trajectories for seizure detection/analysis. We report studies on sequences captured in an epilepsy monitoring unit. Experimental evaluations show that the proposed system has achieved comparable performance to EEG-based systems in detecting motor seizures.

An illumination invariant framework for real-time foreground detection

The detection of interesting objects in a video sequence is a challenging problem, with many applications in automated surveillance and video analytic systems. Background subtraction methods have proven to be useful as a way to differentiate foreground objects from the background, but suffer from various shortcomings that hamper performance in practice. We propose a generalized framework to handle some of the greater challenges facing background modeling systems. This includes object persistence and reintegration, illumination robustness and resistance to environmental effects. In this paper we describe an interaction framework between an illumination invariant and a color-based model, with a novel feature clustering feedback that dynamically influences model updating and by proxy provides object validation. The framework was designed with a strong real-time emphasis, and the principles remain applicable by collapsing each module to a simpler real-time variant.

Bayesian Tracking for Video Analytics

Visual tracking represents the basic processing step for most video analytics applications where the aim is to automatically understand the actions occurring in a monitored scene. Consequently, the performances of these applications are significantly dependent on the accuracy and robustness of the tracking algorithm. Bayesian state estimation and probabilistic graphical models (PGMs) have proved to be very powerful and appropriate mathematical tools to efficiently solve the inference problem of motion estimation by combining object dynamics and observations. In this article, the impact of these signal processing techniques on the development of recent tracking algorithms is shown and a categorization of the most common approaches is proposed. This categorization intends to logically organize different concepts related to Bayesian visual tracking to give a global overview to the reader. Finally, general considerations on the design of visual trackers for video analytics systems are discussed, focusing on the tradeoff that is usually performed between the accuracy of the target motion assumptions and the robustness of the object appearance representation.

우수농산물 관리제도의 안전성 인증기능 보완을 위한 시험 모형연구

현재 운용되고 있는 우수농산물 관리제도는 식품안전시스템으로써 소비자의 신뢰를 확보하는 데 주요한 두 가지 문제점이 있다. 첫 번째는 제도의 근간이 되는 생산이력데이타의 입력이 농민과 유통업자등의 직접입력에 의존하기 때문에 자료의 정확성을 검증하기 불가하다는 것이며 두 번째는 오염된 농산물을 추적하여 회수할 수 없다는 것이다. 본 논문에서는 이러한 문제를 해결하기 위한 방안으로써 정보기술과 농업현장기술을 결합한 IT융합모델을 고안하였다. 이 모델에서는 영상분석시스템을 이용하여 현장 영농 활동을 분류하고 생산이력데이타를 자동생성하며, USN을 기반으로 한 실시간 이력추적 시스템을 통해 유통 중인 농산물의 현재위치와 안전성 관리 자료를 실시간 전송하도록 되어있다. There are two major problems with current National GAP system. One was false in traceability record because it was written or inputted by farmers or distributers and no other measures to check the accuracy was valid. The other was incapability of tracking back and recalling the contaminated agricultural products. For solving these matters, IT convergence model which combined information technology with agricultural experience is elaborated. In IT convergence model, video analytic system classifies every activity depending on the pre-programmed farming process and create the traceability data automatically. Also real time trace system based on USN would solve the problem of tracking back. This system transmits the present location and monitors data of agricultural products from farm to table at all times.