Surveillance Video Sequences Research Articles

Pose-oriented scene-adaptive matching for abnormal event detection

For intelligent surveillance systems, abnormal event detection automatically analyses surveillance video sequences and detects abnormal objects or unusual human actions at the frame level. Due to the lack of labelled data, most approaches are semi-supervised based on reconstruction or prediction methods. However, these methods may not generalize well to unseen scene contexts. To address this issue, we present a novel self and mutual scene-adaptive matching method for abnormal event detection. In the framework, we propose synergistic pose estimation and object detection, which effectively integrates human pose and object detection information to improve pose estimation accuracy. Then, the poses are resized to reduce the spatial distance between the source and target domains. The improved pose sequences are further fed into a spatio-temporal graph convolutional network to extract the geometric features. Finally, the features are embedded in a clustering layer to classify action types and compute normality scores. The training data is taken from the training part of common video anomaly detection datasets: UCSD PED1 & PED2, CHUK Avenue, and ShanghaiTech Campus. The proposed framework is evaluated on video sequences with unseen scene contexts in the UCSD PED2 and ShanghaiTech Campus datasets. The detection accuracy and efficiency are also evaluated in detail, and the proposed method for abnormal event detection achieves the highest AUC performance, 84.6%, on the ShanghaiTech Campus dataset and relatively high AUC performance, 96.9% and 74.8%, on UCSD PED2 & PED1 datasets. Compared with other state-of-the-art works, the performance analysis and results confirm the robustness and effectiveness of our proposed framework for cross-scene abnormal event detection.

Research on Pedestrian Re-identification Techniques in Dynamic Scenes Using Convolutional Neural Networks

Abstract Pedestrian re-recognition is the process of retrieving pedestrians with the same identity information as a given pedestrian from a cross-domain view candidate image dataset or a non-overlapping surveillance video sequence using computer vision techniques. The goal of this paper is to use convolutional neural network techniques to re-recognize pedestrians in dynamic scenes. Through the use of convolutional calculations, activation function selection, and other techniques, this paper provides basic technical support for the research of pedestrian re-recognition technology. A Siamese network is obtained by applying convolutional neural networks to pedestrian recognition as the main discriminative model for subsequent research. In order to effectively solve the problem of occlusion, illumination, and other possible interference with the recognition effect in dynamic scenes, this paper adopts the image enhancement method of random erasure and introduces the attention mechanism to improve the robustness of the model to the occlusion of pedestrian images. Through the examination of the model on the dataset of the average accuracy mean (MAP) and other indicators and the actual application in the construction site and other scenes, it is proved that the pedestrian re-recognition model used in this paper has a more significant recognition performance compared with other models, and can still maintain more than 80% of the accuracy rate under the application of dynamic and complex scenes.

Adapting to the Dark: A Novel Adaptive Low Light Illumination Correction Algorithm for Video Sequences in Wireless Communications

The lower image quality and higher noise levels of low light images can have a substantial effect on wireless communication. Images typically have less brightness, less clarity, and more picture noise in low light situations. The precision and dependability of picture-based communication may be impacted by this decline in image quality. Low light levels can be detrimental to video chats since it becomes difficult for the camera to get clear pictures of the user's face. This may lead to decreased video quality, trouble with facial recognition, and a generally worse user experience when communicating over video. Low light illumination correction algorithms are frequently made to work quickly and effectively in order to enable real-time video-based wireless communication. Correcting uneven illumination or low light in images is crucial for various fields; as such images pose challenges for human recognition, computer vision algorithms, and multimedia algorithms. An adaptive illumination correction algorithm based on a simple log transformation approach enhanced by a hyperbolic beta transformation and a logarithmic image processing is presented in this research. It has an adaptive optimization function that adjusts itself based on the image’s illumination and reflectance. Low computation and minimal parameter optimization are the hallmarks of this method. A variety of illuminated images and video sequences from various datasets are used to evaluate the suggested adaptive illumination approach. This adaptive illumination approach's experimental findings are impressive compared to those produced using prior techniques. The results reveal that the proposed system can adapt to various image-based communication and surveillance video sequences taken under various illumination conditions.

Saliency-Aided Online RPCA for Moving Target Detection in Infrared Maritime Scenarios.

Moving target detection (MTD) is a crucial task in computer vision applications. In this paper, we investigate the problem of detecting moving targets in infrared (IR) surveillance video sequences captured using a steady camera in a maritime setting. For this purpose, we employ robust principal component analysis (RPCA), which is an improvement of principal component analysis (PCA) that separates an input matrix into the following two matrices: a low-rank matrix that is representative, in our case study, of the slowly changing background, and a sparse matrix that is representative of the foreground. RPCA is usually implemented in a non-causal batch form. To pursue a real-time application, we tested an online implementation, which, unfortunately, was affected by the presence of the target in the scene during the initialization phase. Therefore, we improved the robustness by implementing a saliency-based strategy. The advantages offered by the resulting technique, which we called "saliency-aided online moving window RPCA" (S-OMW-RPCA) are the following: RPCA is implemented online; along with the temporal features exploited by RPCA, the spatial features are also taken into consideration by using a saliency filter; the results are robust against the condition of the scene during the initialization. Finally, we compare the performance of the proposed technique in terms of precision, recall, and execution time with that of an online RPCA, thus, showing the effectiveness of the saliency-based approach.

Apron surveillance video coding based on compositing virtual reference frame with object library

AbstractA video coding framework for the apron surveillance scene has been proposed in this paper, which aims to improve coding efficiency by eliminating long‐term redundancy at the object level. To achieve this goal, this study first develops an existing block‐based hybrid video coding framework by exploiting the video redundancy on the object level to perform video coding. Second, an object‐library mechanism is designed to collect the representative object images as coding references on larger temporal and spatial scales. Finally, a virtual reference frame, which blends background and foreground references from the object library, is adaptively composited according to the video content to improve the inter‐prediction performance. Preliminary experimental results demonstrate that the proposed method achieves a high BD rate reduction of up to 23.97% in apron surveillance video sequences, compared to the standard high efficiency video coding (HEVC).

Neural networks contribution in face mask detection to reduce the spread of COVID-19.

In front of COVID-19 propagation, we can protect our self by taking precautionary measures such as wearing face masks. It may be mandatory in particular public place although some persons ignore this rule. Several research in face mask detection area have emerged and most of studies are based on deep learning. In this paper, we present a method to detect whether person wear a mask or not to prevent the propagation of virus. The approach is based on combination of Pulse Couple Neural Network and Fully Connected Neural Network and the processing is divided in three steps: geometrical, feature extraction and decision. The geometrical module selects the Region of Interest for given image and the feature extraction module composed by Pulse Couple Neural Network extracts all pertinent information which will be used by the last module for decision. This decision module makes directly a decision in case of non-complex classification without neural network training overwise the Fully Connected Neural Network continues the treatment. The input image may be captured from video surveillance sequence, the system triggers a signal alarm once a person doesn't wear face mask. Our proposed approach was tested with different datasets like Kaggle, AIZOO, Moxa3K, Real-World Masked Face Dataset, Medical Masks Dataset, Face Mask Dataset and the accuracy varies from 83.2% to 100% with minimum computation time.

An improved Gaussian Mixture Model with post-processing for multiple object detection in surveillance video analytics

Gaussian Mixture Model (GMM) is an effective method for extracting foreground objects from video sequences. However, GMM fails to detect the object in challenging scenarios like the presence of shadow, occlusion, complex backgrounds, etc. To handle these challenges, intrinsic and extrinsic enhancement is required in traditional GMM. This paper presents a novel framework that combines improved GMM with postprocessing for multiple object detection. In the proposed system, GMM with parameter initialization is considered an intrinsic improvement. Video preprocessing and postprocessing are considered extrinsic improvements. Integration of morphological operation with GMM helps for better segmentation than traditional GMM, and it also helps to increase detection performance by reducing false positives. Video preprocessing is the process of noise removal that prepares input video ready for further processing. In the final step gradient of morphological operations is used for postprocessing. The proposed approach was tested on challenging surveillance video sequences from benchmark datasets such as PETS 2009 and CD 2014(Change Detection). The experimental results are compared using ground truth and performance evaluation metrics. The results show that the proposed approach performs better than GMM, and the method can detect the object effectively even in illumination variation and partial occlusion.

Key Frame Extraction Algorithm of Surveillance Video Based on Quaternion Fourier Significance Detection

With the improvement of people's security awareness, numerous monitoring equipment has been put into use, resulting in the explosive growth of surveillance video data. Key frame extraction technology is a paramount technology for improving video storage efficiency and enhancing the accuracy of video retrieval. It can extract key frame sets that can express video content from massive videos. However, the existing key frame extraction algorithms of surveillance video still have deficiencies, such as the destruction of image information integrity and the inability to extract key frames accurately. To this end, this paper proposes a key frame extraction algorithm of surveillance video based on quaternion Fourier saliency detection. Firstly, the algorithm used colors, and intensity features to perform quaternion Fourier transform on surveillance video sequences. Next, the phase spectrum of the quaternion Fourier transformed image was obtained, and he image visual saliency map was obtained according to the quaternion Fourier phase spectrum. Then, the image visual saliency map of two adjacent frames is used to characterize the change of target motion state. Finally, the frames that can accurately express the motion state of the target are selected as key frames. The experimental results show that the method proposed in this paper can accurately capture the changes of the local motion state of the target while maintaining the integrity of the image information.

Intelligent Power Grid Video Surveillance Technology Based on Efficient Compression Algorithm Using Robust Particle Swarm Optimization

Companies that produce energy transmit it to any or all households via a power grid, which is a regulated power transmission hub that acts as a middleman. When a power grid fails, the whole area it serves is blacked out. To ensure smooth and effective functioning, a power grid monitoring system is required. Computer vision is among the most commonly utilized and active research applications in the world of video surveillance. Though a lot has been accomplished in the field of power grid surveillance, a more effective compression method is still required for large quantities of grid surveillance video data to be archived compactly and sent efficiently. Video compression has become increasingly essential with the advent of contemporary video processing algorithms. An algorithm’s efficacy in a power grid monitoring system depends on the rate at which video data is sent. A novel compression technique for video inputs from power grid monitoring equipment is described in this study. Due to a lack of redundancy in visual input, traditional techniques are unable to fulfill the current demand standards for modern technology. As a result, the volume of data that needs to be saved and handled in live time grows. Encoding frames and decreasing duplication in surveillance video using texture information similarity, the proposed technique overcomes the aforementioned problems by Robust Particle Swarm Optimization (RPSO) based run-length coding approach. Our solution surpasses other current and relevant existing algorithms based on experimental findings and assessments of different surveillance video sequences utilizing varied parameters. A massive collection of surveillance films was compressed at a 50% higher rate using the suggested approach than with existing methods.

A combined multiple action recognition and summarization for surveillance video sequences

Human action recognition and video summarization represent challenging tasks for several computer vision applications including video surveillance, criminal investigations, and sports applications. For long videos, it is difficult to search within a video for a specific action and/or person. Usually, human action recognition approaches presented in the literature deal with videos that contain only a single person, and they are able to recognize his action. This paper proposes an effective approach to multiple human action detection, recognition, and summarization. The multiple action detection extracts human bodies’ silhouette, then generates a specific sequence for each one of them using motion detection and tracking method. Each of the extracted sequences is then divided into shots that represent homogeneous actions in the sequence using the similarity between each pair frames. Using the histogram of the oriented gradient (HOG) of the Temporal Difference Map (TDMap) of the frames of each shot, we recognize the action by performing a comparison between the generated HOG and the existed HOGs in the training phase which represents all the HOGs of many actions using a set of videos for training. Also, using the TDMap images we recognize the action using a proposed CNN model. Action summarization is performed for each detected person. The efficiency of the proposed approach is shown through the obtained results for mainly multi-action detection and recognition.

Analysis of Automatic Annotations of Real Video Surveillance Images

The results of the analysis of the automatic annotations of real video surveillance sequences are presented. The annotations of the frames of surveillance sequences of the parking lot of a university campus are generated. The purpose of the analysis is to evaluate the quality of the descriptions and analyze the correspondence between the semantic content of the images and the corresponding annotation. To perform the tests, a fixed camera was placed in the campus parking lot and video sequences of about 20 minutes were obtained, later each frame was annotated individually and a text repository with all the annotations was formed. It was observed that it is possible to take advantage of the properties of the video to evaluate the performance of the annotator and the example of the crossing of a pedestrian is presented as an example for its analysis.

Super-Resolution using Deep Learning to Support Person Identification in Surveillance Video

Recently, video surveillance systems have been perceived as important technical tools that play a fundamental role in protecting people and assets. In particular, the recorded surveillance video sequences are used as evidence to solve violation, theft and criminal cases. Therefore, the identification of the person present on the crime scene becomes a critical task. In this paper, we proposed a Deep learning-based Super-Resolution system that aims to enhance the faces images captured from surveillance video in order to support suspect identification. The proposed system relies on an image processing technique called Super-Resolution that consists of recovering high-resolution images from low-resolution ones. More specifically, we used the Very-Deep Super-Resolution (VDSR) neural network to enhance the image quality. The proposed model was trained with CelebA faces dataset and used to enhance the resolution of the QMUL-SurvFace dataset. It yielded a Peak Signal-to-Noise Ratio (PSNR) improvement of 7% and Structural Similarity Index (SSIM) improvement of 3%. Most importantly, it increased the face recognition rate by 45.7%.

Motion segmentation-based surveillance video compression using adaptive particle swarm optimization

Video surveillance is one of the widely used and most active research applications of computer vision. Although lots of works have been done in the area of smart surveillance, but still there is a need of effective compression technique for compact archival and efficient transmission of vast amount of surveillance video data. In this work, we propose a hybrid video compression approach with the help of foreground motion compensation for the above application. This method works effectively by including the advantages of both block-based and object-based coding techniques as well as reducing the drawbacks of both. The proposed method first segments the foreground moving objects from the background with the help of adaptive thresholding-based optical flow techniques. Next, it determines the contour of the segmented foreground regions with the help of Freeman chain code. Subsequently, block-based motion estimation and compensation using variants of particle swarm optimization are computed. After that, motion failure areas are detected using change detection method, and finally, DCT and Huffman coding-based entropy encoding are done to compactly represent the data. Experimental results and analyses on different surveillance video sequences using Wilcoxon’s rank-sum test, PSNR and SSID show that our method outperforms other recent and relevant existing techniques.

An Optimized CNN based Real World Anomaly Detection in Surveillance Videos

Anomaly detection in automated surveillance video is an extremely monotonous process for monitoring for crowded scenes and surveillance videos are capable to incarcerate a mixture of sensible anomalies. An appropriate machine learning technique can help to train the Anomaly Detection System (ADS) in identifying anomalous activities during surveillance. To this end, we present an anomaly detection system that can be used as a tool for anomaly detection in surveillance videos using the concept of artificial intelligence. The main intention of the proposed anomaly detection system is to improve the detection time and accuracy by using the concept of Convolutional Neural Network (CNN) as artificial intelligence technique. In this paper we present a CNN based Anomaly Detection System (CNN-ADS), which is the combination of multiple layer of hidden unit with the optimized MSER feature by using Genetic Algorithm (GA). Here CNN is used for classifying the activity into normal and abnormal from the surveillance videos based on the fitness function of GA which is used for the selection of optimal MSER feature sets. Further, Self adaptive genetic algorithm (SAGA) is adopted to efficiently solve optimization problems in the continuous search domain to select the best possible feature to segregate the pattern of normal and abnormal activities. The main contribution of this research is validation of proposed system for the large scale data and we introduce a new large-scale dataset of 128 hours of videos. Dataset consists of 1900 long and untrimmed real-world surveillance videos, with 13 sensible anomalies such as road accident, burglary, fighting, robbery, etc. as well as normal activities. The experimental results of the planned system show that our CNN-ADS for anomaly detection achieve essential improvement on anomaly detection presentation as compared to the state-of-the-art approaches. The dataset is available at: https://webpages.uncc.edu/cchen62/dataset.html. In this paper, to validate the proposed ADS we provide the comparison of existing results of several recent deep learning baselines on anomalous activity detection. The real-time ADS in surveillance video sequences using SAGA based CNN with MSER feature extraction technique is implemented using Image Processing Toolbox within Matlab Software.

AED-Net: An Abnormal Event Detection Network

Abstract It has long been a challenging task to detect an anomaly in a crowded scene. In this paper, a self-supervised framework called the abnormal event detection network (AED-Net), which is composed of a principal component analysis network (PCAnet) and kernel principal component analysis (kPCA), is proposed to address this problem. Using surveillance video sequences of different scenes as raw data, the PCAnet is trained to extract high-level semantics of the crowd’s situation. Next, kPCA, a one-class classifier, is trained to identify anomalies within the scene. In contrast to some prevailing deep learning methods, this framework is completely self-supervised because it utilizes only video sequences of a normal situation. Experiments in global and local abnormal event detection are carried out on Monitoring Human Activity dataset from University of Minnesota (UMN dataset) and Anomaly Detection dataset from University of California, San Diego (UCSD dataset), and competitive results that yield a better equal error rate (EER) and area under curve (AUC) than other state-of-the-art methods are observed. Furthermore, by adding a local response normalization (LRN) layer, we propose an improvement to the original AED-Net. The results demonstrate that this proposed version performs better by promoting the framework’s generalization capacity.

Foreground target extraction method based on neighbourhood pixel intensity correction

ABSTRACT The automatic processing and prediction of monitoring video information have attracted great attention in many fields, so how to extract the foreground and target information effectively and quickly is very important. In this paper, the surveillance video sequence is taken as the processing object, and a novel foreground extraction method is proposed for the dynamic background problem. The method is divided into two steps. Firstly, in order to solve the problem of the foreground and background moving, a foreground detection method based on neighbourhood pixel intensity correction is proposed. Then, the grey projection method is used to eliminate the video jitter caused by camera shake to achieve foreground extraction. The experimental results show that the proposed method has a wide application range and can extract the foreground target clearly and accurately when both the foreground and the background are moving and video jitter.

Moving object detection algorithm based on pixel spatial sample difference consensus

Moving object detection is an essential component for security video surveillance system and other computer vision applications. Although the latest object detection methods get promising detection expectations, however, accurate detection is still a tricky problem due to various challenges such as aperture effects, illumination variations, camouflage issues and retention problems in unconstrained video environments. In this paper, we propose a brand-new theoretical framework for foreground object detection based on the stable spatial relationship between current pixel and the randomly selected pixels in current frame. Different from the existing methods which determine the moving object area by comparing each pixel value with its surrounding pixels or by comparing two pixel values occupying the same positions in adjacent frames, the proposed algorithm sets up a spatial sample set for each individual pixel and defines Spatial Sample Difference Consensus (SSDC), which denotes changes of stable spatial relationship rather than direct changes in pixel values. Thus, the proposed algorithm computes the SSDC between two adjacent frames to subtract the moving objects. The experiments on recent data-set in both indoor and outdoor surveillance video sequences show that the proposed method achieved promising performance after compared with several state-of-the art methods.

An effective algorithm to detect both smoke and flame using color and wavelet analysis

Fire detection is an important task in many applications. Smoke and flame are two essential symbols of fire in images. In this paper, we propose an algorithm to detect smoke and flame simultaneously for color dynamic video sequences obtained from a stationary camera in open space. Motion is a common feature of smoke and flame and usually has been used at the beginning for extraction from a current frame of candidate areas. The adaptive background subtraction has been utilized at a stage of moving detection. In addition, the optical flow-based movement estimation has been applied to identify a chaotic motion. With the spatial and temporal wavelet analysis, Weber contrast analysis and color segmentation, we achieved moving blobs classification. Real video surveillance sequences from publicly available datasets have been used for smoke detection with the utilization of our algorithm. We also have conducted a set of experiments. Experiments results have shown that our algorithm can achieve higher detection rate of 87% for smoke and 92% for flame.

Data Association and Prediction for Tracking Multiple Objects

Objectives: Tracking moving objects is essential for high level computer vision analysis, like object behavior interpretation or gait recognition etc. In this paper, a new method to track multiple moving objects in the surveillance video sequence is proposed. Methods/Statistical Analysis: Object tracking is done by extracting color moments feature from the segmented foreground object and associating individual objects in the successive frames using nearest neighbor classifier and Chi- Square dissimilarity measure. Further, object tracking during occlusion has been addressed by predicting the missing state of the occluded object by applying Lagrange’s polynomial extrapolation using the apriori knowledge. Findings: The proposed method is evaluated using several challenging sequences of the benchmark IEEE PETS, IEEE CHANGE DETECTION, and EPFL dataset. Further, comparative evaluation with contemporary methods has been carried out to corroborate the efficacy of the proposed method. Application/Improvements: This work focuses on devising a new method to track multiple moving objects using color moments and Lagrange’s extrapolation framework which works for a complex environment and in the presence of occlusions. Further, the shadow elimination task after the motion segmentation could be considered as an improvement step in this work which will aid in better tracking results.

Motion Segmentation from Surveillance Video using modified Hotelling's T-Square Statistics

Motion segmentation is an important task in video surveillance and in many high-level vision applications. This paper proposes two generic methods for motion segmentation from surveillance video sequences captured from different kinds of sensors like aerial, Pan Tilt and Zoom (PTZ), thermal and night vision. Motion segmentation is achieved by employing Hotelling's T-Square test on the spatial neighborhood RGB color intensity values of each pixel in two successive temporal frames. Further, a modified version of Hotelling's T-Square test is also proposed to achieve motion segmentation. On comparison with Hotelling's TSquare test, the result obtained by the modified formula is better with respect to computational time and quality of the output. Experiments along with the qualitative and quantitative comparison with existing method have been carried out on the standard IEEE PETS (2006, 2009 and 2013) and IEEE Change Detection (2014) dataset to demonstrate the efficacy of the proposed method in the dynamic environment and the results obtained are encouraging.