Objects In Surveillance Videos Research Articles

Image convolution techniques integrated with YOLOv3 algorithm in motion object data filtering and detection.

In order to address the challenges of identifying, detecting, and tracking moving objects in video surveillance, this paper emphasizes image-based dynamic entity detection. It delves into the complexities of numerous moving objects, dense targets, and intricate backgrounds. Leveraging the You Only Look Once (YOLOv3) algorithm framework, this paper proposes improvements in image segmentation and data filtering to address these challenges. These enhancements form a novel multi-object detection algorithm based on an improved YOLOv3 framework, specifically designed for video applications. Experimental validation demonstrates the feasibility of this algorithm, with success rates exceeding 60% for videos such as "jogging", "subway", "video 1", and "video 2". Notably, the detection success rates for "jogging" and "video 1" consistently surpass 80%, indicating outstanding detection performance. Although the accuracy slightly decreases for "Bolt" and "Walking2", success rates still hover around 70%. Comparative analysis with other algorithms reveals that this method's tracking accuracy surpasses that of particle filters, Discriminative Scale Space Tracker (DSST), and Scale Adaptive Multiple Features (SAMF) algorithms, with an accuracy of 0.822. This indicates superior overall performance in target tracking. Therefore, the improved YOLOv3-based multi-object detection and tracking algorithm demonstrates robust filtering and detection capabilities in noise-resistant experiments, making it highly suitable for various detection tasks in practical applications. It can address inherent limitations such as missed detections, false positives, and imprecise localization. These improvements significantly enhance the efficiency and accuracy of target detection, providing valuable insights for researchers in the field of object detection, tracking, and recognition in video surveillance.

A Review on the Identification and Classification of Activities for the Purpose of Tracking Objects in Video Surveillance

An increasing number of academics are becoming interested in the field of image processing since object tracking in video surveillance is a significant application and a growing subject of research in the field of technique known as video tracking is one that involves the utilization of a camera in order to find a moving object or many moving things throughout the course of time. As a result of its key qualities, video surveillance serves a variety of functions, including but not limited to human-computer interactions, security and surveillance, video communication, traffic management, and public areas such as airports, underground stations, and large-scale events, amongst others. Keeping track of a target in a congested area is still a difficult video surveillance task. The processing framework for video surveillanceis completed by a sequential flow that includes the detection of moving objects, their classification, tracking, and behavior identification. This study examines tracking techniques, classifies them into many categories, and concentrates on significant and practical tracking techniques. This paper discusses several tracking systems, including active contour-based, area-based, and others, along with their advantages and disadvantages. Various tracking techniques are discussed along with thorough descriptions. Finally, researchers presentan overview of potential research areas after reviewing general tactics and a scan of the literature on various methodologies.

Mayfly Optimization with Deep Learning-based Robust Object Detection and Classification on Surveillance Videos

Surveillance videos are recordings captured by video recording devices for monitoring and securing an area or property. These videos are frequently used in applications, involving law enforcement, security systems, retail analytics, and traffic monitoring. Surveillance videos can provide valuable visual information for analyzing patterns, identifying individuals or objects of interest, and detecting and investigating incidents. Object detection and classification on video surveillance involves the usage of computer vision techniques to identify and categorize objects within the video footage. Object detection algorithms are employed to locate and identify objects within each frame. These algorithms use various techniques, namely bounding box regression, Convolutional Neural Networks (CNNs), and feature extraction to detect objects of interest. This study presents the Mayfly Optimization with Deep Learning-based Robust Object Detection and Classification (MFODL-RODC) method on surveillance videos. The main aim of the MFODL-RODC technique lies in the accurate classification and recognition of objects in surveillance videos. To accomplish this, the MFODL-RODC method follows a two-step process, consisting of object detection and object classification. The MFODL-RODC method uses the EfficientDet object detector for the object detection process. Besides, the classification of detected objects takes place using the Variational Autoencoder (VAE) model. The MFO algorithm is employed to enrich the performance of the VAE model. The simulation examination of the MFODL-RODC technique is performed on benchmark datasets. The extensive results accentuated the improved performance of the MFODL-RODC method over other existing algorithms with an output of 98.89%.

Object tracking in surveillance video based on Siamese network considering geographical spatiotemporal constraints

AbstractIdentifying and tracking objects in surveillance videos is an important means of mining information during surveillance. Currently, most object‐tracking methods rely only on image features, which cannot accurately express the motion of the tracked object in real geographical scenes and are easily influenced by occlusion and surrounding objects having similar features. However, tracked objects, such as pedestrians and vehicles, usually move in geographical space with fixed patterns of motion, and the motion in a short time is constrained by geographical space and time, the motion trajectory is predictable, and the range of motion is limited. Therefore, based on the SiamFC object tracking framework, this study introduces geographical spatiotemporal constraints into the tracking framework and proposes the GeoSiamFC method. The objective of this is to: (1) construct the mapping relationship between geographical space and image space to solve the problem that the pixel movement within the image after perspective imaging cannot accurately express the motion of the tracked object in a real geographical scene; (2) add candidate search areas according to the predicted trajectory location to correct the tracking errors caused by the occlusion of the object; and (3) focus on the search for the range of motion of the mapped pixel within the image space according to the limited geographical range of motion of the tracked objects in a certain time to reduce the interference of similar objects within the search area. In this study, separate experiments were conducted on a common test dataset using multiple methods to deal with challenges such as occlusion and illumination changes. In addition, a robust test dataset with noise addition and luminance adjustment based on the common test dataset was used. The results show that GeoSiamFC outperforms other object‐tracking methods in the common test dataset with a precision score of 0.995 and a success score of 0.756 compared with most other object‐tracking algorithms under the base condition of using only shallow networks. Moreover, GeoSiamFC maintained the highest precision score (0.970) and high success score (0.734) in the more challenging robust test dataset as well. The tracking speed of 59 frames per second far exceeds the real‐time requirement of 25 FPS. Geographical spatiotemporal constraints were considered to improve tracker performance while providing real‐time feedback on the motion trajectory of the target in geographical space. Thus, the proposed method is suitable for real‐time tracking of the motion trajectory of a target in real geographical scenes in various surveillance videos.

Human Behavior Recognition of Video Surveillance System Based on Neural Network

Video surveillance system is widely used in modern society. Human behavior recognition based on neural network uses computer vision and image processing technology to analyze and judge the behavior of different objects in surveillance video. In order to improve the recognition accuracy of human behavior, this paper makes an in-depth study on the design of video surveillance system and target tracking using the relevant algorithms of neural network. In this paper, the application of relevant neural network algorithms and the technology of character behavior recognition are analyzed by means of experimental method comparison and algorithm comparison. The experimental data shows that in the detection of abnormal behavior, the highest accuracy rate of human behavior recognition is 90% by using the improved hu invariant moment model.

A video anomaly detection method based on percentile loss training and attention mechanism

Currently, the mainstream methods for video anomaly detection include frame reconstruction and video stream prediction. These methods are always confronted with the problem of over-generalization, which makes the network reconstruct or predict both normal and anomalous modes well and makes it difficult to distinguish the normal and anomalous modes. In this paper, we propose a video anomaly detection model based on the U-Net architecture, in which percentile loss (PL) training is introduced to weaken the generalization of the network by diminishing the influence of complex regular modes that tend to be anomalous on parameter update, and in turn weaken the model’s ability to predict anomalous modes. Moreover, based on the fact that most of the objects in video surveillance are stationary and abnormal behaviors tend to occur in moving objects, a variance attention module is used after the encoder module to make the network focus on moving objects that tend to contain abnormal modes. The proposed framework has been evaluated on publicly available real-world anomaly detection datasets including CUHK Avenue, UCSD and ShanghaiTech. The experiments show that the proposed algorithm improves the detection performance compared with the state-of-the-art algorithms.

Abnormal behavior detection of stationary objects in surveillance videos with visualization and classification

SUMMARYAnomaly detection in video systems has been popular over several years. It is still challenging to detect anomalies in a static object. To manage this objective, we focus on changes in the position of a stationary object in videos. In a normal scenario, the pixel values of the static object are fixed while in abnormal motion the fixed values change. We introduce a new concept to determine anomalies based on manual annotations in each video frame, only over a part of a static object in a frame such that it can be taken as a reference for the whole. Through color channel splitting we determine mask image, from which handcrafted features such as scratch area, perimeter, equivalent diameter and density are calculated. In the next step, we analyze frame‐wise changes in feature values using a linear regression model, feature values are constant when the object remains stationary while there is a rise or fall in values when an object changes location. We classify feature values through anomaly scores and thresholds. In this model, we are evaluating our proposed framework on 12 real‐time video datasets. Results are compared with existing techniques which are outperforming in terms of accuracy, mean square error and area under the curve.

Extracting Objects’ Spatial–Temporal Information Based on Surveillance Videos and the Digital Surface Model

Surveillance systems focus on the image itself, mainly from the perspective of computer vision, which lacks integration with geographic information. It is difficult to obtain the location, size, and other spatial information of moving objects from surveillance systems, which lack any ability to couple with the geographical environment. To overcome such limitations, we propose a fusion framework of 3D geographic information and moving objects in surveillance video, which provides ideas for related research. We propose a general framework that can extract objects’ spatial–temporal information and visualize object trajectories in a 3D model. The framework does not rely on specific algorithms for determining the camera model, object extraction, or the mapping model. In our experiment, we used the Zhang Zhengyou calibration method and the EPNP method to determine the camera model, YOLOv5 and deep SORT to extract objects from a video, and an imaging ray intersection with the digital surface model to locate objects in the 3D geographical scene. The experimental results show that when the bounding box can thoroughly outline the entire object, the maximum error and root mean square error of the planar position are within 31 cm and 10 cm, respectively, and within 10 cm and 3 cm, respectively, in elevation. The errors of the average width and height of moving objects are within 5 cm and 2 cm, respectively, which is consistent with reality. To our knowledge, we first proposed the general fusion framework. This paper offers a solution to integrate 3D geographic information and surveillance video, which will not only provide a spatial perspective for intelligent video analysis, but also provide a new approach for the multi-dimensional expression of geographic information, object statistics, and object measurement.

Classification of Rigid and Non-Rigid Objects Using CNN

Classifying moving objects in video surveillance can be difficult, and it is challenging to classify hard and soft objects with high Accuracy. Here rigid and non-rigid objects are limited to vehicles and people. CNN is used for the binary classification of rigid and non-rigid objects. A deep-learning system using convolutional neural networks was trained using python and categorized according to their appearance. The classification is supplemented by the use of a data set, which contains two classes of images that are both rigid and not rigid that differ by illuminations.

Automatic textual description of interactions between two objects in surveillance videos

The purpose of our work is to automatically generate textual video description schemas from surveillance video scenes compatible with police incidents reports. Our proposed approach is based on a generic and flexible context-free ontology. The general schema is of the form [actuator] [action] [over/with] [actuated object] [+ descriptors: distance, speed, etc.]. We focus on scenes containing exactly two objects. Through elaborated steps, we generate a formatted textual description. We try to identify the existence of an interaction between the two objects, including remote interaction which does not involve physical contact and we point out when aggressivity took place in these cases. We use supervised deep learning to classify scenes into interaction or no-interaction classes and then into subclasses. The chosen descriptors used to represent subclasses are keys in surveillance systems that help generate live alerts and facilitate offline investigation.

Foreground Objects Detection by U-Net with Multiple Difference Images

In video surveillance, robust detection of foreground objects is usually done by subtracting a background model from the current image. Most traditional approaches use a statistical method to model the background image. Recently, deep learning has also been widely used to detect foreground objects in video surveillance. It shows dramatic improvement compared to the traditional approaches. It is trained through supervised learning, which requires training samples with pixel-level assignment. It requires a huge amount of time and is high cost, while traditional algorithms operate unsupervised and do not require training samples. Additionally, deep learning-based algorithms lack generalization power. They operate well on scenes that are similar to the training conditions, but they do not operate well on scenes that deviate from the training conditions. In this paper, we present a new method to detect foreground objects in video surveillance using multiple difference images as the input of convolutional neural networks, which guarantees improved generalization power compared to current deep learning-based methods. First, we adjust U-Net to use multiple difference images as input. Second, we show that training using all scenes in the CDnet 2014 dataset can improve the generalization power. Hyper-parameters such as the number of difference images and the interval between images in difference image computation are chosen by analyzing experimental results. We demonstrate that the proposed algorithm achieves improved performance in scenes that are not used in training compared to state-of-the-art deep learning and traditional unsupervised algorithms. Diverse experiments using various open datasets and real images show the feasibility of the proposed method.

Foreground Gating and Background Refining Network for Surveillance Object Detection.

Detecting objects in surveillance videos is an important problem due to its wide applications in traffic control and public security. Existing methods tend to face performance degradation because of false positive or misalignment problems. We propose a novel framework, namely, Foreground Gating and Background Refining Network (FG-BR Net), for surveillance object detection (SOD). To reduce false positives in background regions, which is a critical problem in SOD, we introduce a new module that first subtracts the background of a video sequence and then generates high-quality region proposals. Unlike previous background subtraction methods that may wrongly remove the static foreground objects in a frame, a feedback connection from detection results to background subtraction process is proposed in our model to distill both static and moving objects in surveillance videos. Furthermore, we introduce another module, namely, the background refining stage, to refine the detection results with more accurate localizations. Pairwise non-local operations are adopted to cope with the misalignments between the features of original and background frames. Extensive experiments on real-world traffic surveillance benchmarks demonstrate the competitive performance of the proposed FG-BR Net. In particular, FG-BR Net ranks on the top among all the methods on hard and sunny subsets of the UA-DETRAC detection dataset, without any bells and whistles.

Minimization of Storage on Moving Objects in Video Surveillance

Minimization of Storage on Moving Objects in Video Surveillance

ESTHER: Joint Camera Self-Calibration and Automatic Radial Distortion Correction From Tracking of Walking Humans

Camera calibration and radial distortion correction are the crucial prerequisites for many applications in image big data and computer vision. Many existing works rely on the Manhattan world assumption to estimate the camera parameters automatically; however, they may perform poorly when there was lack of man-made structure in the scene. As walking humans are the common objects in video surveillance, they have also been used for camera self-calibration, but the main challenges include the noise reduction for the estimation of vanishing points, the relaxation of assumptions on unknown camera parameters, and the radial distortion correction. In this paper, we present a novel framework for camera self-calibration and automatic radial distortion correction. Our approach starts with the reliable human body segmentation that is facilitated by robust object tracking. Mean shift clustering and Laplace linear regression are, respectively, introduced in the estimation of the vertical vanishing point and the horizon line. The estimation of distribution algorithm, an evolutionary optimization scheme, is then utilized to optimize the camera parameters and the distortion coefficients, in which all the unknowns in camera projection can be fine-tuned simultaneously. Experiments on the three public benchmarks and our own captured dataset demonstrate the robustness of the proposed method. The superiority of this algorithm is also verified by the capability of reliably converting 2D object tracking into 3D space.

Spatiotemporal Coherence-Based Annotation Placement for Surveillance Videos

In this paper, we propose a novel annotation placement approach for revealing information about foreground objects in surveillance videos. To arrange positions of annotations, spatiotemporal coherence between annotations and foreground objects is applied. The annotation placement problem is formulated as an optimization problem with respect to spatiotemporal coherence of annotations and foreground objects. The optimization problem is effectively solved using Markov random fields. To the best of our knowledge, this paper is the first work that discusses and solves the annotation placement problem for surveillance videos by considering the relationships between annotations and foreground objects with trajectories. As shown in the experiments, the proposed approach can arrange annotations based on the moving trajectories of foreground objects and prevent the occlusions between different annotations and foreground objects. It also achieves better quantitative and qualitative results compared with state-of-the-art approaches.

Integration of GIS and Moving Objects in Surveillance Video

This paper discusses the integration of a geographic information system (GIS) and moving objects in surveillance videos (“moving objects” hereinafter) by using motion detection, spatial mapping, and fusion representation techniques. This integration aims to overcome the limitations of conventional video surveillance systems, such as low efficiency in video searching, redundancy in video data transmission, and insufficient capability to position video content in geographic space. Furthermore, a model for integrating GIS and moving objects is established. The model includes a moving object extraction method and a fusion pattern for GIS and moving objects. From the established integration model, a prototype of GIS and moving objects (GIS–MOV) system is constructed and used to analyze the possible applications of the integration of GIS and moving objects.

Implementation of moving object detection using angle change method This paper conclude the work in terms of input and output parameter that has been considered while detecting the moving object in video surveillance. It also provides with a look up of our work area. This paper presents a discussion of various techniques like Background subtraction, Block matching, Frame differencing, Optical

Implementation of moving object detection using angle change method This paper conclude the work in terms of input and output parameter that has been considered while detecting the moving object in video surveillance. It also provides with a look up of our work area. This paper presents a discussion of various techniques like Background subtraction, Block matching, Frame differencing, Optical

Intelligent Video Surveillance System Based on Moving Object Detection and Tracking

Aiming at the problem that traditional video surveillance system is not intelligent, a new idea of intelligent video surveillance system is proposed. Analysis of the commonly used moving object detection and tracking methods, using three frame difference and Mean Shift method for automatic detection and tracking of moving objects in video surveillance. A suspicious event rules corresponding to the definition of all kinds of security threats, application of moving object detection and tracking method, the realization of the monitoring video system of automatic alarm function, can improve the level of intelligent video surveillance system.

The Analysis of Target Trajectory Based on Genetic Evolutionary and Particle Filter

Aiming at the problem of trajectory recognition of moving objects in video surveillance, an analysis algorithm of the target trajectory based on genetic evolutionary and particle filter is presented, which is combined with genetic algorithm. Firstly generate initial parent particle population in the reference frame, use preferred evolution strategies of genetic algorithm, and select excellent particles of offspring. Then do not the corresponding calculations on the basis of crossover and mutation rules for the unselected particles until reach the number of iterations. At last, calculate object states by the weights of each particle and obtain the target trajectory. The new algorithm is not affected by the background changes. Compared with the existing algorithms, the simulation results show that it easy to realize and has more stable tracking effect.

Detection of Moving Objects in Surveillance Video by Integrating Bottom-up Approach with Knowledge Base

In the modern age, where every prominent and populous area of a city is continuously monitored, a lot of data in the form of video has to be analyzed. There is a need for an algorithm that helps in the demarcation of the abnormal activities, for ensuring better security. To decrease perceptual overload in CCTV monitoring, automation of focusing the attention on significant events happening in overpopulated public scenes is also necessary. The major challenge lies in differentiating detecting of salient motion and background motion. This paper discusses a saliency detection method that aims to discover and localize the moving regions for indoor and outdoor surveillance videos. This method does not require any prior knowledge of a scene and this has been verified with snippets of surveillance footages.