Objects In Video Sequences Research Articles

Self-supervised spatial–temporal feature enhancement for one-shot video object detection

One-shot video object detection is a task that aims to locate and identify objects in video sequences given only a single video sample for each class. Exploration in this field is still in its infancy and previous few-shot video object detection methods have limitation in this task. In this paper, we propose the Self-supervised Feature Enhancement (SFE) framework to address one-shot video object detection task. SFE includes two important modules: Hybrid Spatial Self-supervised Feature Enhancement (HSSFE) and Dynamic Temporal Self-supervised Feature Enhancement (DTSFE). HSSFE enhances features from a spatial perspective with spatial self-supervised auxiliary tasks at frame and instance levels. DTSFE on the other hand enhances features from a temporal perspective with memory-based self-supervised constraint for the same object across different frames. We have conducted experiments on multiple benchmarks, and the results demonstrate that our method achieves state-of-the-art performance.

Moving object detection based on bioinspired background subtraction

Flying insects rely mainly upon visual motion to detect and track objects. There has been a lot of research on fly inspired algorithms for object detection, but few have been developed based on visual motion alone. One of the daunting difficulties is that the neural and circuit mechanisms underlying the foreground-background segmentation are still unclear. Our previous modeling study proposed that the lobula held parallel pathways with distinct directional selectivity, each of which could retinotopically discriminate figures moving in its own preferred direction based on relative motion cues. The previous model, however, did not address how the multiple parallel pathways gave the only detection output at their common downstream. Since the preferred directions of the pathways along either horizontal or vertical axis were opposite to each other, the background moving in the opposite direction to an object also activated the corresponding lobula pathway. Indiscriminate or ungated projection from all the pathways to their downstream would mix objects with the moving background, making the previous model fail with non-stationary background. Here, we extend the previous model by proposing that the background motion-dependent gating of individual lobula projections is the key to object detection. Large-field lobula plate tangential cells are hypothesized to perform the gating to realize bioinspired background subtraction. The model is shown to be capable of implementing a robust detection of moving objects in video sequences with either a moving camera that induces translational optic flow or a static camera. The model sheds light on the potential of the concise fly algorithm in real-world applications.

Detection and localization of anomalous objects in video sequences using vision transformers and U-Net model

Detection and localization of anomalous objects in video sequences using vision transformers and U-Net model

Block based motion estimation model using CNN with representative point matching algorithm for object tracking in videos

Motion estimation is considered significant for tracking the movement of an object in video sequences, and it is widely used in various video processing applications. Traditionally, many researchers focus on pixel-based motion estimation for object tracking, but it experienced increased computation time and cost. To reduce computation time, the utilization of a block-based motion estimation approach for object tracking is a recent trend. The existing block-based approach faces difficulty in finding representative points within the intensity domain. Therefore, this current research merged the deep learning approach with a block-matching algorithm for achieving efficient object tracking. In this proposed work, initially, video sequences are collected from a benchmark video dataset. Then, the acquired video sequences are segmented into frames. From the segmented frames, current and previous frames are considered for motion estimation. Frames are sent for the data augmentation process in which the process of flipping, cropping, and rotation is carried out. Then, the augmented frames are sent into Convolutional Neural Network (CNN) for feature extraction. Representative Point Matching (RPM) is used to estimate the motion vector based on the extracted features. After estimating the motion vector, the similarity between two consecutive frames is found using Structural Similarity Index (SSIM) technique. Finally, based on the similarity score, the movement of an object in the video is tracked effectively. Simulation analysis of the proposed block-based motion estimation model is done by evaluating some performance metrics. RMSE, PSNR, Execution Time, SSIM, and accuracy obtained for the proposed model are 27.5, 26.5 db, 31 sec, 0.91, and 94 %. This analysis suggested that the proposed CNN-RPM motion estimation model performs better in tracking the movement of the object.

HAtt-Flow: Hierarchical Attention-Flow Mechanism for Group-Activity Scene Graph Generation in Videos.

Group-activity scene graph (GASG) generation is a challenging task in computer vision, aiming to anticipate and describe relationships between subjects and objects in video sequences. Traditional video scene graph generation (VidSGG) methods focus on retrospective analysis, limiting their predictive capabilities. To enrich the scene-understanding capabilities, we introduced a GASG dataset extending the JRDB dataset with nuanced annotations involving appearance, interaction, position, relationship, and situation attributes. This work also introduces an innovative approach, a Hierarchical Attention-Flow (HAtt-Flow) mechanism, rooted in flow network theory to enhance GASG performance. Flow-attention incorporates flow conservation principles, fostering competition for sources and allocation for sinks, effectively preventing the generation of trivial attention. Our proposed approach offers a unique perspective on attention mechanisms, where conventional "values" and "keys" are transformed into sources and sinks, respectively, creating a novel framework for attention-based models. Through extensive experiments, we demonstrate the effectiveness of our Hatt-Flow model and the superiority of our proposed flow-attention mechanism. This work represents a significant advancement in predictive video scene understanding, providing valuable insights and techniques for applications that require real-time relationship prediction in video data.

Vehicle Counting Method Based On Gaussian Mixture Models And Blob Analysis

Combination of a Gaussian Mixture Models and Entropy to detect vehicles in motion from any sort of video sequences. The proposed method is composed of four major steps: background subtraction, improved GMM, Blob Analysis, Voting Scheme processing. An implementation of proposed technique has been performed using MATLAB R2017a. Vehicle detection process provides relevant information about traffic patterns, crash occurrences and traffic peak times in roadways. Detection of number of vehicles and traffic monitoring will be helpful by implementing motion and object detection using background subtraction, Monitoring traffic from video footages will provide a greater reality in terms of vehicle count. Gaussian Mixture Model is most advanced model in motion detection due to the reliability that it has shown in the background extraction and foreground segmentation process. Here we are going to find motion of object in video sequences under both static camera arrangement and dynamic background sequences by using background subtraction method and blob analysis approach and analyzing them

Detection and Tracking of Moving Object-Using Kalman Filter Enhancement (KF) by Grasshopper Optimization Algorithm

Today, the process of tracking moving objects in Video sequence has many real applications such as robots' systems, surveillance systems, and monitoring systems, visual information processing, and so more. The first stage in tracking moving object systems is detection the target in Video sequence and images and the second stage is to track the identified object. In this regard, Automatic detection and tracking of object is an attractive scope in now researches, so the proposed method based on the detection-tracking target moving objects by using the Kalman Improved Filter (KF) method. The Kalman filter, assuming initial state and noise covariance parameter for detection the object, which are critical parameters for estimating speed. For successful tracking by Kalman filter, the noise covariance matrix must be optimized. Therefore, in many studies, different methods based on optimization and metaheuristic algorithms have been proposed to increase the performance of Kalman filter method. In this research to adjust, the noise covariance is of the Kalman filter for object tracking and improve the initial parameters of it, using grasshopper optimization algorithm (GOA). Here considered not only the properties of the object, but also the estimation of the motion of the object to speed up the search process. To compare the new method's efficiency and accuracy, we used MATLAB R2019b software. The results show that the proposed method has improved at least 10% in accuracy compared to RMOT and CMOT and 20% in recovery compared to Recall. In addition, the proposed method has at least a 2% improvement in Recall parameters and accuracy compared to the evaluated article.

Selected Three Frame Difference Method for Moving Object Detection

Three frame difference is one of the well-known method that is used to perform moving object detection. According to the theory, the presence of a moving object is estimated by subtracting consecutive three image frames that provide moving object edges. However, these edges do not give complete information of the moving object which means that the method leads to loss of information. Some post-processing methods such as morphological operations, optical flow and combining these techniques are necessary to be apply for obtaining complete information of moving object. In this paper, we present a new approach called Selected Three Frame Difference (STFD) to detect moving object in video sequences without any post processing operations. We initially propose an algorithm that selects three images considering the local maximum value of frame differences. Instead of using consecutive three frames, these three selected image differences that include non-overlapping object frames are applied to the logical and operator. We mathematically prove that the entire moving object is always detected in the second selected image. We tested the proposed method on public benchmark and real datasets collected from our laboratory. To validate the performance of our approach, we also compared with three frame difference method on all dataset.

Background Subtraction of an Indian Classical Dance Videos using Adaptive Temporal Averaging Method

A slew of motion detection methods have been proposed in recent years. The background includes some constraints such as changes in illumination, shadow, cluttered the background, scene change and speed of dance between hand gestures and body gestures are different. One of the most basic methods for background subtraction is temporal averaging. We looked at a new adaptive temporal averaging approach in this paper. To identify moving objects in video sequences, an adaptive temporal averaging technique is used. Depending upon the speed of the technique we proposed a Gaussian distribution technique. Gaussian distribution done background subtraction depending upon active pixels it differentiates whether it is a background or foreground. The background model's update rate has been modified to be adaptive and determined by pixel difference .Our aim is to improve the method's F-measure by making it more adaptable to various scene scenarios. The experiment results are shown and evaluated. The proposed method and the original method's quality parameters are compared

Anomalous object detection by active search with PTZ cameras

Due to the large amount of visual information generated daily, proposals that automatically analyze and process data are becoming increasingly necessary. This work focuses on the detection of anomalous objects in video sequences captured by PTZ (pan-tilt-zoom) cameras, considering as anomalies the objects which belong to categories that should not appear in a specific scene (e.g. pedestrians on a highway). There is a lack in the previous literature of a principled approach for the control of PTZ cameras that takes advantage of the recent developments in deep learning-based object detection. Our proposal aims to fill this gap by offering a probabilistic framework where the guidance of PTZ cameras is accommodated. The proposed methodology involves three different modules. An object detection stage, where deep learning networks are used to detect the objects that appear in the scene; an anomalous detection module, where a mixture of Dirichlet distributions is considered to detect automatically, and without supervised training, those detected objects which are likely to be anomalous; and finally, a PTZ camera controller which allows to follow and focus on the object considered as the most probably anomalous in the scene. The experimental results show the performance and viability of our proposal, which outperforms several competitors from qualitative and quantitative points of view.

MS-Faster R-CNN: Multi-Stream Backbone for Improved Faster R-CNN Object Detection and Aerial Tracking from UAV Images

Tracking objects across multiple video frames is a challenging task due to several difficult issues such as occlusions, background clutter, lighting as well as object and camera view-point variations, which directly affect the object detection. These aspects are even more emphasized when analyzing unmanned aerial vehicles (UAV) based images, where the vehicle movement can also impact the image quality. A common strategy employed to address these issues is to analyze the input images at different scales to obtain as much information as possible to correctly detect and track the objects across video sequences. Following this rationale, in this paper, we introduce a simple yet effective novel multi-stream (MS) architecture, where different kernel sizes are applied to each stream to simulate a multi-scale image analysis. The proposed architecture is then used as backbone for the well-known Faster-R-CNN pipeline, defining a MS-Faster R-CNN object detector that consistently detects objects in video sequences. Subsequently, this detector is jointly used with the Simple Online and Real-time Tracking with a Deep Association Metric (Deep SORT) algorithm to achieve real-time tracking capabilities on UAV images. To assess the presented architecture, extensive experiments were performed on the UMCD, UAVDT, UAV20L, and UAV123 datasets. The presented pipeline achieved state-of-the-art performance, confirming that the proposed multi-stream method can correctly emulate the robust multi-scale image analysis paradigm.

Object detection and tracking in video sequences: formalization, metrics and results

One of the promising areas of development and implementation of artificial intelligence is the automatic detection and tracking of moving objects in video sequence. The paper presents a formalization of the detection and tracking of one and many objects in video. The following metrics are considered: the quality of detection of tracked objects, the accuracy of determining the location of the object in a frame, the trajectory of movement, the accuracy of tracking multiple objects. Based on the considered generalization, an algorithm for tracking people has been developed that uses the tracking through detection method and convolutional neural networks to detect people and form features. Neural network features are included in a composite descriptor that also contains geometric and color features to describe each detected person in the frame. The results of experiments based on the considered criteria are presented, and it is experimentally confirmed that the improvement of the detector operation makes it possible to increase the accuracy of tracking objects. Examples of frames of processed video sequences with visualization of human movement trajectories are presented.

Robust Segmentation and Classification of Moving Objects from Surveillance Video

Object segmentation and classification in the video sequence is a classical and critical problem that is constantly addressed due to vast real-time applications such as autonomous vehicles, smart surveillance, etc. Segmentation and classification of moving objects in video sequences captured from real-time non-constraint sequences is a challenging task. This paper presents a moving object segmentation and classification method for video sequences captured in a real-time environment. The key contributions of the paper include a method to detect and segment motion regions by applying the non-parametric Kolmogorov–Smirnov statistical test in the Spatio-temporal domain and a probabilistic neural network-based classification method to classify the moving objects into various classes. Promising results are obtained by experimentation using challenging PETS and Change Detection datasets. To corroborate the efficacy, a comparative analysis with contemporary method is also performed.

DeepFoveaNet: Deep Fovea Eagle-Eye Bioinspired Model to Detect Moving Objects.

Birds of prey especially eagles and hawks have a visual acuity two to five times better than humans. Among the peculiar characteristics of their biological vision are that they have two types of foveae; one shallow fovea used in their binocular vision, and a deep fovea for monocular vision. The deep fovea allows these birds to see objects at long distances and to identify them as possible prey. Inspired by the biological functioning of the deep fovea a model called DeepFoveaNet is proposed in this paper. DeepFoveaNet is a convolutional neural network model to detect moving objects in video sequences. DeepFoveaNet emulates the monocular vision of birds of prey through two Encoder-Decoder convolutional neural network modules. This model combines the capacity of magnification of the deep fovea and the context information of the peripheral vision. Unlike algorithms to detect moving objects, ranked in the first places of the Change Detection database (CDnet14), DeepFoveaNet does not depend on previously trained neural networks, neither on a huge number of training images for its training. Besides, its architecture allows it to learn spatiotemporal information of the video. DeepFoveaNet was evaluated in the CDnet14 database achieving high performance and was ranked as one of the ten best algorithms. The characteristics and results of DeepFoveaNet demonstrated that the model is comparable to the state-of-the-art algorithms to detect moving objects, and it can detect very small moving objects through its deep fovea model that other algorithms cannot detect.

Moving Object Segmentation Using Level Set Algorithm with GWO-KFCM Clustering

Video analytics is widely used to automatically analyze the videos to extract the required information and detect the various events object identification and traffic analysis. The segmentation of the image is referring to extracting the required region from an image. The major objective of the segmentation process is to cluster the images without being affected by the noises. The detection of the moving objects is a challenging task in video analysis due to the dynamic background of the video. The major drawback of the existing Kernel Fuzzy C-Means (KFCM) clustering is initialization of random centroids which increases the execution time to identify the segmented portions. In this research, the Grey Wolf Optimization (GWO) algorithm used to initialize the centroids of required clusters in KFCM and Level Set (LS) Algorithm is used to segment the objects in video sequence. The proposed KFCM-GWO-LS is implemented for moving and static object detection in the videos obtained from SBM-RGBD dataset. For object detection, determining central clusters are important which is performed by using KFCM. GWO helps in finding the best centroids clusters by matching with KFCM. The centroids clusters are segmented by using LS algorithm which undergoes over segmentation problem that is overcome by GWO. As all the three techniques are dependent on one another hybrid of all these techniques obtains better results. The proposed KFCM-GWO-LS is evaluated in terms of Recall, Specificity, Precision and F-measure and the experimental results showed that the proposed method improved the system performance from 0.3 % to 14.35 % compared to existing methods Multi-Sensor Scheduler Algorithm and Statistical Inference Theory model.

Reliability Analysis of Multi-Objective Spatio-Temporal Segmentation of Human Motion in Video Sequences

In view of the problem of uneven distribution of edge contour of multi-target human motion image in video sequence, which leads to the decline of target detection ability, an algorithm of multi-target spatial-temporal segmentation of human motion in video sequence based on edge contour feature detection and block fusion is proposed. Firstly, a multi-target spatial-temporal detection model of human motion in video sequence was constructed, extracting video image frame sequence, using discrete frame fusion method to segment and fuse moving target image, matching moving multi-target in video sequence, secondly segmenting motion features in moving target image, combining with SURF algorithm (speeded up robust features, accelerated robust features) to detect and extract human motion objects in video sequence. The experimental results show that the gray histogram of human motion multi-target space-time segmentation is close to the original image histogram, and the detection and recognition ability of human motion target is improved.

Learning attention for object tracking with adversarial learning network

Artificial intelligence has been widely studied on solving intelligent surveillance analysis and security problems in recent years. Although many multimedia security approaches have been proposed by using deep learning network model, there are still some challenges on their performances which deserve in-depth research. On the one hand, high computational complexity of current deep learning methods makes it hard to be applied to real-time scenario. On the other hand, it is difficult to obtain the specific features of a video by fine-tuning the network online with the object state of the first frame, which fails to capture rich appearance variations of the object. To solve above two issues, in this paper, an effective object tracking method with learning attention is proposed to achieve the object localization and reduce the training time in adversarial learning framework. First, a prediction network is designed to track the object in video sequences. The object positions of the first ten frames are employed to fine-tune prediction network, which can fully mine a specific features of an object. Second, the prediction network is integrated into the generative adversarial network framework, which randomly generates masks to capture object appearance variations via adaptively dropout input features. Third, we present a spatial attention mechanism to improve the tracking performance. The proposed network can identify the mask that maintains the most robust features of the objects over a long temporal span. Extensive experiments on two large-scale benchmarks demonstrate that the proposed algorithm performs favorably against state-of-the-art methods.

AN IMPROVED MOVING OBJECTS DETECTION ALGORITHM IN VIDEO SEQUENCES

Context. The implementation of video analytics functions in video surveillance systems makes it possible to increase the efficiency of these systems. One of the functions of these intelligent video surveillance systems is to detect dynamic objects in the surveillance sectors of video surveillance cameras. Existing methods of background subtractoin and object recognition have important disadvantages that limit their application in practice: under low contrast algorithms can not select an object from the background; some moving objects can be recognized as a background, algorithms critical to lighting conditions, and so on. Therefore, an important task is to develop and improve methods for detecting dynamic objects in video sequences.Objective. The research is devoted to the development of an improved method for detecting dynamic objects in video sequences.Method. For moving objects detection in video sequences we used background subtraction methods based on pixel-by-pixel analysis of frames using elements of the expert systems theory.Results. In this paper, we propose an improved method for detecting dynamic objects in video sequences, which is based on the ViBe algorithm. The proposed approach differs from original the using of U*V*W* color model, using double threshold levels and some elements of theory expert systems for removal of vaguenesses in pixel classification (Dempster-Shafer theory) and dynamic method for updating background pixel models. Proposed algorithm include following stages: initialization of the background model (for each pixel with known parameters, the number of previous values in the current frame is stored); foreground detection; the next step is a calculation amounts of points, that belong to the foreground and to the background. For removal of vaguenesses in pixel classification we used some elements of Dempster-Shafer theory. After initialization of the background model and foreground detection next stage is updating background model. For this we used a three-level constructed neighborhood of the studied pixel and used of the even distribution of random values is into each of three levels.Conclusions. Experimental research of the improved algorithm in comparing to original ViBe conducted with the use of test frames from a set of CDNET in the various variants of environment and with the different variants of discriminability. The consolidated results specify on the improvement of results of an offer method as compared to original ViBe on the average on 6,7%.

Optimization of a Tracking System Based on a Network of Cameras

Tracking the motion of objects in video sequences is an important problem of computer vision that has a wide range of applications. The key points in tracking systems is the detection of an object and, if it was detected repeatedly, its reidentification. A fast correctly working tracking system that uses a number of cameras is described. The system includes detection and segmentation of objects in images, construction of their appearance descriptors, comparison of each new object with earlier collected objects, and making a decision about their reidentification. The basic system configuration is implemented in which the state-of-the art detection algorithms and models for constructing the appearance descriptors are used as the constituent parts. Based on this, the system as a whole and some of its modules are modified. A computational experiment that quantitatively confirms the advantages of the modified system over the basic system is performed.

Asynchronous Semantic Background Subtraction.

The method of Semantic Background Subtraction (SBS), which combines semantic segmentation and background subtraction, has recently emerged for the task of segmenting moving objects in video sequences. While SBS has been shown to improve background subtraction, a major difficulty is that it combines two streams generated at different frame rates. This results in SBS operating at the slowest frame rate of the two streams, usually being the one of the semantic segmentation algorithm. We present a method, referred to as “Asynchronous Semantic Background Subtraction” (ASBS), able to combine a semantic segmentation algorithm with any background subtraction algorithm asynchronously. It achieves performances close to that of SBS while operating at the fastest possible frame rate, being the one of the background subtraction algorithm. Our method consists in analyzing the temporal evolution of pixel features to possibly replicate the decisions previously enforced by semantics when no semantic information is computed. We showcase ASBS with several background subtraction algorithms and also add a feedback mechanism that feeds the background model of the background subtraction algorithm to upgrade its updating strategy and, consequently, enhance the decision. Experiments show that we systematically improve the performance, even when the semantic stream has a much slower frame rate than the frame rate of the background subtraction algorithm. In addition, we establish that, with the help of ASBS, a real-time background subtraction algorithm, such as ViBe, stays real time and competes with some of the best non-real-time unsupervised background subtraction algorithms such as SuBSENSE.