Intermittent Object Motion Research Articles

Moving Object Detection for Complex Scenes by Merging BG Modeling and Deep Learning Method

Abstract In recent years, many studies have attempted to use deep learning for moving object detection. Some research also combines object detection methods with traditional background modeling. However, this approach may run into some problems with parameter settings and weight imbalances. In order to solve the aforementioned problems, this paper proposes a new way to combine ViBe and Faster-RCNN for moving object detection. To be more specific, our approach is to confine the candidate boxes to only retain the area containing moving objects through traditional background modeling. Furthermore, in order to make the detection able to more accurately filter out the static object, the probability of each region proposal then being retained. In this paper, we compare four famous methods, namely GMM and ViBe for the traditional methods, and DeepBS and SFEN for the deep learning-based methods. The result of the experiment shows that the proposed method has the best overall performance score among all methods. The proposed method is also robust to the dynamic background and environmental changes and is able to separate stationary objects from moving objects. Especially the overall F-measure with the CDNET 2014 dataset (like in the dynamic background and intermittent object motion cases) was 0,8572.

Change detection of pulmonary embolism using isomeric cluster and computer vision

<p>Visual change detection functions in X-ray analytics and computer vision attempt to divide X-ray images toward front and backside areas. There are various difficulties in change detection such as weather changes and shadows; real-time processing; intermittent object motion; lighting variation; and diverse object forms. Traditionally, this issue has been addressed via backdrop modeling methods and the creation of custom features. We present a new feature descriptor called pulmonary embolism detection using isomeric cluster (PEDIC), uses the concept of isomerism. The isomeric and cluster isomerism characteristics of the PEDIC are distinguish it from other graphs. At isomeric thetical orientations, the cluster pattern corresponds to consecutive differences in pixel intensity between the two images. Also, the clusters are oppositely orientated, and both clusters conform to a specified isomeric feature. The local area's lines and corner point information are identified and recorded using the PEDIC in several different directions. We introduced multiresolution PEDIC, which incorporates the multiresolution Gaussian filter to achieve increased resilience in the system. We expanded our research to include rotation-invariant characteristics. We also proposed inter-PEDIC and intra-PEDIC to identify motion changes in X-ray sequences, which allowed them to extract spatiotemporal characteristics.</p>

An Empirical Review of Deep Learning Frameworks for Change Detection: Model Design, Experimental Frameworks, Challenges and Research Needs

Visual change detection, aiming at segmentation of video frames into foreground and background regions, is one of the elementary tasks in computer vision and video analytics. The applications of change detection include anomaly detection, object tracking, traffic monitoring, human machine interaction, behavior analysis, action recognition, and visual surveillance. Some of the challenges in change detection include background fluctuations, illumination variation, weather changes, intermittent object motion, shadow, fast/slow object motion, camera motion, heterogeneous object shapes and real-time processing. Traditionally, this problem has been solved using hand-crafted features and background modelling techniques. In recent years, deep learning frameworks have been successfully adopted for robust change detection. This article aims to provide an empirical review of the state-of-the-art deep learning methods for change detection. More specifically, we present a detailed analysis of the technical characteristics of different model designs and experimental frameworks. We provide model design based categorization of the existing approaches, including the 2D-CNN, 3D-CNN, ConvLSTM, multi-scale features, residual connections, autoencoders and GAN based methods. Moreover, an empirical analysis of the evaluation settings adopted by the existing deep learning methods is presented. To the best of our knowledge, this is a first attempt to comparatively analyze the different evaluation frameworks used in the existing deep change detection methods. Finally, we point out the research needs, future directions and draw our own conclusions.

Automatic Change Detection System over Unmanned Aerial Vehicle Video Sequences Based on Convolutional Neural Networks.

In recent years, the use of unmanned aerial vehicles (UAVs) for surveillance tasks has increased considerably. This technology provides a versatile and innovative approach to the field. However, the automation of tasks such as object recognition or change detection usually requires image processing techniques. In this paper we present a system for change detection in video sequences acquired by moving cameras. It is based on the combination of image alignment techniques with a deep learning model based on convolutional neural networks (CNNs). This approach covers two important topics. Firstly, the capability of our system to be adaptable to variations in the UAV flight. In particular, the difference of height between flights, and a slight modification of the camera’s position or movement of the UAV because of natural conditions such as the effect of wind. These modifications can be produced by multiple factors, such as weather conditions, security requirements or human errors. Secondly, the precision of our model to detect changes in diverse environments, which has been compared with state-of-the-art methods in change detection. This has been measured using the Change Detection 2014 dataset, which provides a selection of labelled images from different scenarios for training change detection algorithms. We have used images from dynamic background, intermittent object motion and bad weather sections. These sections have been selected to test our algorithm’s robustness to changes in the background, as in real flight conditions. Our system provides a precise solution for these scenarios, as the mean F-measure score from the image analysis surpasses 97%, and a significant precision in the intermittent object motion category, where the score is above 99%.

SEnDec: An Improved Image to Image CNN for Foreground Localization

Although it is not immediately intuitive that Deep Convolutional Neural Networks (DCNNs) can yield adequate feature representation for a Foreground Localization (FGL) task, recent architectural and algorithmic advancements in Deep Learning (DL) have shown that the DCNNs have become forefront methodology for this pixel-level classification problem. In FGL, the DCNNs face an inherent trade-off between moving objects, i.e., the foreground (FG) and the non-static background (BG) scenes, through learning from local- and global-level features. Driven by the latest success of the innovative structures for image classification and semantic segmentation, this work introduces a novel architecture, called Slow Encoder-Decoder (sEnDec) that aims to improve the learning capacity of a traditional image-to-image DCNN. The proposed model subsumes two subnets for contraction (encoding) and expansion (decoding), wherein both phases, it employs an intermediate feature map up-sampling and residual connections. In this way, the lost structural details due to spatial subsampling are recovered. It helps to get a more delineated FG region. The experimental study is carried out with two variants of the proposed model: one with strided convolution (conv) and the other with max pooling for spatial subsampling. A comparative analysis on sixteen benchmark video sequences, including baseline, dynamic background, camera jitter, shadow effects, intermittent object motion, night videos, and bad weather show that the proposed sEnDec model performs very competitively against the prior- and state-of-the-art approaches.

Video Foreground Extraction Using Multi-View Receptive Field and Encoder–Decoder DCNN for Traffic and Surveillance Applications

The automatic detection of foreground (FG) objects in videos is a demanding area of computer vision, with essential applications in video-based traffic analysis and surveillance. New solutions have attempted exploiting deep neural network (DNN) for this purpose. In DNN, learning agents, i.e., features for video FG object segmentation is nontrivial, unlike image segmentation. It is a temporally processed decision-making problem, where the agents involved are the spatial and temporal correlations of the FG objects and the background (BG) of the scene. To handle this and to overcome the conventional DL models’ poor delineation at the borders of FG regions due to fixed-view receptive filed-based learning, this work introduces a Multi-view Receptive Field Encoder-Decoder Convolutional Neural Network called MvRF-CNN. The main contribution of the model is harnessing multiple views of convolutional (conv) kernels with residual feature fusions at early, mid and late stages in an encoder-decoder (EnDec) architecture. It enhances the ability of the model to learn condition-invariant agents resulting in highly delineated FG masks when compared to the existing approaches from heuristic- to DL-based techniques. The model is trained with sequence-specific labeled samples to predict scene-specific pixel-level labels of FG objects in near static scenes with a minute dynamism. The experimental study on 37 video sequences from traffic and surveillance scenarios that include complex environments, viz. dynamic background, camera jittery, intermittent object motion, scenes with cast shadows, night videos, and lousy weather proves the effectiveness of the model. The study covers two input configurations: a 3-channel (RGB) single frame and a 3-channel double-frame with a BG such that two consecutive grayscale frames stacked with a prior BG model. The ablation investigations are also conducted to show the importance of transfer learning (TL) and mid-fusion approaches for enhancing the segmentation performance and the model's robustness on failure modes: when there is lack of manually annotated hard ground truths (HGT) and testing the model under non-scene-specific videos. In overall, the model achieves a figure-of-merit of ${95\%}$ and 42 $FPS$ of mean average performance.

Locally Statistical Dual-Mode Background Subtraction Approach

Due to the variety of background model in the real world, detecting changes in a video cannot be addressed exhaustively by a simple background subtraction method, especially with several motion detection challenges, such as dynamic background, camera jitter, intermittent object motion, and so on. In this paper, we propose an efficient background subtraction method, namely locally statistical dual-mode (LSD), for detecting moving objects in video-based surveillance systems. The method includes a local intensity pattern comparison algorithm for foreground segmentation by analyzing the homogeneity of intensity patterns of the input frame and the background model, in which the homogeneity is calculated by the mean and standard deviation of pixel intensity. Besides that, a dual-mode scheme is developed to temporally update the background model for the short- and long-term scenarios corresponding to sudden and gradual changes in the background. The advantage of this scheme is the allowance of updating the model in both pixel- and frame-wise manners simultaneously. The parameters used in both the local intensity pattern comparison algorithm and the dual-mode background model updating scheme are estimated for every input frame consecutively based on local and global statistical information of segmentation result. In experiments, the proposed LSD method is extensively evaluated on the Wallflower and CDnet2014 datasets; and remarkable performance demonstrates its preeminence to the many state-of-the-art background subtraction approaches in terms of segmentation accuracy and computational complexity.

An Adaptive Background Modeling Method for Foreground Segmentation

Background modeling has played an important role in detecting the foreground for video analysis. In this paper, we presented a novel background modeling method for foreground segmentation. The innovations of the proposed method lie in the joint usage of the pixel-based adaptive segmentation method and the background updating strategy, which is performed in both pixel and object levels. Current pixel-based adaptive segmentation method only updates the background at the pixel level and does not take into account the physical changes of the object, which may result in a series of problems in foreground detection, e.g., a static or low-speed object is updated too fast or merely a partial foreground region is properly detected. To avoid these deficiencies, we used a counter to place the foreground pixels into two categories (illumination and object). The proposed method extracted a correct foreground object by controlling the updating time of the pixels belonging to an object or an illumination region respectively. Extensive experiments showed that our method is more competitive than the state-of-the-art foreground detection methods, particularly in the intermittent object motion scenario. Moreover, we also analyzed the efficiency of our method in different situations to show that the proposed method is available for real-time applications.

Long-Term Stationary Object Detection Based on Spatio-Temporal Change Detection

We present a block-wise approach to detect stationary objects based on spatio-temporal change detection. First, block candidates are extracted by filtering out consecutive blocks containing moving objects. Then, an online clustering approach groups similar blocks at each spatial location over time via statistical variation of pixel ratios. The stability changes are identified by analyzing the relationships between the most repeated clusters at regular sampling instants. Finally, stationary objects are detected as those stability changes that exceed an alarm time and have not been visualized before. Unlike previous approaches making use of Background Subtraction, the proposed approach does not require foreground segmentation and provides robustness to illumination changes, crowds and intermittent object motion. The experiments over an heterogeneous dataset demonstrate the ability of the proposed approach for short- and long-term operation while overcoming challenging issues.

Moving object detection method based on complementary multi resolution background models

A novel moving object detection method was proposed in order to adapt the difficulties caused by intermittent object motion, thermal and dynamic background sequences. Two groups of complementary Gaussian mixture models were used. The ghost and real static object could be classified by comparing the similarity of the edge images further. In each group, the multi resolution Gaussian mixture models were used and dual thresholds were applied in every resolution in order to get a complete object mask without much noise. The computational color model was also used to depress illustration variations and light shadows. The proposed method was verified by the public test sequences provided by the IEEE Change Detection Workshop and compared with three state-of-the-art methods. Experimental results demonstrate that the proposed method is better than others for all of the evaluation parameters in intermittent object motion sequences. Four and two in the seven evaluation parameters are better than the others in thermal and dynamic background sequences, respectively. The proposed method shows a relatively good performance, especially for the intermittent object motion sequences.