Robust Moving Object Detection Research Articles

DeepFTSG: Multi-stream Asymmetric USE-Net Trellis Encoders with Shared Decoder Feature Fusion Architecture for Video Motion Segmentation

Discriminating salient moving objects against complex, cluttered backgrounds, with occlusions and challenging environmental conditions like weather and illumination, is essential for stateful scene perception in autonomous systems. We propose a novel deep architecture, named DeepFTSG, for robust moving object detection that incorporates single and multi-stream multi-channel USE-Net trellis asymmetric encoders extending U-Net with squeeze and excitation (SE) blocks and a single shared decoder network for fusing multiple motion and appearance cues. DeepFTSG is a deep learning based approach that builds upon our previous hand-engineered flux tensor split Gaussian (FTSG) change detection video analysis algorithm which won the CDNet CVPR Change Detection Workshop challenge competition. DeepFTSG generalizes much better than top-performing motion detection deep networks, such as the scene-dependent ensemble-based FgSegNet_v2, while using an order of magnitude fewer weights. Short-term motion and longer-term change cues are estimated using general-purpose unsupervised methods—flux tensor and multi-modal background subtraction, respectively. DeepFTSG was evaluated using the CDnet-2014 change detection challenge dataset, the largest change detection video sequence benchmark with 12.3 billion labeled pixels, and had an overall F-measure of 97%. We also evaluated the cross-dataset generalization capability of DeepFTSG trained solely on CDnet-2014 short video segments and then evaluated on unseen SBI-2015, LASIESTA and LaSOT benchmark videos. On the unseen SBI-2015 dataset, DeepFTSG had an F-measure accuracy of 87%, more than 30% higher compared to the top-performing deep network FgSegNet_v2 and outperforms the recently proposed KimHa method by 17%. On the unseen LASIESTA, DeepFTSG had an F-measure of 88% and outperformed the best recent deep learning method BSUV-Net2.0 by 3%. On the unseen LaSOT with axis-aligned bounding box ground-truth, network segmentation masks were converted to bounding boxes for evaluation, DeepFTSG had an F-Measure of 55%, outperforming KimHa method by 14% and FgSegNet_v2 by almost 1.5%. When a customized single DeepFTSG model is trained in a scene-dependent manner for comparison with state-of-the-art approaches, then DeepFTSG performs significantly better, reaching an F-Measure of 97% on SBI-2015 (+ 10%) and 99% on LASIESTA (+ 11%). The source code, pre-trained weights, and video demo for DeepFTSG are available at https://github.com/CIVA-Lab/DeepFTSG.

Robust moving object detection based on fusing Atanassov's Intuitionistic 3D Fuzzy Histon Roughness Index and texture features

Background modeling is a crucial step in various computer vision applications such as video surveillance, object tracking, and moving object detection. Classifying image pixels as foreground or background is yet a challenging task particularly in complicated situations such as illumination variations, rippling water, camera jitter, and the presence of fast and slow moving objects. Therefore, for a better detection of the moving objects, the multi-modal nature of a scene in those intricate situations should be modeled by multiple models for each image pixel. To this end, in this article, we improve our previous work by fusing color features and texture features using Choquet fuzzy integral. Thereby, our proposed spatial color features that are described by Atanassov's Intuitionistic 3D Fuzzy Histon Roughness Index are fused by the texture features extracted using a covariance matrix. As handling multi-modal background updating is an arduous task, we also propose a new model updating for tackling various challenges such as model initializing with moving objects, existence of fast and slow moving objects in a scene, and existence of the moving objects that stop for a while. We intensively evaluate our proposed approach on diverse benchmark datasets. Experimental results demonstrate the robustness and supremacy of our proposed approach compared to its previous version and the state-of-the-art algorithms in the field.

OBJECT DETECTION UNDER MOVING CLOUD SHADOWS IN WAMI

Abstract. For a reliable and robust moving object detection, the subtraction of a precisely modeled background is crucial in wide-area motion imagery (WAMI). Even the most successful background subtraction algorithms that are designed to model highly-dynamic environments cannot cope with rapidly changing scenery, such as moving cloud shadows, which has different characteristics from dynamic textures. This paper presents a novel method to detect moving objects and to eliminate false alarms under moving cloud shadow regions in gray-level video sequences. The proposed method uses the relation between reflectance values of the shadowed and well-illuminated sequences of the regions in the video frame. A modified adaptive region growing approach, which extends from seed points, is designed to obtain the moving parts of the cloud shadows without presuming the geometric structure of the clouds. In order to determine the moving border of the cloud shadows, where false alarms typically occur, the cloud shadow motion should be detected. As the last stage of the proposed method, real moving objects in the scene are tried to be discriminated from false alarms by exploiting the relation of intensity ratios between the object candidate and its surroundings. The accuracy and computational efficiency of the proposed approach make it a reliable and feasible approach to be used in real-time surveillance solutions.

A Three-Way Optimization Technique for Noise Robust Moving Object Detection Using Tensor Low-Rank Approximation, l1/2, and TTV Regularizations

The rising demand for surveillance systems naturally necessitates more efficient and noise robust moving object detection (MOD) systems from the captured video streams. Inspired by the challenges in MOD which are yet to be addressed properly, this paper proposes a new MOD scheme using l1/2 regularization in the tensor framework. It takes advantage of the special features of tensor singular value decomposition ( t -SVD) along with regularizations using l1/2 -norm with half thresholding operation and tensor total variation (TTV) to develop a noise robust MOD system with improved detection accuracy. While t -SVD exploits the spatio-temporal correlation of the video background, l1/2 regularization provides noise robustness besides removing the sparser but discontinuous dynamic elements in the spatio-temporal direction. Moreover, TTV enhances the spatio-temporal continuity and fills up the gaps due to the lingering objects and thereby extracting the foreground precisely. The proposed three-way optimization method is designed to address both static and dynamic background cases of MOD separately with the intention to reduce the misclassifications due to moving/cluttered background. The brilliance of this method is confirmed by the impressive visual quality of the background/foreground separation, noise robustness, reduced computational complexity, and rapid response. The quantitative evaluation discloses the predominance of the proposed method with respect to the state-of-the-art techniques.

Robust moving objects detection in long-distance imaging through turbulent medium

The tough challenges of detecting moving objects in long-distance imaging stems from the optical turbulence effects, which leads to blurred image, flickering pixel value and shifted object positions. To address this problem, we propose a coarse-to-fine method based on dynamic background subtraction and pipeline filter. First, we model the turbulent foregrounds with a specific mixture of Gaussian (MoG) distribution, which is regularized by online low-rank subspace factorization. Moreover, we embed a transformation operator into the model to ensure that the low-rank space is not damaged in practical camera jitters or rotation. Then, to avoid the interference of residual turbulence and marginal noise, a method of Variable Weighted Pipeline Filter (VWPF) is proposed to discriminate the true from false objects by taking full advantage of objects’ spatial-temporal property. Compared with other state-of-art methods, the proposed method shows better false alarm and miss detection rates.

Illumination Invariant Motion Detection and Tracking Using SMDWT and a Dense Disparity-Variance Method

The navigation management systems in autonomous vehicles should be able to gather solid information about the immediate environment of the vehicle, discern ambulance from a delivery truck, and react in a proper manner to handle any difficult situation. Separating such information from a vision controlled system is a computationally demanding task for heavy traffic areas in the real world environmental conditions. In such a scenario, we need a robust moving object detection tracking system. To achieve this, we can make use of stereo vision-based moving object detection and tracking, utilizing symmetric mask-based discrete wavelet transform to deal with illumination changes, low memory requirement, and fake motion avoidance. The accurate motion detection in complex dynamic scenes is done by the combined background subtraction and frame differencing technique. For the fast motion track, we can employ a dense disparity-variance method. This SMDWT-based object detection has a maximum and minimum accuracy of 99.62% and 94.95%, respectively. The motion track has the highest accuracy of 79.47% within the time frame of 28.03 seconds. The lowest accuracy of the system is 62.01% within the time frame of 34.46 seconds. From the analysis, it is clear that this proposed method exceptionally outperforms the existing monocular and dense stereo object tracking approaches in terms of low computational cost, high accuracy, and in handling the dynamic environments.

Approach for moving object detection using visible spectrum and thermal infrared imaging

Moving object detection is one of the most promising research areas, which is required in different applications, such as video monitoring and surveillance systems, human activity recognition systems, vehicle counting, and anomaly detection. Various methods for object detection using single sensor and a few using multimodal techniques have been reported in the literature. However, such systems fail to handle adverse or challenging atmospheric conditions such as illumination variations, scale and appearance change of objects or targets, occlusions, and camouflaged conditions. We have presented an approach for the detection of moving objects using structural similarity metric (SSIM) and Gaussian mixture model (GMM). SSIM is used to compute similarity between reference mean background frame and foreground frame of visible spectrum (VIS) and thermal infrared (IR) independently. The computation of similarity measure is performed in an image spatial domain. The threshold results of SSIM are fused together using different pixel-level fusion methods such logical “OR,” discrete wavelet transform, and principal components analysis. Temporal analysis is performed to eliminate noise and false positives (unwanted background regions) using GMM on fused results. We have compared the results with recent methods for different complex scenarios and found out that approximately F -measure increases up to 80%. Hence, the proposed method proves to be a robust moving object detection technique in multimodality domain.

Atanassov's intuitionistic fuzzy histon for robust moving object detection

Histon and feature extracted by it, Basic Histon Roughness Index (BHRI), have been previously employed in image segmentation and moving object detection with outstanding performances. This work incorporates Atanassov's Intuitionistic Fuzzy Sets (A-IFS) theory to the concept of histon and extends 3D Basic Histon Roughness Index (3DBHRI) to Atanassov's Intuitionistic 3D Basic Histon Roughness Index. In addition, an adaptive Gaussian membership function is proposed for constructing Atanassov's Intuitionistic 3D Fuzzy Histon Roughness Index (A-IA3DFHRI). A-IFS provides a flexible framework to deal with hesitancy along with the vagueness originating from the imperfect, imprecise and noisy information. The proposed model has been rigorously tested and compared with several previous state-of-the-art models and shows significant performance improvements.

Local-to-global background modeling for moving object detection from non-static cameras

This paper investigates efficient and robust moving object detection from non-static cameras. To tackle the motion of background caused by moving cameras and to alleviate the interference of noises, we propose a local-to-global background model for moving object detection. Firstly, motion compensation based local location-specific background model is deployed to roughly detect the foreground regions in non-static cameras. More specifically, the local background model is built for each pixel and represented by a set of pixel values drawn from its location and neighborhoods. Each pixel can be classified as foreground or background pixel according to the compensated background model based on the fast optical flow. Secondly, we estimate the global background model by the rough superpixel-based background regions to further separate foregrounds from background accurately. In particular, we use the superpixel to generate the initial background regions based on the detection results generated by local background model to alleviate the noises. Then, a Gaussian Mixture Model (GMM) is estimated for the backgrounds on superpixel level to refine the foreground regions. Extensive experiments on newly created dataset, including 10 challenging video sequences recorded in PTZ cameras and hand-held cameras, suggest that our method outperforms other state-of-the-art methods in accuracy.

Robust moving object detection based on spatio‐temporal confidence relationship

It is still an open problem in the context of complex scenarios like dynamic background and illumination variations, although numerous moving object detection schemes have been demonstrated. Great efforts have been made to develop some probability distribution pixels obey, for which samples are collected from spatial domain and/or temporal domain. However, significant attention has not been paid to the confidence relationship between pixels and its spatio-temporal neighbours in previous works. In this Letter, a confidence relationship model is proposed to complete the moving object detection task in complex environments. Experiments on typical surveillance scenes verify that the proposed algorithm has attractive robustness and high accuracy for illumination variations and dynamic background against state-of-the-art methods.

Moving object detection using unstable camera for video surveillance systems

This paper presents a robust moving object detection method by compensating for motion of an unstable camera. Assuming that global camera motion results in affine transform between two successive frames, local affine motions are separately estimated in multiple pre-specified regions for fast, robust estimation of the global motion. The global camera motion is then estimated by the least squares method using the pre-estimated multiple local affine motions. Given a current frame as the reference, the subsequent frame is registered to the current frame using the estimated global motion. The moving objects are finally detected using difference of Gaussian and non-parametric kernel density estimation from the set of registered three frames. Experimental results show that the proposed method can robustly detect moving objects in unstable imaging environment for intelligent surveillance systems using various types of cameras including pan-tilt-zoom (PTZ) and unmanned aerial vehicle (UAV) cameras.

Moving Object Detection and Segmentation using Frame Differencing and Summing Technique

technology of motion detection has become one of the important research areas in computer vision. In surveillance video this has got a number of applications which range from indoor and outdoor security environment, traffic control, behavior detection during spot activities and for compression of video. In this paper simple but robust moving object detection and segmentation algorithm is proposed. The algorithm is based on the background subtraction. A differencing and summing technique (DST) has been used for the moving object detection and segmentation. This method is simple and low in computational complexity as compared to traditional object identification and segmentation techniques. The experimental results show that the proposed method work efficiently in identifying and segmenting moving objects, both in indoor as well as in outdoor environment with static background.

Moving Object Detection with Fixed Camera and Moving Camera for Automated Video Analysis

Detection of moving objects in a video sequence is a difficult task and robust moving object detection in video frames for video surveillance applications is a challenging problem. Object detection is a fundamental step for automated video analysis in many vision applications. Object detection in a video is usually performed by object detectors or background subtraction techniques. Frequently, an object detector requires manual labeling, while background subtraction needs a training sequence. To automate the analysis, object detection without a separate training phase becomes a critical task. This paper presents a survey of various techniques related to moving object detection and discussed the optimization process that can lead to improved object detection and the speed of formulating the low rank model for detected object.

Robust moving object detection against fast illumination change

To solve the problem due to fast illumination change in a visual surveillance system, we propose a novel moving object detection algorithm for which we develop an illumination change model, a chromaticity difference model, and a brightness ratio model. When fast illumination change occurs, background pixels as well as moving object pixels are detected as foreground pixels. To separate detected foreground pixels into moving object pixels and false foreground pixels, we develop a chromaticity difference model and a brightness ratio model that estimates the intensity difference and intensity ratio of false foreground pixels, respectively. These models are based on the proposed illumination change model. Based on experimental results, the proposed method shows excellent performance under various illumination change conditions while operating in real-time.

Fast and Robust Moving Objects Detection based on Non-parametric Background Modeling

Fast and reliable detection of moving objects is one of the important requirements for many computer vision and video analysis applications. Mean shift based non-parametric background modeling supports more sensitive and robust detection in dynamic outdoor scenes. However it is prohibitive to real-time applications such as video surveillance. This paper aims to deal with the limitation of high computational complexity. Firstly, coarse to fine methods are proposed to avoid raster scanning entire image. Foreground pixels are detected in coarse level to roughly locate the foreground objects in the image, and then fine detection is performed on the corresponding blocks gradually. Secondly ， fast mean shift approach is presented according to temporal dependencies. Mean shift iterations are performed starting from incoming data and the modes obtained last time. The experimental results show that the proposed algorithm is effective and efficient in dynamic environment. The proposed algorithm has been applied to move objects detection in our real-time marine video surveillance system.

Computer vision system for in-house video surveillance

In-house video surveillance to control the safety of people living in domestic environments is considered. In this context, common problems and general purpose computer vision techniques are discussed and implemented in an integrated solution comprising a robust moving object detection module which is able to disregard shadows, a tracking module designed to handle large occlusions, and a posture detector. These factors, shadows, large occlusions and people's posture, are the key problems that are encountered with in-house surveillance systems. A distributed system with cameras installed in each room of a house can be used to provide full coverage of people's movements. Tracking is based on a probabilistic approach in which the appearance and probability of occlusions are computed for the current camera and warped in the next camera's view by positioning the cameras to disambiguate the occlusions. The application context is the emerging area of domotics (from the Latin word domus, meaning ‘home’, and informatics). In particular, indoor video surveillance, which makes it possible for elderly and disabled people to live with a sufficient degree of autonomy, via interaction with this new technology, which can be distributed in a house at affordable costs and with high reliability.