Scene Background Initialization Research Articles

Quaternion-based dynamic mode decomposition for background modeling in color videos

Scene Background Initialization (SBI) is one of the challenging problems in computer vision. Dynamic mode decomposition (DMD) is a recently proposed method to robustly decompose a video sequence into the background model and the corresponding foreground part. However, this method needs to convert the color image into a grayscale image for processing, which leads to the neglect of the coupling information between the three channels of the color image. In this study, we propose a quaternion-based DMD (Q-DMD), which extends the DMD by quaternion matrix analysis, so as to ultimately preserve the inherent color structure of the color image and the color video. We exploit the standard eigenvalues of the quaternion matrix to compute its spectral decomposition and calculate the corresponding Q-DMD modes and eigenvalues. The results on the publicly available benchmark datasets prove that our Q-DMD outperforms the exact DMD method, and experiment results also demonstrate that the performance of our approach is comparable to that of the state-of-the-art ones.

Singular Spectrum Analysis for Background Initialization with Spatio-Temporal RGB Color Channel Data.

In video processing, background initialization aims to obtain a scene without foreground objects. Recently, the background initialization problem has attracted the attention of researchers because of its real-world applications, such as video segmentation, computational photography, video surveillance, etc. However, the background initialization problem is still challenging because of the complex variations in illumination, intermittent motion, camera jitter, shadow, etc. This paper proposes a novel and effective background initialization method using singular spectrum analysis. Firstly, we extract the video’s color frames and split them into RGB color channels. Next, RGB color channels of the video are saved as color channel spatio-temporal data. After decomposing the color channel spatio-temporal data by singular spectrum analysis, we obtain the stable and dynamic components using different eigentriple groups. Our study indicates that the stable component contains a background image and the dynamic component includes the foreground image. Finally, the color background image is reconstructed by merging RGB color channel images obtained by reshaping the stable component data. Experimental results on the public scene background initialization databases show that our proposed method achieves a good color background image compared with state-of-the-art methods.

Real-time scene background initialization based on spatio-temporal neighborhood exploration

In this paper, we address the problem of scene background initialization to define a background model free from foreground objects. The complexity of this task resides in the continuous clutter of the scene by moving and stationary objects. To face this challenge, we propose a robust real-time iterative model completion method based on online block-level processing to initialize the background with low computational cost. First, temporal data analysis is conducted to cluster similar blocks. Meanwhile, a two-folded inter-block spatial neighborhood exploration is performed. It aims to capture relationships among neighboring clusters and reduce the number of candidate clusters employed in the next phase. Then, a smoothness analysis between neighboring locations is performed to iteratively reconstruct the background based on a newly proposed edge matching metric and an inter-block color discontinuity. Extensive evaluations of the proposed approach on the public Scene Background Initialization 2015 dataset and on the Scene Background Modeling Contest 2016 dataset revealed a performance superior or comparable to state-of-the-art methods.

A robust background initialization algorithm with superpixel motion detection

Scene background initialization allows the recovery of a clear image without foreground objects from a video sequence, which is generally the first step in many computer vision and video processing applications. The process may be strongly affected by some challenges such as illumination changes, foreground cluttering, intermittent movement, etc. In this paper, a robust background initialization approach based on superpixel motion detection is proposed. Both spatial and temporal characteristics of frames are adopted to effectively eliminate foreground objects. A subsequence with stable illumination condition is first selected for background estimation. Images are segmented into superpixels to preserve spatial texture information and foreground objects are eliminated by superpixel motion filtering process. A low-complexity density-based clustering is then performed to generate reliable background candidates for final background determination. The approach has been evaluated on SBMnet dataset and it achieves a performance superior or comparable to other state-of-the-art works with faster processing speed. Moreover, in those complex and dynamic categories, the algorithm produces the best results showing the robustness against very challenging scenarios.

LaBGen-P-Semantic: A First Step for Leveraging Semantic Segmentation in Background Generation

Given a video sequence acquired from a fixed camera, the stationary background generation problem consists of generating a unique image estimating the stationary background of the sequence. During the IEEE Scene Background Modeling Contest (SBMC) organized in 2016, we presented the LaBGen-P method. In short, this method relies on a motion detection algorithm for selecting, for each pixel location, a given amount of pixel intensities that are most likely static by keeping the ones with the smallest quantities of motion. These quantities are estimated by aggregating the motion scores returned by the motion detection algorithm in the spatial neighborhood of the pixel. After this selection process, the background image is then generated by blending the selected intensities with a median filter. In our previous works, we showed that using a temporally-memoryless motion detection, detecting motion between two frames without relying on additional temporal information, leads our method to achieve the best performance. In this work, we go one step further by developing LaBGen-P-Semantic, a variant of LaBGen-P, the motion detection step of which is built on the current frame only by using semantic segmentation. For this purpose, two intra-frame motion detection algorithms, detecting motion from a unique frame, are presented and compared. Our experiments, carried out on the Scene Background Initialization (SBI) and SceneBackgroundModeling.NET (SBMnet) datasets, show that leveraging semantic segmentation improves the robustness against intermittent motions, background motions and very short video sequences, which are among the main challenges in the background generation field. Moreover, our results confirm that using an intra-frame motion detection is an appropriate choice for our method and paves the way for more techniques based on semantic segmentation.

Extensive Benchmark and Survey of Modeling Methods for Scene Background Initialization.

Scene background initialization is the process by which a method tries to recover the background image of a video without foreground objects in it. Having a clear understanding about which approach is more robust and/or more suited to a given scenario is of great interest to many end users or practitioners. The aim of this paper is to provide an extensive survey of scene background initialization methods as well as a novel benchmarking framework. The proposed framework involves several evaluation metrics and state-of-the-art methods, as well as the largest video data set ever made for this purpose. The data set consists of several camera-captured videos that: 1) span categories focused on various background initialization challenges; 2) are obtained with different cameras of different lengths, frame rates, spatial resolutions, lighting conditions, and levels of compression; and 3) contain indoor and outdoor scenes. The wide variety of our data set prevents our analysis from favoring a certain family of background initialization methods over others. Our evaluation framework allows us to quantitatively identify solved and unsolved issues related to scene background initialization. We also identify scenarios for which state-of-the-art methods systematically fail.

Background estimation by weightless neural networks

Initial background estimation in video processing serves as bootstrapping model for moving object detection based on background subtraction. In long-term videos, the initial background model may require a constant update as long as the environmental changes happen (slowly or suddenly). In this paper we approach the background initialization together with its constant updating in time by modeling video background as ever-changing states of weightless neural networks. The result is a background estimation method based on a weightless neural network, called BEWiS. The approach proposed in this work is simple: background estimation at each pixel is carried out by weightless neural networks designed to learn pixel color frequency during video play, and all networks share the same rule for memory retention during training. This approach has the advantage of providing a useful background model at the very beginning of the video, since it operates in unsupervised mode. On the other hand, depending on the video scene, the pixel-level learning rule can be tuned to tackle the specificities and difficulties of the scene. The approach presented in this work has been experimented on the public Scene Background Initialization 2015 dataset and on the Scene Background Modeling Contest 2016 dataset, and it showed a performance comparable or superior to state-of-the-art methods.

Matrix and tensor completion algorithms for background model initialization: A comparative evaluation

Background model initialization is commonly the first step of the background subtraction process. In practice, several challenges appear and perturb this process, such as dynamic background, bootstrapping, illumination changes, noise image, etc. In this context, we investigate the background model initialization as a reconstruction problem from missing data. This problem can be formulated as a matrix or tensor completion task where the image sequence (or video) is revealed as partially observed data. In this paper, the missing entries are induced from the moving regions through a simple joint motion-detection and frame-selection operation. The redundant frames are eliminated, and the moving regions are represented by zeros in our observation model. The second stage involves evaluating twenty-three state-of-the-art algorithms comprising of thirteen matrix completion and ten tensor completion algorithms. These algorithms aim to recover the low-rank component (or background model) from partially observed data. The Scene Background Initialization data set was selected in order to evaluate this proposal with respect to the background model challenges. Our experimental results show the good performance of LRGeomCG method over its direct competitors.