Depth extended online RPCA with spatiotemporal constraints for robust background subtraction

Abstract

The detection of moving objects is the first step in video surveillance systems. But due to the challenging backgrounds such as illumination conditions, color saturation, and shadows, etc., the state of the art methods do not provide accurate segmentation using only a single camera. Recently, subspace learning model such as Robust Principal Component analysis (RPCA) shows a very nice framework towards object detection. But, RPCA presents the limitations of computational and memory issues due to the batch optimization methods, and hence it cannot process high dimensional data. Recent research on RPCA methods such as Online RPCA (OR-PCA) alleviates the traditional RPCA limitations. However, OR-PCA using only color or intensity features shows a weak performance specially when the background and foreground objects have a similar color or shadows appear in the background scene. To handle these challenges, this paper presents an extension of OR-PCA with the integration of depth and color information for robust background subtraction. Depth is less affected by shadows or background/foreground color saturation issues. However, the foreground object may not be detected when it is far from the camera field as depth is less useful without color information. We show that the OR-PCA including spatiotemporal constraints provides accurate segmentation with the utilization of both color and depth features. Experimental evaluations on a well-defined benchmark dataset with other methods demonstrate that our proposed technique is a top performer using color and range information.