Background Subtraction Based on Deep Pixel Distribution Learning

Abstract

Previous approaches to background subtraction typically address the problem by formulating a representation of the background, and comparing the background to new frames. In this work, we focus on the essence of background subtraction, which is the classification of a pixel's current observation in comparison to historical observations, and propose a Deep Pixel Distribution Learning (DPDL) model for background subtraction. In the DPDL model, a novel pixel-based feature, called the Random Permutation of Temporal Pixels (RPoTP), is used to represent the distribution of past observations for a particular pixel, in which the temporal correlation between observations is deliberately obfuscated. Subsequently a convolutional neural network (CNN) is used to learn the distribution for determining whether the current observation is foreground or background, with the random permutation enabling the framework to focus primarily on the distribution of observations, rather than be misled by learning spurious temporal correlations. In addition, the pixel-wise representation allows for a large number of RPoTP features to be captured even with a limited number of groundtruth frames, with the DPDL model being effective even with only a single groundtruth frame. The proposed framework is able to achieve promising results in diverse natural scenes, and a comprehensive evaluation on standard benchmarks demonstrates the superiority of our work to state-of-the-art methods. The source code ispublicly available at https://github.com/zhaochenqiu/DPDL