Robust salient motion detection in non-stationary videos via novel integrated strategies of spatio-temporal coherency clues and low-rank analysis

Abstract

Listen

Translate article

Salient motion detection is vital for security surveillance, pattern and motion recognition, traffic control, human–computer interaction, etc. Although such a subject has been very well investigated for analysis of stationary videos, many technical challenges still prevail when correctly handling and analyzing non-stationary videos recorded by hand-hold and pan-tilt-zoom cameras. To ameliorate, this paper develops a novel and robust salient motion detection method (especially valuable for quantitative analysis of non-stationary videos) by employing new computational strategies, including low-rank analysis aided by the divide-and-conquer approach, and exploration of the space–time semantic coherency. The key idea in our new approach is to respectively conduct multi-purpose low-rank analysis over a temporal series of well-decomposed frame-batches that have relatively-consistent backgrounds. First, we conduct bilateral random projection (BRP)-based low-rank analysis to accurately keep track of short-term stable-background observations, which consist of frames with similar global appearance and small local variations. Then, to eliminate the side effects due to visual variations induced by view angle changes, we incorporate the low-rank background prior into previous short-term observation to guide robust principal component analysis (RPCA) low-rank revealing based robust salient motion detection over current short-term observation. Meanwhile, a series of saliency clues extracted from the stabilized short-term observations are leveraged to expedite the proper updating of the low-rank background information, which enables us to effectively combat several obstinate problems. Finally, we conduct comprehensive experiments on the public CD2014 benchmark and other five non-stationary videos recorded from the hand-hold camera, and make extensive and quantitative evaluations with six state-of-the-art methods. Experimental results indicate that our method not only outperforms all other methods in the case of non-stationary videos but also obtains outstanding performance for stationary videos.