Spatial Sparsity-Induced Prediction (SIP) for Images and Video: A Simple Way to Reject Structured Interference

Abstract

We propose a prediction technique that is geared toward forming successful estimates of a signal based on a correlated anchor signal that is contaminated with complex interference. The corruption in the anchor signal involves intensity modulations, linear distortions, structured interference, clutter, and noise just to name a few. The proposed setup reflects nontrivial prediction scenarios involving images and video frames where statistically related data is rendered ineffective for traditional methods due to cross-fades, blends, clutter, brightness variations, focus changes, and other complex transitions. Rather than trying to solve a difficult estimation problem involving nonstationary signal statistics, we obtain simple predictors in linear transform domain where the underlying signals are assumed to be sparse. We show that these simple predictors achieve surprisingly good performance and seamlessly allow successful predictions even under complicated cases. None of the interference parameters are estimated as our algorithm provides completely blind and automated operation. We provide a general formulation that allows for nonlinearities in the prediction loop and we consider prediction optimal decompositions. Beyond an extensive set of results on prediction and registration, the proposed method is also implemented to operate inside a state-of-the-art compression codec and results show significant improvements on scenes that are difficult to encode using traditional prediction techniques.