Video restoration based on a novel second order nonlocal total variation model

Abstract

Total variation and its variants have been widely used in the video/image restoration area in the past decades. Among them, the nonlocal total variation model introduces penalization on nonlocal gradients and demonstrates remarkable performance gain in many applications. However, this approach tends to suppress intensity-changes of visual contents, and hence cannot restore complicated visual contents well enough. To address this issue, this paper proposes a novel second order nonlocal total variation model for video restoration problems. Firstly, the directed space-time nonlocal gradients are defined in our model to formulate the spatio-temporal intensity-changes of video contents. Secondly, mean-value approximations of these nonlocal gradients are introduced into the model. Based on them, we build up a new model that consists of both first order and second order nonlocal regularization terms. Furthermore, for the purpose of adapting to specific visual contents, it is also augmented with a content-adaptive modification. These features make the proposed second order nonlocal total variation model a high order generalization of the original one. Experimental results on various video restoration problems show that the proposed model significantly improves the restoration qualities, compared with other state-of-the-art approaches.