Multi-frame super resolution via deep plug-and-play CNN regularization

Abstract

Abstract Because of the ill-posedness of multi-frame super resolution (MSR), the regularization method plays an important role in the MSR field. Various regularization terms have been proposed to constrain the image to be estimated. However, artifacts also exist in the estimated image due to the artificial tendency in the manually designed prior model. To solve this problem, we propose a novel regularization-based MSR method with learned prior knowledge. By using the variable splitting technique, the fidelity term and regularization term are separated. The fidelity term is associated with an “ L 2 {L^{2}} - L 2 {L^{2}} ” form sub-problem. Meanwhile, the sub-problem respect to regularization term is a denoising problem, which can be solved by denoisers learned from a deep convolutional neural network. Different from the traditional regularization methods which employ hand-crafted image priors, in this paper the image prior model is replaced by learned prior implicitly. The two sub-problems are solved alternately and iteratively. The proposed method cannot only handle complex degradation model, but also use the learned prior knowledge to guide the reconstruction process to avoid the artifacts. Both the quantitative and qualitative results demonstrate that the proposed method gains better quality than the state-of-the-art methods.