Frequency domain based super-resolution method for mixed-resolution multi-view images

Abstract

Super-resolution (SR) techniques, which are based on single or multi-frame low-resolution (LR) images, have been extensively investigated in the last two decades. Mixed-resolution multiview video format plays an important role in three-dimensional television (3DTV) coding scheme. Previous work considers multiview or multi-camera images and videos at the same resolution, which performs well under the planar model without or with little projection error among the videos captured by different cameras. In recent years, several researchers have discussed the SR problem in mixed-resolution multi-view video format, where the superresolved image is created using the up-sampled version of the LR image and the high frequency components extracted from the warped image in the adjacent high-resolution (HR) views. Unfortunately, the output HR images suffer from artifacts caused by depth error. To obtain the detailed texture and edge information from the HR image as much as possible, while preserving the structure of the LR image, a novel SR reconstruction algorithm is proposed. The algorithm is composed of three components: the structure term, the detail information term, and the regularization term. The first term preserves the structure similarity of the LR image; the second term extracts detailed information from the adjacent HR image; and the last term ensures the uniqueness of the solution. Experimental results show the effectiveness and robustness of the proposed algorithm, which achieves high performance both subjectively and objectively.