Improved video image by pixel-based learning for super-resolution

Abstract

In recent years, the resolution of display devices has been extremely increased. The resolution of video camera (except very expensive one), however, is quite lower than that of display since it is difficult to achieve high spatial resolution with specific frame rate (e.g. 30 frames per second) due to the limited bandwidth. The resolution of image can be increased by interpolation, such as bi-cubic interpolation, but in this method it is known that the edges of image are blurred. To create plausible high-frequency details in the blurred image, super-resolution technique has been studied for a long time. In this paper, we proose a new algorithm for video super-resolution by considering multi-sensor camera system. The multi-sensor camera can capture two types video sequence as follow; (a) high-resolution with low frame rate luminance sequence, (b) low-resolution with high frame rate color sequences. The training pairs for super-resolution are obtained from these two sequences. The relationships between the high- and low-resolution frames are trained using pixel-based feature named "texton" and stored in the database with their spatial distribution. The low-resolution sequences are then represented with texton and each texton is substituted by searching the trained database to create high-resolution features in output sequences. The experimental results showed that the proposed method can well reproduce both the detail regions and sharp edges of the scene. It was also shown that the PSNR of the image obtained by proposed method is improved compared to the image by bi-cubic interpolation method.