Generation of high resolution image based on accumulated feature trajectory

Abstract

The proposed method creates a high-resolution (HR) image on the basis of the frame registration of multiple low-resolution (LR) images. Not only does the super-resolution (SR) method based on using multiple LR images generally enhance the restored HR image quality compared to that based on using a single LR image, but it also increases the complexity and frame memory for hardware implementation. In order to generate an HR image, the multi-frame SR method has to estimate all motion vectors (MVs) between the target LR image and all the reference LR images. Additionally, the total frame memories used for storing LR images have to be preset according to the number of all the reference LR images. Therefore, the proposed multi-frame SR method focuses on a real-time and low frame memory system, thereby reducing the number of motion estimation (ME) operations and the total frame memory required, and preserving the image quality in an HR image restoration. First, we classify the input LR image into a feature and a uniform region in order to reduce the frame memory because the performance of SR algorithms is predominantly affected by restoring a feature region rather than a uniform region. Accordingly, we only save and use the feature region of the multiple LR images and not the uniform region for restoring an HR image. Next, the MV of each feature is estimated frame-wise to reduce the complexity of ME, and these MVs are accumulated as the feature trajectories through multiple LR frames. In the proposed method, the ME operation is conducted once between the reference LR image and the target LR image, and the estimated MVs are linked to the feature trajectories. These accumulated feature trajectories are used for generating an HR image. Experimental results show that the proposed multi-frame SR method can reduce the complexity and frame memory to one-third, while the quality of the restored HR image is equal to that obtained by using the conventional SR methods.