Super-Resolution Reconstruction for Mixed Resolution Videos Using Key Frames and Adaptive Detail Warping

Abstract

AbstractIn the mixed resolution coding approach, one type of scalable video coding, a mixed resolution video sequence contains low-resolution (LR) nonkey video frames at high frame rate and periodic high-resolution (HR) key video frames at low frame rate in order to reduce bit rate and encoding complexity. At the decoder, LR nonkey video frames can be upsampled to the corresponding HR video frames by neighboring HR key video frames via video super-resolution. In this study, a video super-resolution reconstruction approach for mixed resolution videos using key frames and adaptive detail warping is proposed. Resolution enhancement is realized by incorporating the high-frequency details into the HR nonkey video frames that are directly upsampled by same conventional spatial interpolation method. The corresponding high-frequency details of each nonkey frame are derived by adaptive detail warping of the high-frequency details extracted from two (forward and backward) neighboring HR key frames, using the non-local-means concept and motion estimation. Based on the experimental results obtained in this study, the performance of the proposed approach is better than those of three comparison approaches.KeywordsSuper-resolution reconstructionMixed resolution videoKey framesNonkey framesNonlocal meansAdaptive detail warping