Block unshifting high-accuracy motion estimation: A new method adapted to super-resolution enhancement

Abstract

Sub-pixel motion estimation plays a vital role in a multitude of video applications, including encoding, audiovisual archiving/heritage and super-resolution enhancement. Most existing block-based methods rely on the implicit assumption that blocks can be accurately predicted through appropriate shifts. In particular, shifted blocks in the target frame are estimated from the associated anchor frame blocks. The present paper introduces a different strategy, which discards this assumption and treats anchor and target frame blocks equally, as sub-pixel shifted versions of an unavailable implied block. The new method attempts to construct this implied block and, by calculating the “imaginary” motion vectors that relate it to the two existing blocks, it estimates the wanted motion vectors more accurately. This approach aims at extracting motion vectors that more accurately represent the actual movements of objects, minimizing the interpolation error that is associated with sub-pixel shifting, which manifests as blurring and a lowering of contrast. The new method focuses on accurate motion estimation, paying less attention to the associated computational load. Hence, the approach is both inspired from, and proposed for, super-resolution enhancement scenarios, where higher definition motion image sequences are estimated from their available lower definition counterparts. In order to implement the new strategy, an algorithm for reversing the bilinear sub-pixel shift of a block (unshifting) is implemented and validated. Comparisons between original blocks of images and blocks that have been shifted and unshifted back to their original coordinates showcase the accuracy of the unshifting process. The proposed motion estimation method is evaluated through a number of different experimental assessment procedures and metrics, comparing it to existing high-accuracy state-of-the-art motion estimation methods.