Video stabilization based on image registration

Abstract

We address the problem of video stabilization via image registration. We propose a single convolutional neural network, which is simultaneously a dense feature descriptor and a keypoint detector, to find reliable keypoints and their features under each frames. To obtain accurate keypoint localization, the authors leverage the inherent feature hierarchy to restore spatial resolution and low-level details. Compared with traditional detectors, the obtained keypoints are more stable for the following image registration. Based on the correspondences collected from registration results, we propose a technique for video stabilization which is a spatial smooth sparse motion field with motion vectors only at the mesh vertexes. In practice, the authors assign each vertex an unique motion vector via their neighboring correspondences and a median filter. The video stabilization is performed on the vertex profiles, which are motion vectors collected at the same vertex location over time. The quantitative and qualitative evaluations show that the proposed registration-based video stabilization method can produce comparable results with the state-of-the-art methods and achieve more stable performance on challenging situations.