Robust Learning-Based Camera Motion Characterization Scheme With Applications to Video Stabilization

Abstract

This paper investigates a novel learning-based camera motion characterization scheme along with its application to the video stabilization problem. The proposed characterization scheme represents the compressed domain block motion vectors (MVs) using polar angle and magnitude histograms. Discriminative features from these two histograms are extracted and fed to a supervised learning-based hierarchical classifier for recognizing the six camera motion patterns. A comparative analysis with an existing scheme is carried out to support and validate the proposed characterization scheme. The proposed scheme works at the frame level by classifying the inter-frame camera motion patterns. This scheme is extended to classify the video segments and a novel application to video stabilization is investigated. An experimental analysis of a number of test sequences captured using a handheld video camera shows that by characterizing the smooth and jittery motions, selective video stabilization could be carried out only on those video segments that have been degraded. This approach of selective video stabilization saves considerable amount of computational time compared with running the stabilization algorithm on the entire video sequence, as proposed in the literature. The proposed strategy of using a classification scheme prior to applying the video stabilization routine offers a new paradigm to the conventional video stabilization problem. Experimental validation carried out using exhaustive search motion estimation obtained block MVs, and H.264/Advanced Video Coding-obtained MVs shows that by using the idea of selective video stabilization, up to 62% reduction in processing time can be achieved compared with video stabilization approaches wherein the entire video sequence is processed.