Deformable Shape Preserving Video Retargeting With Salient Curve Matching

Abstract

Video retargeting is dedicated to resizing the resolution of videos in a content aware manner, and it involves three critical challenges, namely, visual saliency preservation, deformation prevention, and temporal consistency persistence. Existing retargeting algorithms include seam carving based approaches, warping based approaches, and cropping based approaches, which mainly concentrate on saliency preservation and fail to prevent the deformation of salient shapes. This paper proposes a deformable shape preserving video retargeting scheme where salient curves extracted from frames are protected from deformation by minimizing the matching cost of curves in the original frames and the retargeted frames. Correspondingly, a curve matching algorithm is developed to generate the deformation cost which is invariant to translation, rotation and scaling with the Bookstein coordinate transform. In turn, the deformation cost of salient curves will be incorporated into the energy map of seam carving. Furthermore, the proposed scheme defines a temporal energy term to penalize the change of the relative position of curves in consecutive retargeted frames with respect to the original frames. Extensive experiments are validated by the visual comparison, user evaluation, deformation analysis and temporal consistency evaluation, which prove that the proposed scheme outperforms state-of-the-art video retargeting methods.