Prediction based seam carving for video retargeting

Abstract

This paper presents a prediction based spatio-temporal seam carving scheme for video retargeting. It resizes the video maintaining appropriate balance between spatial and temporal coherence. In a video frame, the proposed approach finds a ‘temporal’ seam by using Kalman filter estimation and then modifies it with the help of ‘spatial’ seam considering both spatial and temporal coherency. Unlike image retargeting, it is of utmost importance in retargeting a video frame to consider temporal coherency along with spatial coherency to remove or replicate unimportant background portion. This will ensure that insignificant amount of motion artifacts are introduced during resizing. The proposed Kalman filter based approach not only predicts a spatio-temporal seam to mark a portion of the frame where there is more possibility of having spatially and temporally coherent seam, but also has low time complexity. The proposed approach outperforms other state-of-the-art video retargeting methods which is illustrated by experimental results.