Video retargeting through spatio‐temporal seam carving using Kalman filter

Abstract

A Kalman filter-based spatio-temporal seam determination scheme is proposed in this study for video retargeting through seam carving. In probably a first, the authors’ retargeting approach is designed to be hardware friendly, and to achieve spatial and temporal coherence through an optimal solution with explicit mechanisms to reduce jitter and structural distortion. In a video frame, while spatially coherent seam is determined by the popular approach of seam energy minimisation, temporally coherent seam is determined using Kalman prediction and updation processes. Further, these spatial and temporal seams are combined judiciously to obtain the spatio-temporal seam to be removed/repeated for decreasing/increasing the frame size. The authors show that the proposed Kalman filter-based approach has less theoretical complexity compared to the existing. Extensive experimental results show that the proposed approach consistently outperforms the state-of-the-art, both qualitatively and quantitatively in performance, and in computational time.