Depth Trajectory-Aware Stereoscopic Video Retargeting

Xuejin Wang,Pengfei Li,Feng Shao

doi:10.1109/access.2021.3059477

Xuejin Wang, Pengfei Li + Show 1 more

Open Access

https://doi.org/10.1109/access.2021.3059477

Copy DOI

Abstract

The image/video retargeting (resolution adaptation) problem—adaption of source data (image/video) to different screen resolution—has garnered wide attention due to compelling applications in intelligent adaptive display. Most of the current image/video displaying application achieves the adaption through scaling source uniformly. Compared with traditional image/video retargeting, stereoscopic video retargeting poses new challenges that should preserve shape and depth of salient objects with temporal coherence. In this paper, we propose a depth trajectory-aware stereoscopic video retargeting method by imposing constraints to optimize the coordinates and depths simultaneously. To reduce visual discomfort induced by fast depth motion, our method uses a temporal depth distortion energy to optimize the depth trajectory in temporal direction. As a result, our solution can preserve shape, depth and temporal fidelity of salient objects simultaneously with comfortable visual experience.

Highlights

3D vision aims to provide real-world experience to users involving 3D video content and display technologies
Since there are only a few works on stereoscopic video retargeting that is available for comparison, we compare our method with two traditional methods: uniform scaling (US) and seam carving (SC) [38]
We evaluate the performance of stereoscopic video retargeting from shape preservation, depth preservation and temporal coherence

Summary

SPATIAL DEPTH DISTORTION

Without considering the depth motion in temporal direction, to maintain stereoscopic properties from the original video, the disparity distributions for a specific frame before and after retargeting should be consistent. We design a spatial depth distortion energy to enforce the disparities of all vertices be consistent with the target ones. We enforce depth preservation constraint using a few control points via grid warping. It can be represented as a linear combination of four vertices around it:. By controlling the keypoints, i=1 the coordinates of the vertices are adjusted synchronously to control the width and height of the grid. The spatial depth distortion energy is calculated by summing all keypoints across all frames: ESD = (

TEMPORAL DEPTH DISTORTION

OPTIMIZATION

EXPERIMENTAL RESULTS

CONCLUSION