Disparity manipulation for stereo images and video

Abstract

We develop a general framework for disparity manipulation and morphing for stereo images and video. The framework consists of three parts: disparity map generation, disparity map manipulating/editing, and stereo image synthesis. We first discuss disparity map generation techniques for different original input data types, including monoscopic images, monoscopic video, stereo image pairs, and stereo video. Then, we describe three methods for user manipulation of the disparity map. In the first, the user employs an interactive object-selecting tool by inputting seed points near the desired object boundary. Given the selected objects, the user defines input-output disparity mapping curves for each object. In the second method, the user arbitrary manipulates a 3D disparity surface and our system calculates the new 3D surface after the user editing. A third method provides conversions between the two common stereo camera capture setups: "toein" and "off-axis" (we present their mathematical description). We show several morphed disparity map examples for each disparity manipulation method. Finally, we describe disparity-based image rendering to synthesize new stereo image pairs from given original stereo image pairs based on a morphed disparity map. The synthesis method includes image warping, data-filling and disparity map smoothing procedures.