Stereo image analysis for multi-viewpoint telepresence applications

Abstract

Abstract An improved method for combined motion and disparity estimation in stereo sequences to synthesize temporally and perspectively intermediate views is presented. The main problems of matching methods for motion and disparity analysis are summarised. The improved concept is based on a modified block matching algorithm in which a cost function consisting of feature- and area-based correlation together with an appropriately weighted temporal smoothness term is applied. Considerable improvements have been obtained with respect to the motion and disparity assignments by introducing a confidence measure to evaluate the reliability of estimated correspondences. In occluded image areas, enhanced results are obtained applying an edge-assisted vector interpolation strategy. Two different image synthesis concepts are presented. The first concept is suitable for processing natural stereo sequences. It comprises the detection of covered and uncovered image areas caused by motion or disparity. This information is used to switch between different interpolation and extrapolation modes during the computation of intermediate views. The proposed object-based approach is suitable for processing typical video conference scenes containing extremely large occluded image regions and keeping implementation costs low. A set of stereo sequences has been processed. The performed computer simulations show that a continuous motion parallax can be obtained with good image quality by using sequences taken with stereo cameras having large interaxial distances.