Choose your own viewpoint: A high-quality/low-complexity free-viewpoint 3D visual system

Abstract

Choosing one's own viewpoint when watching a video program has long been a desire for viewers. To achieve this goal, view synthesis and depth map generation are two fundamental techniques. View synthesis is a signal processing procedure which creates dense virtual views based on sparse real views. Each object inside a frame is warped to a proper position according to its depth information to form the viewpoint changing perception for viewers. Hence, the correctness of depth map influences the view synthesis quality. To increase the accuracy of depth map, this paper proposes an edge-adaptive block matching scheme cooperated with an unreliable region repairing approach. The former avoid finding local minimum in stereo matching, and the latter repairs the errors caused by occlusion regions. As for view synthesis, this paper proposes a special warping method that can detect errors caused by boundary mismatches of objects between corresponding depth and color images to improve quality of the synthesized view. Besides, we also propose a compensative-filling method that can fix tiny cracks due to round-off errors. Because of these two features, the proposed view synthesis becomes more robust to tolerate errors inside depth maps when compared with previous schemes. Both the depth generation and view synthesis are extremely complex computations. Therefore, this paper also proposes a low-complexity computing technology based on group-of-pixels which increases 30 times of performance for depth map generation, and reduces 60% computation time of view synthesis.