A hybrid method of image synthesis in IBR for novel viewpoints

Abstract

Due to visibility change and surface enlargement in producing a novel view from a new viewpoint, 3D re-projection from one reference image in IBMR inevitably produces holes in the destination image. Even worse, exposure errors occur when a background region occluded is visible in a desired image because of the absence of some background elements in the reference image. The general solution to this kind of problems is to use multiple images from different viewpoints as input source. By doing so however, the rendering cost would increase with the number of reference images and the composition algorithm has to rely on the z-buffer processing.In fact, plenty of redundant information exists among different reference images. Seeking for a nice way to extract the information needed in the novel view from the reference images is the key issue in solving the problem. In this paper, we propose a new method of image synthesis from multiple reference images. The method combines forward warping and backward warping to fulfil the image composition task for a novel viewpoint. The primary inspiration behind the development of our image synthesis method comes from a fact that the polygon edge geometry may indicate where an exposure and, possible an exposure error occur in the destination image if object silhouettes are prior known. The feature that intersection between scanline and polygons must be in pairs is employed to distinguish holes caused by enlargement of surfaces from those by visibility change. Different heuristic methods are used to choose one image as a primary reference image which shares the most resemblance with the destination image, and other reference images for filling different kinds of holes. Depth continuity along scanline and the depth information already present in the destination image are employed to accelerate the searching process of the corresponding pixels for filling holes.