Perceptual Completion of Occluded Surfaces

Abstract

Researchers in computer vision have primarily studied the problem of visual reconstruction of environmental structure that is plainly visible. In this paper, the conventional goals of visual reconstruction are generalized to include both visible and occluded forward facing surfaces. This larger fraction of the environment is termed theanterior surfaces. Because multiple anterior surface neighborhoods project onto a single image neighborhood wherever surfaces overlap, surface neighborhoods and image neighborhoods are not guaranteed to be in one-to-one correspondence, as conventional “shape-from” methods assume. The result is that the topology of three-dimensional scene structure can no longer be taken for granted, but must be inferred from evidence provided by image contours. In this paper, we show that the boundaries of the anterior surfaces can be represented in viewer-centered coordinates as alabeled knot-diagram. Where boundaries are not occluded and where surface reflectance is distinct from that of the background, boundaries will be marked by image contours. However, where boundaries are occluded, or where surface reflectance matches background reflectance, there will be no detectable luminance change in the image. Deducing the complete image trace of the boundaries of the anterior surfaces under these circumstances is called thefigural completionproblem. The second half of this paper describes a computational theory of figural completion. In more concrete terms, the problem of computing a labeled knot-diagram representing an anterior scene from a set of contour fragments representing image luminance boundaries is investigated. A working model is demonstrated on a variety of illusory contour displays. The experimental system employs a two-stage process of completion hypothesis and combinatorial optimization. The labeling scheme is enforced by a system of integer linear inequalities so that the final organization is the optimal feasible solution of an integer linear program.