Carved visual hulls for high-accuracy image-based modeling

Abstract

Listen

Translate article

The relative accuracy of high-end laser range scanners can be as high as 1/10,000, allowing the construction of high-accuracy solid models of complex shapes from registered depth maps [Levoy et al. 2000]. Comparable accuracy levels can be achieved using “ordinary” cameras and sophisticated photogrammetric methods, but these typically output a relatively sparse set of points and require markers [Uffenkamp 1993]. Computer vision approaches to image-based modeling from calibrated photographs construct solid object models and do not need markers [Baumgart 1974; Kutulakos and Seitz 2000], but their relative accuracy is typically below 1/200. [Hernandez and Schmitt 2004] propose to use the visual hull [Baumgart 1974] to initialize the deformation of a surface mesh under the influence of rimand photo-consistency constraints expressed by gradient flow forces (see [Keriven 2002] for a related approach). Although this method yields excellent results, its reliance on iterative refinement makes it susceptible to local minima. To overcome this problem, we propose a combination of global and local optimization techniques to carve the surface of the visual hull. The algorithm proposed in [Lazebnik 2002] is first used to construct a combinatorial mesh representation of the visual hull surface in terms of polyhedral cone strips meeting at frontier points where two visual rays are tangent to the surface (Figure 2). Photoconsistency constraints are then used to refine this initial surface in three consecutive steps: (1) dynamic programming is used to identify the rims where the surface grazes the visual hull as the most photo-consistent paths between successive frontier points in the corresponding strips; (2) with the rims now fixed, the visual hull is carved by graph cuts [Boykov and Kolmogorov 2003] to globally minimize the image discrepancy of the surface and recover the main surface features, including concavities—which, unlike convex and saddle-shaped parts of the surface, are not captured by the visual hull; and (3) iterative (local) refinement [Keriven 2002] is finally used to recover surface detail.