Shape From Silhouette Research Articles

SSfS: Segmented Shape from Silhouette Reconstruction of the Human Body.

Three-dimensional (3D) shape estimation of the human body has a growing number of applications in medicine, anthropometry, special effects, and many other fields. Therefore, the demand for the high-quality acquisition of a complete and accurate body model is increasing. In this paper, a short survey of current state-of-the-art solutions is provided. One of the most commonly used approaches is the Shape-from-Silhouette (SfS) method. It is capable of the reconstruction of dynamic and challenging-to-capture objects. This paper proposes a novel approach that extends the conventional voxel-based SfS method with silhouette segmentation—segmented Shape from Silhouette (sSfS). It allows the 3D reconstruction of body segments separately, which provides significantly better human body shape estimation results, especially in concave areas. For validation, a dataset representing the human body in 20 complex poses was created and assessed based on the quality metrics in reference to the ground-truth photogrammetric reconstruction. It appeared that the number of invalid reconstruction voxels for the sSfS method was 1.7 times lower than for the state-of-the-art SfS approach. The root-mean-square (RMS) error of the distance to the reference surface was also 1.22 times lower.

Generation and quality improvement of 3D models from silhouettes of 2D images

The shape-from-silhouette (SFS) method has been widely used in 3D shape reconstruction. It uses silhouettes of a series of 2D images captured from multiple viewpoints of an object to generate a 3D model that describes the visual hull of the object. The SFS method faces an inherent problem that virtual features appear all over the model. In addition, concavities on the object may wrongly be modeled as convex shapes because they are invisible on image silhouettes. The purpose of this study is to propose a method to generate a 3D model from silhouettes of multiple images and propose a quality improvement method to overcome the above-mentioned problems. The 3D modeling method focuses on accurate evaluation of 3D points intersected by all polyhedra from different views and the removal of poor meshes on triangulation. The quality improvement method is essentially an iterative procedure, which for smoothing the model and eliminating virtual features and artifacts, while preserving the consistency of all silhouettes. The proposed method is to be used for product presentations in e-commerce, in which the 3D model must be covered with color texture of an object. Several examples are presented to illustrate the capability of the proposed method.

Fast and accurate triangular model generation for the shape-from-silhouette technique

ABSTRACTIn this study, we propose a method for generating triangular meshes of objects based on the shape-from-silhouette (SFS) technique. The proposed algorithm uses the direct intersection of multiple sets of infinite polygons from the silhouettes of two-dimensional (2D) images to acquire the surface points of an object. The surface points are then triangulated according to the topological relationship of the obtained points and the polygons corresponding to each of the points. In addition, a comprehensive study between the proposed method and the marching-cubes method was conducted. The main advantages of the proposed method are that all 3D points are precisely located on the silhouettes of 2D images, and that the number of vertices on the triangular model is very efficient compared to that for the results from the marching-cubes method. In contrast, the marching-cubes method faces noise and resolution problems, where the former is because the vertices are interpolated, and the latter is because the ma...

Digital Shape Reconstruction of a Micro-Sized Machining Tool Using Light-Field Microscopy

In this paper, we propose a digital shape reconstruction method for micro-sized 3D (three-dimensional) objects based on the shape from silhouette (SFS) method that reconstructs the shape of a 3D model from silhouette images taken from multiple viewpoints. In the proposed method, images used in the SFS method are depth images acquired with a light-field microscope by digital refocusing (DR) of a stacked image along the axial direction. The DR can generate refocused images from an acquired image by an inverse ray tracing technique using a microlens array. Therefore, this technique provides fast image stacking with different focal planes. Our proposed method can reconstruct micro-sized object models including edges, convex shapes, and concave shapes on the surface of an object such as micro-sized defects so that damaged structures in the objects can be visualized. Firstly, we introduce the SFS method and the light-field microscope for 3D shape reconstruction that is required in the field of micro-sized manufacturing. Secondly, we show the developed experimental equipment for microscopic image acquisition. Depth calibration using a USAF1951 test target is carried out to convert relative value into actual length. Then 3D modeling techniques including image processing are implemented for digital shape reconstruction. Finally, 3D shape reconstruction results of micro-sized machining tools are shown and discussed.

Visual Hull-Based Geometric Data Compression of a 3-D Object

As image-based 3-D modeling is used in a variety of applications, accordingly, the compression of 3-D object geometry represented by multiple images becomes an important task. This paper presents a model-based approach to predict the geometric structure of an object using its visual hull. A visual hull is a geometric entity generated by shape-from-silhouette (SFS), and consequently it largely follows the overall shape of the object. The construction of a visual hull is computationally inexpensive and a visual hull can be encoded with relatively small amount of bits because it can be represented with 2-D silhouette images. Therefore, when it comes to the predictive compression of object’s geometric data, the visual hull should be an effective predictor. In the proposed method, the geometric structure of an object is represented by a layered depth image (LDI), and a visual hull from the LDI data is computed via silhouette generation and SFS. The geometry of an object is predicted with the computed visual hull, and the visual hull data with its prediction errors are encoded. Simulation results show that the proposed predictive coding based on visual hull outperforms the previous image-based methods and the partial surface-based method.

Image Enhancement for the 3-D Reconstruction in the Uncontrolled Environment using Shape from Silhouette

Among the multiple models for 3-D shape reconstruction, Shape from silhouette (SFS) is one of the fast and simple 3D shape rendering techniques as compare to other approaches. In SFS model multiple images captured from different viewpoints in a controlled environment are used as input data at the front end to extract silhouette and are free of noises. Silhouette extraction from such well-defined input data is easy and accurate, having no loss of information while extracting the silhouette. On the other hand images from uncontrolled environment involve many degradation factors. Common and frequently degradation factors are motion blur and noise addition which effects acquired image quality, illumination and resolution seriously. The proposed work is an effort to extract useful information from such environmentally variant images. The successful reconstruction of the image is main emphasis.

3D Shape from Silhouettes in Water for Online Novel-view Synthesis

This paper is aimed at presenting a new algorithm for full 3D shape reconstruction and online free-viewpoint rendering of objects in water. The key contributions are (1) a new calibration model for the refractive projection, and (2) a new 3D shape reconstruction algorithm based on shape-from-silhouette (SfS) concept. We also propose an online free-viewpoint rendering system as a practical application.

Computer-generated holograms using multiview images captured by a small number of sparsely arranged cameras

Computer-generated holograms (CGHs) using multiview images (MVIs) are holograms generated with multiple ordinary cameras. This process typically requires a huge number of cameras arranged at high density. In this paper, we propose a method to improve CGH using MVIs that obtains the MVIs by using voxel models rather than cameras. In the proposed method the voxel model is generated using the shape-from-silhouette (SFS) technique. We perform SFS using a small number of cameras arranged sparsely to create voxel models of objects and then generate the required number of images from these models by volume rendering. This enables us to generate CGHs using MVIs with just a small number of sparsely arranged cameras. Moreover, the proposed method arrange CGHs using MVIs at arbitrary positions.

An octree-based method for shape from inconsistent silhouettes

Shape-from-Silhouette (SfS) is the widely known problem of obtaining the 3D structure of an object from its silhouettes. Two main approaches can be employed: those based on voxel sets, which perform an exhaustive search of the working space, and those based on octrees, which perform a top-down analysis that speeds up the computation. The main problem of both approaches is the need for perfect silhouettes to obtain accurate results. Perfect background subtraction hardly ever happens in realistic scenarios, so these techniques are restricted to controlled environments where the consistency hypothesis can be assumed. Recently, some approaches (all of them based on voxel sets) have been proposed to solve the problem of inconsistency. Their main drawback is the high computational cost required to perform an exhaustive analysis of the working space. This paper proposes a novel approach to solve SfS with inconsistent silhouettes from an octree based perspective. The inconsistencies are dealt by means of the Dempster–Shafer (DS) theory and we employ a Butterworth function for adapting threshold values in each resolution level of the octree. The results obtained show that our proposal provides higher reconstruction quality than the standard octree based methods in realistic environments. When compared to voxel set approaches that manage inconsistency, our method obtains similar results with a reduction in the computing time of an order of magnitude.

System for reconstruction of three-dimensional micro objects from multiple photographic images

We introduce a system to reconstruct a three-dimensiojnal (3D) polygonal model of 3D micro objects with outer dimensions ranging from several hundred microns to several millimeters from multiple two-dimensional (2D) images of an object taken from different views. The data acquisition system consists of a digital microscope that captures still images at a resolution of 1600 × 1200 pixels and a computer-controlled turntable. We employ the shape-from-silhouette (SFS) method to construct a voxel-based 3D model from silhouette images. The concave shapes are further carved by using the space carving technique. In order to make the resulting model compatible with a commercial CAD/CAM system, the voxel model is converted into a triangular mesh using the marching cubes algorithm. Because the mesh generated from the voxel model by using the marching cubes algorithm inherits the staircase effect, the mesh is adjusted to recover the object precisely by using silhouette images. Finally, we evaluate the accuracy of the proposed method. The reconstructed models of complex micro objects indicate the effectiveness of the 3D shape reconstruction system for micro objects.

Shape from silhouette using Dempster–Shafer theory

This work proposes a novel shape from silhouette (SfS) algorithm using the Dempster–Shafer (DS) theory for dealing with inconsistent silhouettes. Standard SfS methods makes assumptions about consistency in the silhouettes employed. However, total consistency hardly ever happens in realistic scenarios because of inaccuracies in the background subtraction or occlusions, thus leading to poor reconstruction outside of controlled environments. Our method classify voxels using the DS theory instead of the traditional intersection of all visual cones. Sensors reliability is modelled taking into account the positional relationships between camera pairs and voxels. This information is employed to determine the degree in which a voxel belongs to a foreground object. Finally, evidences collected from all sensors are fused to choose the best hypothesis that determines the voxel state. Experiments performed with synthetic and real data show that our proposal outperforms the traditional SfS method and other techniques specifically designed to deal with inconsistencies. In addition, our method includes a parameter for adjusting the precision of the reconstructions so that it could be adapted to the application requirements.

3D Shape Reconstruction from 2D Images

AbstractWe present a method of reconstructing a 3D model from several 2D images of an object taken from different views. The data acquisition system consists of a camera on a tripod, and a computer-controlled turntable. We employ the Shape-From-Silhouette (SFS) method, which constructs a voxel-based 3D model from silhouette images. The concave shapes are further carved by using the space carving technique. In order to make the resulting model compatible with the CAD/CAM system, the voxel model is converted into a triangular mesh using the Marching Cubes algorithm. Since the mesh generated from the voxel model by using the Marching Cubes algorithm inherits the staircase effect, the mesh is adjusted to recover the object precisely by using silhouette images. Finally, we evaluate the accuracy of the proposed method. The reconstructed models of complex objects show the effectiveness of the proposed inexpensive 3D shape reconstruction system.

Shape from silhouettes based on a centripetal pentahedron model

In this paper we present a novel volumetric shape from silhouette (SfS) algorithm based on a centripetal pentahedron model (pent-model). The pent-model is an object-centered volumetric model composed of a set of pentahedrons cut from the centripetal triangular pyramids, which together partition the 3D space. The SfS algorithm first computes the pyramids by constructing a geodesic sphere. These pyramids are then projected onto the image planes of all cameras. The intersections between the projected pyramids and the silhouettes, which are a set of hexagons, are computed. This process can be performed very efficiently with pre-computed polar silhouette graphs (PSGs) and reduced PSGs. The hexagons are then back-projected into the 3D space, where the intersections are calculated and the pent-model is derived. After that, a mesh surface model can be extracted by marching pentahedrons. Our algorithm has the combined advantages of robustness, speediness and preciseness. Experimental results based on both synthetic images and real photos are presented.

Shape from inconsistent silhouette

Shape from silhouette (SfS) is the general term used to refer to the techniques that obtain a volume estimate from a set of binary images. In a first step, a number of images are taken from different positions around the scene of interest. Later, each image is segmented to produce binary masks, also called silhouettes, to delimit the objects of interest. Finally, the volume estimate is obtained as the maximal one which yields the silhouettes. The set of silhouettes is usually considered to be consistent which means that there exists at least one volume which completely explains them. However, silhouettes are normally inconsistent due to inaccurate calibration or erroneous silhouette extraction techniques. In spite of that, SfS techniques reconstruct only that part of the volume which projects consistently in all the silhouettes, leaving the rest unreconstructed. In this paper, we extend the idea of SfS to be used with sets of inconsistent silhouettes. We propose a fast technique for estimating that part of the volume which projects inconsistently and propose a criteria for classifying it by minimizing the probability of miss-classification taking into account the 2D error detection probabilities of the silhouettes. A number of theoretical and empirical results are given, showing that the proposed method reduces the reconstruction error.

3313 視体積交差法を用いた3D-CADモデルの作成

We present a method to reconstruct a voxel-based 3D model from several 2D images of an object of different views. The data acquisition system consists of a camera on a tripod, and a computer controlled turntable. We employ a so called Shape-From-Silhouette (SFS) method which constructs a voxel-based 3D model from silhouette images. The obtained voxel model is further carved by the voxel coloring technique. As a final step, we convert the voxel-based model to a triangular mesh representation using the marching cubes method followed by a mesh smoothing. The reconstructed models of complex objects show the effectiveness of our inexpensive data acquisition system.

Shape-From-Silhouette Across Time Part II: Applications to Human Modeling and Markerless Motion Tracking

In Part I of this paper we developed the theory and algorithms for performing Shape-From-Silhouette (SFS) across time. In this second part, we show how our temporal SFS algorithms can be used in the applications of human modeling and markerless motion tracking. First we build a system to acquire human kinematic models consisting of precise shape (constructed using the temporal SFS algorithm for rigid objects), joint locations, and body part segmentation (estimated using the temporal SFS algorithm for articulated objects). Once the kinematic models have been built, we show how they can be used to track the motion of the person in new video sequences. This marker-less tracking algorithm is based on the Visual Hull alignment algorithm used in both temporal SFS algorithms and utilizes both geometric (silhouette) and photometric (color) information.

Shape-From-Silhouette Across Time Part I: Theory and Algorithms

Shape-From-Silhouette (SFS) is a shape reconstruction method which constructs a 3D shape estimate of an object using silhouette images of the object. The output of a SFS algorithm is known as the Visual Hull (VH). Traditionally SFS is either performed on static objects, or separately at each time instant in the case of videos of moving objects. In this paper we develop a theory of performing SFS across time: estimating the shape of a dynamic object (with unknown motion) by combining all of the silhouette images of the object over time. We first introduce a one dimensional element called a Bounding Edge to represent the Visual Hull. We then show that aligning two Visual Hulls using just their silhouettes is in general ambiguous and derive the geometric constraints (in terms of Bounding Edges) that govern the alignment. To break the alignment ambiguity, we combine stereo information with silhouette information and derive a Temporal SFS algorithm which consists of two steps: (1) estimate the motion of the objects over time (Visual Hull Alignment) and (2) combine the silhouette information using the estimated motion (Visual Hull Refinement). The algorithm is first developed for rigid objects and then extended to articulated objects. In the Part II of this paper we apply our temporal SFS algorithm to two human-related applications: (1) the acquisition of detailed human kinematic models and (2) marker-less motion tracking.

Three-dimensional shape reconstruction from images by shape-from-silhouette technique and iterative triangulation

We propose an image-based three-dimensional shape determination system. The shape, and thus the three-dimensional coordinate information of the 3-D object, is determined solely from captured images of the 3-D object from a prescribed set of viewpoints. The approach is based on the shape-from-silhouette (SFS) technique, and the efficacy of the SFS method is tested using a sample data set. The extracted three-dimensional shape is modeled with polygons generated by a new iterative triangulation algorithm, and the polygon model can be exported to commercial software. The proposed system may be used to visualize the 3-D object efficiently, or to quickly generate initial CAD data for reverse engineering purposes, including three dimensional design applications such as 3-D animation and 3-D games.