Visual Hull Reconstruction Research Articles

Geometry characterization and thermal radiation prediction of buoyant square and line diffusion flames based on 3D flame reconstruction

This paper presents an experimental study on the instantaneous and mean geometrical features and radiation properties of buoyant square and line diffusion flames with heat release rates (Q˙) of 5.0–30.0 kW using the visual hull reconstruction method. Equal fuel exit areas were selected for the square and linear burners to reveal the effect of fire source shape on the combustion process. Results show that the reconstructed three-dimensional (3D) instantaneous flames could well reflect the temporal evolutions of the square and line flames. The instantaneous flame surface area and volume exhibit excellent synchronization with time. Consistent with the predictions based on the turbulent mixing concepts, the mean flame surface area and volume of the square flame are lower than the line flame under equal Q˙. The heat release rates per unit surface area remain unchanged, and the heat release rates per unit volume decrease slightly with Q˙ for the square and line flames. The mean beam lengths, calculated from the mean flame surface area and volume of the 3D flame shape, increase with Q˙ steadily for the two types of flames. A 3D flame radiation model is developed based on the reconstructed 3D instantaneous flame shape. It could accurately predict the instantaneous and time-averaged radiative heat fluxes in the near-field and far-field for axisymmetric and non-axisymmetric flames.

Diamond Inclusion Reconstruction

In this paper, we make an improvement on the conventional visual hull reconstruction method that runs on a single consumer graphics card. The target application is a high-precision diamond inclusion reconstruction system. One major contribution of this paper is an evaluation system of voxels for high-precision reconstruction. In contrast to existing approaches, it allows us to reconstruct the thin structure of the diamond inclusion. Based on this, we obtain a significant improvement in reconstruction precision, especially for thin inclusion.

Fast High Resolution Volume Carving for 3D Plant Shoot Reconstruction

Volume carving is a well established method for visual hull reconstruction and has been successfully applied in plant phenotyping, especially for 3d reconstruction of small plants and seeds. When imaging larger plants at still relatively high spatial resolution (≤1 mm), well known implementations become slow or have prohibitively large memory needs. Here we present and evaluate a computationally efficient algorithm for volume carving, allowing e.g., 3D reconstruction of plant shoots. It combines a well-known multi-grid representation called “Octree” with an efficient image region integration scheme called “Integral image.” Speedup with respect to less efficient octree implementations is about 2 orders of magnitude, due to the introduced refinement strategy “Mark and refine.” Speedup is about a factor 1.6 compared to a highly optimized GPU implementation using equidistant voxel grids, even without using any parallelization. We demonstrate the application of this method for trait derivation of banana and maize plants.

14-3 ボクセル空間における幾何的制約を考慮したVisual Hullの高精度化に関する一検討(第14部門 コンピュータビジョン・符号化)

14-3 ボクセル空間における幾何的制約を考慮したVisual Hullの高精度化に関する一検討(第14部門 コンピュータビジョン・符号化)

Quantitative wake analysis of a freely swimming fish using 3D synthetic aperture PIV

Synthetic aperture PIV (SAPIV) is used to quantitatively analyze the wake behind a giant danio (Danio aequipinnatus) swimming freely in a seeded quiescent tank. The experiment is designed with minimal constraints on animal behavior to ensure that natural swimming occurs. The fish exhibits forward swimming and turning behaviors at speeds between 0.9 and 1.5 body lengths/second. Results show clearly isolated and linked vortex rings in the wake structure, as well as the thrust jet coming off of a visual hull reconstruction of the fish body. As a benchmark for quantitative analysis of volumetric PIV data, the vortex circulation and impulse are computed using methods consistent with those applied to planar PIV data. Volumetric momentum analysis frameworks are discussed for linked and asymmetric vortex structures, laying a foundation for further volumetric studies of swimming hydrodynamics with SAPIV. Additionally, a novel weighted refocusing method is presented as an improvement to SAPIV reconstruction.

Camera calibration with translational planar object for visual hull reconstruction

Camera calibration with translational planar object for visual hull reconstruction

GPU-based parallel construction of compact visual hull meshes

Building a visual hull model from multiple two-dimensional images provides an effective way of understanding the three-dimensional geometries inherent in the images. In this paper, we present a GPU accelerated algorithm for volumetric visual hull reconstruction that aims to harness the full compute power of the many-core processor. From a set of binary silhouette images with respective camera parameters, our parallel algorithm directly outputs the triangular mesh of the resulting visual hull in the indexed face set format for a compact mesh representation. Unlike previous approaches, the presented method extracts a smooth silhouette contour on the fly from each binary image, which markedly reduces the bumpy artifacts on the visual hull surface due to a simple binary in/out classification. In addition, it applies several optimization techniques that allow an efficient CUDA implementation. We also demonstrate that the compact mesh construction scheme can easily be modified for also producing a time- and space-efficient GPU implementation of the marching cubes algorithm.

Real-time voxel-based visual hull reconstruction

High-definition 3D video for real-time applications is one of the features explored by the next generation of telecommunication systems. From an implementation point for view, the most restrictive limitations of previously published methods are processing speed and memory bandwidth. In this paper we present several algorithmic improvements for Visual Hull (VH) reconstruction using a voxel-based approach that reduces the resource consumption. We also present FPGA and GPU based implementations. These improvements have been shown to outperform processing speeds of the latest publications, allowing real-time and high-definition implementations. The proposed approach allows a 256×256×128 reconstruction volume to be obtained in only 33.55ms in a FPGA.

Tele-Immersive Interaction with Intelligent Virtual Agents Based on Real-Time 3D Modeling

To enable intelligent agents interacting smoothly with human users, researchers have been deploying novel interaction modalities (e.g. non-verbal cue, vision and touch) in addition to agents’ conversational skills. Models of multi-modality interaction can enhance agents’ real-time perception, cognition and reaction towards the user. In this paper we report a novel tele-immersive interaction system developed using real-time 3D modelling techniques. In such system user’s full body is reconstructed using multi-view cameras and CUDA based visual hull reconstruction algorithm. User’s mesh model is then loaded into a virtual environment for interacting with an autonomous agent. Technical details and initial results of the system are illustrated in this paper. Following that a novel interaction scenario is proposed which links the virtual agent with a remote physical robot who takes the role of mediating interactions between two geographically separated users. Finally we discuss in depth the implications of such human-agent interaction and possible future improvements and directions.

Recovery and Reasoning About Occlusions in 3D Using Few Cameras with Applications to 3D Tracking

In this work we propose algorithms to learn the locations of static occlusions and reason about both static and dynamic occlusion scenarios in multi-camera scenes for 3D surveillance (e.g., reconstruction, tracking). We will show that this leads to a computer system which is able to more effectively track (follow) objects in video when they are obstructed from some of the views. Because of the nature of the application area, our algorithm will be under the constraints of using few cameras (no more than 3) that are configured wide-baseline. Our algorithm consists of a learning phase, where a 3D probabilistic model of occlusions is estimated per-voxel, per-view over time via an iterative framework. In this framework, at each frame the visual hull of each foreground object (person) is computed via a Markov Random Field that integrates the occlusion model. The model is then updated at each frame using this solution, providing an iterative process that can accurately estimate the occlusion model over time and overcome the few-camera constraint. We demonstrate the application of such a model to a number of areas, including visual hull reconstruction, the reconstruction of the occluding structures themselves, and 3D tracking.

Parameterised structured light imaging for depth edge detection

This reported research features parameterised structured light imaging that is practically useful for detecting depth edges. Given input parameters such as the range of distances of an object from the camera/projector and minimum detectable depth difference, the presented method is capable of computing an optimal pattern width and the number of structured light images that are needed to detect all depth edges in the specified range of distances that have at least the given detectable depth difference. Application of this parameter control to the detection of silhouette edges for visual hull reconstruction shows the effectiveness of the method.

Self-Calibration of Biplanar Radiographic Images Through Geometric Spine Shape Descriptors

This paper presents a novel self-calibration method of an X-ray scene applied for the 3-D reconstruction of the scoliotic spine. Current calibration techniques either use a cumbersome calibration apparatus or depend on manually identified landmarks to determine the geometric configuration, thus limiting routine clinical evaluation. The proposed approach uses high-level information automatically extracted from biplanar X-rays to solve the radiographic scene parameters. We first present a segmentation method that takes into account the variable appearance and geometry of a scoliotic spine in order to isolate and extract the silhouettes of the anterior vertebral body. By incorporating prior anatomical information through a Bayesian formulation of the morphological distribution, a multiscale spine segmentation framework is proposed for scoliotic patients. An iterative nonlinear optimization procedure, integrating a 3-D visual hull reconstruction and geometrical torsion properties of the spine, is then applied to globally refine the geometrical parameters of the 3-D viewing scene and obtain the optimal 3-D reconstruction. An experimental comparison with data provided from reference synthetic models yields similar accuracy on the retroprojection of low-level primitives such as anatomical landmarks identified on each vertebra (2.2 mm). Results obtained from a clinical validation on 60 pairs of uncalibrated digitized X-rays of adolescents with scoliosis show that the 3-D reconstructions from the new system offer geometrically accurate models with insignificant differences for 3-D clinical indexes commonly used in the evaluation of spinal deformities. The reported experiments demonstrate a viable and accurate alternative to previous reconstruction techniques, offering the first automatic approach for routine 3-D clinical assessment in radiographic suites.

Automated hull reconstruction motion tracking (HRMT) applied to sideways maneuvers of free-flying insects

Flying insects perform aerial maneuvers through slight manipulations of their wing motions. Because such manipulations in wing kinematics are subtle, a reliable method is needed to properly discern consistent kinematic strategies used by the insect from inconsistent variations and measurement error. Here, we introduce a novel automated method that accurately extracts full, 3D body and wing kinematics from high-resolution films of free-flying insects. This method combines visual hull reconstruction, principal components analysis, and geometric information about the insect to recover time series data of positions and orientations. The technique has small, well-characterized errors of under 3 pixels for positions and 5 deg. for orientations. To show its utility, we apply this motion tracking to the flight of fruit flies, Drosophila melanogaster. We find that fruit flies generate sideways forces during some maneuvers and that strong lateral acceleration is associated with differences between the left and right wing angles of attack. Remarkably, this asymmetry can be induced by simply altering the relative timing of flips between the right and left wings, and we observe that fruit flies employ timing differences as high as 10% of a wing beat period while accelerating sideways at 40% g.

Incorporating object-centered sampling and Delaunay tetrahedrization for visual hull reconstruction

In this paper we present a novel shape from silhouette algorithm. For an object to be modeled, the algorithm first computes a cloud of points located on a pencil of rays and distributed evenly on the visual hull surface, inside and outside the visual hull. Then Delaunay tetrahedrization is applied to the point cloud to partition its convex hull into a set of tetrahedrons. Finally, outlier tetrahedrons are removed by tetrahedron peeling, and a mesh model of the visual hull is extracted. The algorithm is robust, free from discretization artifacts, and produces a mesh model composed of well-shaped triangles.

Synthesis of Silhouettes and Visual Hull Reconstruction for Articulated Humans

In this paper, we propose a complete framework for improved synthesis and understanding of the human pose from a limited number of silhouette images. It combines the active image-based visual hull (IBVH) algorithm and a contour-based body part segmentation technique. We derive a simple, approximate algorithm to decide the extrinsic parameters of a virtual camera, and synthesize the turntable image collection of the person using the IBVH algorithm by actively moving the virtual camera on a properly computed circular trajectory around the person. Using the turning function distance as the silhouette similarity measurement, this approach can be used to generate the desired pose-normalized images for recognition applications. In order to overcome the inability of the visual hull (VH) method to reconstruct concave regions, we propose a contour-based human body part localization algorithm to segment the silhouette images into convex body parts. The body parts observed from the virtual view are generated separately from the corresponding body parts observed from the input views and then assembled together for a more accurate VH reconstruction. Furthermore, the obtained turntable image collection helps to improve the body part segmentation and identification process. By using the inner distance shape context (IDSC) measurement, we are able to estimate the body part locations more accurately from a synthesized view where we can localize the body part more precisely. Experiments show that the proposed algorithm can greatly improve body part segmentation and hence shape reconstruction results.

The Theory and Practice of Coplanar Shadowgram Imaging for Acquiring Visual Hulls of Intricate Objects

Acquiring 3D models of intricate objects (like tree branches, bicycles and insects) is a challenging task due to severe self-occlusions, repeated thin structures, and surface discontinuities. In theory, a shape-from-silhouettes (SFS) approach can overcome these difficulties and reconstruct visual hulls that are close to the actual shapes, regardless of the complexity of the object. In practice, however, SFS is highly sensitive to errors in silhouette contours and the calibration of the imaging system, and has therefore not been used for obtaining accurate shapes with a large number of views. In this work, we present a practical approach to SFS using a novel technique called coplanar shadowgram imaging that allows us to use dozens to even hundreds of views for visual hull reconstruction. A point light source is moved around an object and the shadows (silhouettes) cast onto a single background plane are imaged. We characterize this imaging system in terms of image projection, reconstruction ambiguity, epipolar geometry, and shape and source recovery. The coplanarity of the shadowgrams yields unique geometric properties that are not possible in traditional multi-view camera-based imaging systems. These properties allow us to derive a robust and automatic algorithm to recover the visual hull of an object and the 3D positions of the light source simultaneously, regardless of the complexity of the object. We demonstrate the acquisition of several intricate shapes with severe occlusions and thin structures, using 50 to 120 views.

GPU-Based Optimization of a Free-Viewpoint Video System

We present a method for optimizing the reconstruction and rendering of 3D objects from multiple images by utilizing the latest features of consumer-level graphics hardware based on shader model 4.0. We accelerate visual hull reconstruction by rewriting a shape-from-silhouette algorithm to execute on the GPU's parallel architecture. Rendering is optimized through the application of geometry shaders to generate billboarding micr facets textured with captured images. We also present a method for handling occlusion in the camera selection process that is optimized for execution on the GPU. Execution time is further improved by rendering intermediate results directly to texture to minimize the number of data transfers between graphics and main memory. We show our GPU based system to be significantly more efficient than a purely CPUbased approach, due to the parallel nature of the GPU, while maintaining graphical quality.

A Markerless Motion Capture System to Study Musculoskeletal Biomechanics: Visual Hull and Simulated Annealing Approach

Human motion capture is frequently used to study musculoskeletal biomechanics and clinical problems, as well as to provide realistic animation for the entertainment industry. The most popular technique for human motion capture uses markers placed on the skin, despite some important drawbacks including the impediment to the motion by the presence of skin markers and relative movement between the skin where the markers are placed and the underlying bone. The latter makes it difficult to estimate the motion of the underlying bone, which is the variable of interest for biomechanical and clinical applications. A model-based markerless motion capture system is presented in this study, which does not require the placement of any markers on the subject's body. The described method is based on visual hull reconstruction and an a priori model of the subject. A custom version of adapted fast simulated annealing has been developed to match the model to the visual hull. The tracking capability and a quantitative validation of the method were evaluated in a virtual environment for a complete gait cycle. The obtained mean errors, for an entire gait cycle, for knee and hip flexion are respectively 1.5 degrees (+/-3.9 degrees ) and 2.0 degrees (+/-3.0 degrees ), while for knee and hip adduction they are respectively 2.0 degrees (+/-2.3 degrees ) and 1.1 degrees (+/-1.7 degrees ). Results for the ankle and shoulder joints are also presented. Experimental results captured in a gait laboratory with a real subject are also shown to demonstrate the effectiveness and potential of the presented method in a clinical environment.