Image-based Visual Hull Research Articles

Temporal Coherence in Image-Based Visual Hull Rendering

Image-based visual hull rendering is a method for generating depth maps of a desired viewpoint from a set of silhouette images captured by calibrated cameras. It does not compute a view-independent data representation, such as a voxel grid or a mesh, which makes it particularly efficient for dynamic scenes. When users are captured, the scene is usually dynamic, but does not change rapidly because people move smoothly within a subsecond time frame. Exploiting this temporal coherence to avoid redundant calculations is challenging because of the lack of an explicit data representation. This paper analyzes the image-based visual hull algorithm to find intermediate information that stays valid over time and is, therefore, worth to make explicit. We then derive methods that exploit this information to improve the rendering performance. Our methods reduce the execution time by up to 25 percent. When the user's motions are very slow, reductions of up to 50 percent are achieved.

Virtual Try-On through Image-Based Rendering

Virtual try-on applications have become popular because they allow users to watch themselves wearing different clothes without the effort of changing them physically. This helps users to make quick buying decisions and, thus, improves the sales efficiency of retailers. Previous solutions usually involve motion capture, 3D reconstruction or modeling, which are time consuming and not robust for all body poses. Our method avoids these steps by combining image-based renderings of the user and previously recorded garments. It transfers the appearance of a garment recorded from one user to another by matching input and recorded frames, image-based visual hull rendering, and online registration methods. Using images of real garments allows for a realistic rendering quality with high performance. It is suitable for a wide range of clothes and complex appearances, allows arbitrary viewing angles, and requires only little manual input. Our system is particularly useful for virtual try-on applications as well as interactive games.

Image-based visual hull of a tennis racket

Researchers are often interested in tracking object movement in sport. This could be useful to identify equipment designs that match the technique of players. Previously, stereo camera systems have been used to track markers attached to striking implements to measure their movement in three dimensions. However, manual selection of markers on the image plane can be time consuming and inaccurate. There is potential however to reduce these drawbacks associated with marker-based analysis by tracking a striking implement using its visual hull. The closest geometric approximation of an object that can be reconstructed using only its silhouette images is its visual hull. Early applications of visual hulls include size and shape estimation of objects such as stones. Recently, subject specific visual hulls constructed from multiple camera views combined with anatomical tracking algorithms have measured human motion through markerless motion capture. However, multiple camera systems are not practical for real play conditions in most sports. The application of visual hulls to measure movements of striking implements used in sport has not yet been explored. A set of calibrated views of a tennis racket were captured and segmented into binary images to obtain silhouettes. The visual hull of a tennis racket was constructed by intersection of the volume of space formed by back-projecting the silhouettes from all input views. This research is the first stage in the development of a system that measures movement of a striking implement in real play conditions by combining its visual hull with footage from a single camera.

Bottom and Concave Surface Rendering in Image-based Visual Hull

Bottom and concave shapes on object surface are difficult to reconstruct in image-based visual hull method. In this paper, we propose a simple but efficient method to solve these problems in regular image-based visual hull framework. With the help of a simple image acquiring platform which involves a glass and a mirror, we can capture images of the object from both upper and lower side at the same time. Using these images, silhouette cones necessary for reconstructing the bottom and the concave surfaces could be generated. Therefore the final rendering result of the object can be significantly improved in accuracy and reality, especially in the parts of bottom and concaves.

Synthesis of Silhouettes and Visual Hull Reconstruction for Articulated Humans

In this paper, we propose a complete framework for improved synthesis and understanding of the human pose from a limited number of silhouette images. It combines the active image-based visual hull (IBVH) algorithm and a contour-based body part segmentation technique. We derive a simple, approximate algorithm to decide the extrinsic parameters of a virtual camera, and synthesize the turntable image collection of the person using the IBVH algorithm by actively moving the virtual camera on a properly computed circular trajectory around the person. Using the turning function distance as the silhouette similarity measurement, this approach can be used to generate the desired pose-normalized images for recognition applications. In order to overcome the inability of the visual hull (VH) method to reconstruct concave regions, we propose a contour-based human body part localization algorithm to segment the silhouette images into convex body parts. The body parts observed from the virtual view are generated separately from the corresponding body parts observed from the input views and then assembled together for a more accurate VH reconstruction. Furthermore, the obtained turntable image collection helps to improve the body part segmentation and identification process. By using the inner distance shape context (IDSC) measurement, we are able to estimate the body part locations more accurately from a synthesized view where we can localize the body part more precisely. Experiments show that the proposed algorithm can greatly improve body part segmentation and hence shape reconstruction results.