3D Depth Map Research Articles

Three-Dimensional Depth Map Motion Estimation and Compensation for 3D Video Compression

In this paper, we propose a new approach to 3D depth map motion estimation and compensation for 3D video compression. 3D video provides realistic vision and will be a feature of future video displays. The many kinds of 3D formats include multiview 3D, single view with a depth map, time division multiple 3D, and so on. 3D video requires huge quantities of data and requires a great deal of storage space to store 3D information. In addition, when 3D video is transmitted, the huge amount of data should be compressed to reduce bandwidth usage. In order to solve this problem, we adopt single view with a depth map 3D format and design a 3D depth map compression scheme for 3D video. We consider depth map motion estimation and compensation to achieve temporal compression of 3D video.

A Robust Method for Hand Tracking Using Mean-shift Algorithm and Kalman Filter in Stereo Color Image Sequences

Real-time hand tracking is a challenging task in many computer vision applications such as gesture recognition. This paper proposes a robust method for hand tracking in a complex environment using Mean-shift analysis and Kalman filter in conjunction with 3D depth map. The depth information solve the overlapping problem between hands and face, which is obtained by passive stereo mea- suring based on cross correlation and the known calibration data of the cameras. Mean-shift analysis uses the gradient of Bhattacharyya coefficient as a similarity function to derive the candidate of the hand that is most similar to a given hand target model. And then, Kalman filter is used to estimate the position of the hand target. The results of hand tracking, tested on various video sequences, are robust to changes in shape as well as partial occlusion. Keywords—Computer Vision and Image Analysis, Object Track- ing, Gesture Recognition.

스테레오 적외선 조명 및 단일카메라를 이용한 3차원 환경인지

This paper describes a new sensor system for 3D environment perception using stereo structured infrared light sources and a camera. Environment and obstacle sensing is the key issue for mobile robot localization and navigation. Laser scanners and infrared scanners cover <TEX>$180^{\circ}$</TEX> and are accurate but too expensive. Those sensors use rotating light beams so that the range measurements are constrained on a plane. 3D measurements are much more useful in many ways for obstacle detection, map building and localization. Stereo vision is very common way of getting the depth information of 3D environment. However, it requires that the correspondence should be clearly identified and it also heavily depends on the light condition of the environment. Instead of using stereo camera, monocular camera and two projected infrared light sources are used in order to reduce the effects of the ambient light while getting 3D depth map. Modeling of the projected light pattern enabled precise estimation of the range. Two successive captures of the image with left and right infrared light projection provide several benefits, which include wider area of depth measurement, higher spatial resolution and the visibility perception.

Fault-tolerant 3D Mapping with Application to an Orchard Robot

In this paper we present a geometric reasoning method for dealing with noise as well as faults present in 3D depth maps. These maps are acquired using stereo-vision sensors, but our framework makes no assumption about the origin of the underlying data. The method is based on observations made on the environment from different camera poses (viewpoints), where the occupied space as well as uncertainties in the range measurement are modelled using dynamic octree structures. This scheme allows us to detect and diagnose faulty range measurements in an efficient manner. We present results on the acquisition of comprehensive 3D maps for an agricultural robot operating in an orchard.

적외선 조명 및 단일카메라를 이용한 입체거리 센서의 개발

This paper describes a new sensor system for 3D range measurement using the structured infrared light. Environment and obstacle sensing is the key issue for mobile robot localization and navigation. Laser scanners and infrared scanners cover <TEX>$180^{\circ}$</TEX> and are accurate but too expensive. Those sensors use rotating light beams so that the range measurements are constrained on a plane. 3D measurements are much more useful in many ways for obstacle detection, map building and localization. Stereo vision is very common way of getting the depth information of 3D environment. However, it requires that the correspondence should be clearly identified and it also heavily depends on the light condition of the environment. Instead of using stereo camera, monocular camera and the projected infrared light are used in order to reduce the effects of the ambient light while getting 3D depth map. Modeling of the projected light pattern enabled precise estimation of the range. Identification of the cells from the pattern is the key issue in the proposed method. Several methods of correctly identifying the cells are discussed and verified with experiments.

A Novel H.264 Based Motion Vector Recovery Method for 3D Video Transmission

The performance of 3D video transmission over error-prone channels is limited by the channel noise. Error concealment is an effective method to combat these channel errors. In H.264 video coding, motion vectors (MVs) are highly significant for the matching of the missing video block. Thus how to recover the lost MV is an important issue for error concealment in 3D video communications. In this paper, we propose a new method to recover the corrupted MVs for 3D video transmission, which is based on the traditional boundary matching algorithm (BMA) for the 2D video. Our proposed depth based BMA (DBMA) method will make full use of the relationships between the 2D video and depth map to recover the lost MV based on BMA. Experimental results show that it is highly effective and significantly outperforms other existing MV recovery methods for 3D video error concealment at different block loss ratios (BLRs)1.

A Two-Level Generative Model for Cloth Representation and Shape from Shading

In this paper, we present a two-level generative model for representing the images and surface depth maps of drapery and clothes. The upper level consists of a number of folds which will generate the high contrast (ridge) areas with a dictionary of shading primitives (for 2D images) and fold primitives (for 3D depth maps). These primitives are represented in parametric forms and are learned in a supervised learning phase using 3D surfaces of clothes acquired through photometric stereo. The lower level consists of the remaining flat areas which fill between the folds with a smoothness prior (Markov random field). We show that the classical ill-posed problem-shape from shading (SFS) can be much improved by this two-level model for its reduced dimensionality and incorporation of middle-level visual knowledge, i.e., the dictionary of primitives. Given an input image, we first infer the folds and compute a sketch graph using a sketch pursuit algorithm as in the primal sketch [10], [11]. The 3D folds are estimated by parameter fitting using the fold dictionary and they form the "skeleton" of the drapery/cloth surfaces. Then, the lower level is computed by conventional SFS method using the fold areas as boundary conditions. The two levels interact at the final stage by optimizing a joint Bayesian posterior probability on the depth map. We show a number of experiments which demonstrate more robust results in comparison with state-of-the-art work. In a broader scope, our representation can be viewed as a two-level inhomogeneous MRF model which is applicable to general shape-from-X problems. Our study is an attempt to revisit Marr's idea [23] of computing the 2(1/2)D sketch from primal sketch. In a companion paper [2], we study shape from stereo based on a similar two-level generative sketch representation.

Advancing state-of-the-art 3D human facial recognition

Automated human face recognition has numerous applications in areas of security and surveillance, database retrieval, and human-computer interaction. Over two decades of research have resulted in successful techniques for recognizing color/intensity 2D frontal facial images.1 However, performance declines severely with variations in pose, expression, and ambient illumination.2 Achieving robust and accurate automatic face recognition remains an important and open problem. Recently, researchers have proposed the use of 3D models.3 In addition to providing explicit information about the shape of the face, thesemodels can easily correct for pose by rigid rotation in 3D space, and are scale and illumination invariant. However, successful 3D facial recognition techniques2 based on rigid surface matching suffer from an exceptionally high computational cost that makes them unsuitable for real-time operation. Existing 3D techniques are also unable to handle changes in facial expression. We have attempted to resolve some of these issues. Features for face recognition are numerical quantities that vary considerably between individuals yet remain constant for different instances of the same individual. To date, 3D recognition algorithms that are based on local features have not embodied a sound understanding of discriminatory facial structural characteristics. As a starting point, we extensively investigated the existing literature on anthropometric facial proportions.5 We identified measurements reported to be highly diverse across different age and gender cohorts and ethnic groups. We employed these to design novel and effective algorithms.4 Specifically, we used 3D euclidean and along-the-surface geodesic distances between 25 manually located facial fiducial points (i.e., loci of interest, see Figure 1), associated with the highly variable anthropometric measurements, as features for face recognition.4 We studied geodesic distances because a recent study6 suggests that changes in expression may be modeled as isometric deformations of the face, under which geodesic Figure 1. (a) The facial fiducial points associated with discriminatory facial anthropometric measurements on a color image. (b) These points on a facial 3D depth map image (reproduced from Gupta et al.4).

Geomatics for Precise 3D Breast Imaging

Canadian women have a one in nine chance of developing breast cancer during their lifetime. Mammography is the most common imaging technology used for breast cancer detection in its earliest stages through screening programs. Clusters of microcalcifications are primary indicators of breast cancer; the shape, size and number may be used to determine whether they are malignant or benign. However, overlapping images of calcifications on a mammogram hinder the classification of the shape and size of each calcification and a misdiagnosis may occur resulting in either an unnecessary biopsy being performed or a necessary biopsy not being performed. The introduction of 3D imaging techniques such as standard photogrammetry may increase the confidence of the radiologist when making his/her diagnosis. In this paper, traditional analytical photogrammetric techniques for the 3D mathematical reconstruction of microcalcifications are presented. The techniques are applied to a specially designed and constructed x-ray transparent Plexiglas phantom (control object). The phantom was embedded with 1.0 mm x-ray opaque lead pellets configured to represent overlapping microcalcifications. Control points on the phantom were determined by standard survey methods and hand measurements. X-ray films were obtained using a LORAD M-III mammography machine. The photogrammetric techniques of relative and absolute orientation were applied to the 2D mammographic films to analytically generate a 3D depth map with an overall accuracy of 0.6 mm. A Bundle Adjustment and the Direct Linear Transform were used to confirm the results.

LAMP: 3D layered, adaptive-resolution, and multi-perspective panorama—a new scene representation

A compact visual representation, called the 3D layered, adaptive-resolution, and multi-perspective panorama (LAMP), is proposed for representing large-scale 3D scenes with large variations of depths and obvious occlusions. Two kinds of 3D LAMP representations are proposed: the relief-like LAMP and the image-based LAMP. Both types of LAMPs concisely represent almost all the information from a long image sequence. Methods to construct LAMP representations from video sequences with dominant translation are provided. The relief-like LAMP is basically a single extended multi-perspective panoramic view image. Each pixel has a pair of texture and depth values, but each pixel may also have multiple pairs of texture-depth values to represent occlusion in layers, in addition to adaptive resolution changing with depth. The image-based LAMP, on the other hand, consists of a set of multi-perspective layers, each of which has a pair of 2D texture and depth maps, but with adaptive time-sampling scales depending on depths of scene points. Several examples of 3D LAMP construction for real image sequences are given. The 3D LAMP is a concise and powerful representation for image-based rendering.

2P2-L05 三次元距離画像の RRT 探索によるヒューマノイドロボットの腕の動作計画

2P2-L05 三次元距離画像の RRT 探索によるヒューマノイドロボットの腕の動作計画