Camera Projection Research Articles

3D reconstruction of specular surfaces using a calibrated projector–camera setup

Structured light is widely used for shape measurement of beamless surfaces using the triangulation principle. In the case of specular surfaces deflectometry is an appropriate method. Hereby the reflection of a light pattern is observed by a camera. The distortion of the reflected pattern is evaluated to obtain information about the reflecting surface. An important requirement for a 3d reconstruction of a specular surface by deflectometry is a calibrated measurement setup. We propose a method for the overall calibration of a setup consisting of a structured light source, a projection screen and a camera. We consider all extrinsic and intrinsic parameters for the optical mapping including a distortion model for the projector and for the camera, respectively. Given the deflectometric data obtained by the calibrated setup two methods are described which allow the 3d reconstruction of points on a specular surface. This is not trivial as a reconstruction using solely deflectometric data shows ambiguity. In the first method screen displacement between two deflectometric measurements is used to overcome this ambiguity. In the second method curvature-like features on the surface are determined to serve as starting points for a region growing approach. Results of the reconstruction of a specular surface are shown and the performance of the described reconstruction methods are compared to each other.

Nonintrusive iris image acquisition system based on a pan-tilt-zoom camera and light stripe projection

Although iris recognition is one of the most accurate biometric technologies, it has not yet been widely used in practical applications. This is mainly due to user inconvenience during the image acquisition phase. Specifically, users try to adjust their eye position within small capture volume at a close distance from the system. To overcome these problems, we propose a novel iris image acquisition system that pro- vides users with unconstrained environments: a large operating range, enabling movement from standing posture, and capturing good-quality iris images in an acceptable time. The proposed system has the follow- ing three contributions compared with previous works: 1 the capture volume is significantly increased by using a pan-tilt-zoom PTZ camera guided by a light stripe projection, 2 the iris location in the large capture volume is found fast due to 1-D vertical face searching from the user's horizontal position obtained by the light stripe projection, and 3 zoom- ing and focusing on the user's irises at a distance are accurate and fast using the estimated 3-D position of a face by the light stripe projection and the PTZ camera. Experimental results show that the proposed sys- tem can capture good-quality iris images in 2.479 s on average at a distance of 1.5 to 3 m, while allowing a limited amount of movement by the user.

Improved robust and accurate camera calibration method used for machine vision application

Camera calibration is an important step for vision-based measurement applications. A well-known flexible camera calibration method is analyzed that uses the checkerboard pattern plane and in which the camera can be moved freely. When using a perspective projection camera model, characteristics of both the objective plane and the image plane are utilized and accurate results can be obtained. However, the method's results may fail when the rotation angles of the planar pattern are small, and the distortion coefficients obtained under the perspective projection model can not be used for a real-time vision application. We solve the ill-conditioned equations using the genetic algorithm, and the correct camera parameters are always obtained. We compute the distortion coefficients of the inverse projection model, which can be used for general vision applications. The influence of the corner detection precision is taken into consideration. Simulation shows that the best results may be obtained when the planar pattern is placed in a close range and its rotation angle is small. Simulations and real-world experiments illustrate that the improved calibration algorithm can always obtain robust and accurate results.

Probing Metagenomics by Rapid Cluster Analysis of Very Large Datasets

BackgroundThe scale and diversity of metagenomic sequencing projects challenge both our technical and conceptual approaches in gene and genome annotations. The recent Sorcerer II Global Ocean Sampling (GOS) expedition yielded millions of predicted protein sequences, which significantly altered the landscape of known protein space by more than doubling its size and adding thousands of new families (Yooseph et al., 2007 PLoS Biol 5, e16). Such datasets, not only by their sheer size, but also by many other features, defy conventional analysis and annotation methods.Methodology/Principal FindingsIn this study, we describe an approach for rapid analysis of the sequence diversity and the internal structure of such very large datasets by advanced clustering strategies using the newly modified CD-HIT algorithm. We performed a hierarchical clustering analysis on the 17.4 million Open Reading Frames (ORFs) identified from the GOS study and found over 33 thousand large predicted protein clusters comprising nearly 6 million sequences. Twenty percent of these clusters did not match known protein families by sequence similarity search and might represent novel protein families. Distributions of the large clusters were illustrated on organism composition, functional class, and sample locations.Conclusion/SignificanceOur clustering took about two orders of magnitude less computational effort than the similar protein family analysis of original GOS study. This approach will help to analyze other large metagenomic datasets in the future. A Web server with our clustering results and annotations of predicted protein clusters is available online at http://tools.camera.calit2.net/gos under the CAMERA project.

What can we learn about the scene structure from three orthogonal vanishing points in images

The problem of 3D Euclidean reconstruction of structured scenes from uncalibrated images based on the property of vanishing points is studied in this paper. Under the assumption of three-parameter-camera model with varying parameters, we prove that under an arbitrary preassigned world coordinate system, the camera projection matrix of an image can be uniquely determined from three mutually orthogonal vanishing points obtained from the image. When multiple images of the object are present, it is proved that the global consistent projection matrices can be recovered if an arbitrary reference point in space is observed across the views. For the scenario with multiple objects, we may reconstruct each object individually by the proposed method, then register and align them together via the technique of visual metrology so as to obtain the 3D structure of the entire scene. Compared with previous stereovision techniques, the proposed method avoids the bottleneck problem of feature matching and is easy to implement, therefore, more accurate, robust and realistic results are expected. Extensive experiments on synthetic and real world images validate the effectiveness of the proposed method.

Heterogeneous Fusion of Omnidirectional and PTZ Cameras for Multiple Object Tracking

Dual-camera systems have been widely used in surveillance because of the ability to explore the wide field of view (FOV) of the omnidirectional camera and the wide zoom range of the PTZ camera. Most existing algorithms require a priori knowledge of the omnidirectional camera's projection model to solve the nonlinear spatial correspondences between the two cameras. To overcome this limitation, two methods are proposed: 1) geometry and 2) homography calibration, where polynomials with automated model selection are used to approximate the camera's projection model and spatial mapping, respectively. The proposed methods not only improve the mapping accuracy by reducing its dependence on the knowledge of the projection model but also feature reduced computations and improved flexibility in adjusting to varying system configurations. Although the fusion of multiple cameras has attracted increasing attention, most existing algorithms assume comparable FOV and resolution levels among multiple cameras. Different FOV and resolution levels of the omnidirectional and PTZ cameras result in another critical issue in practical tracking applications. The omnidirectional camera is capable of multiple object tracking while the PTZ camera is able to track one individual target at one time to maintain the required resolution. It becomes necessary for the PTZ camera to distribute its observation time among multiple objects and visit them in sequence. Therefore, this paper addresses a novel scheme where an optimal visiting sequence of the PTZ camera is obtained so that in a given period of time the PTZ camera automatically visits multiple detected motions in a target-hopping manner. The effectiveness of the proposed algorithms is illustrated via extensive experiments using both synthetic and real tracking data and comparisons with two reference systems.

Obtaining external parameters of an omnidirectional camera with a fish-eye lens using contour matching

This paper presents a novel omnidirectional camera calibration technique based on automatic contour matching. In this proposed method, the initial parameters, translation and rotation, are computed using the epipolar constraint from matched feature points. By using a normal vector to a 3-D plane connecting the contour with the center of the camera projection model, after contour segmentation, candidate contours are chosen to establish a correspondence between views. In order to find the corresponding end points of the contours among the views, the initial parameters and their active matching windows are utilized. Finally, the camera motions are estimated by minimizing the angular error function between the epipolar plane and the backprojected end-point vectors of the corresponding contours. Experiments on synthetic and real images demonstrate that the proposed algorithm obtains more precise camera parameters than previous methods.

Optimal keyframe selection algorithm for three-dimensional reconstruction in uncalibrated multiple images

A keyframe selection algorithm is the process of selecting the essential image for 3-D reconstruction from among many uncalibrated images. Camera autocalibration between images is also essential for 3-D reconstruction. This paper proposes a keyframe selection algorithm that selects the best image to reduce reprojection error. The camera projection matrix in the selected keyframe arises from a full camera autocalibration process. From the camera projection matrix, which is assumed exact, the algorithm calculates the fundamental matrix using algebraic derivation. By this process, false matching is eliminated and finally a 3-D data set is obtained. In our experimental results, the proposed algorithm needs less time than other algorithms; it also has fewer errors in the reconstructed 3-D data. The fundamental matrix that is gained from algebraic derivation takes less time than other algorithms, and it has same average error as the others.

Recovering epipolar direction from two affine views of a planar object

The mainstream approach to estimate epipolar geometry from two views requires matching the projections of at least four non-coplanar points in the scene, assuming a full projective camera model. Our work deviates from this in three respects: affine camera, planar scene and active contour tracking. A B-spline is fitted to a planar contour, which is tracked using a Kalman filter. The corresponding control points are used to compute the affine transformation between images. We prove that the affine epipolar direction can be computed as one of the eigenvectors of this affine transformation, provided camera motion is free of cyclorotation. A Staübli robot is used to obtain calibrated image streams, which are used as ground truth to evaluate the performance of the method, and to test its limiting conditions in practice. The fact that our method and the gold standard algorithm produce comparable results shows the potential of our proposal.

Accurate calibration for a camera–projector measurement system based on structured light projection

The accurate calibration for a camera–projector measurement system based on structured light projection is important to the system measurement accuracy. This study proposes an improved systematic calibration method focusing on three key factors: calibration model, calibration artifact and calibration procedures. The calibration model better describes the camera and projector imaging process by considering higher to fourth order radial and tangential lens distortion. The calibration artifact provides a sufficient number of accurate 3D reference points uniformly distributed in a common world coordinate system. And the calibration procedures calibrate the camera and projector simultaneously based on the same reference points to eliminate the influences of the camera calibration error on the projector calibration. The experiments demonstrate that our calibration method can improve the measurement accuracy by 47%.

VISION-BASED VELOCITY ESTIMATION FOR UNMANNED GROUND VEHICLES

A novel vision-based velocity estimation technique is presented in this paper. A monocular camera rigidly attached to an unmanned ground vehicle (UGV) is used to capture image sequences of the terrain surface and compute the image velocities using an optical flow method. Combining with the proposed camera model, the velocity of the UGV can be directly estimated. This velocity estimation method is validated over coarse sand, fine sand and mixture of coarse sand and gravel separately. Estimated velocities are compared to measured velocities from highly accurate optical encoders, showing the maximum error is less than 1.5%. The effect of feature window size and the distance between the camera projection center and the terrain surface on the velocity estimation is investigated. Random white noise is added to test the robustness of the algorithm and the results are encouraging. The proposed velocity estimation method has many promising potential applications.

Image enhancement using second generation wavelet super resolution

Second generation wavelet based super resolution algorithms developed in this research are presented in this paper. Generalization of the Second generation wavelet super resolution (SGWSR) algorithm to handle arbitrarily irregular 2-D grids as opposed to the semi-regular grids handled in the previous algorithm is introduced. A consequence is the ability to incorporate the projective camera motion model into the framework. Simultaneous noise filtering with super resolution is achieves. The generalization to the arbitrary 2-D version is subsequently described. Simulation results which demonstrate the improved performance of the developed super resolution algorithms in comparison to other approaches are also presented. Key words: Super resolution, second generation wavelets, blind deconvolution, optimal thresholding, blur support determination

Character animation from 2D pictures and 3D motion data

This article presents a new method to animate photos of 2D characters using 3D motion capture data. Given a single image of a person or essentially human-like subject, our method transfers the motion of a 3D skeleton onto the subject's 2D shape in image space, generating the impression of a realistic movement. We present robust solutions to reconstruct a projective camera model and a 3D model pose which matches best to the given 2D image. Depending on the reconstructed view, a 2D shape template is selected which enables the proper handling of occlusions. After fitting the template to the character in the input image, it is deformed as-rigid-as-possible by taking the projected 3D motion data into account. Unlike previous work, our method thereby correctly handles projective shape distortion. It works for images from arbitrary views and requires only a small amount of user interaction. We present animations of a diverse set of human (and nonhuman) characters with different types of motions, such as walking, jumping, or dancing.

Projective Visual Hulls

This article presents a novel method for computing the visual hull of a solid bounded by a smooth surface and observed by a finite set of cameras. The visual hull is the intersection of the visual cones formed by back-projecting the silhouettes found in the corresponding images. We characterize its surface as a generalized polyhedron whose faces are visual cone patches; edges are intersection curves between two viewing cones; and vertices are frontier points where the intersection of two cones is singular, or intersection points where triples of cones meet. We use the mathematical framework of oriented projective differential geometry to develop an image-based algorithm for computing the visual hull. This algorithm works in a weakly calibrated setting----that is, it only requires projective camera matrices or, equivalently, fundamental matrices for each pair of cameras. The promise of the proposed algorithm is demonstrated with experiments on several challenging data sets and a comparison to another state-of-the-art method.

Characterization of septal penetration in 511 keV SPECT

Single photon emission computed tomography (SPECT) with 511 keV photons is a challenging modality and collimators for this purpose require trade-offs among resolution, sensitivity and septal penetration. While PET is the modality of choice for imaging at 511 keV, there are some procedures, e.g., dual-isotope imaging, in which 511 keV SPECT has a role. To measure the imaging properties of a VPC-93 SPECT collimator designed for imaging at 511 keV and to isolate the effects of septal penetration. NaI gamma camera projection images of (18)F (511 keV) and (99m)Tc (140 keV) point sources were measured and the corresponding modulation transfer functions calculated. The projection images were reconstructed via filtered back-projection to obtain the tomographic three-dimensional (3-D) point spread function. Differences between the 511 and 140 keV results were attributed mainly to septal penetration. Contrast measurements were made separately using (18)F and (99m)Tc of a 20 cm phantom containing hot spheres and a warm background. Both isotopes were also used in imaging studies of a 3-D Hoffman brain phantom. Reconstructed 511 keV point source images were spatially extended with more than half of the total reconstructed counts appearing away from the point source region. The number of false counts contained in the image as a function of distance from the true source location remains approximately constant for large distances out to at least 14 cm. Septal penetration results in a rapid roll-off with spatial frequency of collimator response. The response of the collimator to 511 keV photons falls to half of its 0-frequency response at 0.03 cm(-1). For 140 keV photons this value is 0.20 cm(-1). A result is reduced image contrast as measured in the phantom sphere studies. Septal penetration causes image degradation through large-scale blurring. Image noise characteristics are modified and correlations are extended into many transaxial planes. Both 2-D and 3-D point spread functions for 511 and 140 keV photons using the VPC-93 collimator have been measured. Septal penetration unfavourably affects image resolution and changes image noise characteristics. Without compensation, the effects of septal penetration are readily apparent in images of real objects.

EM enhancement of 3D head pose estimated by point at infinity

Head pose estimation is a key task for visual surveillance, HCI and face recognition applications. In this paper, a new approach is proposed for estimating 3D head pose from a monocular image. The approach assumes the full perspective projection camera model. Our approach employs general prior knowledge of face structure and the corresponding geometrical constraints provided by the location of a certain vanishing point to determine the pose of human faces. To achieve this, eye-lines, formed from the far and near eye corners, and mouth-line of the mouth corners are assumed parallel in 3D space. Then the vanishing point of these parallel lines found by the intersection of the eye-line and mouth-line in the image can be used to infer the 3D orientation and location of the human face. In order to deal with the variance of the facial model parameters, e.g. ratio between the eye-line and the mouth-line, an EM framework is applied to update the parameters. We first compute the 3D pose using some initially learnt parameters (such as ratio and length) and then adapt the parameters statistically for individual persons and their facial expressions by minimizing the residual errors between the projection of the model features points and the actual features on the image. In doing so, we assume every facial feature point can be associated to each of features points in 3D model with some a posteriori probability. The expectation step of the EM algorithm provides an iterative framework for computing the a posterori probabilities using Gaussian mixtures defined over the parameters. The robustness analysis of the algorithm on synthetic data and some real images with known ground-truth are included.

High-end aerial digital cameras and their impact on the automation and quality of the production workflow

The IGN digital camera project was established in the early 1990s. The first research surveys were carried out in 1996 and the digital camera was first used in production in 2000. In 2004 approximately 10 French departments (accounting for 10% of the territory) were covered with a four-head camera system and since summer 2005 all IGN imagery has been acquired digitally. Nevertheless the camera system is still evolving, with tests on new geometric configurations being continuously undertaken. The progressive integration of the system in IGN production workflows has allowed IGN to keep the system evolving in accordance with production needs. Remaining problems are due to specific camera characteristics such as CCD format, the optical quality of off-the-shelf lenses, and because some production tools are ill-adapted to digital images with a large dynamic range. However, when considering the pros and cons of integrating these images into production lines, the disadvantages are largely balanced by the numerous benefits this technology offers.

Superimposition and projective transformation of 3D object

Superimposition is an efficient method for evaluation of coincidence between a skull and a photo portrait. The principle of superimposition method lies in the projection of the skull into the face image. During the projection of an object with a perspective camera, the mapping of a three-dimensional object into a two-dimensional image takes place. The acquired images of the same object are more or less distorted due to various photographic conditions, due to extrinsic and intrinsic parameters of the camera. The distortions have important influence onto reliability of human identification by the superimposition method. Mathematically we can describe most of the distortions. On the basis of the description the divergences could be simulated and in some cases eliminated by geometric transformation of the compared images. We are presenting a mathematical model of the standard projective camera and the mathematical description of distortions which are important for the superimposition process. The results show the distortions and the elimination of the distortions by means of the projection model.

Fast 3D profilometer based upon the projection of a single fringe pattern and absolute calibration

We present a whole-field, 3D optical digitizer based on phase profilometry and on absolute calibration, which performs shape measurement through the projection of a single pattern of Ronchi fringes, and an absolute calibration of the optical head. A suitably developed procedure, based on accurate fringe analysis, implements phase extraction and allows univocal indexing of the field of view. Accurate models express, in a global reference system, both extrinsic and intrinsic parameters of the camera–projector pair. Proper calibration is performed for lens distortion compensation, and to obtain the absolute measurement of the shapes within a global reference system. The technique requires the acquisition of a single image, shows good robustness against fine variations of the fringe period and adapts well to the measurement of free-form shapes in a very short time. The system is portable, fully reconfigurable depending on the measurement requirements and suitable for a low-cost implementation. In this paper, we present the phase extraction method and the system calibration, and highlight the measurement performance of the digitizer in different set-ups, using free-form objects requiring multi-view acquisition.

Experimental study on the sensitivity of autocalibration to projective camera model parameters

Existing methods of 3-D object modeling and recovering 3-D data from uncalibrated 2-D images are subject to errors introduced by assumptions about camera parameters and mismatches in finding point pairs in the images. In this study, we experimentally evaluate the effect of each of these assumptions together with the inaccuracy in the measurements in the images. Sensitivity of reconstruction errors to inaccuracies in the estimation of camera parameters and mismatches due to noise in input data is measured using a linear and two nonlinear autocalibration methods for a projective camera. Our experimental results show that some assumptions such as a vanishing skew can be safely made; however, other parameters such as principal point location are quite sensitive to wrong assumptions.