Using vanishing points for camera calibration

In this paper, we show how to calibrate a camera and to recover the geometry and the photometry (textures) of objects from a single image. The aim of this work is to make it possible walkthrough and augment reality in a 3D model reconstructed from a single image. The calibration step does not need any calibration target and makes only four assumptions: (1) the single image contains at least two vanishing points, (2) the length (in 3D space) of one line segment (for determining the translation vector) in the image is known, (3) the principle point is the center of the image, and (4) the aspect ratio is fixed by the user. Each vanishing point is determined from a set of parallel lines. These vanishing points help determine a 3D world coordinate system R o. After having computed the focal length, the rotation matrix and the translation vector are evaluated in turn for describing the rigid motion between R o and the camera coordinate system R c. Next, the reconstruction step consists in placing, rotating, scaling, and translating a rectangular 3D box that must fit at best with the potential objects within the scene as seen through the single image. With each face of a rectangular box, a texture that may contain holes due to invisible parts of certain objects is assigned. We show how the textures are extracted and how these holes are located and filled. Our method has been applied to various real images (pictures scanned from books, photographs) and synthetic images.

For images taken in man-made scenes, vanishing points and focal length of camera play important roles in scene understanding. In this paper, we present a novel method to quickly, accurately and simultaneously estimate three orthogonal vanishing points (TOVPs) and focal length from single images. Our method is based on the following important observations: If we establish a polar coordinate system on the image plane whose origin is at the image center, angle coordinates of vanishing points can be robustly estimated by seeking peaks in a histogram. From the detected angle coordinates, altitudes of a triangle formed by TOVPs are determined. Novel constraints on both vanishing points and focal length could be obtained from the three altitudes. By using the constraints, radial coordinates of TOVPs and focal length can be estimated simultaneously. Our method decomposes a 2D Hough parameter space into two cascaded 1D Hough parameter spaces, which makes our method much faster and more robust than previous methods without losing accuracy. Enormous experiments on real images have been done to test feasibility and correctness of our method.

In this work, we propose a novel method for automatic camera calibration, mainly for surveillance cameras. The calibration consists in observing objects on the ground plane of the scene; in our experiments, vehicles were used. However, any arbitrary rigid objects can be used instead, as verified by experiments with synthetic data. The calibration process uses convolutional neural network localisation of landmarks on the observed objects in the scene and the corresponding 3D positions of the localised landmarks � thus fine-grained classification of the detected vehicles in the image plane is done. The observation of the objects (detection, classification and landmark detection) enables to determine all typically used camera calibration parameters (focal length, rotation matrix, and translation vector). The experiments with real data show slightly better results in comparison with state-of-the-art work, however with an extreme speed-up. The calibration error decreased from 3.01% to 2.72% and 1223 � faster computation was achieved.

Camera and laser rangefinder (LRF) are widely used in various mobilized systems, such as intelligent vehicle, autonomous robot, etc. And extrinsic calibration is essential and basically the first step to integrate image and 3D LIDAR data. This paper presents a flexible method for extrinsic calibration of a pin-hole camera and a 3D Laser Rangefinder (LRF). The method extends the chessboard pattern, which is originally used for camera calibration, for camera and 2D LRF calibration. It requires at least 3 input planes to determine the rotation and translation between these two sensors. The proposed method formulates the extrinsic calibration problem as to register point correspondences in the dual 3D space instead of plane registration in 3D space. And the rotation matrix and the translation vector are estimated separately. The extrinsic calibration results allow the registration of image and 3D LIDAR data by mapping all the LIDAR data onto the imaging plane. The proposed method has been tested with both simulation and real data. The experimental results show that the proposed algorithm is both accurate and practical.

Interests of auto-calibration have been increased in several camera systems. This paper presents a novel self-calibration method using fast and accurate vanishing point detection algorithm that works in manmade environments. The proposed algorithm estimates focal length assuming that the principal point is the center of an image to satisfy the orthogonality of three vanishing points. By using proposed vanishing point detection algorithm and minimization of the proposed objective function, the proposed system detects accurate vanishing points with focal length outperforming other methods. The proposed vanishing point detection algorithm detects vanishing points by using J-linkage based method that is more delicate by fragmentation and re-merging strategies. The proposed objective function finally detects vanishing points that meets orthogonality among estimated hypotheses for vanishing points by checking several geometric relationships. We believe that the proposed method can be used for automatic camera calibration, localization of a camera in an autonomous navigation system, and three-dimensional reconstruction of a single-view image.

3-D camera calibration using vanishing point concept

The line structure light three-dimensional reconstruction system is a kind of three-dimensional non-contact measurement system, which has the advantages of high precision, high speed, small damage to objects and strong adaptability. Camera calibration is a major factor that constrains the accuracy of 3D measurement systems. The camera calibration is based on the pinhole imaging model, and through a series of complex calculations, the camera’s internal parameters (focal length, distortion coefficient) and external parameters (rotation matrix and translation vector). The different calibration methods use different calibration targets, which can be divided into 3D calibration targets, 2D calibration targets, and one-dimensional calibration targets according to the characteristics of the calibration targets. This paper mainly discusses: calibration content and significance, calibration methods for different targets and evaluation methods for calibration of different targets. Firstly, the content and significance of calibration are expounded. Then, according to different calibration targets, the calibration algorithm is analyzed. Finally, the calibration algorithm is analyzed and summarized, and the development trends, advantages and disadvantages of different calibration methods are pointed out.

This paper proposes a calibration system consisting of three components: a quasi-linear intrinsic calibrator, a linear extrinsic calibrator and a nonlinear L-M optimizer. The focal length is evaluated form vanishing points. Then the rotation matrix and the translation vector are estimated linearly. At last a Levenberg-Marquardt optimization is performed to refine the extrinsic parameters by minimizing the reprojection error. The parameterization of the rotation matrix is discussed in detail, and two parameterization methods, Euler-Angle and Axis-Angle are compared. Experimental results prove that the system can calibrate the cameras precisely.

A widely used approach for estimating camera orientation is to use the points at infinity, i.e., the vanishing points (VPs). Enforcement of the orthogonal constraint between the VPs, known as the Manhattan world constraint, enables an estimation of the drift-free camera orientation to be achieved. However, in practical applications, this approach is neither effective (because of noisy parallel line segments) nor performable in non-Manhattan world scenes. To overcome these limitations, we propose a novel method that jointly estimates the VPs and camera orientation based on sequential Bayesian filtering. The proposed method does not require the Manhattan world assumption, and can perform a highly accurate estimation of camera orientation. In order to enhance the robustness of the joint estimation, we propose a keyframe-based feature management technique that removes false positives from parallel line clusters and detects new parallel line sets using geometric properties such as the orthogonality and rotational dependence for a VP, a line, and the camera rotation. In addition, we propose a 3-line camera rotation estimation method that does not require the Manhattan world assumption. The 3-line method is applied to the RANSAC-based outlier rejection technique to eliminate outlier measurements; therefore, the proposed method achieves accurate and robust estimation of the camera orientation and VPs in general scenes with non-orthogonal parallel lines. We demonstrate the superiority of the proposed method by conducting an extensive evaluation using synthetic and real datasets and by comparison with other state-of-the-art methods.

An efficient detection of vanishing points using inverted coordinates image space

Within the computer vision field, estimating image vanishing points has many applications regarding robotic navigation, camera calibration, image understanding, visual measurement, 3D reconstruction, among others. Different methods for detecting vanishing points relies on accumulator space techniques, while others employ a heuristic approach such as RANSAC. Nevertheless, these types of methods suffer from low accuracy or high computational cost. To explore a different technique, this paper focuses on improving the efficiency of the metaheuristic search for vanishing points by using a recently proposed population‐based method: The Teaching Learning Based Optimisation algorithm (TLBO). The TLBO algorithm is a metaheuristic technique inspired by the teaching–learning process. In our method, the TLBO algorithm is used after a line segment detection, to cluster line segments according to their more optimal vanishing point. Thus, our algorithm detects both orthogonal and nonorthogonal vanishing points in real images. To corroborate the performance of our proposed algorithm, different comparison and tests with other approaches were carried out. The results validate the accuracy and efficiency of our proposed method. Our approach had an average computational time of1.42 seconds and obtained a cumulative focal length error of 1 pixel, and cumulative angular error of 0.1°.

The problem of estimating the motion and orientation parameters of a rigid object from two m - D point set patterns is of significant importance in medical imaging, electrocardiogram (ECG) alignment, and fingerprint matching. The rigid parameters can be defined by an m times m rotation matrix, a diagonal m times m scale matrix, and an m times 1 translation vector. All together, the total number of parameters to be found is m(m + 2). Several least squares based algorithms have recently appeared in the literature. These algorithms are all based on a singular value decomposition (SVD) of the m times m cross-covariance matrix between the two data sets. However, there are cases where the SVD based algorithms return a reflection matrix rather than a rotation matrix. Some authors have introduced a simple correction for guarding against such cases. Other types of algorithm are based on unit quaternions which guarantee obtaining a true rotation matrix. In this paper we introduce a principal component based registration algorithm which is solved in closed-form. By using matrix vectorization properties the problem can be cast as one of finding a rank-1 symmetric projection matrix. This is equivalent to solving a Sylvester equation with equality constraints. Once the solution is obtained, we apply the inverse vectorization operation to estimate the rotation and scale matrices, along with the translation vector. We apply the proposed algorithm to the alignment of ECG signals and compare the results to those obtained by the SVD and quaternion based algorithms

A completely linear approach for binocular CCD ranging system calibration is presented in this paper, and it improves the Tsai's calibration algorithm. First, we are solving the other internal and external camera parameters based on Radial-Array-Constraints and first-order radial distortion model, which include focal length, first-order radial distortion parameter, rotation matrix and translation vector. Next, the calibration of binocular CCD ranging system includes baseline of the high precision binocular ranging system, relative position of two image plane and optical axis angle of two cameras. Finally, basing on the results of calibration, we are amending the binocular ranging formula. Experiments show that the precision is higher for most of the camera calibration, the method has wide applicative value. The application of this method, the actual measurement accuracy of 3km target is less than 0.5%, it can satisfy the requirements of high precision of binocular CCD ranging system.

A technique of estimating fisheye camera parameters, which uses three orthogonal vanishing points from one single image of 3D calibration pattern, is presented in this paper. For an image of 3D calibration pattern, three orthogonal vanishing points are firstly calculated using the image of the scene including three groups of parallel lines, whose directions are different. And then the basic intrinsic parameters of fisheye camera are estimated from three vanishing points. Thus, the distorted model parameters of camera can be refined from the objective function. Finally, rotation matrix between fisheye camera and 3D calibration pattern can be computed from three orthogonal directions on the 3D pattern and their three vanishing points. Experimental results show that this method is simple and feasible.

The Convolutional Neural Networks (CNNs) have made monocular image processing a powerful obstacle detector, but in order to transform these results into 3D data robust automatic calibration is needed. This paper proposes an unassisted camera calibration algorithm, based on analyzing image sequences acquired from naturalistic driving. The focal distance is computed based on the mean lateral displacement of similar features in consecutive frames, compared with the yaw rate of the vehicle. The height and pitch angle are computed based on the distribution of the lane width values on the image lines, assuming an average lane width is known. The lane markings are detected using the edges on the road, already segmented using the CNN. The yaw angle is computed using the vanishing point (VP) detection, which is performed using the direction of the road gradients. The pitch angle value is dynamically corrected using the VP, and using comparisons between the past frame and the current frame, under multiple correction hypotheses.