Camera Intrinsic Parameters Research Articles

STRUCTURELESS BUNDLE ADJUSTMENT WITH SELF-CALIBRATION USING ACCUMULATED CONSTRAINTS

Abstract. Bundle adjustment based on collinearity is the most widely used optimization method within image based scene reconstruction. It incorporates observed image coordinates, exterior and intrinsic camera parameters as well as object space coordinates of the observed points. The latter dominate the resulting nonlinear system, in terms of the number of unknowns which need to be estimated. In order to reduce the size of the problem regarding memory footprint and computational effort, several approaches have been developed to make the process more efficient, e.g. by exploitation of sparsity or hierarchical subdivision. Some recent developments express the bundle problem through epipolar geometry and scale consistency constraints which are free of object space coordinates. These approaches are usually referred to as structureless bundle adjustment. The number of unknowns in the resulting system is drastically reduced. However, most work in this field is focused on optimization towards speed and considers calibrated cameras, only. We present our work on structureless bundle adjustment, focusing on precision issues as camera calibration and residual weighting. We further investigate accumulation of constraint residuals as an approach to decrease the number of rows of the Jacobian matrix.

STRUCTURELESS BUNDLE ADJUSTMENT WITH SELF-CALIBRATION USING ACCUMULATED CONSTRAINTS

Bundle adjustment based on collinearity is the most widely used optimization method within image based scene reconstruction. It incorporates observed image coordinates, exterior and intrinsic camera parameters as well as object space coordinates of the observed points. The latter dominate the resulting nonlinear system, in terms of the number of unknowns which need to be estimated. In order to reduce the size of the problem regarding memory footprint and computational effort, several approaches have been developed to make the process more efficient, e.g. by exploitation of sparsity or hierarchical subdivision. Some recent developments express the bundle problem through epipolar geometry and scale consistency constraints which are free of object space coordinates. These approaches are usually referred to as structureless bundle adjustment. The number of unknowns in the resulting system is drastically reduced. However, most work in this field is focused on optimization towards speed and considers calibrated cameras, only. We present our work on structureless bundle adjustment, focusing on precision issues as camera calibration and residual weighting. We further investigate accumulation of constraint residuals as an approach to decrease the number of rows of the Jacobian matrix.

On-demand calibration and evaluation for electromagnetically tracked laparoscope in augmented reality visualization.

Common camera calibration methods employed in current laparoscopic augmented reality systems require the acquisition of multiple images of an entire checkerboard pattern from various poses. This lengthy procedure prevents performing laparoscope calibration in the operating room (OR). The purpose of this work was to develop a fast calibration method for electromagnetically (EM) tracked laparoscopes, such that the calibration can be performed in the OR on demand. We designed a mechanical tracking mount to uniquely and snugly position an EM sensor to an appropriate location on a conventional laparoscope. A tool named fCalib was developed to calibrate intrinsic camera parameters, distortion coefficients, and extrinsic parameters (transformation between the scope lens coordinate system and the EM sensor coordinate system) using a single image that shows an arbitrary portion of a special target pattern. For quick evaluation of calibration results in the OR, we integrated a tube phantom with fCalib prototype and overlaid a virtual representation of the tube on the live video scene. We compared spatial target registration error between the common OpenCV method and the fCalib method in a laboratory setting. In addition, we compared the calibration re-projection error between the EM tracking-based fCalib and the optical tracking-based fCalib in a clinical setting. Our results suggest that the proposed method is comparable to the OpenCV method. However, changing the environment, e.g., inserting or removing surgical tools, might affect re-projection accuracy for the EM tracking-based approach. Computational time of the fCalib method averaged 14.0s (range 3.5s-22.7s). We developed and validated a prototype for fast calibration and evaluation of EM tracked conventional (forward viewing) laparoscopes. The calibration method achieved acceptable accuracy and was relatively fast and easy to be performed in the OR on demand.

A flexible technique based on fundamental matrix for camera self-calibration with variable intrinsic parameters from two views

We propose a new self-calibration technique for cameras with varying intrinsic parameters that can be computed using only information contained in the images themselves. The method does not need any a priori knowledge on the orientations of the camera and is based on the use of a 3D scene containing an unknown isosceles right triangle. The importance of our approach resides at minimizing constraints on the self-calibration system and the use of only two images to estimate these parameters. This method is based on the formulation of a nonlinear cost function from the relationship between two matches which are the projection of two points representing vertices of an isosceles right triangle, and the relationship between the images of the absolute conic. The resolution of this function enables us to estimate the cameras intrinsic parameters. The algorithm is implemented and validated on several sets of synthetic and real image data.

A Linear Approach for Depth and Colour Camera Calibration Using Hybrid Parameters

Many recent applications of computer graphics and human computer interaction have adopted both colour cameras and depth cameras as input devices. Therefore, an effective calibration of both types of hardware taking different colour and depth inputs is required. Our approach removes the numerical difficulties of using non-linear optimization in previous methods which explicitly resolve camera intrinsics as well as the transformation between depth and colour cameras. A matrix of hybrid parameters is introduced to linearize our optimization. The hybrid parameters offer a transformation from a depth parametric space (depth camera image) to a colour parametric space (colour camera image) by combining the intrinsic parameters of depth camera and a rotation transformation from depth camera to colour camera. Both the rotation transformation and intrinsic parameters can be explicitly calculated from our hybrid parameters with the help of a standard QR factorisation. We test our algorithm with both synthesized data and real-world data where ground-truth depth information is captured by Microsoft Kinect. The experiments show that our approach can provide comparable accuracy of calibration with the state-of-the-art algorithms while taking much less computation time (1/50 of Herrera’s method and 1/10 of Raposo’s method) due to the advantage of using hybrid parameters.

Reconstruction of 3D Scenes by Camera Self-Calibration and Using Genetic Algorithms

In this paper, we address a problem of reconstruction of three-dimensional scenes from images taken by cameras, with varying parameters, from different views. This method is based on the projection of 3D points in the image planes. The relationships between the matches and the camera parameters are used to formulate a nonlinear equation system. This system is transformed into an objective function, which is minimized by a genetic algorithm to estimate the intrinsic and extrinsic camera parameters. Finally, the coordinates of 3D points of the scene are obtained by solving a linear equation system. The experiments on synthetic and real data show the quality of this work and the good obtained results.

Cross-ratio–based line scan camera calibration using a planar pattern

A flexible new technique is proposed to calibrate the geometric model of line scan cameras. In this technique, the line scan camera is rigidly coupled to a calibrated frame camera to establish a pair of stereo cameras. The linear displacements and rotation angles between the two cameras are fixed but unknown. This technique only requires the pair of stereo cameras to observe a specially designed planar pattern shown at a few (at least two) different orientations. At each orientation, a stereo pair is obtained including a linear array image and a frame image. Radial distortion of the line scan camera is modeled. The calibration scheme includes two stages. First, point correspondences are established from the pattern geometry and the projective invariance of cross-ratio. Second, with a two-step calibration procedure, the intrinsic parameters of the line scan camera are recovered from several stereo pairs together with the rigid transform parameters between the pair of stereo cameras. Both computer simulation and real data experiments are conducted to test the precision and robustness of the calibration algorithm, and very good calibration results have been obtained. Compared with classical techniques which use three-dimensional calibration objects or controllable moving platforms, our technique is affordable and flexible in close-range photogrammetric applications.

A Decoupled Calibration Method for Camera Intrinsic Parameters and Distortion Coefficients

Camera calibration is a necessary process in the field of vision measurement. In this paper, we propose a flexible and high-accuracy method to calibrate a camera. Firstly, we compute the center of radial distortion, which is important to obtain optimal results. Then, based on the radial distortion of the division model, the camera intrinsic parameters and distortion coefficients are solved in a linear way independently. Finally, the intrinsic parameters of the camera are optimized via the Levenberg-Marquardt algorithm. In the proposed method, the distortion coefficients and intrinsic parameters are successfully decoupled; calibration accuracy is further improved through the subsequent optimization process. Moreover, whether it is for relatively small image distortion or distortion larger image, utilizing our method can get a good result. Both simulation and real data experiment demonstrate the robustness and accuracy of the proposed method. Experimental results show that the proposed method can be obtaining a higher accuracy than the classical methods.

カメラを回転させた時の特徴点軌跡を用いた魚眼カメラの内部パラメータ推定手法の実機実験による検証

カメラを回転させた時の特徴点軌跡を用いた魚眼カメラの内部パラメータ推定手法の実機実験による検証

A Bridge Deflection Monitoring System Based on CCD

For long-term monitoring of the midspan deflection of Songjiazhuang cloverleaf junction on 309 national roads in Zibo city, this paper proposes Zhang’s calibration-based DIC deflection monitoring method. CCD cameras are used to track the change of targets’ position, Zhang’s calibration algorithm is introduced to acquire the intrinsic and extrinsic parameters of CCD cameras, and the DIC method is combined with Zhang’s calibration algorithm to measure bridge deflection. The comparative test between Zhang’s calibration and scale calibration is conducted in lab, and experimental results indicate that the proposed method has higher precision. According to the deflection monitoring scheme, the deflection monitoring software for Songjiazhuang cloverleaf junction is developed by MATLAB, and a 4-channel CCD deflection monitoring system for Songjiazhuang cloverleaf junction is integrated in this paper. This deflection monitoring system includes functions such as image preview, simultaneous collection, camera calibration, deflection display, and data storage. In situ deflection curves show a consistent trend; this suggests that the proposed method is reliable and is suitable for the long-term monitoring of bridge deflection.

Camera Self-Calibration with Varying Intrinsic Parameters by an Unknown Three-Dimensional Scene

In the present paper, we will propose a new and robust method of camera self-calibration having varying intrinsic parameters from a sequence of images of an unknown 3D object. The projection of two points of the 3D scene in the image planes is used to determine the projection matrices. The present method is based on the formulation of a non linear cost function from the determination of a relationship between two points of the scene with their opposite relative to the axis of abscise and their projections in the image planes. The resolution of this function with genetic algorithm enables us to estimate the intrinsic parameters of different cameras. The important of our approach reside in the use of a single pair of images which provides fewer equations, simplifies the mathematical complexities and minimizes the execution time of the application, the use of the data of the first image only without the use of the data of the second image, the use of any camera which makes the intrinsic parameters variable not constant and the use of a 3D scene reduces the planarity constraints. The experimental results on synthetic and real data prove the performance and robustness of our approach.

Effect of camera temperature variations on stereo-digital image correlation measurements.

In laboratory and especially non-laboratory stereo-digital image correlation (stereo-DIC) applications, the extrinsic and intrinsic parameters of the cameras used in the system may change slightly due to the camera warm-up effect and possible variations in ambient temperature. Because these camera parameters are generally calibrated once prior to measurements and considered to be unaltered during the whole measurement period, the changes in these parameters unavoidably induce displacement/strain errors. In this study, the effect of temperature variations on stereo-DIC measurements is investigated experimentally. To quantify the errors associated with camera or ambient temperature changes, surface displacements and strains of a stationary optical quartz glass plate with near-zero thermal expansion were continuously measured using a regular stereo-DIC system. The results confirm that (1) temperature variations in the cameras and ambient environment have a considerable influence on the displacements and strains measured by stereo-DIC due to the slightly altered extrinsic and intrinsic camera parameters; and (2) the corresponding displacement and strain errors correlate with temperature changes. For the specific stereo-DIC configuration used in this work, the temperature-induced strain errors were estimated to be approximately 30-50 με/°C. To minimize the adverse effect of camera temperature variations on stereo-DIC measurements, two simple but effective solutions are suggested.

Method to solve intrinsic geometric parameters of a paracatadioptric camera using 3D lines

The paracatadioptric projection of a 3D line is a conic. In this paper, the constraint relation of three conics formed by three 3D lines according to the paracatadioptric projection is divided into three situations. In the first, three groups of vanishing points in orthogonal directions are formed by the principle point coordinates and six intersection points of the three conics. In the second, six groups of vanishing points in orthogonal directions are determined by common diameters on the unit sphere and tangents at intersection points of the three conics. In the third, three groups of circular points are formed by each side of a complete quadrangle formed by two circular points and the intersection points of the three conics. These three methods can solve the camera intrinsic parameters effectively and accurately, as verified by simulation and real data applications.

Adaptive Image-Based Trajectory Tracking Control of Wheeled Mobile Robots With an Uncalibrated Fixed Camera

In this paper, the uncalibrated image-based trajectory tracking control problem of wheeled mobile robots will be studied. The motion of the wheeled mobile robot can be observed using an uncalibrated fixed camera on the ceiling. Different from traditional vision-based control strategies of wheeled mobile robots in the fixed camera configuration, the camera image plane is not required to be parallel to the motion plane of the wheeled mobile robots and the camera can be placed at a general position. To guarantee that the wheeled mobile robot can efficiently track its desired trajectory, which is specified by the desired image trajectory of a feature point at the forward axis of the wheeled mobile robot, we will propose a new adaptive image-based trajectory tracking control approach without the exact knowledge of the camera intrinsic and extrinsic parameters and the position parameter of the feature point. To eliminate the nonlinear dependence on the unknown parameters from the closed-loop system, a depth-independent image Jacobian matrix framework for the wheeled mobile robots will be developed such that unknown parameters in the closed-loop system can be linearly parameterized. In this way, adaptive laws can be designed to estimate the unknown parameters online, and the depth information of the feature point can be allowed to be time varying in this case. The Lyapunov stability analysis will also be performed to show asymptotical convergence of image position and velocity tracking errors of the wheeled mobile robot. The simulation results based on a two-wheeled mobile robot will be given in this paper to illustrate the performance of the proposed approach as well. The experimental results based on a real wheeled mobile robot will also be provided to validate the proposed approach.

Intrinsic parameter determination of a paracatadioptric camera by the intersection of two sphere projections.

In this paper, a linear calibration method is proposed for a paracatadioptric camera using the images of two spheres. Two spheres are selected in space, and the two groups of their projection circles on the unit viewing sphere are made to intersect at four points. The quadrilateral consisting of four points is a rectangle, so a group of orthogonal directions can be determined in space to obtain a group of orthogonal vanishing points in the paracatadioptric image plane. Because of the relationship between orthogonal vanishing points and intrinsic camera parameters, the intrinsic parameters of a paracatadioptric camera can be linearly solved by at least five views satisfying the above conditions. First, one estimates the sphere images and their antipodal sphere images. Second, by solving the intersection of the images of two spheres and the intersection of the images of their antipodal spheres, a group of orthogonal vanishing points can be obtained in the image plane. Finally, by taking the relationship between the orthogonal vanishing points and the intrinsic camera parameters as constraint conditions, the intrinsic parameters of the paracatadioptric camera can be obtained. Simulation results and real image data demonstrate the effectiveness of our new algorithms.

Non-iterative method for camera calibration.

This paper presents a new and effective technique to calibrate a camera without nonlinear iteration optimization. To this end, the centre-of-distortion is accurately estimated firstly. Based on the radial distortion division model, point correspondences between model plane and its image were used to compute the homography and distortion coefficients afterwards. Once the homographies of calibration images are obtained, the camera intrinsic parameters are solved analytically. All the solution techniques applied in this calibration process are non-iterative that do not need any initial guess, with no risk of local minima. Moreover, estimation of the distortion coefficients and intrinsic parameters could be successfully decoupled, yielding the more stable and reliable result. Both simulative and real experiments have been carried out to show that the proposed method is reliable and effective. Without nonlinear iteration optimization, the proposed method is computationally efficient and can be applied to real-time online calibration.

A novel camera calibration technique based on differential evolution particle swarm optimization algorithm

Camera calibration is one of the fundamental issues in computer vision and aims at determining the intrinsic and exterior camera parameters by using image features and the corresponding 3D features. This paper proposes a relationship model for camera calibration in which the geometric parameter and the lens distortion effect of camera are taken into account in order to unify the world coordinate system (WCS), the camera coordinate system (CCS) and the image coordinate system (ICS). Differential evolution is combined with particle swarm optimization algorithm to calibrate the camera parameters effectively. Experimental results show that the proposed algorithm has a good optimization ability to avoid local optimum and can complete the visual identification tasks accurately.

A SEMI-AUTOMATIC PROCEDURE FOR TEXTURING OF LASER SCANNING POINT CLOUDS WITH GOOGLE STREETVIEW IMAGES

Abstract. We introduce a method to texture 3D urban models with photographs that even works for Google Streetview images and can be done with currently available free software. This allows realistic texturing, even when it is not possible or cost-effective to (re)visit a scanned site to take textured scans or photographs. Mapping a photograph onto a 3D model requires knowledge of the intrinsic and extrinsic camera parameters. The common way to obtain intrinsic parameters of a camera is by taking several photographs of a calibration object with a priori known structure. The extra challenge of using images from a database such as Google Streetview, rather than your own photographs, is that it does not allow for any controlled calibration. To overcome this limitation, we propose to calibrate the panoramic viewer of Google Streetview using Structure from Motion (SfM) on any structure of which Google Streetview offers views from multiple angles. After this, the extrinsic parameters for any other view can be calculated from 3 or more tie points between the image from Google Streetview and a 3D model of the scene. These point correspondences can either be obtained automatically or selected by manual annotation. We demonstrate how this procedure provides realistic 3D urban models in an easy and effective way, by using it to texture a publicly available point cloud from a terrestrial laser scan made in Bremen, Germany, with a screenshot from Google Streetview, after estimating the focal length from views from Paris, France.

Accurate Self-calibration of Camera with Variable Intrinsic Parameters from Unknown 3D Scene

In this article we present a practical self-calibration method of camera having variable intrinsic parameters. This method is an extension of a method which we have already published. The main idea of our method addressed in this paper is the use of interest points automatically detected in two images of an unknown 3D scene to self-calibrate the camera. These interest points are the projections of the vertices of unknown 3D parallelograms (each triplet of interest points is the projections of three vertices of a unique 3D parallelogram). The projection of the vertices of parallelograms in the two image planes of the camera allows us to obtain two equations based on the intrinsic camera parameters. Seeing that our camera can change its intrinsic parameters from a view to another so the number of unknowns is ten hence the need to use at least five parallelograms in the scene to obtain other equations in function of intrinsic camera parameters. The paper tests the performance of the proposed approach on the both synthetic and real data.

Catadioptric 전방향 스테레오 영상시스템 및 3차원 좌표 복원

An image acquisition by using an optical mirror is called as a catadioptric method. The catadioptric imaging method is generally used for acquisition of 360-degree all directional visual information in an image. An exemplar omnidirectional optical mirror is the bowl-shaped hyperbolic mirror. In this paper, a single camera omnidirectional stereo imaging method is studied with an additional concave lens. It is possible to obtain 3 dimensional coordinates of environmental objects from the omnidirectional stereo image by matching the stereo image having different view points. The omnidirectional stereo imaging system in this paper is cost-effective and relatively easy for correspondence matching because of consistent camera intrinsic parameters in the stereo image. The parameters of the imaging system are extracted through 3-step calibration and the performance for 3-dimensional coordinates reconstruction is verified through experiments in this paper. Measurable range of the proposed imaging system is also presented by depth-resolution analysis.