Lens Distortion Model Research Articles

A Bi-Radial Model for Lens Distortion Correction of Low-Cost UAV Cameras

Recently developed cameras in the low-cost sector exhibit lens distortion patterns that cannot be handled well with established models of radial lens distortion. This study presents an approach that divides the image sensor and distortion modeling into two concentric zones for the application of an extended radial lens distortion model. The mathematical model is explained in detail and it was validated on image data from a DJI Mavic Pro UAV camera. First, the special distortion pattern of the camera was examined by decomposing and analyzing the residuals. Then, a novel bi-radial model was introduced to describe the pattern. Eventually, the new model was integrated in a bundle adjustment software package. Practical tests revealed that the residuals of the bundle adjustment could be reduced by 63% with respect to the standard Brown model. On the basis of external reference measurements, an overall reduction in the residual errors of 40% was shown.

Calibration method for fringe projection profilometry based on rational function lens distortion model

The calibration of the fringe projection profilometry (FPP) system is susceptible to the camera lens distortion. This paper presents a calibration method based on the rational function lens distortion model. In this method, we integrate the rational function into the mathematical expression of the out-of-plane height to mitigate the impact of lens distortion. With arbitrarily arranged measurement components, we can flexibly measure the three-dimensional (3D) shape of objects with no expensive auxiliaries, high-precision setups, or complicated projector calibration. First, we project modulated sinusoidal fringe patterns with high frequency to the concentric circle grids calibration board to accurately estimate the unwrapped phase value, the pixel coordinate, and the out-of-plane height at each feature point. Then, the 3D coordinates can be recovered by determining the coefficients of the high order phase, coordinate, and height (PCH) mapping. Experiments on computer simulation and real data demonstrate that the modified calibration contributes to the flexibility and accuracy of the 3D shape measurement.

Sample balancing of curves for lens distortion modeling and decoupled camera calibration

State-of-the-art camera calibration methods usually estimate the pinhole model and lens distortion together by strict point correspondence. However, in many applications, only part of the full calibration result is required. Simpler calibration patterns like straight lines are studied in many works to calibrate only the lens distortion but lack precision compared with conventional methods. In this paper, we study the geometric invariants of line straightness and proposed a method to accurately estimate the normal vector of the original line from the distorted curve. We discovered that unbalanced samples have a significant influence on estimation accuracy. We proposed a sample balancing method and linear COD determination methods based on normal geometry. The proposed method requires little positioning restrictions of the camera or the calibration pattern, and lines are very easily found or established patterns. The proposed method is demonstrated with both synthetic data and real images. We show by experiment that camera calibration can be decoupled, and more flexible calibration patterns can be used for each stage.

Non-metric correction method for complete lens distortion using dual-characteristics joint measurement

In practical vision systems, due to the limitation of manufacturing technology, charge-coupled device pixel cells are difficult to satisfy the standard square, and the lens inevitably has imaging nonlinearity, which leads to geometric distortion of the image. To ensure the accuracy of subsequent visual processing, it is necessary to correct the image distortion. A non-metric correction method for complete lens distortion using dual-characteristics joint measurement is proposed. We not only considered the influence of the distortion center in the correction process but also constructed a mapping matrix to standardize the pixel cells, thus establishing a complete lens distortion model. Simultaneously, the linear residual function of the dual-characteristics joint measurement was defined according to the geometric constraint between the vanishing point and the straightness. In addition, the distortion model coefficients were determined by nonlinear iterative optimization. Extensive experiment results demonstrated that the proposed method enables effective and accurate lens distortion correction, which is significantly superior to the existing single characteristic measurement distortion correction approach.

Enhanced calibration of camera setups for high-performance visual odometry

For robots and autonomous system that rely on visual data for operating in the real world, camera calibration is an indispensable step as it relates image information to the geometric structure of the 3D world. Although it is convenient to consider a several decades old problem as something that is swiftly solvable with a dedicated toolbox, we should still push calibration methods to their practical limits in order to gain valuable insights, and especially when robots are operating in circumstances that concern human safety. In this paper we propose a camera setup calibration procedure with emphasis on visual odometry accuracy. We focus on target-based calibration and two popular datasets are used for evaluating visual odometry and SLAM algorithms, namely the EuRoC and KITTI datasets. Our procedure consists of: (i) introducing a novel highly accurate corner detection algorithm robust to challenging illumination conditions, (ii) investigating different lens distortion models, (iii) incorporating static and dynamic board deformation models, (iv) ex-post analysis of reprojection error sensitivity and calibration parameter uncertainty, and (v) grid search method based on odometry accuracy when board poses do not constrain calibration parameters well enough. The whole process significantly reduced the reprojection error when calibrating the camera setups of the EuRoC and KITTI datasets. We tested four different odometries, namely SOFT, ORB-SLAM2, VINS-FUSION, and VISO2—all four showed higher accuracy with the proposed calibration parameters. Moreover, with the proposed calibration method our SOFT2 scored 0.53% in translation and 0.0009 deg/m in rotation error rendering it currently the highest ranking algorithm on the KITTI scoreboard.

Sample Balancing of Curves for Lens Distortion Modeling and Decoupled Camera Calibration

Sample Balancing of Curves for Lens Distortion Modeling and Decoupled Camera Calibration

A new phase to height model in fringe projection profilometry by considering radial distortion of camera lens

In fringe projection profilometry, the straight-shaped fringe patterns predefined in computer are projected onto the object surface and the distorted fringe patterns reflected from the surface are captured by camera. The phase information obtained by analysis of the modulated fringe pattern is converted to the height information by a certain phase to height model. Here, the accuracy of the height information is affected by the lens distortion. So it arises the problems to consider the lens distortion during capturing the fringe pattern and to use correct phase to height model. In this paper a phase to height model is proposed, which uses 9 coefficients, and a new method for removing the radial distortion of camera lens is suggested by ignoring the tangential distortion and approximating the total lens distortion model to a 5th-order equation. Simulation and experiment results demonstrate that the proposed method is available and effective in three-dimensional shape measurement.

A virtual environment for evaluation of computer vision algorithms under general airborne camera imperfections

AbstractThis paper describes a camera simulation framework for validating machine vision algorithms under general airborne camera imperfections. Lens distortion, image delay, rolling shutter, motion blur, interlacing, vignetting, image noise, and light level are modelled. This is the first simulation that considers all temporal distortions jointly, along with static lens distortions in an online manner. Several innovations are proposed including a motion tracking system allowing the camera to follow the flight log with eligible derivatives. A reverse pipeline, relating each pixel in the output image to pixels in the ideal input image, is developed. It is shown that the inverse lens distortion model and the inverse temporal distortion models are decoupled in this way. A short-time pixel displacement model is proposed to solve for temporal distortions (i.e. delay, rolling shutter, motion blur, and interlacing). Evaluation is done by several means including regenerating an airborne dataset, regenerating the camera path on a calibration pattern, and evaluating the ability of the time displacement model to predict other frames. Qualitative evaluations are also made.

DoF-Dependent and Equal-Partition Based Lens Distortion Modeling and Calibration Method for Close-Range Photogrammetry.

Lens distortion is closely related to the spatial position of depth of field (DoF), especially in close-range photography. The accurate characterization and precise calibration of DoF-dependent distortion are very important to improve the accuracy of close-range vision measurements. In this paper, to meet the need of short-distance and small-focal-length photography, a DoF-dependent and equal-partition based lens distortion modeling and calibration method is proposed. Firstly, considering the direction along the optical axis, a DoF-dependent yet focusing-state-independent distortion model is proposed. By this method, manual adjustment of the focus and zoom rings is avoided, thus eliminating human errors. Secondly, considering the direction perpendicular to the optical axis, to solve the problem of insufficient distortion representations caused by using only one set of coefficients, a 2D-to-3D equal-increment partitioning method for lens distortion is proposed. Accurate characterization of DoF-dependent distortion is thus realized by fusing the distortion partitioning method and the DoF distortion model. Lastly, a calibration control field is designed. After extracting line segments within a partition, the de-coupling calibration of distortion parameters and other camera model parameters is realized. Experiment results shows that the maximum/average projection and angular reconstruction errors of equal-increment partition based DoF distortion model are 0.11 pixels/0.05 pixels and 0.013°/0.011°, respectively. This demonstrates the validity of the lens distortion model and calibration method proposed in this paper.

Augmented-reality-assisted workpiece localization in rod-type flexible fixtures

This study proposes an augmented reality (AR)-assisted workpiece-localization technique for rod-type flexible fixtures used in automotive and aerospace industries, for machining thin-walled components. The machining accuracy of a component depends considerably on the profile error and envelopment condition of the blank workpiece. An AR virtual reference of the component is created in the image space using a pinhole camera with a lens distortion model that visually assists the rough placement of the workpiece at the desired location. The fine workpiece-localization process is formulated as a nonlinear optimization problem to minimize the profile error. The problem is subjected to a point-in-polygon constraint in the image space to guarantee the envelopment requirement. The proposed method is validated using an aircraft body panel. It uses only the measurement data acquired from the upper surface of the panel and the AR reference and is a more flexible and accurate solution than the locating-pin method.

Geometric modelling and calibration of a spherical camera imaging system

AbstractThe Ladybug5 is an integrated, multi‐camera system that features a near‐spherical field of view. It is commonly deployed on mobile mapping systems to collect imagery for 3D reality capture. This paper describes an approach for the geometric modelling and self‐calibration of this system. The collinearity equations of the pinhole camera model are augmented with five radial lens distortion terms to correct the severe barrel distortion. Weighted relative orientation stability constraints are added to the self‐calibrating bundle adjustment solution to enforce the angular and positional stability between the Ladybug5’s six cameras. Centimetre‐level 3D reconstruction accuracy can be achieved, with image‐space precision and object‐space accuracy improved by 92%and 93%, respectively, relative to a two‐term lens distortion model. Sub‐pixel interior orientation stability and millimetre‐level relative orientation stability were also demonstrated over a 10‐month period.

Assessing Stability Performance of Non-Metric Camera’s Lens Distortion Model during UAV Flight Missions

The process and definition of camera calibration have change greatly over recent years. Aerial metric cameras calibration, for which laboratory and field calibration procedures were a separate process, was performed before and independent of any actual mapping data collection using precise calibration fixtures with an assumption that the camera parameters determined would remain valid for a significant period. In contrast, non-metric cameras are characterized by unstable intrinsic parameters over the times and they are vulnerable to the engine and other vibrations during flight data acquisitions. Moreover, there is no standard calibration procedures exist for these cameras. But, since non-metric camera self-calibration augments the concept of calibration as a part of the measurement process, it can determine the camera intrinsic parameters at the time of the data acquisition as long as highly-convergent geometry and the use of multiple exposures are employed. Therefore, this paper investigates variations of the lens distortion components with object distance within the photographic field by using the self-calibration method. The use of redundant flight paths and tilted camera is also incorporated to ascertain the stability of the principal distance and the principal points during the flight mission. During the experiments, a series of flight mission is conducted to photograph test field areas from over a relatively flat area to highly mountainous one. It is revealed that the radial, decentering, and affinity distortion parameters are more stable than that of the principal distance and principal points against vibrations.

Development of an Image Grating Sensor for Position Measurement.

In this research paper, a precision position-measurement system based on the image grating technique is presented. The system offers a better robustness and flexibility for 1D position measurement compared to a conventional optical encoder. It is equipped with an image grating attached to a linear stage as the target feature and a line scan camera as the stationary displacement reader. By measuring the position of the specific feature in the image and applying a subpixel image registration method, the position of the linear stage can be obtained. In order to improve the computational efficiency, the calculations for pattern correlation and subpixel registration are performed in the frequency domain. An error compensation method based on a lens distortion model is investigated and implemented to improve the measurement accuracy of the proposed system. Experimental data confirms the capability of the developed image grating system as ±0.3 µm measurement accuracy within a 50 mm range and ±0.2 µm measurement accuracy within a 25 mm range. By applying different optics, the standoff distance, measurement range, and resolution can be customized to conform to different precision measurement applications.

A weighting intersection point prediction iteration optimization algorithm used in photogrammetry for port hoisting machinery

As a mature technology, photogrammetry is widely applied in today's engineering measurement field. However, because of the limitation of the port condition, it is impossible to obtain photos that are taken at ideal angles and distances when photogrammetry is used to measure port hoisting machinery, and this leads to invalid measurements with low accuracy data. To solve this problem, a new algorithm is proposed in this work. First, the proposed method introduces redundant measurements through an intersection point prediction algorithm to improve the measurement data’s accuracy. Second, a weighting algorithm based on the lens distortion model is then provided to further improve accuracy. Third, an iterative method is established from the threshold setting method based on the weighting algorithm. Thus, the quality of the final measurement could be controllable. Finally, an experiment is devised for the characteristics of the algorithm and the port condition. The results demonstrate that the method described in this paper significantly improves the accuracy of the measuring results of photogrammetry while photos used for the calculation were taken at unsatisfactory angles and distances caused by the limitation of the port condition.

Analysis and simplification of lens distortion model for the scheimpflug imaging system calibration

Abstract The Scheimpflug imaging system has been widely applied in many fields, such as 3D reconstruction and tomographic PIV. Modeling and calibration are essential steps for further analysis of the Scheimpflug imaging system. In this paper, lens distortion model of the Scheimpflug imaging system is established and simplified for calibration. Firstly, lens distortion model of the Scheimpflug imaging system is analyzed and established, which is proved to be different from the conventional imaging system. Secondly, the established lens distortion model is simplified to reduce the complexity of the calibration for the infinite parameters. The application condition of the simplified lens distortion model is also analyzed. Finally, calibration experiment is implemented for the Scheimpflug imaging system with the simplified lens distortion model. The high calibration accuracy verifies that the simplified lens distortion model is feasible.

Comparison of pre- and self-calibrated camera calibration models for UAS-derived nadir imagery for a SfM application

Structure from Motion (SfM) is a tool being increasingly utilised in geosciences for high-resolution three-dimensional mapping of landscapes. However, a number of authors have demonstrated that broad-scale systematic deformations, in the form of ‘doming’ and ‘bowling’, can occur when applied to linear (low-amplitude, feature-limited) topographies. In such contexts, a more rigorous lens calibration and ground control point acquisition process is required, which means that application of SfM to environments such as tidal flats or desert plains can be challenging. Uncertainties in elevation models generated through SfM were investigated here in the context of the low elevation, micro-topographic environment of saltmarsh. Eight digital surface models (DSMs) were generated for a saltmarsh site in the Deben Estuary (Suffolk, UK) using imagery acquired by a low-cost consumer grade unmanned aerial system (UAS). The results provide clear illustration of the systematic bowling effect following self-calibration during bundle adjustment. This was due to poor estimations of distortion parameters in the camera model. Deformation was most pronounced when UAS-GPS data were used for georeferencing. The use of dGPS-determined ground control points improved the DSM, but did not fully mitigate the deformations. By introducing a pre-calibrated model, derived using a typical checkerboard routine, deformation was significantly mitigated. These results were tested in both the commercial Agisoft PhotoScan® and open-source Micmac software. When self-calibration was used, Micmac generated significantly more accurate DSMs because a more complex lens distortion model could be implemented. The results show that when mapping flat topographies, pre-calibration of the camera model out-performs self-calibration. However, if pre-calibration is not possible, a complex distortion model (such as Micmac’s Four model) can be utilised to limit deformation. The results of the software analysis concluded there is no one-size fits all software solution, and therefore customisable open-source systems offer many potential benefits.

MODULAR BUNDLE ADJUSTMENT FOR PHOTOGRAMMETRIC COMPUTATIONS

Abstract. In this paper we investigate how the residuals in bundle adjustment can be split into a composition of simple functions. According to the chain rule, the Jacobian (linearisation) of the residual can be formed as a product of the Jacobians of the individual steps. When implemented, this enables a modularisation of the computation of the bundle adjustment residuals and Jacobians where each component has limited responsibility. This enables simple replacement of components to e.g. implement different projection or rotation models by exchanging a module. The technique has previously been used to implement bundle adjustment in the open-source package DBAT (Börlin and Grussenmeyer, 2013) based on the Photogrammetric and Computer Vision interpretations of Brown (1971) lens distortion model. In this paper, we applied the technique to investigate how affine distortions can be used to model the projection of a tilt-shift lens. Two extended distortion models were implemented to test the hypothesis that the ordering of the affine and lens distortion steps can be changed to reduce the size of the residuals of a tilt-shift lens calibration. Results on synthetic data confirm that the ordering of the affine and lens distortion steps matter and is detectable by DBAT. However, when applied to a real camera calibration data set of a tilt-shift lens, no difference between the extended models was seen. This suggests that the tested hypothesis is false and that other effects need to be modelled to better explain the projection. The relatively low implementation effort that was needed to generate the models suggest that the technique can be used to investigate other novel projection models in photogrammetry, including modelling changes in the 3D geometry to better understand the tilt-shift lens.

Geometric model and assessment of a dual‐fisheye imaging system

AbstractThis paper presents a technique for the calibration of a 360° dual imaging system (Ricoh Theta S) composed of two cameras with fisheye lenses specially mounted in a common structure. The experiments are based on calibration with a bundle adjustment using the equidistant fisheye lens model and relative orientation stability constraints, such as distance, base elements and rotation matrix between individual cameras’ coordinate systems. As a result, an equidistant mathematical model using the Conrady–Brown lens distortion model was shown to be suitable for estimating the interior and relative orientation parameters of the Ricoh Theta S. The main contribution presented in this work is a simultaneous bundle adjustment approach with stability constraints on base elements and rotation matrix angles, which achieved accuracies comparable with conventional photogrammetric calibration with perspective cameras.

基于BP神经网络的物镜畸变改正模型精度探讨

基于BP神经网络的物镜畸变改正模型精度探讨

제르니케 기울기 함수를 이용한 결상광학계의 왜곡보정

We present a method of highly accurate distortion correction of imaging systems by exploiting the gradient of Zernike polynomials. Distortion correction is one of the critical steps, along with the derivation of intrinsic and extrinsic parameters via nonlinear optimization, in the procedure of imaging system calibration. Compared to the conventional lens distortion model, based on two-dimensional distortion correction, the gradient of Zernike polynomials enables a higher order of distortion representation. Theoretically, the gradient of the Zernike polynomial provides complete sets of zero and non-zero divergence orthogonal functions, which could replace the transverse and radial distortion terms of the conventional lens distortion model, respectively. We implemented a practical imaging system calibration setup based on the method introduced in this paper, and the result shows an accuracy of 0.1 pixel, which is better than conventional method.