Geometric Camera Calibration Research Articles

Geometric Wide-Angle Camera Calibration: A Review and Comparative Study

Wide-angle cameras are widely used in photogrammetry and autonomous systems which rely on the accurate metric measurements derived from images. To find the geometric relationship between incoming rays and image pixels, geometric camera calibration (GCC) has been actively developed. Aiming to provide practical calibration guidelines, this work surveys the existing GCC tools and evaluates the representative ones for wide-angle cameras. The survey covers the camera models, calibration targets, and algorithms used in these tools, highlighting their properties and the trends in GCC development. The evaluation compares six target-based GCC tools, namely BabelCalib, Basalt, Camodocal, Kalibr, the MATLAB calibrator, and the OpenCV-based ROS calibrator, with simulated and real data for wide-angle cameras described by four parametric projection models. These tests reveal the strengths and weaknesses of these camera models, as well as the repeatability of these GCC tools. In view of the survey and evaluation, future research directions of wide-angle GCC are also discussed.

A Perceptual Measure for Deep Single Image Camera and Lens Calibration.

Image editing and compositing have become ubiquitous in entertainment, from digital art to AR and VR experiences. To produce beautiful composites, the camera needs to be geometrically calibrated, which can be tedious and requires a physical calibration target. In place of the traditional multi-image calibration process, we propose to infer the camera calibration parameters such as pitch, roll, field of view, and lens distortion directly from a single image using a deep convolutional neural network. We train this network using automatically generated samples from a large-scale panorama dataset, yielding competitive accuracy in terms of standard l2 error. However, we argue that minimizing such standard error metrics might not be optimal for many applications. In this work, we investigate human sensitivity to inaccuracies in geometric camera calibration. To this end, we conduct a large-scale human perception study where we ask participants to judge the realism of 3D objects composited with correct and biased camera calibration parameters. Based on this study, we develop a new perceptual measure for camera calibration and demonstrate that our deep calibration network outperforms previous single-image based calibration methods both on standard metrics as well as on this novel perceptual measure. Finally, we demonstrate the use of our calibration network for several applications, including virtual object insertion, image retrieval, and compositing.

Infrared Camera Geometric Calibration: A Review and a Precise Thermal Radiation Checkerboard Target

Different infrared (IR) planar geometric calibration targets have been developed over the years that exploit a well-established and flexible optical camera geometric calibration procedure following the pinhole approximation. This geometric calibration is typically neglected in IR cameras, due to the relatively low resolution of thermal images and the complex IR targets needed for the geometric calibration in comparison to the optical targets. In this study, a thorough literature review of numerous IR camera geometric calibration targets, along with their respective outcomes, were summarized and leveraged to deliver a practical checkerboard target for less experienced end users, while offering the lowest reprojection errors. It was concluded that the fabrication of high emissivity contrast and precise square points of intersection within a checkerboard pattern extends the accuracy of capturing these control points in a thermal image for an optimized IR camera geometric calibration. Accordingly, two simple planar checkerboard targets were fabricated using laser engraving and ultraviolet (UV) printing technologies on a polished stainless steel (SS304) plate. The UV-printed checkerboard target on a polished metallic alloy delivered the lowest mean reprojection error (MRE) of 0.057 pixels and the lowest root mean square error (RMSE) of reprojection of 0.063 pixels, with a standard deviation lower than 0.003 pixels. The UV-printed design offers better accuracy than any other checkerboard calibration target, and comparable results to the best prominent circular pattern results reported in the literature.

Accuracy Evaluation and Prediction of Single-Image Camera Calibration

This paper proposes an application to statistically predict the accuracy of single-image geometric camera calibration that uses given 2D-3D correspondences. Deriving both camera intrinsics and extrinsics from correspondences between a single image and a 3D shape, is important for the scene analysis when the optical system of the camera is lost, such as in the analyses of traffic accidents. It is unclear whether the single-image calibration will be successful in practice, particularly when the number of 2D-3D correspondences is small, even if we could assign accurate correspondences by manual labor. To this end, we perform a systematic evaluation of the camera parameter accuracy using synthetic environments. Based on the statistics observed during the experiments, our application predicts the calibration accuracy from simple variables ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g</i> ., the area that correspondences could be given). Since the prediction process does not rely on 3D shapes, it provides an estimate of the success of the calibration before time-consuming processes, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., 3D scanning and 2D-3D correspondence mapping.

Improved blur circle detection method for geometric calibration of multifocus light field cameras

Multifocus light field cameras are increasingly gaining attention in the field of computational imaging and machine vision due to their ability to capture spatial and angular information of light rays. Accurate calibration of the geometric parameters of multifocus light field cameras is the prerequisite for implementing computational imaging techniques, such as digital refocusing, depth reconstruction, and three-dimensional reconstruction. We propose an improved blur circle detection method for the calibration of multifocus light field cameras. The circular Hough transform-based circle detection method is used to obtain the subimage centers accurately in white raw images. On this basis, the subimages in the checkerboard images are classified, and the image corners are detected to determine the clusters of image corners corresponding to checkerboard corners. Then, the centers of plenoptic disc features are used as projection points of checkerboard corners on the light field image to calculate the camera pose. Levenberg–Marquardt method is further used to solve the calibration model. Finally, the calibration experiments of multifocus light field cameras are carried out to evaluate the improved blur circle detection method. Experimental results indicated that the improved blur circle detection method has higher calibration accuracy of multifocus light field camera than the blur circle detection method, while the mean reprojection error of the microlens centers and the image corners are 0.12 and 0.33 pixels, and the mean relative distance error between adjacent checkerboards is 2.53%.

The Conception of Test Fields for Fast Geometric Calibration of the FLIR VUE PRO Thermal Camera for Low-Cost UAV Applications.

The dynamic evolution of photogrammetry led to the development of numerous methods of geometric calibration of cameras, which are mostly based on building flat targets (fields) with features that can be distinguished in the images. Geometric calibration of thermal cameras for UAVs is an active research field that attracts numerous researchers. As a result of their low price and general availability, non-metric cameras are being increasingly used for measurement purposes. Apart from resolution, non-metric sensors do not have any other known parameters. The commonly applied process is self-calibration, which enables the determining of the approximate elements of the camera’s interior orientation. The purpose of this work was to analyze the possibilities of geometric calibration of thermal UAV cameras using proposed test field patterns and materials. The experiment was conducted on a FLIR VUE PRO thermal camera dedicated to UAV platforms. The authors propose the selection of various image processing methods (histogram equalization, thresholding, brightness correction) in order to improve the quality of the thermograms. The consecutive processing methods resulted in over 80% effectiveness on average by 94%, 81%, and 80 %, respectively. This effectiveness, for no processing and processing with the use of the filtering method, was: 42% and 38%, respectively. Only high-pass filtering did not improve the obtained results. The final results of the proposed method and structure of test fields were verified on chosen geometric calibration algorithms. The results of fast and low-cost calibration are satisfactory, especially in terms of the automation of this process. For geometric correction, the standard deviations for the results of specific methods of thermogram sharpness enhancement are two to three times better than results without any correction.

Internal Geometric Quality Improvement of Optical Remote Sensing Satellite Images with Image Reorientation

When the in-orbit geometric calibration of optical satellite cameras is not performed in a precise or timely manner, optical remote sensing satellite images (ORSSIs) are produced with inaccurate camera parameters. The internal orientation (IO) biases of ORSSIs caused by inaccurate camera parameters show a discontinuous distorted characteristic and cannot be compensated by a simple orientation model. The internal geometric quality of ORSSIs will, therefore, be worse than expected. In this study, from the ORSSI users’ perspective, a feasible internal geometric quality improvement method is presented for ORSSIs with image reorientation. In the presented method, a sensor orientation model, an external orientation (EO) model, and an IO model are successively established. Then, the EO and IO model parameters are estimated with ground control points. Finally, the original image is reoriented with the estimated IO model parameters. Ten HaiYang-1C coastal zone imager (CZI) images, a ZiYuan-3 02 nadir image, a GaoFen-1B panchromatic image, and a GaoFen-1D panchromatic image, were tested. The experimental results showed that the IO biases of ORSSIs caused by inaccurate camera parameters could be effectively eliminated with the presented method. The IO accuracies of all the tested images were improved to better than 1.0 pixel.

An Improved Blur Circle Detection Method for Geometric Calibration of Multi-Focus Light Field Cameras

An Improved Blur Circle Detection Method for Geometric Calibration of Multi-Focus Light Field Cameras

Systematic approach for thermal imaging camera calibration for machine vision applications

In order to estimate the intrinsic and extrinsic parameters of a camera, the geometric calibration method for a camera is required to map 3D world coordinate points onto 2D image coordinate points. In this work, a systematic approach for designing a calibration board is proposed to calibrate the thermal imaging cameras that include both mid-wave and long-wave infrared waveband (MWIR and LWIR). The low-cost materials such as thermocol, white glue, and aluminum foil sheet are used for our calibration board design and these materials have not been used for calibrating thermal cameras to the best of our knowledge. To achieve geometric calibration of thermal cameras, our calibration board has generated strong thermal image contrast and has displayed consistent checkerboard corner detection. The proposed method is tested in both indoor and outdoor environments with MWIR and LWIR cameras. The single camera calibrator tool and stereo camera calibrator tool in MATLAB are used for this work and it exhibited that the overall mean reprojection error was ≤ 0.40 pixel. The new board’s thermal imaging quality is compared to an existing paint-based calibration board. In terms of precision, low-cost construction and easy reusability, the proposed calibration board is proven to be more beneficial.

Iterative feature detection of a coded checkerboard target for the geometric calibration of infrared cameras

Abstract. The geometric calibration of cameras becomes necessary when images should be undistorted, geometric image information is needed or data from more than one camera have to be fused. This process is often done using a target with a checkerboard or circular pattern and a given geometry. In this work, a coded checkerboard target for thermal imaging cameras and the corresponding image processing algorithm for iterative feature detection are presented. It is shown that, due in particular to the resulting better feature detectability at image borders, lower uncertainties in the estimation of the distortion parameters are achieved.

A Spliced Satellite Optical Camera Geometric Calibration Method Based on Inter-Chip Geometry Constraints

When in orbit, spliced satellite optical cameras are affected by various factors that degrade the actual image stitching precision and the accuracy of their data products. This is a major bottleneck in the current remote sensing technology. Previous geometric calibration research has mostly focused on stitched satellite images and has largely ignored the inter-chip relationship among original multi-chip images, resulting in accuracy loss in geometric calibration and subsequent image products. Therefore, in this paper, a novel geometric calibration method is proposed for spliced satellite optical cameras. The integral geometric calibration model was developed on inter-chip geometry constraints among multi-chip images, including the corresponding external and internal calibration models. The proposed approach improves uncontrolled geopositioning accuracy and enhances mosaic precision at the same time. For evaluation, images from the optical butting satellite ZiYuan-3 (ZY-3) and mechanical interleaving satellite Tianhui-1 (TH-1) were used for the experiments. Multiple sets of satellite data of the Songshan Calibration field and other regions were used to evaluate the reliability, stability, and applicability of the calibration parameters. The experiment results found that the proposed method obtains reliable camera alignment angles and interior calibration parameters and generates high-precision seamless mosaic images. The calibration scheme is not only suitable for mechanical interleaving cameras with large geometric displacement among multi-chip images but is also effective for optical butting cameras with minor chip offset. It also significantly improves uncontrolled geopositioning accuracy for both types of spliced satellite images. Moreover, the proposed calibration procedure results in multi-chip satellite images being seamlessly stitched together and mosaic errors within one pixel.

Corner cases and limitations using a DOE based geometric camera calibration

Several years ago, the geometric calibration of cameras based on diffractive optical elements was invented, and since October 2020 the first product is commercially available.A laser beam is expanded, and the plane wave falls onto a diffractive optical element. The DOE generates a regular grid of light dots that virtually originates from infinity.This structure is then captured with the device under test and the dot positions are detected. From the positions, the required values can be calculated.The potential of the method, the compactness of the setup and the ease of use have brought up many desires that so far had not been addressed.Amongst these are: &lt;list list-type="order"&gt; &lt;list-item&gt;Calibration of extreme wide field of view cameras &gt; 140°.&lt;/list-item&gt; &lt;list-item&gt;Calibration of cameras/lens combinations with a large entrance pupil.&lt;/list-item&gt; &lt;list-item&gt;Increased camera DOE distance to, e.g., measure cameras behind a windshield in automotive applications.&lt;/list-item&gt; &lt;list-item&gt;Camera pairs with a stereo base significantly exceeding 60 mm.&lt;/list-item&gt; &lt;list-item&gt;Deriving the point spread function of the system at every light dot to use the method for more than just distortion measurement, e.g., MTF determination or visualization.&lt;/list-item&gt; &lt;/list&gt;There are also a few limitations compared to the conventional methods: &lt;list list-type="lower-alpha"&gt; &lt;list-item&gt;Measurement at infinity only&lt;/list-item&gt; &lt;list-item&gt;Stereo basis cannot be measured due to translation invariance of the method&lt;/list-item&gt; &lt;list-item&gt;Determination of chromatic aberration&lt;/list-item&gt; &lt;list-item&gt;Limited application of a single DOE (due to resolution of the camera and field of view)&lt;/list-item&gt; &lt;/list&gt;All these desires and limitations are discussed, and solutions are presented where possible.

Target Analysis for the Multispectral Geometric Calibration of Cameras in Visual and Infrared Spectral Range

While for the geometric calibration of cameras in the visual spectral range simple printed targets with chessboard patterns have become established over the years, the spectrum of targets used for cameras in the infrared range is broadly diversified. Within the scope of this article different target types are compared with each other in the four spectral ranges visual (VIS), near infrared (NIR), medium-wave infrared (MWIR) and long-wave infrared (LWIR). For this purpose, three targets are manufactured and evaluated with regard to the detectability of the features and the achievable calibration results. The use of a single target, which provides good results in all ranges, is of particular interest for the extrinsic calibration of cameras of different wavelengths to each other. It is shown that the contrast-to-noise ratio and different OpenCV feature detection algorithms are the reasons for the varying calibration quality with different targets.

Challenges in UAS-Based TIR Imagery Processing: Image Alignment and Uncertainty Quantification

Thermal infrared measurements acquired with unmanned aerial systems (UAS) allow for high spatial resolution and flexibility in the time of image acquisition to assess ground surface temperature. Nevertheless, thermal infrared cameras mounted on UAS suffer from low radiometric accuracy as well as low image resolution and contrast hampering image alignment. Our analysis aims to determine the impact of the sun elevation angle (SEA), weather conditions, land cover, image contrast enhancement, geometric camera calibration, and inclusion of yaw angle information and generic and reference pre-selection methods on the point cloud and number of aligned images generated by Agisoft Metashape. We, therefore, use a total amount of 56 single data sets acquired on different days, times of day, weather conditions, and land cover types. Furthermore, we assess camera noise and the effect of temperature correction based on air temperature using features extracted by structure from motion. The study shows for the first time generalizable implications on thermal infrared image acquisitions and presents an approach to perform the analysis with a quality measure of inter-image sensor noise. Better image alignment is reached for conditions of high contrast such as clear weather conditions and high SEA. Alignment can be improved by applying a contrast enhancement and choosing both, reference and generic pre-selection. Grassland areas are best alignable, followed by cropland and forests. Geometric camera calibration hampers feature detection and matching. Temperature correction shows no effect on radiometric camera uncertainty. Based on a valid statistical analysis of the acquired data sets, we derive general suggestions for the planning of a successful field campaign as well as recommendations for a suitable preprocessing workflow.

Geometric stitching of a HaiYang-1C ultra violet imager with a distorted virtual camera.

The HaiYang-1C (HY-1C) ultra violet imager (UVI) consists of five independent cameras with a designed total image swath of approximately 3000 km. In order to obtain a complete seamless image formed by the five sub-images, a feasible geometric stitching method for the HY-1C UVI with a distorted virtual camera is proposed. First, we perform the absolute geometric calibration of camera 3 and the relative geometric calibration of cameras 1, 2, 4, and 5. Then, a distorted virtual camera is assigned. Finally, the five sub-images are stitched together with the distorted virtual camera. Three HY-1C UVI images were tested. The experimental results showed that the georeferencing accuracy of the stitched images was better than 1 pixel. Compared with the conventional stitching method with an undistorted virtual camera, the ground sampling distance differences of the five cameras obtained by the proposed method were reduced from 23%, 37%, 53%, 37%, and 25% to 6%, 6%, 1%, 7%, and 8%, respectively.

A new dimension in geometric camera calibration

There are many test charts and software to determine the intrinsic geometric calibration of a camera including distortion. But all of these setups have a few problems in common. They are limited to finite object distances and require large test charts for calibrations at greater distances combined with powerful and uniform illumination. On production lines the workaround for this problem is often times the use of a relay lens which itself introduces geometric distortions and therefore inaccuracies that need to be compensated for. A solution to overcome these problems and limitations has originally been developed for space applications and has already become a common method for the calibration of satellite cameras. We have now turned the lab setup on an optical bench into a commercially available product that can be used for the calibration of a huge variety of cameras for different applications. This solution is based on a diffractive optical element (DOE) that gets illuminated by a plane wave generated with an expanded laser diode beam. In addition to the conventional methods the proposed one also provides the extrinsic orientation of the camera and therefore allows the adjustment of cameras to each other.

Validation methods for geometric camera calibration

We have developed a semi-automatic annotation tool – “CVL Annotator” – for bounding box ground truth generation in videos. Our research is particularly motivated by the need for reference annotations of challenging nighttime traffic scenes with highly dynamic lighting conditions due to reflections, headlights and halos from oncoming traffic. Our tool incorporates a suite of different state-of-the-art tracking algorithms in order to minimize the amount of human input necessary to generate high-quality ground truth data. We focus our user interface on the premise of minimizing user interaction and visualizing all information relevant to the user at a glance. We perform a preliminary user study to measure the amount of time and clicks necessary to produce ground truth annotations of video traffic scenes and evaluate the accuracy of the final annotation results.

Online Correction of the Mutual Miscalibration of Multimodal VIS–IR Sensors and 3D Data on a UAV Platform for Surveillance Applications

Unmanned aerial vehicles (UAVs) are widely used to protect critical infrastructure objects, and they are most often equipped with one or more RGB cameras and, sometimes, with a thermal imaging camera as well. To obtain as much information as possible from them, they should be combined or fused. This article presents a situation in which data from RGB (visible, VIS) and thermovision (infrared, IR) cameras and 3D data have been combined in a common coordinate system. A specially designed calibration target was developed to enable the geometric calibration of IR and VIS cameras in the same coordinate system. 3D data are compatible with the VIS coordinate system when the structure from motion (SfM) algorithm is used. The main focus of this article is to provide the spatial coherence between these data in the case of relative camera movement, which usually results in a miscalibration of the system. Therefore, a new algorithm for the detection of sensor system miscalibration, based on phase correlation with automatic calibration correction in real time, is introduced.

Camera Geometric Calibration Using Dynamic Single-Pixel Illumination With Deep Learning Networks

Traditional methods of geometric camera calibration (GCC) are based on angle measurements (AM) or diffractive optical elements (DOE). However, the AM-based approach has the low accuracy and reliability because the vibration of rotating mechanisms, i.e., turntables and angle measuring accuracy easily influence the calibration accuracy. The DOE failure easily occurs in the calibration process when some micro-apertures are blocked by dust particles. In this paper, a new method for the GCC by means of single pixel illumination in a deep neural network (DNN) is presented. A closed-loop calibration link is composed of a single stimulus input produced by a single pixel generator and a collimator, an uncalibrated camera, and a DNN. The dynamic single-pixel illumination forms the different stimulus input of the DNN at different stimulus time. The synaptic weights (the camera interior parameters) of multilayer perceptron are adjusted until the cost function is minimized. This method is able to avoid the aforementioned shortcoming of conventional calibration methods. This method can be especially used for on-ground calibration of remote sensing cameras but in principle also suitable for on-orbit GCC and other cameras.

AUTOMATIC CAMERA SYSTEM CALIBRATION WITH A CHESSBOARD ENABLING FULL IMAGE COVERAGE

Abstract. Geometric camera calibration is a mandatory prerequisite for many applications in computer vision and photogrammetry. Especially when requiring an accurate camera model the effort for calibration can increase dramatically. For the calibration of the stereo-camera used for optical navigation a new chessboard based approach is presented. It is derived from different parts of existing approaches which, taken separately, are not able to meet the requirements. Moreover, the approach adds one novel main feature: It is able to detect all visible chessboard fields with the help of one or more fiducial markers simply sticked on a chessboard (AprilTags). This allows a robust detection of one or more chessboards in a scene, even from extreme perspectives. Except for the acquisition of the calibration images the presented approach enables a fully automatic calibration. Together with the parameters of the interior and relative orientation the full covariance matrix of all model parameters is calculated and provided, allowing a consistent error propagation in the whole processing chain of the imaging system. Even though the main use case for the approach is a stereo camera system it can be used for a multi-camera system with any number of cameras mounted on a rigid frame.