Camera Calibration Method Research Articles

Image super-resolution based on improved ESRGAN and its application in camera calibration

Camera calibration is a key technique in the field of computer vision. The Zhang’s method is currently the most commonly used camera calibration method, but its accuracy is mainly dependent on the image quality, thusly constrained by camera hardware conditions. Considering that high-performance cameras are often expensive, a cost-effective method to improve the quality of calibration images is of great practical value. Without changing the existing hardware conditions, image super-resolution can be considered to enhance the quality of the calibration image. Image super-resolution (SR) is the process of reconstructing an image from low resolution (LR) to high resolution (HR). It can enhance the clarity and detail of the calibration images, which will be beneficial in improving the accuracy of camera calibration. However, there is very little research on the application of image super-resolution to camera calibration. Therefore, this study innovatively proposed a method and process for applying image super-resolution in camera calibration. Firstly, the ESRGAN (Enhanced Super-Resolution Generative Adversarial Networks) was optimized and improved, followed by the proposal of the MMRSRGAN (Multi-scale Multi-adaptive-weights Residual Super-Resolution Generative Adversarial Networks). Two methods for local image super-resolution were also proposed for efficient training and testing of datasets, and two evaluation metrics based on reprojection error were proposed evaluate the effectiveness of the proposed model. Training and testing experiments of image super-resolution for camera calibration were conducted on both virtual and real image datasets. The proposed MMRSRGAN was compared with several advanced image super-resolution networks developed in recent years across multiple dimensions and indicators. The positive results have demonstrated the feasibility of applying image super-resolution to camera calibration and showcased the good performance of the MMRSRGAN.

A flexible camera calibration method for pose vision measurement system of roadheader

Cameras used for visual measurement in underground roadway construction sites are difficult to calibrate. Therefore, this study proposed an on-site camera calibration method that does not require manual intervention. First, the virtual point and line features on the plane target are constructed using pole-polar constraints. Second, a homography model based on point-and-line features is established. Finally, a combined objective function based on the dynamic weight coefficient is proposed to optimise the calibration results further. Relevant simulations and experiments showed that the maximum internal parameter error obtained using the proposed method was 0.463 %, and the maximum position estimation error of the roadheader was 30 mm. In addition, the results show that this method is superior to other methods in terms of calibration accuracy and robustness, solving the difficulty of camera calibration in underground roadway construction sites and guaranteeing high-precision pose vision measurement.

A binocular camera calibration method based on circle detection

Camera calibration is a crucial step in binocular measurement, and the accuracy of camera calibration largely determines the measurement accuracy of binocular vision. However, the calibration accuracy of existing calibration methods is difficult to satisfy the requirements of variable calibration environments and engineering applications. Therefore, based on the principle of Zhang's calibration method, a calibration method is proposed by combining bundle adjustment and diagonal constraints of the calibration target. Firstly, the improved Canny edge extraction algorithm is used to obtain the sub-pixel center of mass of ellipse (CME). Then, the Zhang's calibration method is used to obtain the initial values of the calibration parameters. The camera calibration parameters are optimized by using the bundle adjustment. Finally, diagonal constraints are used to further optimize the extrinsic camera position parameters. The experimental results show that compared to the other excellent methods, the proposed method can significantly improve calibration accuracy and has certain engineering application value.

A calibration method for telecentric line-scan cameras with an enhanced dynamic imaging model and initial estimates derived from direct linear transformation

This study presents a calibration method for telecentric line-scan cameras, incorporating a dynamic imaging model and initial estimates derived from direct linear transformation. The enhanced dynamic imaging model includes an inclination parameter instead of velocities in three orthogonal spatial directions. It is applicable to any direction of motion, considers lens distortion, and accommodates the line sensor at an arbitrary position relative to the optical axis. Employing the enhanced camera model, the initial values of telecentric line-scan camera parameters are determined derived from direct linear transformation. Sensitivity of our method with respect to the noise level of image points and the number of pattern planes are assessed through numerical simulations, while effectiveness and robustness are validated through experiments using real-world data, achieving a reprojection error of 0.6386 pixels and a standard deviation of 0.0160 pixels. The versatility of the proposed method can be extended to measurement applications utilizing flatbed scanners.

Comparison of two 3D calibration methods for thermal imaging cameras to track bat flight paths near wind turbines

Zusammenfassung 3D Kameras mit geringer Auflösung, die in einem großen Außenvolumen eingesetzt werden sollen, benötigen große interaxiale Abstände von einigen Metern zwischen den Kameras. Das ist zum Beispiel der Fall, wenn 3D Fledermaus- Flugbahnen mit Wärmebildkameras in der Nähe von Windenergieanlagen erforscht werden, um deren Auswirkungen auf die Tiere zu untersuchen. Derartige Kamera- Systeme müssen nach erfolgter Montage der Kameras im Feld 3D kalibriert werden. Dazu haben wir zwei Methoden identifiziert, die große interaxiale Abstände ermöglichen: Eine Methode (easyWand) basiert auf Landmarken die in beiden Kameras sichtbar sind und eine weitere von uns entwickelte Methode (Batflight3D) basiert auf dem 3D Flugpfad einer Drohne. In dieser Publikation versuchen wir einen möglichst repräsentativen quantitativen Vergleich dieser Methoden, indem wir die 2D Korrespondenzen des jeweiligen Drohnenflugpfades für beide Methoden nutzen, um sie so direkt miteinander vergleichen zu können. Die Ergebnisse zeigen, dass Batflight3D für den oben genannten Anwendungsfall wesentlich genauere und auch verlässlichere Ergebnisse liefert, aber auf die Möglichkeit eines Drohnenfluges und GPS-Empfangs angewiesen ist.

Catadioptric omnidirectional thermal odometry in dynamic environment

This paper presents a catadioptric omnidirectional thermal odometry (COTO) system that estimates the six degrees of freedom (DoF) pose of a camera using only omnidirectional thermal images in visually degraded, fast-motion, and dynamic environments. First, we design and fabricate a central hyperbolic catadioptric omnidirectional thermal camera that captures surrounding thermal images with 360° horizontal field of view (FoV), and improve the omnidirectional camera model and calibration method to obtain high-precision camera intrinsic parameter. Second, we propose the epipolar curve constraint combining with omnidirectional thermal object detection to significantly reduce the interference of moving objects on pose estimation. Third, the implemented COTO pipeline consists of photometric calibration, dynamic region removal, tracking and mapping to overcome the drawbacks of photometric inconsistency and large distortion in omnidirectional thermal images. Experiments have been conducted on a total of 17 sequences of Lab, Outdoor and Driving, amounting to more than 60,000 omnidirectional thermal images of real environments. The experimental results indicate that the proposed COTO system has excellent localization accuracy and unparalleled robustness over the current state-of-the-art methods. The average localization accuracy measured by the absolute trajectory error (ATE) is less than 15 cm from the ground truth in both Lab and Outdoor sequences. In addition, COTO was the only system with complete and successful tracking in all sequences. The system can be used as an innovative localization solution, particularly in challenging environments with changes in ambient light, rapid vehicle motion, and moving object interference, which can be a difficult problem for visual odometry to solve.

Implicit calibration method for underwater stereo cameras

Underwater stereo cameras can effectively capture intricate environments with restricted accessibility, offering an appealing solution for precise perception. Stereo imaging is however susceptible to distortions caused by the refraction of incoming rays. These distortions are nonlinear and challenge the standard single viewpoint projection assumption. In this paper, we propose a data-driven implicit calibration method for underwater stereo cameras. To address the imaging characteristics and aberration distributions across different coordinates of underwater stereo cameras, we have developed the corresponding coordinates regression network and fusion strategy, thereby converting the calibration process into network-based learning. Secondly, we designed an underwater self-luminous calibration target system and the underwater corner point extraction strategy for sample dataset acquisition. We evaluated the proposed method comprehensively in terms of measurement, camera posture estimation, and 3D reconstruction, and compared it with other explicit calibration methods. The experimental results show that the proposed implicit calibration method is superior to other explicit calibration. We demonstrate with real experiments that our method enables efficient camera calibration for underwater vision applications.

Motion-error-free calibration of event camera systems using a flashing target

Event cameras, inspired by biological vision, offer high dynamic range, excellent temporal resolution, and minimal data redundancy. Precise calibration of event camera systems is essential for applications such as 3D vision. The cessation of extra gray frame production in popular models like the dynamic vision sensor (DVS) poses significant challenges to achieving high-accuracy calibration. Traditional calibration methods, which rely on motion to trigger events, are prone to movement-related errors. This paper introduces a motion-error-free calibration method for event cameras using a flashing target produced by a standard electronic display that elicits high-fidelity events. We propose an improved events-accumulator to reconstruct gray images with distinct calibration features and develop an optimization method that adjusts camera parameters and control point positions simultaneously, enhancing the calibration accuracy of event camera systems. Experimental results demonstrated higher accuracy compared to the traditional motion-based calibration method (reprojection error: 0.03 vs. 0.96 pixels). The 3D reconstruction error remained around 0.15 mm, significantly improving over the motion-based method’s 8.00 mm. Additionally, the method’s adaptability for hybrid calibration in event-based stereovision systems was verified (e.g., with frame cameras or projectors).

Hybrid calibration method for a single camera stereo digital image correlation system

In single camera stereo digital image correlation systems, the existing calibration methods have problems with poor accuracy and violation of theoretical constraints, so we have proposed a hybrid calibration method. In this study, we first explain the principles of the existing single camera stereo measurement calibration methods. Then, on the basis of the classical camera calibration method combined with the characteristics of planar mirror reflection imaging, a hybrid calibration method is proposed that not only satisfies the constraints of the same intrinsic parameters but also has higher accuracy. Next, we complete the mathematical model of the hybrid calibration method and explain the specific calibration process. Finally, the experimental results of point distance reconstruction and static displacement measurement show that, compared with the existing calibration methods, the hybrid calibration method has lower error and higher reconstruction accuracy.

Evaluating Linear Coral Growth Estimation Using Photogrammetry and Alternative Point Cloud Comparison Methods

Abstract. Corals are critical reef-building organisms, providing essential habitat and ecosystem services. Tracking coral growth over time indicates coral reef health, which can be measured using various established techniques. Several coral growth-related studies have successfully applied photogrammetry to a particular coral of various types. While the focus of previous work was on standardised data processing and, to a certain degree, on the assessment of different point cloud comparison methods (Lange et al. 2022), little attention has been given to the impact of camera calibration. This study measured the annual linear extension of five Acropora spp. colonies using photogrammetry and evaluated all stages of imagery processing. A high focus was given to the analysis of the camera calibration method and the validation of camera parameters derived using an in-situ calibration of coral images with scale bars placed in the camera's field of view. We demonstrate that this method is as reliable as the calibration using a calibration frame. This study also examined the impact of the different point cloud comparison methods for Acropora spp. More specifically, the derived point clouds are compared by applying the point-to-point and point-to-model methods and manually selecting 12 coral branch tips. Histograms derived from the comparison methods were analysed and deemed a suitable and efficient alternative approach for measuring the maximum growth rate of mature colonies over shorter time periods (1 year or less).

Vortex-induced vibration dynamic characteristics monitoring with AI-based target object detection for long-span bridges

This research proposes an innovative framework for monitoring bridge Vortex-induced Vibration (VIV) dynamic characteristics utilizing AI-based machine-learning target object detection and tracking techniques with keypoint detection. Two camera calibration methods are implemented for cases with and without intrinsic and extrinsic camera setup information based on homography matrix conversion and distance-based conversion. The framework is verified on a video recording of a real-bridge VIV event and a simulation animation of bridge VIV with a peak/trough-based statistical method for calculating VIV frequency and amplitude. Accurate detection is achieved in both cases with short video durations. The framework demonstrates great potential for real-time bridge VIV monitoring, requiring minimal camera calibration and a straightforward device setup. It offers reliable and accurate results while remaining cost-effective.

An edge feature-based external calibration of monocular camera and LiDAR using in-orbit data

Multimodal perception represents a vital approach for observing non-cooperative targets in space. The accurate determination of the extrinsic parameters for monocular camera and LiDAR systems serves as the cornerstone for achieving effective space multimodal perception. However, due to factors like launch and operation, these extrinsic parameters can experience drift, thereby compromising the overall performance of the multimodal perception system. To tackle this challenge, this study introduces a novel approach: an edge feature-based external calibration method for monocular camera and LiDAR using in-orbit data. The proposed methodology involves several steps. Firstly, edges are extracted from the image, followed by distance transformation based on the edge image. The subsequent integration of the edge image and distance image yields the search scene image. Then, edge feature is extracted from the point cloud data. Finally, after constructing the objective function, genetic algorithms are used to determine precise external parameters. Simulation results underscore the effectiveness of the proposed algorithm, affirming its ability to achieve high-precision calibration results for external parameters even within intricate spatial environments.

Comprehensive Review of Camera Calibration Methods for Three-dimensional Imaging Technology

Comprehensive Review of Camera Calibration Methods for Three-dimensional Imaging Technology

LECalib: Line-based event camera calibration

Camera calibration is an essential prerequisite for event-based vision applications. Current event camera calibration methods typically involve using flashing patterns, reconstructing intensity images, and utilizing the features extracted from events. Existing methods are generally time-consuming and require manually placed calibration objects, which cannot meet the needs of rapidly changing scenarios. In this paper, we propose a line-based event camera calibration framework exploiting the geometric lines of commonly-encountered objects in man-made environments, e.g., doors, windows, boxes, etc. Different from previous methods, our method detects lines directly from event streams and leverages an event-line calibration model to generate the initial guess of camera parameters, which is suitable for both planar and non-planar lines. Then, a non-linear optimization is adopted to refine camera parameters. Both simulation and real-world experiments have demonstrated the feasibility and accuracy of our method, with validation performed on monocular and stereo event cameras. The source code is released at https://github.com/Zibin6/line_based_event_camera_calib.

A Straightforward Camera Calibration Method Based on a Single Low-cost Cubical Target

Abstract. Calibrating optical sensors with common targets facilitates the efficient and convenient acquisition of the sensor's internal parameters. In this paper, we present a new method of camera calibration utilizing a low-cost foamy cube, in a form of dice, which is based on the fact that arrangement of pip and cubical die surfaces is mutually orthogonal. Initially, each face and pips are identified through the color information on the die’s surfaces. Subsequently, the centers of pips are corrected using a circular projection model, and radial distortion coefficients are estimated based on centers’ one-to-one correspondences. After that, the tangent information between pairs of pips on orthogonal dice faces are utilized to compute vanishing points, leading to estimation of intrinsic parameters. Experimental results demonstrate that our method has similar effects compared to well-known checkerboard calibration method, reaching an average relative error of 2.43%, simplifying the calibration process in practical applications and showcasing good practicality and robustness.

A Robust Camera Self-calibration Method Based on Circular Oblique Images

Abstract. It is crucial to calibrate the camera’s intrinsic orientation elements and distortion parameters to ensure the photogrammetric accuracies. However, using nadir images to perform this task often leads to correlation between the intrinsic and extrinsic orientation elements, which will result in different camera calibration results by using different self-calibration strategies. It even has an impact on the follow-up processes and makes the product accuracy declined. To overcome this challenge, a robust camera calibration method based on circular oblique images was developed in this study. Firstly, circular oblique images with different viewing angles and camera distances were captured by unmanned aerial vehicle, following a specially designed circular flight path. Then the camera parameters were solved through the self-calibration bundle adjustment based on the circular oblique images. The experiments were carried out to compare the robustness and accuracy of nadir-image-based and circular-oblique-image-based methods. The standard deviation of focal lengths solved by different self-calibration strategies reduced from 12.99 pixels to 1.72 pixels, proving that the proposed method weakens the correlation between the intrinsic and extrinsic orientation elements and has strong robustness. The accuracy of aerial triangulation based on the camera parameters solved by the proposed method improved from 34.7 cm to 3.5cm, illustrating the effectiveness of the proposed method.

A novel camera calibration method based on known rotations and translations

It is often difficult to align the camera optical center with the rotation center of the motion mechanism during active vision calibration, and this dis-alignment could lead to inaccuracy of the constraint equations and calibration results. To circumvent such issues, this paper proposes a novel method for active vision camera calibration and facilitates its real-world implementation. In this method, the rotational motion axis is assumed not necessarily to pass through the camera’s optical center, and the constraint equations are established by correlating image matched points before and after camera motions. The displacements caused by rotation are incorporated into the constraint equations to improve the accuracy of calibration. Numerical simulation and experiments are conducted. The feasibility of this proposed method is justified by using both synthetic and experimental data, and the results show that this method could estimate camera intrinsic parameters with high accuracy. Furthermore, it is found that small rotation angles can better the calibration accuracy even under the influence of external noises. This study suggests that this novel method has great potential for visual and image applications.

High-precision binocular camera calibration method based on a 3D calibration object.

A high-precision binocular camera calibration method is proposed to address the issues of poor calibration accuracy and large calibration errors in current practical applications. This method uses a triangular stereo sphere as the calibration object and employs steps, such as ellipse fitting, Cholesky decomposition, homography matrix solution, and nonlinear optimization, to compute the intrinsic and extrinsic parameters, distortion parameters, and relative pose of the binocular camera. Moreover, this method simplifies the correspondences between primitives, enabling simultaneous calibration of multiple viewpoint cameras. This method is also suitable for both binocular cameras consisting of two different structured monocular cameras and those composed of two image sensors within the same structure. Experimental results showed that this method outperforms traditional algorithms in terms of binocular camera calibration accuracy, calibration errors between left and right cameras, and robustness, resulting in a significant improvement in overall algorithm performance.

Camera calibration based on lightweight fan-shaped target detection and fitness-distance-balance chaotic marine predators algorithm

Camera calibration represents a critical stage in visual measurement, directly impacting the measurement precision. Addressing the limitations of checkerboard and circular targets, resulting in low calibration accuracy and eccentricity errors, this study introduces a camera calibration method that relies on lightweight fan-shaped target detection and Fitness-Distance-Balance Chaotic Marine Predators Algorithm(FDBCMPA). Starting with the improvement of camera calibration targets by refining and designing fan-shaped targets, this paper aims to tackle eccentricity errors through hardware-based solutions. In the process of extracting target feature points, a lightweight object detection algorithm is devised to capture individual fan-shaped pattern positions from images, transitioning from global multi-shape feature detection to localized single-pattern feature extraction. This approach mitigates the distortions inherent in traditional checkerboard targets during global linear feature imaging and partially alleviates interference from background environments and similar features. Following the initial camera calibration, the FDBCMPA is employed to further optimize the calibration outcomes, thereby boosting the calibration method's performance. Experimental findings highlight that the enhanced object detection method in this study attains detection accuracy comparable to mainstream methods, with distinct advantages in detection speed, parameters, and FLOPs, rendering it more suitable for use in camera calibration. Additionally, the FDBCMPA demonstrates improvements in convergence results compared to MPA, PSO, TACPSO, Chimp, and other methods. The final camera calibration results demonstrate an impressive reprojection error of 0.0738 pixels through the utilization of the proposed camera calibration method introduced in this paper, and substantiate the practicality and effectiveness of the calibration method proposed in this study.

Based on an Overview of Crack Detection

This Crack detection, as an important part of concrete structural health monitoring, aims to reflect the stress state and damage degree of the structure. Traditional concrete crack detection mainly relies on manual visual identification, which has low detection efficiency and accuracy. In addition, manual visual detection has problems such as being greatly affected by lighting conditions, being unable to cover overhead locations such as bridge towers and high piers, and being highly subjective. In recent years, in order to solve the above problems, researchers at home and abroad have developed concrete crack detection equipment based on digital image technology, such as inspection vehicles equipped with high-resolution cameras, unmanned aerial vehicles, crawling robots, and so on. Meanwhile, efficient and accurate crack detection algorithms are the basis for realizing accurate crack identification. How to balance the detection speed and accuracy has been the focus of academic attention. This paper reviews the research progress of concrete crack detection equipment based on digital image technology at home and abroad in recent years, including camera platform, calibration method, preprocessing, traditional and deep learning algorithms, crack feature extraction, image stitching, and 3D representation and monitoring of cracks. In addition, the deficiencies in the research are summarized.