Extrinsic Calibration Parameters Research Articles

Online extrinsic parameters calibration of on-board stereo cameras based on certifiable optimization

The extrinsic parameters of on-board stereo cameras can be slightly altered due to temperature fluctuations, vibrations, and accidental impacts during automobile driving, leading to significant performance loss in dense stereo matching. In this paper, we propose an online calibration method based on certifiable optimization to address this issue. Initially, sparse feature points are collected based on plane distribution and disparity. A robust optimization model is then developed to minimize the epipolar error, utilizing iterative local optimization to eliminate outliers and determine the 5DOF extrinsic parameters. Subsequently, a relaxation problem is constructed using inliers, and global optimization is performed to certify that the locally optimal results are indeed globally optimal. Comparative experimental results demonstrate that the proposed method offers high accuracy and reliability. Additionally, the quality of the disparity map generated by our calibration method is comparable to that achieved through offline calibration.

Camera LiDAR calibration: an automatic and accurate method with novel PLE metrics

Abstract Camera-LiDAR calibration is a crucial aspect of perception systems and sensor fusion in various fields, facilitating the fusion of data from cameras and LiDAR sensors for applications such as autonomous vehicles, robotics, and augmented reality. In this paper, we presented a novel multi-modal fusion method that introduces an efficient camera-LiDAR joint calibration technique using a simple checkerboard. Our method estimates the 3D rigid transformation of the camera system with respect to the LiDAR frame by establishing 2D-to-3D point correspondences for geometric calibration. We improved the idea of the traditional PNP (Perspective-N-Point) algorithm by employing point cloud segmentation and clustering to re-match the checkerboard. When compared with the calibration methods in the Autoware and MATLAB calibration toolbox, the reprojection error was improved by 38.13\% and 58.30\% respectively. The average translation error was improved by 78.26\% and 51.61\% respectively, while the average rotation error is improved by 75\% and 60.78\% respectively, which has a great improvement in all three areas. At the same time, we propose a new evaluation metric for joint calibration results based on reprojected images: PLE (Pixel-Level Evaluation) index to reflect the accuracy of the joint calibration of the camera and LiDAR. An automatic calibration software has been developed for the calibration of the camera's intrinsic parameters as well as for the joint calibration of the camera-LIDAR extrinsic parameters. We have extensively validated our algorithm using the Intel RealSense Depth D435 Camera and LeiShen C16-151B LiDAR.

A Review of Deep Learning-Based LiDAR and Camera Extrinsic Calibration

Extrinsic parameter calibration is the foundation and prerequisite for LiDAR and camera data fusion of the autonomous system. This technology is widely used in fields such as autonomous driving, mobile robots, intelligent surveillance, and visual measurement. The learning-based method is one of the targetless calibrating methods in LiDAR and camera calibration. Due to its advantages of fast speed, high accuracy, and robustness under complex conditions, it has gradually been applied in practice from a simple theoretical model in just a few years, becoming an indispensable and important method. This paper systematically summarizes the research and development of this type of method in recent years. According to the principle of calibration parameter estimation, learning-based calibration algorithms are divided into two categories: accurate calibrating estimation and relative calibrating prediction. The evolution routes and algorithm frameworks of these two types of algorithms are elaborated, and the methods used in the algorithms’ steps are summarized. The algorithm mechanism, advantages, limitations, and applicable scenarios are discussed. Finally, we make a summary, pointing out existing research issues and trends for future development.

Camera-IMU extrinsic calibration method based on intermittent sampling and RANSAC optimization

Accurately calibrate the extrinsic parameters between a camera and an Inertial Measurement Unit (IMU) is a prerequisite for achieving sensor data fusion in visual-inertial navigation systems. Due to delays introduced by triggering, transmission, and other factors, the sampled times of the camera and IMU do not align with the system timestamps, leading to a decrease in the accuracy of extrinsic calibration. Existing calibration methods are unable to avoid the temporal misalignment between the camera and IMU. This paper introduces a novel intermittent calibration sampling method to address temporal misalignment challenges. The approach entails detecting motion-stillness transition points in the IMU data as keyframe segmentation criteria, thereby decoupling data acquisition from system timestamps and aligning it with the sensor’s motion process. Subsequently, the robot hand-eye calibration method is applied to the extrinsic calibration of the Camera-IMU, combining the Random Sample Consensus algorithm to filter the poses of the camera and IMU. This process achieves precise calibration of extrinsic parameters. Experimental validation shows that the algorithm proposed in this paper enhances calibration accuracy when compared to the widely used Kalibr Camera-IMU calibration toolbox. The rotational extrinsic parameter accuracy increased by 181.25%, and the translational extrinsic parameter accuracy improved by 168.33%. These findings provide effective technical means for precise measurement of Camera-IMU extrinsic parameters.

High-recall calibration monitoring for stereo cameras

Cameras are the prevalent sensors used for perception in autonomous robotic systems, but their initial calibration may degrade over time due to dynamic factors. This may lead to a failure of downstream tasks, such as simultaneous localization and mapping (SLAM) or object recognition. Hence, a computationally lightweight process that detects the decalibration is of interest. We describe a modification of StOCaMo, an online calibration monitoring procedure for a stereoscopic system. The method uses robust kernel correlation based on epipolar constraints; it validates extrinsic calibration parameters on a single frame with no temporal tracking. In this paper, we present a modified StOCaMo with an improved recall rate on small decalibrations through a confirmation technique based on resampled variance. With fixed parameters learned on a realistic synthetic dataset from CARLA, StOCaMo and its proposed modification were tested on multiple sequences from two real-world datasets: KITTI and EuRoC MAV. The modification improved the recall of StOCaMo by 25 % (to 91 % and 82 %, respectively), and the accuracy by 12 % (to 94.7 % and 87.5 %, respectively), while labeling at most one-third of the input data as uninformative. The upgraded method achieved the rank correlation between StOCaMo V-index and downstream SLAM error of 0.78 (Spearman).

Online Calibration of Extrinsic Parameters for Solid-State LIDAR Systems.

This work addresses the challenge of calibrating multiple solid-state LIDAR systems. The study focuses on three different solid-state LIDAR sensors that implement different hardware designs, leading to distinct scanning patterns for each system. Consequently, detecting corresponding points between the point clouds generated by these LIDAR systems-as required for calibration-is a complex task. To overcome this challenge, this paper proposes a method that involves several steps. First, the measurement data are preprocessed to enhance its quality. Next, features are extracted from the acquired point clouds using the Fast Point Feature Histogram method, which categorizes important characteristics of the data. Finally, the extrinsic parameters are computed using the Fast Global Registration technique. The best set of parameters for the pipeline and the calibration success are evaluated using the normalized root mean square error. In a static real-world indoor scenario, a minimum root mean square error of 7 cm was achieved. Importantly, the paper demonstrates that the presented approach is suitable for online use, indicating its potential for real-time applications. By effectively calibrating the solid-state LIDAR systems and establishing point correspondences, this research contributes to the advancement of multi-LIDAR fusion and facilitates accurate perception and mapping in various fields such as autonomous driving, robotics, and environmental monitoring.

Automatic calibration and association for roadside radar and camera based on fluctuating traffic volume

Accurate perception of the movement and appearance of vehicles depends on the robustness and reliability of the extrinsic parameters calibration in a multi-sensor fusion scenario. However, conventional calibration methods require manual acquisition of prior information, leading to high labor costs and low calibration accuracy. Therefore, we proposed an automatic coarse-to-fine calibration method for roadside radar and camera sensors to lower costs and improve accuracy. Next, an association strategy based on fluctuating traffic volumes was also developed to assist in robust target matching during the coarse-to-fine calibration process. Finally, extrinsic parameters between the radar coordinate system and camera coordinate system were calibrated through double rotations of the position vectors obtained from each system. To verify the proposed method, an experiment was conducted on a pedestrian bridge using an uncalibrated 4D millimeter-wave radar and a traffic monocular camera. The results showed that our proposed method reduced the interquartile range of the roll angle by 41.5% compared to a state-of-the-art neural network method. It also outperformed the manual calibration method by 2.47% in terms of the average reprojection error.

Imaging radar and LiDAR image translation for 3-DOF extrinsic calibration

The integration of sensor data is crucial in the field of robotics to take full advantage of the various sensors employed. One critical aspect of this integration is determining the extrinsic calibration parameters, such as the relative transformation, between each sensor. The use of data fusion between complementary sensors, such as radar and LiDAR, can provide significant benefits, particularly in harsh environments where accurate depth data is required. However, noise included in radar sensor data can make the estimation of extrinsic calibration challenging. To address this issue, we present a novel framework for the extrinsic calibration of radar and LiDAR sensors, utilizing CycleGAN as a method of image-to-image translation. Our proposed method employs translating radar bird-eye-view images into LiDAR-style images to estimate the 3-DOF extrinsic parameters. The use of image registration techniques, as well as deskewing based on sensor odometry and B-spline interpolation, is employed to address the rolling shutter effect commonly present in spinning sensors. Our method demonstrates a notable improvement in extrinsic calibration compared to filter-based methods using the MulRan dataset.

Automatic Targetless Monocular Camera and LiDAR External Parameter Calibration Method for Mobile Robots

With the continuous development and popularization of sensor-fusion technology for mobile robots, the application of camera and light detection and ranging (LiDAR) fusion perception has become particularly important. Moreover, the calibration of extrinsic parameters between the camera and LiDAR is a crucial prerequisite for fusion. Although traditional target-based calibration methods have been widely adopted, their cumbersome operation and high costs necessitate the development of more efficient and flexible calibration methods. To address this problem, this study proposed a two-stage calibration method based on motion and edge matching. In the first stage, the preliminary estimation of the extrinsic parameters between the camera and LiDAR was performed by matching visual odometry and LiDAR odometry using a hand–eye target method. In the second stage, the calibration results from the first stage were further refined by matching the image edges and discontinuous depth point clouds. The calibration system was then tested in both simulated and actual environments. The experimental results showed that this method, which did not require specially structured targets, could achieve highly flexible and robust automated calibration. Compared to other advanced methods, the accuracy of the proposed method was higher.

FusionCalib: Automatic extrinsic parameters calibration based on road plane reconstruction for roadside integrated radar camera fusion sensors

Calibrating the relative positional posture of the roadside sensor with respect to the road is crucial for target tracking and sensor fusion. However, most previous studies rely on prior information or require additional manual measurements, which increases calibration error and labor costs. This paper proposes an automatic extrinsic parameters calibration method for roadside integrated radar camera fusion sensors for the first time. In particular, we leverage the radar camera fusion framework to automatically reconstruct the road plane. Specifically, to integrate sensing abilities, a stable bidirectional selection association method is adopted to match radar and camera trajectories. Based on the association results, the Extended Kalman Filter (EKF) is adopted to fuse pixel positions and three-dimensional (3D) velocities of vehicles, and then a series of 3D positions of vehicle bottoms are obtained to reconstruct the road plane. Experimental results show that our proposed method achieves automatic and accurate calibration of extrinsic parameters.

A fast and accurate camera-IMU calibration method for localization system

<p>Autonomous driving has spurred the development of sensor fusion techniques, which combine data from multiple sensors to improve system performance. In particular, a localization system based on sensor fusion, such as Visual Simultaneous Localization and Mapping (VSLAM), plays a crucial role in environment perception and serves as the foundation for decision-making and motion control in intelligent vehicles. The accuracy of extrinsic calibration parameters between the camera and IMU is of utmost importance for precise positioning in VSLAM systems. However, existing calibration methods are often time-consuming, rely on complex optimization techniques, and are sensitive to noise and outliers, leading to potential degradation in system performance. To address these challenges, this paper presents a fast and accurate camera-IMU calibration method based on space coordinate transformation constraints and SVD (Singular Value Decomposition) tricks. The method involves constructing constraint equations by ensuring the equality of rotation and transformation matrices between camera frames and IMU coordinates at different time instances. Subsequently, the external parameters of the camera-IMU system are solved using quaternion transformation and SVD techniques. To validate the proposed method, experiments were conducted using the ROS (Robot Operating System) platform, where camera images and velocity, acceleration, and angular velocity data from the IMU were recorded in a ROS bag file. The results demonstrate that the proposed method achieves reliable camera-IMU calibration parameters, requiring less tuning time and exhibiting reduced uncertainty.</p>

A non-coplanar high-precision calibration method for cameras based on an affine coordinate correction model

Traditional non-coplanar calibration methods such as Tsai’s method have many problems, such as insufficient calibration accuracy, inconvenient operation, inaccurate models, etc. This paper proposes a new high-precision non-coplanar calibration method that aims to solve these problems. Like Tsai’s method, the proposed calibration method utilizes a one-dimensional displacement stage and a two-dimensional plane target to generate a virtual 3D feature point sequence. As an improvement, an affine coordinate correction model is applied to ensure the accuracy and orthogonality of the obtained virtual 3D coordinates. A novel and accurate camera calibration model is further established. Compared with Tsai’s model, which uses a radial alignment constraint and ignores the orthonormal constraint of the rotation matrix, the proposed calibration model fully considers the degrees of freedom of the camera’s parameters to be calibrated, as well as the lens’s nonlinear distortion parameters. More accurate analytical solutions of intrinsic and extrinsic parameters can be obtained with the proposed calibration model. Finally, a novel high-precision non-coplanar calibration method is proposed based on the proposed calibration model. The reprojection experiment proves that the calibration accuracy of this calibration method is better than that of Tsai’s and Zhang’s calibration methods under the same calibration conditions. As a supplement, a novel binocular camera system extrinsic parameter calibration method with known intrinsic parameters is proposed. With accurate intrinsic and extrinsic parameters, the binocular camera system’s relative measurement accuracy could be within 1/10 000. Overall, this method can be used in experimental and industrial applications that require high-precision calibration parameters.

High-Accuracy Patternless Calibration of Multiple 3-D LiDARs for Autonomous Vehicles

This paper proposes a new method for estimating the extrinsic calibration parameters between any pair of multi-beam LiDAR sensors on a vehicle. Unlike many state-of-the-art works, this method does not use any calibration pattern or reflective marks placed in the environment to perform the calibration; in addition, the sensors do not need to have overlapping fields of view. An iterative ICP-based process is used to determine the values of the calibration parameters, resulting in better convergence and improved accuracy. Furthermore, a setup based on the CARLA simulator is introduced to evaluate the approach, enabling quantitative assessment with ground-truth data. The results show an accuracy comparable with other approaches that require more complex procedures and have a more restricted range of applicable setups. This work also provides qualitative results on a real setup, where the alignment between the different point clouds can be visually checked. The open-source code is available at https://github.com/midemig/pcd_calib.

Extrinsic calibration of a camera and a 2D laser range finder using ping pong balls and the corner of a room

Three different methods for extrinsic calibration of a 2D laser range finder and a camera using ping pong balls and the corner of a room are presented and compared with four recent state-of-the-art methods. In the first method, a servo motor and gimbal were used to rotate the laser rangefinder to produce a 3D point cloud from the test field. In the second method, the laser range finder was solely used to get a 2D point cloud of the ping pong balls. In the third approach by employing the pyramid that was built in the room's corner and resolving the perspective-three-point problem, the 2D LRF calibration parameters with respect to the test field were determined with just one scan. In comparison to the estimated values produced by the third approach, the extrinsic calibration parameters derived with the 3D point cloud were more accurate. The second approach produced the worst outcomes, although it was the most straightforward approach.

An incremental LiDAR/POS online calibration method

Extrinsic Calibration between LiDAR and POS (position and orientation system) is a fundamental prerequisite for varieties of MLS (mobile laser scanner) applications. Due to the sparse structure of LiDAR data, the current calibration methods relying on common point feature matching are unreliable, and the low accuracy POS results make the extrinsic calibration of MLS system more challenging. In this paper, we propose an incremental estimation method of six degree of freedom extrinsic transformation of LiDAR and POS. Firstly, the POS-SLAM is used to accumulate LiDAR scans as online sub maps. Attitudes of the carrier are calculated by using GNSS/INS loose combination method of bidirectional adjustment, and scans are associated with sub map based on the time interpolation. Then, the extrinsic calibration parameters are estimated by optimizing corresponding points difference between SLAM and MLS coordinate frame. Finally, field tests have been conducted to the proposed method. RMS between the map by the calibrated MLS and by the static measurement is 0.57 cm. The results demonstrate that the accuracy and robustness of our calibration approach are sufficient for mapping requirement of MLS.

Remote 3D Displacement Sensing for Large Structures with Stereo Digital Image Correlation

The work performance of stereo digital image correlation (stereo-DIC) technologies, especially the operating accuracy and reliability in field applications, is not fully understood. In this study, the key technologies of the field remote 3D displacement sensing of civil structures based on stereo-DIC have been proposed. An image correlation algorithm is incorporated in improving the matching accuracy of control points. An adaptive stereo-DIC extrinsic parameter calibration method is developed by fusing epipolar-geometry-based and homography-based methods. Furthermore, a reliable reference frame that does not require artificial markers is established based on Euclidean transformation, which facilitates in-plane and out-of-plane displacement monitoring for civil structures. Moreover, a camera motion correction is introduced by considering background points according to the camera motion model. With an experiment, the feasibility and accuracy of the proposed system are validated. Moreover, the system is applied to sense the dynamic operating displacement of a 2 MW wind turbine’s blades. The results show the potential capability of the proposed stereo-DIC system in remote capturing the full-field 3D dynamic responses and health status of large-scale structures.

Robot-mounted sensing and local calibration for high-accuracy manufacturing

Positional accuracy is critically important for application of industrial robots in fixtureless precision manufacturing; thus, this paper presents a method for simultaneously calibrating the kinematic parameters of robot and robot-mounted measurement devices. In the modeling step, the Modified Denavit–Hartenberg parameters are extended to include the world and tool frames within a minimal parameter set using a systematic approach. Closed-loop calibration is performed using the data collected in the local target workspace using the robot-mounted measurement device. The presented modeling and calibration methods are applied to a case study in which a machining task is performed using a DENSO VS-6556 W industrial robot equipped with a cutting tool, force sensor, and laser profile scanner. In the case study, local models are calibrated for the laser and tool frames using common 1-2-3 blocks as calibration artifacts . The robot/laser model is identified by scanning the artifacts with a robot-mounted laser scanner , and the robot/tool model is calibrated by touching the artifacts with a robot-mounted force sensor. In this application, the robot/laser model is used to scan and register a sheet metal workpiece, and the robot/tool model is used to plan a tool path to machine the periphery. Positional accuracy of the tool path was evaluated to be ± 0.15 mm within the target workspace, which shows significant improvement compared with the mean error of 0.8 mm achieved by global calibration of the robot/tool model. • Simultaneous calibration of robot and extrinsic parameters of robot-mounted sensors. • Closed-loop calibration without external measurement devices. • Local calibration to improve accuracy within target workspace. • Systematic approach to include world and tool/sensor frame in MDH parameter set. • Case study for high-accuracy robotic machining application.

EXTRINSIC CALIBRATION OF ROTATING 2D LASER RANGE FINDER AND CAMERA USING PHOTOGRAMMETRIC TEST FIELD AND PING PONG BALLS

Abstract. In this article, a method for the extrinsic calibration of a 2D laser range finder and a camera is presented. This technique produces a 3D point cloud from the test field by connecting a laser range finder to a servomotor. In this study, ping balls and standard photogrammetric targets were employed as a test field. Ping pong balls are used because they can be easily recognized in data from laser range finder and camera. To calculate extrinsic calibration parameters between a camera and laser range finder, these balls are employed as control points in the data. The extrinsic calibration of the Laser range finder and camera is carried out using the point cloud created from the test field and the photos captured from the test field. In this method, a sphere is fitted to each ping pong ball's points in the 3D point cloud, and the coordinates of that sphere's center are taken to be the coordinates of that ball. By measuring the distances between various targets in the test field, the scale can be resolved. This approach was compared with another state-of-the-art method. The proposed method is more accurate and stable than the alternative way, taking into account the average inaccuracy of check points.

Two-step calibration method of the extrinsic parameters with high accuracy for a bistatic non-orthogonal shafting laser theodolite system.

Motivated by the increasing demands on the precision of 3D large-scale measurement, the extrinsic parameters calibration with high accuracy of the bistatic non-orthogonal shafting laser theodolite (N-theodolite) system is required. A two-step method is proposed to achieve the extrinsic parameters calibration with high accuracy in this paper. In the first step, by analyzing and setting the approximate emitted point during the motion of the laser axis in local space, the calculation of the initial extrinsic parameters can be simplified. In the second step, the above results are taken as the initial values of optimization, and the distances between the spatial laser points provided by PSD sensors with high accuracy in global space are used to construct the unconstrained optimal objective function. The proposed method is validated with the measurement experiment of the bistatic N-theodolite system, the average error of 3D coordinate measurement is less than 0.4 mm, and the average error of distance measurement is less than 0.3 mm within 5 m.

3D-CALI: Automatic calibration for camera and LiDAR using 3D checkerboard

Accurate and robust calibration of intrinsic and extrinsic parameters is crucial to multi-sensor simultaneous localization and mapping (SLAM) systems. However, extrinsic calibration of heterogeneous sensors is challenging owing to the differences in data acquisition methods. Therefore, we propose a more robust and convenient method to calibrate the intrinsic parameters for the camera and extrinsic parameters for cameras and light detection and ranging (LiDAR) sensors based on a 3D checkerboard. We tested the algorithm in simulation and real-world environments. Experiments in the simulation environment showed that our proposed method had similar accuracy as the state-of-the-art calibration method velo2cam_calibration.11https://github.com/beltransen/velo2cam_calibration. However, our proposed algorithm had less interaction and was more automatic. The real-world test also confirmed the results of our proposed method. Thus, the proposed method is a convenient means to calibrate intrinsic parameters for cameras, and extrinsic parameters for cameras and LiDAR sensors based on a 3D checkerboard.