Extrinsic Camera Calibration Research Articles

A Review of Deep Learning-Based LiDAR and Camera Extrinsic Calibration

Extrinsic parameter calibration is the foundation and prerequisite for LiDAR and camera data fusion of the autonomous system. This technology is widely used in fields such as autonomous driving, mobile robots, intelligent surveillance, and visual measurement. The learning-based method is one of the targetless calibrating methods in LiDAR and camera calibration. Due to its advantages of fast speed, high accuracy, and robustness under complex conditions, it has gradually been applied in practice from a simple theoretical model in just a few years, becoming an indispensable and important method. This paper systematically summarizes the research and development of this type of method in recent years. According to the principle of calibration parameter estimation, learning-based calibration algorithms are divided into two categories: accurate calibrating estimation and relative calibrating prediction. The evolution routes and algorithm frameworks of these two types of algorithms are elaborated, and the methods used in the algorithms’ steps are summarized. The algorithm mechanism, advantages, limitations, and applicable scenarios are discussed. Finally, we make a summary, pointing out existing research issues and trends for future development.

Study of Underwater Large-Target Localization Based on Binocular Camera and Laser Rangefinder

Currently, for underwater close-range large-target localization, visual localization techniques fail since large targets completely occupy the camera’s field of view at ultraclose ranges. To address the issue, a multi-stage optical localization method combining a binocular camera and a single-point laser rangefinder is proposed in this paper. The proposed method comprises three parts. First, the imaging model of the underwater camera is modified, and a laser rangefinder is used to further correct the underwater calibration results of the binocular camera. Second, YOLOv8 is applied to recognize the targets to prepare for target localization. Third, extrinsic calibration of the binocular camera and laser rangefinder is performed, and a Kalman filter is employed to fuse the target position information measured by the binocular camera and laser rangefinder. The experimental results show that, compared with using a binocular camera alone, the proposed method can accurately and stably locate the target at close ranges with an average error of only 2.27 cm, without the risk of localization failure, and reduces binocular localization error by 90.57%.

Adaptive Point-Line Fusion: A Targetless LiDAR-Camera Calibration Method with Scheme Selection for Autonomous Driving.

Accurate calibration between LiDAR and camera sensors is crucial for autonomous driving systems to perceive and understand the environment effectively. Typically, LiDAR-camera extrinsic calibration requires feature alignment and overlapping fields of view. Aligning features from different modalities can be challenging due to noise influence. Therefore, this paper proposes a targetless extrinsic calibration method for monocular cameras and LiDAR sensors that have a non-overlapping field of view. The proposed solution uses pose transformation to establish data association across different modalities. This conversion turns the calibration problem into an optimization problem within a visual SLAM system without requiring overlapping views. To improve performance, line features serve as constraints in visual SLAM. Accurate positions of line segments are obtained by utilizing an extended photometric error optimization method. Moreover, a strategy is proposed for selecting appropriate calibration methods from among several alternative optimization schemes. This adaptive calibration method selection strategy ensures robust calibration performance in urban autonomous driving scenarios with varying lighting and environmental textures while avoiding failures and excessive bias that may result from relying on a single approach.

Extrinsic Calibration of Camera and LiDAR Systems With Three‐Dimensional Towered Checkerboards

With the increasing utilization of cameras and three‐dimensional Light Detection and Ranging (LiDAR) systems in perception tasks, the fusion of these two sensor modalities has emerged as a prominent research focus in the fields of robotics and unmanned systems. While various extrinsic calibration methods have been developed, they often suffer from limited accuracy when using low‐resolution LiDAR sensors and require the placement of calibration targets at multiple locations. This paper introduces a novel calibration target known as the Three‐Dimensional Towered Checkerboard (3TC), along with a precise and straightforward extrinsic calibration approach for camera‐LiDAR systems. The 3TC consists of stacked cubes adorned with planar or 2D checkerboards, which provide the known positions of checkerboard corner points in three‐dimensional space. Leveraging the Iterative Closest Point (ICP) algorithm, the proposed method calculates the spatial relationship between LiDAR point cloud data and the 3TC model to infer the positions of checkerboard corner points in the LiDAR coordinate system. Subsequently, the Perspective‐n‐Point (PnP) algorithm is employed to establish the correlation between corner positions in the LiDAR coordinate system and the camera image, given the intrinsic parameters of the camera. By ensuring an adequate number of cubes and 2D checkerboards on a specific 3TC, along with accurately estimated corner point positions in LiDAR, a single frame of data from both the camera and LiDAR facilitates their extrinsic calibration. Experimental validations conducted across diverse camera and LiDAR systems, achieving minimal error close to the theoretical limit of the devices, attest to the robustness and precision of the 3TC and the proposed calibration methodology.

GTSCalib: Generalized Target Segmentation for Target-Based Extrinsic Calibration of Non-Repetitive Scanning LiDAR and Camera

GTSCalib: Generalized Target Segmentation for Target-Based Extrinsic Calibration of Non-Repetitive Scanning LiDAR and Camera

Extrinsic Calibration of High Resolution LiDAR and Camera Based on Vanishing Points

Extrinsic Calibration of High Resolution LiDAR and Camera Based on Vanishing Points

Targetless Extrinsic Calibration of Camera and LiDAR via Identifying True 2-D–3-D Line Matching Among Unknown Line Correspondences of Structured Environments

Targetless Extrinsic Calibration of Camera and LiDAR via Identifying True 2-D–3-D Line Matching Among Unknown Line Correspondences of Structured Environments

Automatic LiDAR–Camera Extrinsic Calibration Using Pseudoimage and Multiple Targets

Automatic LiDAR–Camera Extrinsic Calibration Using Pseudoimage and Multiple Targets

Why You Cannot Rank First: Modifications for Benchmarking Six-Degree-of-Freedom Visual Localization Algorithms

Robust and precise visual localization over extended periods of time poses a formidable challenge in the current domain of spatial vision. The primary difficulty lies in effectively addressing significant variations in appearance caused by seasonal changes (summer, winter, spring, autumn) and diverse lighting conditions (dawn, day, sunset, night). With the rapid development of related technologies, more and more relevant datasets have emerged, which has also promoted the progress of 6-DOF visual localization in both directions of autonomous vehicles and handheld devices.This manuscript endeavors to rectify the existing limitations of the current public benchmark for long-term visual localization, especially in the part on the autonomous vehicle challenge. Taking into account that autonomous vehicle datasets are primarily captured by multi-camera rigs with fixed extrinsic camera calibration and consist of serialized image sequences, we present several proposed modifications designed to enhance the rationality and comprehensiveness of the evaluation algorithm. We advocate for standardized preprocessing procedures to minimize the possibility of human intervention influencing evaluation results. These procedures involve aligning the positions of multiple cameras on the vehicle with a predetermined canonical reference system, replacing the individual camera positions with uniform vehicle poses, and incorporating sequence information to compensate for any failed localized poses. These steps are crucial in ensuring a just and accurate evaluation of algorithmic performance. Lastly, we introduce a novel indicator to resolve potential ties in the Schulze ranking among submitted methods. The inadequacies highlighted in this study are substantiated through simulations and actual experiments, which unequivocally demonstrate the necessity and effectiveness of our proposed amendments.

A Method for Synchronous Automated Extrinsic Calibration of LiDAR and Cameras Based on a Circular Calibration Board

A Method for Synchronous Automated Extrinsic Calibration of LiDAR and Cameras Based on a Circular Calibration Board

LCE-Calib: Automatic LiDAR-Frame/Event Camera Extrinsic Calibration With a Globally Optimal Solution

The combination of light detection and rangings (LiDARs) and cameras enables a mobile robot to perceive environments with multimodal data, becoming a key factor in achieving robust perception. Traditional frame cameras are sensitive to changing illumination conditions, motivating us to introduce novel event cameras to make LiDAR-camera fusion more complete and robust. However, to jointly exploit these sensors, the challenging extrinsic calibration problem should be addressed. This article proposes an automatic checkerboard-based approach to calibrate extrinsics between a LiDAR and a frame/event camera, where the following four contributions are presented: 1) we present an automatic feature extraction and checkerboard tracking method from LiDAR's point clouds; 2) we reconstruct realistic frame images from event streams, applying traditional corner detectors to event cameras; 3) we propose an initialization-refinement procedure to estimate extrinsics using point-to-plane and point-to-line constraints in a coarse-to-fine manner; 4) we introduce a unified and globally optimal solution to address two optimization problems in calibration. Our approach has been validated with extensive experiments on 19 simulated and real-world datasets and outperforms the state-of-the-art.

Extrinsic calibration of 2D laser range finder and camera using photogrammetric test field and point-on-plane constraint

Extrinsic calibration of 2D laser range finder and camera using photogrammetric test field and point-on-plane constraint

Targetless Extrinsic Calibration of Camera and Low-Resolution 3-D LiDAR

Autonomous driving heavily relies on LiDAR and camera sensors, which can significantly improve the performance of perception and navigation tasks when fused together. The success of this cross-modality fusion hinges on accurate extrinsic calibration. In recent years, targetless LiDAR-camera calibration methods have gained increasing attention thanks to their independence from external targets. Nevertheless, developing a targetless method for low-resolution LiDARs remains challenging due to the difficulty in extracting reliable features from point clouds with limited LiDAR beams. In this paper, we propose a robust targetless method to solve this struggling problem. It can automatically estimate accurate LiDAR and camera poses and solve the extrinsic matrix through hand-eye calibration. Moreover, we also carefully analyze pose estimation issues existing in the low-resolution LiDAR and present our solution. Real-world experiments are carried out on a UGV-mounted multi-sensor platform containing a CCD camera and a VLP-16 LiDAR. For evaluation, we use a state-of-the-art target-based calibration approach to generate the ground-truth extrinsic parameters. Experimental results demonstrate that our method achieves low calibration error in both translation (3 cm) and rotation (0.59 degrees).

Automatic extrinsic calibration of terrestrial laser scanner and digital camera by MoG image correlation

Automatic extrinsic calibration of terrestrial laser scanner and digital camera by MoG image correlation

외부 시스템을 이용한 Monocular camera-3D LiDAR 융합 시스템의 Non-overlapping 외부 교정 방법에 대한 연구

This study presents a novel framework for the extrinsic calibration of monocular camera and LiDAR with a non-overlapping field of view. The conventional non-overlapping extrinsic calibration utilizes a motion-based calibration method. However, motion-based calibration has a disadvantage in that the vehicle must be moving. In order to solve extrinsic calibration problems under such constraint configurations, the proposed solution uses external sensors that are located away from vehicle. First, the external camera and vehicle camera utilize an aruco marker. Second, the external LiDAR and vehicle LiDAR utilize a scan matching method. In this study, a Gazebo simulator was utilized to show the accuracy of the proposed method. The external LiDAR used Velodyne HDL 64ch, the vehicle LiDAR used Puck 16ch, and the camera used the Blackfly color sensor from FLIR.

CamMap: Extrinsic Calibration of Non-Overlapping Cameras Based on SLAM Map Alignment

Multiple cameras have emerged as a promising technology for robots and vehicles due to their broad fields of view (FoV) and high resolution. However, there are often limited or no overlapping FoVs among cameras, bringing challenges to estimating extrinsic camera parameters. To overcome this problem, we propose CamMap: a novel 6-degree-of-freedom (DoF) extrinsic calibration pipeline. Following three operating rules, we make a multi-camera rig capture some similar image sequences individually to create sparse feature-based maps with a SLAM system. A two-stage optimization problem is formulated to align the maps and obtain the transformations between them based on bidirectional reprojection. The transformations are exactly the extrinsic parameters. Supporting diverse camera types, the pipeline is available in any texture-rich environment. It can calibrate any number of cameras simultaneously without requiring calibration patterns, synchronization, same resolution and frequency. The pipeline is evaluated on cameras with limited and no overlapping FoVs. In the experiments, we demonstrate our method's accuracy and efficiency. The absolute pose error (APE) between Kalibr and CamMap is less than 0.025.

A Novel and Simplified Extrinsic Calibration of 2D Laser Rangefinder and Depth Camera

It is too difficult to directly obtain the correspondence features between the two-dimensional (2D) laser-range-finder (LRF) scan point and camera depth point cloud, which leads to a cumbersome calibration process and low calibration accuracy. To address the problem, we propose a calibration method to construct point-line constraint relations between 2D LRF and depth camera observational features by using a specific calibration board. Through the observation of two different poses, we construct the hyperstatic equations group based on point-line constraints and solve the coordinate transformation parameters of 2D LRF and depth camera by the least square (LSQ) method. According to the calibration error and threshold, the number of observation and the observation pose are adjusted adaptively. After experimental verification and comparison with existing methods, the method proposed in this paper easily and efficiently solves the problem of the joint calibration of the 2D LRF and depth camera, and well meets the application requirements of multi-sensor fusion for mobile robots.

A4LidarTag: Depth-Based Fiducial Marker for Extrinsic Calibration of Solid-State Lidar and Camera

Visual-based simultaneous localization and mapping (SLAM) systems perform weakly in object tracking and map reconstruction due to the unreliable depth measurement originating from image-only data. Light Detection and Ranging (LiDAR) can be coupled to overcome the drawback of uncertain depth estimation. The prerequisite for performing data fusion is to align visual-Lidar sensors to a specific coordinate system with extrinsic pose by calibrating. The conventional extrinsic calibration frameworks either rely on markers in artificial large-size calibration boards or uncontrollable natural scenes (Fig. 2 ), limiting stability and convenience. In this paper, we have designed a novel marker pattern, A4LidarTag, composed of circular holes. The difference in depth measurement is used to encode location information. Based on A4LidarTag, the automatic extrinsic calibration framework between solid-state Lidar (SSL) and the camera is developed. The proposed framework can be implemented in close range (within 1 m) and on an A4-size calibration board. The average reprojection error resulting from Lidar point clouds projection is about 0.12pixels. Experiments show excellent efficiency and versatility in both indoor and outdoor scenes.

Continuous-Time Spatiotemporal Calibration of a Rolling Shutter Camera-IMU System

The Rolling Shutter (RS) mechanism is widely used in consumer-grade cameras, which are essential parts in smartphones and autonomous vehicles. RS leads to image distortion when the camera moves relative to the scene while capturing images. This effect needs to be considered in structure-from-motion, and vision-aided odometry, for which recent studies have extended Global Shutter (GS) methods to RS cameras. So far, it has been unclear how the RS affects spatiotemporal calibration of the camera in a multi-sensor system, which is crucial for good performance of aforementioned applications. This work takes the camera-IMU system as an example and looks into how the RS affects its spatiotemporal calibration. We develop a continuous-time calibration method for a RS camera-IMU system based on basis splines by using a calibration target. Considering the RS, every observation of a landmark on the target corresponds to a unique camera pose fitted by B-splines. Tests conducted on simulated data generated from public calibration datasets showed that the RS can noticeably affect accuracy of the camera extrinsic parameters, causing errors about 0.5° in orientation and 2 cm in translation with a setting common to smartphone cameras. With real data collected by two industrial camera-IMU systems, we found that considering the RS effect gives more accurate and consistent spatiotemporal calibration. These tests also revealed the impact of alternative IMU models and of the continuous-time versus discrete-time representation on camera extrinsic calibration, and the degradation of visual inertial odometry by extrinsic calibration errors.

Pixel-Level Extrinsic Self Calibration of High Resolution LiDAR and Camera in Targetless Environments

In this letter, we present a novel method for automatic extrinsic calibration of high-resolution LiDARs and RGB cameras in targetless environments. Our approach does not require checkerboards but can achieve pixel-level accuracy by aligning natural edge features in the two sensors. On the theory level, we analyze the constraints imposed by edge features and the sensitivity of calibration accuracy with respect to edge distribution in the scene. On the implementation level, we carefully investigate the physical measuring principles of LiDARs and propose an efficient and accurate LiDAR edge extraction method based on point cloud voxel cutting and plane fitting. Due to the edges' richness in natural scenes, we have carried out experiments in many indoor and outdoor scenes. The results show that this method has high robustness, accuracy, and consistency. It can promote the research and application of the fusion between LiDAR and camera. We have open sourced our code on GitHub <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> to benefit the community.