Automatic Extrinsic Calibration Research Articles

Automatic extrinsic calibration for structured light camera and repetitive LiDARs

Abstract The integration of camera and LiDAR technologies has the potential to significantly enhance construction robots’ perception capabilities by providing complementary construction information. Structured light cameras (SLCs) are a desirable alternative as they provide comprehensive information on construction defects. However, fusing these two types of information depends largely on the sensors’ relative positions, which can only be established through extrinsic calibration. This paper introduces a novel calibration algorithm considering a customized board for SLCs and repetitive LiDARs, which are designed to facilitate the automation of construction robots. The calibration board is equipped with four symmetrically distributed hemispheres, whose centers are obtained by fitting the spheres and adoption with the geometric constraints. Subsequently, the spherical centers serve as reference features to estimate the relationship between the sensors. These distinctive features enable our proposed method to only require one calibration board pose and minimize human intervention. We conducted both simulation and real-world experiments to assess the performance of our algorithm. And the results demonstrate that our method exhibits enhanced accuracy and robustness.

Automatic extrinsic calibration of terrestrial laser scanner and digital camera by MoG image correlation

Automatic extrinsic calibration of terrestrial laser scanner and digital camera by MoG image correlation

Automatic Targetless Calibration for LiDAR and Camera Based on Instance Segmentation

In autonomous vehicles, accurate extrinsic calibration for LiDAR and camera is an essential prerequisite for multi-sensor information fusion. Automatic and targetless extrinsic calibration has become the mainstream of academic research in recent years. However, existing automatic calibration methods that rely on edge or semantic features are unrobust, or require specific scene settings. In this paper, instance segmentation is used for automatic extrinsic calibration of the LiDAR and camera for the first time. Key targets from the segmented instances are extracted and correlated. Regarding the extrinsic calibration as an optimization problem, a novel cost function based on the matching degree of the appearance and centroids from the key targets of the point cloud and image pairs is formulated. Subsequently, differential evolution is used to minimize the cost function to obtain the optimal extrinsic parameters. Extensive experiments on the KITTI dataset and Waymo Open Dataset demonstrate the accuracy and robustness of the proposed method. The MAE of rotation and translation is less than 0.3 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$ ^{\circ }$</tex-math></inline-formula> and 0.05 m respectively, which outperforms semantic-based and edge-based approaches in terms of accuracy.

Automatic Extrinsic Calibration of a Camera and a 2D LiDAR With Point-Line Correspondences

Automatic Extrinsic Calibration of a Camera and a 2D LiDAR With Point-Line Correspondences

Automatic Extrinsic Calibration of Dual LiDARs With Adaptive Surface Normal Estimation

Solutions equipped with multiple light detection and ranging (LiDAR) systems have been widely used in several fields including mobile mapping, navigation, robot, and others. Accurate and robust extrinsic calibration between multiple scanners is necessary for the integration of point cloud data. An automatic method for the calibration of dual LiDARs with adaptive surface normal estimation is presented in this article. Specifically, this approach begins with environment detection from different positions and attitudes. A novel surface normal estimation method is conducted to take account of the uneven distribution of point cloud density and the edge information of planes. Finally, the calibration parameters are calculated by iteratively minimizing the cost function that consists of implicit point-to-plane distances. The experimental results on simulation and real-world data demonstrate that for different types of LiDAR, the proposed algorithm can achieve high-accuracy calibration in different scenes, without manual intervention. The rotation and translation calibration errors between Velodyne LiDARs are less than 1° and 0.02 m, respectively.

Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups

Most sensor setups for onboard autonomous perception are composed of LiDARs and vision systems, as they provide complementary information that improves the reliability of the different algorithms necessary to obtain a robust scene understanding. However, the effective use of information from different sources requires an accurate calibration between the sensors involved, which usually implies a tedious and burdensome process. We present a method to calibrate the extrinsic parameters of any pair of sensors involving LiDARs, monocular or stereo cameras, of the same or different modalities. The procedure is composed of two stages: first, reference points belonging to a custom calibration target are extracted from the data provided by the sensors to be calibrated, and second, the optimal rigid transformation is found through the registration of both point sets. The proposed approach can handle devices with very different resolutions and poses, as usually found in vehicle setups. In order to assess the performance of the proposed method, a novel evaluation suite built on top of a popular simulation framework is introduced. Experiments on the synthetic environment show that our calibration algorithm significantly outperforms existing methods, whereas real data tests corroborate the results obtained in the evaluation suite. Open-source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/beltransen/velo2cam_calibration</uri> .

Accurate and Automatic Extrinsic Calibration for a Monocular Camera and Heterogenous 3D LiDARs

Nowadays, portable cameras and LiDARs are widely applied in robotic applications for environment perception, path planning, and high precision navigation. In such cases, accurate inter-sensor spatial transformation is critical to fuse these sensors seamlessly. However, the feasible and efficient extrinsic calibration of the LiDAR-camera sensor pair is still challenging as it is hard to establish the common feature correspondence between sparse LiDAR point clouds and monocular images. In this contribution, we design a novel calibration board with checkerboard grids and circular holes, through which the proper extrinsic parameters can be obtained automatically by matching the circular hole centers extracted from both images and heterogenous LiDAR scans. Profiting from the versatility and stability of circular hole extraction, the proposed calibration method is suitable for LiDARs with different precision and scanning modes, such as Velodyne LiDAR, Ouster LiDAR, and Livox LiDAR. Both simulation tests and real-world experiments were conducted to evaluate the effectiveness of the proposed calibration method. The simulated experiments indicate that the translation error is less than 0.5 cm, and the rotation error is well below 0.3 degrees. As for real-world experiments, the re-projection error can achieve sub-pixel level with different LiDARs.

A4LidarTag: Depth-Based Fiducial Marker for Extrinsic Calibration of Solid-State Lidar and Camera

Visual-based simultaneous localization and mapping (SLAM) systems perform weakly in object tracking and map reconstruction due to the unreliable depth measurement originating from image-only data. Light Detection and Ranging (LiDAR) can be coupled to overcome the drawback of uncertain depth estimation. The prerequisite for performing data fusion is to align visual-Lidar sensors to a specific coordinate system with extrinsic pose by calibrating. The conventional extrinsic calibration frameworks either rely on markers in artificial large-size calibration boards or uncontrollable natural scenes (Fig. 2 ), limiting stability and convenience. In this paper, we have designed a novel marker pattern, A4LidarTag, composed of circular holes. The difference in depth measurement is used to encode location information. Based on A4LidarTag, the automatic extrinsic calibration framework between solid-state Lidar (SSL) and the camera is developed. The proposed framework can be implemented in close range (within 1 m) and on an A4-size calibration board. The average reprojection error resulting from Lidar point clouds projection is about 0.12pixels. Experiments show excellent efficiency and versatility in both indoor and outdoor scenes.

Automatic Extrinsic Calibration of 3D LIDAR and Multi-Cameras Based on Graph Optimization.

In recent years, multi-sensor fusion technology has made enormous progress in 3D reconstruction, surveying and mapping, autonomous driving, and other related fields, and extrinsic calibration is a necessary condition for multi-sensor fusion applications. This paper proposes a 3D LIDAR-to-camera automatic calibration framework based on graph optimization. The system can automatically identify the position of the pattern and build a set of virtual feature point clouds, and can simultaneously complete the calibration of the LIDAR and multiple cameras. To test this framework, a multi-sensor system is formed using a mobile robot equipped with LIDAR, monocular and binocular cameras, and the pairwise calibration of LIDAR with two cameras is evaluated quantitatively and qualitatively. The results show that this method can produce more accurate calibration results than the state-of-the-art method. The average error on the camera normalization plane is 0.161 mm, which outperforms existing calibration methods. Due to the introduction of graph optimization, the original point cloud is also optimized while optimizing the external parameters between the sensors, which can effectively correct the errors caused during data collection, so it is also robust to bad data.

Pixel-Level Extrinsic Self Calibration of High Resolution LiDAR and Camera in Targetless Environments

In this letter, we present a novel method for automatic extrinsic calibration of high-resolution LiDARs and RGB cameras in targetless environments. Our approach does not require checkerboards but can achieve pixel-level accuracy by aligning natural edge features in the two sensors. On the theory level, we analyze the constraints imposed by edge features and the sensitivity of calibration accuracy with respect to edge distribution in the scene. On the implementation level, we carefully investigate the physical measuring principles of LiDARs and propose an efficient and accurate LiDAR edge extraction method based on point cloud voxel cutting and plane fitting. Due to the edges' richness in natural scenes, we have carried out experiments in many indoor and outdoor scenes. The results show that this method has high robustness, accuracy, and consistency. It can promote the research and application of the fusion between LiDAR and camera. We have open sourced our code on GitHub <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> to benefit the community.

Automatic Targetless Extrinsic Calibration of a 3D Lidar and Camera by Maximizing Mutual Information

This paper reports on a mutual information (MI) based algorithm for automatic extrinsic calibration of a 3D laser scanner and optical camera system. By using MI as the registration criterion, our method is able to work in situ without the need for any specific calibration targets, which makes it practical for in-field calibration. The calibration parameters are estimated by maximizing the mutual information obtained between the sensor-measured surface intensities. We calculate the Cramer-Rao-Lower-Bound (CRLB) and show that the sample variance of the estimated parameters empirically approaches the CRLB for a sufficient number of views. Furthermore, we compare the calibration results to independent ground-truth and observe that the mean error also empirically approaches to zero as the number of views are increased. This indicates that the proposed algorithm, in the limiting case, calculates a minimum variance unbiased (MVUB) estimate of the calibration parameters. Experimental results are presented for data collected by a vehicle mounted with a 3D laser scanner and an omnidirectional camera system.

Automatic Multi-Camera Extrinsic Parameter Calibration Based on Pedestrian Torsors.

Extrinsic camera calibration is essential for any computer vision task in a camera network. Typically, researchers place a calibration object in the scene to calibrate all the cameras in a camera network. However, when installing cameras in the field, this approach can be costly and impractical, especially when recalibration is needed. This paper proposes a novel, accurate and fully automatic extrinsic calibration framework for camera networks with partially overlapping views. The proposed method considers the pedestrians in the observed scene as the calibration objects and analyzes the pedestrian tracks to obtain extrinsic parameters. Compared to the state of the art, the new method is fully automatic and robust in various environments. Our method detect human poses in the camera images and then models walking persons as vertical sticks. We apply a brute-force method to determines the correspondence between persons in multiple camera images. This information along with 3D estimated locations of the top and the bottom of the pedestrians are then used to compute the extrinsic calibration matrices. We also propose a novel method to calibrate the camera network by only using the top and centerline of the person when the bottom of the person is not available in heavily occluded scenes. We verified the robustness of the method in different camera setups and for both single and multiple walking people. The results show that the triangulation error of a few centimeters can be obtained. Typically, it requires less than one minute of observing the walking people to reach this accuracy in controlled environments. It also just takes a few minutes to collect enough data for the calibration in uncontrolled environments. Our proposed method can perform well in various situations such as multi-person, occlusions, or even at real intersections on the street.

Position-Based Visual Servoing of a Mobile Robot with an Automatic Extrinsic Calibration Scheme

SUMMARYIn this paper, we present and implement a novel approach for position-based visual servoing. The challenge of controlling the mobile robot while simultaneously estimating the camera to mobile robot transformation is solved. This is achieved using gradient descent (GD)-based estimation and the sliding-mode approach. The GD approach allows online parameter estimation for controlling the robot to achieve a desired position and orientation. The adaptive nature of the parameters demonstrates the robustness of the system. In contrast to existing work, the proposed technique achieves both estimation and control tasks in a single experiment. Simulation and experimental results are provided to validate the performance of the proposed scheme.

Automatic Extrinsic Calibration of Vision and Lidar by Maximizing Mutual Information

This paper reports on an algorithm for automatic, targetless, extrinsic calibration of a lidar and optical camera system based upon the maximization of mutual information between the sensor‐measured surface intensities. The proposed method is completely data‐driven and does not require any fiducial calibration targets—making in situ calibration easy. We calculate the Cramér‐Rao lower bound (CRLB) of the estimated calibration parameter variance, and we show experimentally that the sample variance of the estimated parameters empirically approaches the CRLB when the amount of data used for calibration is sufficiently large. Furthermore, we compare the calibration results to independent ground‐truth (where available) and observe that the mean error empirically approaches zero as the amount of data used for calibration is increased, thereby suggesting that the proposed estimator is a minimum variance unbiased estimate of the calibration parameters. Experimental results are presented for three different lidar‐camera systems: (i) a three‐dimensional (3D) lidar and omnidirectional camera, (ii) a 3D time‐of‐flight sensor and monocular camera, and (iii) a 2D lidar and monocular camera.