Extrinsic Calibration Research Articles

Motorized Measurement of Deformation on Surface of Revolution With 2-D Laser Profiler

The 2D laser profiler potentially provides an efficient way to measure the profile of a deformed surface of revolution. However, the application is constrained by its finite measurement range, as well as the accumulated errors during device installation, workpiece placement and 3D reconstruction. In this work, an integrated measurement system is designed with high-precision rotary table (HRT) and a motorized vertical stage (MVS), so that in-situ scanning measurement is achieved by coordinated motion of the workpiece and the 2D laser profiler. An extrinsic calibration method is proposed between the 2D laser profiler and the HRT with fittings of a plane and a circle curve by the Leica absolute tracker (LAT), so that large tolerance of installation error is allowed for both MVS and HRT. Eventually, the point cloud of the workpiece with respect to the HRT is virtually relocated on the CAD model by Iterative Closest Point (ICP) algorithm with kd-tree construction, so that the amount of deformation is correctly assessed even when the deformed workpiece is arbitrarily clamped by the 3-jaw chuck. The experiment is conducted on the testbed for measuring the deformation of a wheel rim, which verifies the effectiveness of the developed system and proposed method.

A Single 2-D LiDAR Extrinsic Calibration for Autonomous Mobile Robots

A Single 2-D LiDAR Extrinsic Calibration for Autonomous Mobile Robots

외부 시스템을 이용한 Monocular camera-3D LiDAR 융합 시스템의 Non-overlapping 외부 교정 방법에 대한 연구

This study presents a novel framework for the extrinsic calibration of monocular camera and LiDAR with a non-overlapping field of view. The conventional non-overlapping extrinsic calibration utilizes a motion-based calibration method. However, motion-based calibration has a disadvantage in that the vehicle must be moving. In order to solve extrinsic calibration problems under such constraint configurations, the proposed solution uses external sensors that are located away from vehicle. First, the external camera and vehicle camera utilize an aruco marker. Second, the external LiDAR and vehicle LiDAR utilize a scan matching method. In this study, a Gazebo simulator was utilized to show the accuracy of the proposed method. The external LiDAR used Velodyne HDL 64ch, the vehicle LiDAR used Puck 16ch, and the camera used the Blackfly color sensor from FLIR.

Automatic targetless LiDAR–camera calibration: a survey

The recent trend of fusing complementary data from LiDARs and cameras for more accurate perception has made the extrinsic calibration between the two sensors critically important. Indeed, to align the sensors spatially for proper data fusion, the calibration process usually involves estimating the extrinsic parameters between them. Traditional LiDAR–camera calibration methods often depend on explicit targets or human intervention, which can be prohibitively expensive and cumbersome. Recognizing these weaknesses, recent methods usually adopt the autonomic targetless calibration approach, which can be conducted at a much lower cost. This paper presents a thorough review of these automatic targetless LiDAR–camera calibration methods. Specifically, based on how the potential cues in the environment are retrieved and utilized in the calibration process, we divide the methods into four categories: information theory based, feature based, ego-motion based, and learning based methods. For each category, we provide an in-depth overview with insights we have gathered, hoping to serve as a potential guidance for researchers in the related fields.

Software Tool for the Extrinsic Calibration of Infrared and RGBD Cameras Applied to Thermographic Inspection

Context: Thermographic inspections are currently used to assess energy efficiency in electrical equipment and civil structures or to detect failures in cooling systems and electrical or electronic devices. However, thermal images lack texture details, which does not allow for a precise identification of the geometry of the scene or the objects in it. Method: In this work, the development of the software tool called DepTherm is described. This tool allows performing intrinsic and extrinsic calibration between infrared and RGBD cameras in order to fuse thermal, RGB, and RGBD images, as well as to record thermal and depth data. Additional features include user management, a visualization GUI for all three types of images, database storage, and report generation. Results: In addition to the integration tests performed to validate the functionality of DepTherm, two quantitative tests were conducted in order to evaluate its accuracy. A maximum re-projection error of 1,47±0,64 pixels was found, and the maximum mean error in registering an 11 cm side cube was 4,15 mm. Conclusions: The features of the DepTherm software tool are focused on facilitating thermographic inspections by capturing 3D scene models with thermal data.

Spatio-Temporal Calibration of Multiple Kinect Cameras Using 3D Human Pose.

RGB and depth cameras are extensively used for the 3D tracking of human pose and motion. Typically, these cameras calculate a set of 3D points representing the human body as a skeletal structure. The tracking capabilities of a single camera are often affected by noise and inaccuracies due to occluded body parts. Multiple-camera setups offer a solution to maximize coverage of the captured human body and to minimize occlusions. According to best practices, fusing information across multiple cameras typically requires spatio-temporal calibration. First, the cameras must synchronize their internal clocks. This is typically performed by physically connecting the cameras to each other using an external device or cable. Second, the pose of each camera relative to the other cameras must be calculated (Extrinsic Calibration). The state-of-the-art methods use specialized calibration session and devices such as a checkerboard to perform calibration. In this paper, we introduce an approach to the spatio-temporal calibration of multiple cameras which is designed to run on-the-fly without specialized devices or equipment requiring only the motion of the human body in the scene. As an example, the system is implemented and evaluated using Microsoft Azure Kinect. The study shows that the accuracy and robustness of this approach is on par with the state-of-the-art practices.

Accurate fruit localisation using high resolution LiDAR-camera fusion and instance segmentation

Accurate depth-sensing is crucial in securing a high success rate of robotic harvesting in natural orchard environments. The solid-state LiDAR technique, a recently introduced LiDAR sensor, can perceive high-resolution geometric information of the scenes, which can be utilised to receive accurate depth information. Meanwhile, the fusion of the sensory data from LiDAR and the camera can significantly enhance the sensing ability of the harvesting robots. This work first introduces a LiDAR-camera fusion-based visual sensing and perception strategy to perform accurate fruit localisation in the apple orchards. Two SOTA LiDAR-camera extrinsic calibration methods are evaluated to obtain the accurate extrinsic matrix between the LiDAR and camera. After that, the point clouds and colour images are fused to perform fruit localisation using a one-stage instance segmentation network. In addition, comprehensive experiments show that LiDAR-camera achieves better visual sensing performance in natural environments. Meanwhile, introducing the LiDAR-camera fusion can largely improve the accuracy and robustness of the fruit localisation. Specifically, the standard deviations of fruit localisation using LiDAR-camera at 0.5, 1.2, and 1.8 m are 0.253, 0.230, and 0.285 cm, respectively, during the afternoon with intensive sunlight. This measurement error is much smaller compared with that from Realsense D455. Lastly, visualised point cloud22https://drive.google.com/drive/folders/16NV0Bb6N-zlvJC0bFyu-8pl4Gbh9lyOE?usp=sharing. of the apple trees have been attached to demonstrate the highly accurate sensing results of the proposed Lidar-camera fusion method.

Extrinsic calibration method for 3D scanning system with four coplanar laser profilers

3D scanning is a crucial step to ensuring the machining quality of the workpiece and is an essential part of intelligent manufacturing. However, existing scanning systems usually have only one profiler, which must be combined with a dynamic tracking system to achieve a complete scan of a workpiece. This scanning method has low efficiency and complicated path planning for ring-shaped workpieces. Therefore, in this article, an efficient and high-accuracy 3D scanning system composed of a linear translation stage and four uniformly distributed laser profilers is built, and its extrinsic calibration method is studied. At first, based on the working parameters and spatial layout of multiple profilers, a stereoscopic calibrator composed of three non-collinear target balls (TBs) is designed. Then, a multi-profiler data fusion method is proposed, which utilizes a linear encoder to trigger the four profilers synchronously. Finally, by simultaneously using all data from the multiple profilers and the spherical constraint of each TB, all extrinsic parameters are accurately calibrated at the same time. Experimental results show that the average probing size error of the TB with a 38.1 mm diameter is stable at about 0.007 mm, and its extended uncertainty is about 0.100 mm (k = 2). In addition, standard cylinders and bend tubes are scanned. The results show that the proposed method can meet the high-accuracy calibration requirements of the tube-bending deformation detection system.

Overcoming Bias: Equivariant Filter Design for Biased Attitude Estimation With Online Calibration

Stochastic filters for on-line state estimation are a core technology for autonomous systems. The performance of such filters is one of the key limiting factors to a system's capability. Both asymptotic behavior (e.g., for regular operation) and transient response (e.g., for fast initialization and reset) of such filters are of crucial importance in guaranteeing robust operation of autonomous systems. This letter introduces a new generic formulation for a gyroscope aided attitude estimator using <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N$</tex-math></inline-formula> direction measurements including both body-frame and reference-frame direction type measurements. The approach is based on an integrated state formulation that incorporates navigation, extrinsic calibration for all direction sensors, and gyroscope bias states in a single equivariant geometric structure. This newly proposed symmetry allows modular addition of different direction measurements and their extrinsic calibration while maintaining the ability to include bias states in the same symmetry. The subsequently proposed filter-based estimator using this symmetry noticeably improves the transient response, and the asymptotic bias and extrinsic calibration estimation compared to state-of-the-art approaches. The estimator is verified in statistically representative simulations and is tested in real-world experiments.

Sole GNSS Sensor Extrinsic Calibration Methods for Mobile Mapping Systems

A mobile mapping system (MMS) is a widely-used moving platform for the acquisition of rich geospatial information and requires sensor calibration to guarantee data quality. Global navigation satellite system (GNSS) sensor and light detection and ranging (LiDAR) are important parts of an MMS, and the extrinsic calibration between them can markedly improve its performance. A sole GNSS sensor is generally used in a portable MMS, which is the current trend. However, existing extrinsic calibration methods of GNSS/inertial navigation system (INS) are not applicable for the sole GNSS sensor because the former requires relative six degrees of freedom (DoF) pose output for both sensors and the latter can only provide 3-DoF positions. Here, we propose two sole GNSS sensor extrinsic calibration methods: one is a target-based direct calculation method and the other is a non-target-based automatic iterative method. The experiment results show the two methods have good repeatability, the error of the target-based method is below 3 cm, and the error of the non-target-based method is below 2 cm.

CamMap: Extrinsic Calibration of Non-Overlapping Cameras Based on SLAM Map Alignment

Multiple cameras have emerged as a promising technology for robots and vehicles due to their broad fields of view (FoV) and high resolution. However, there are often limited or no overlapping FoVs among cameras, bringing challenges to estimating extrinsic camera parameters. To overcome this problem, we propose CamMap: a novel 6-degree-of-freedom (DoF) extrinsic calibration pipeline. Following three operating rules, we make a multi-camera rig capture some similar image sequences individually to create sparse feature-based maps with a SLAM system. A two-stage optimization problem is formulated to align the maps and obtain the transformations between them based on bidirectional reprojection. The transformations are exactly the extrinsic parameters. Supporting diverse camera types, the pipeline is available in any texture-rich environment. It can calibrate any number of cameras simultaneously without requiring calibration patterns, synchronization, same resolution and frequency. The pipeline is evaluated on cameras with limited and no overlapping FoVs. In the experiments, we demonstrate our method's accuracy and efficiency. The absolute pose error (APE) between Kalibr and CamMap is less than 0.025.

Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups

Most sensor setups for onboard autonomous perception are composed of LiDARs and vision systems, as they provide complementary information that improves the reliability of the different algorithms necessary to obtain a robust scene understanding. However, the effective use of information from different sources requires an accurate calibration between the sensors involved, which usually implies a tedious and burdensome process. We present a method to calibrate the extrinsic parameters of any pair of sensors involving LiDARs, monocular or stereo cameras, of the same or different modalities. The procedure is composed of two stages: first, reference points belonging to a custom calibration target are extracted from the data provided by the sensors to be calibrated, and second, the optimal rigid transformation is found through the registration of both point sets. The proposed approach can handle devices with very different resolutions and poses, as usually found in vehicle setups. In order to assess the performance of the proposed method, a novel evaluation suite built on top of a popular simulation framework is introduced. Experiments on the synthetic environment show that our calibration algorithm significantly outperforms existing methods, whereas real data tests corroborate the results obtained in the evaluation suite. Open-source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/beltransen/velo2cam_calibration</uri> .

Extrinsic Calibration of Multiple 3D LiDAR Sensors by the Use of Planar Objects.

Three-dimensional light detection and ranging (LiDAR) sensors have received much attention in the field of autonomous navigation owing to their accurate, robust, and rich geometric information. Autonomous vehicles are typically equipped with multiple 3D LiDARs because there are many commercially available low-cost 3D LiDARs. Extrinsic calibration of multiple LiDAR sensors is essential in order to obtain consistent geometric information. This paper presents a systematic procedure for the extrinsic calibration of multiple 3D LiDAR sensors using plane objects. At least three independent planes are required within the common field of view of the LiDAR sensors. The planes satisfying the condition can easily be found on objects such as the ground, walls, or columns in indoor and outdoor environments. Therefore, the proposed method does not require environmental modifications such as using artificial calibration objects. Multiple LiDARs typically have different viewpoints to reduce blind spots. This situation increases the difficulty of the extrinsic calibration using conventional registration algorithms. We suggest a plane registration method for cases in which correspondences are not known. The entire calibration process can easily be automated using the proposed registration technique. The presented experimental results clearly show that the proposed method generates more accurate extrinsic parameters than conventional point cloud registration methods.

3D-CALI: Automatic calibration for camera and LiDAR using 3D checkerboard

Accurate and robust calibration of intrinsic and extrinsic parameters is crucial to multi-sensor simultaneous localization and mapping (SLAM) systems. However, extrinsic calibration of heterogeneous sensors is challenging owing to the differences in data acquisition methods. Therefore, we propose a more robust and convenient method to calibrate the intrinsic parameters for the camera and extrinsic parameters for cameras and light detection and ranging (LiDAR) sensors based on a 3D checkerboard. We tested the algorithm in simulation and real-world environments. Experiments in the simulation environment showed that our proposed method had similar accuracy as the state-of-the-art calibration method velo2cam_calibration.11https://github.com/beltransen/velo2cam_calibration. However, our proposed algorithm had less interaction and was more automatic. The real-world test also confirmed the results of our proposed method. Thus, the proposed method is a convenient means to calibrate intrinsic parameters for cameras, and extrinsic parameters for cameras and LiDAR sensors based on a 3D checkerboard.

3D map building based on extrinsic sensor calibration method and object contour detector with a fully convolutional neural network

In this paper, we propose a simple method to obtain an object’s 3D coordinate information in one image, using one monocular camera and two 2D LiDAR sensors, which are widely used low-cost sensors. An extrinsic calibration method is used for each LiDAR sensor to transfer LiDAR coordinate to camera pixel coordinate in one image. The proportion factor [Formula: see text] is calculated, based on the relationship of pixel from calibration points and distance from detected points of two LiDAR sensors. In order to create a correct 3D map, a deep learning algorithm for real-time object contour detection with a fully Convolutional Neural Network is proposed, through gathering data, labeling data, training model, and testing object contour detector. For 3D map building, by the proportion factor [Formula: see text] and the detector contour feature pixels, the height of the object is obtained. Combining with LiDAR sensor’s detection information, the 3D posture of the object is obtained. Based on the proposed extrinsic sensor calibration method and the object contour detect result, virtual points of the object are calculated. Finally, the experiment results show the 3D coordinate information of the detected single or multiple objects. Assembling sensors and a controller on a Kobuki robot, the indoor 3D map is built based on the proposed methods.

Robots’ State Estimation and Observability Analysis Based on Statistical Motion Models

This article presents a generic motion model to capture mobile robots’ dynamic behaviors (translation and rotation). The model is based on statistical models driven by white random processes and is formulated into a full state estimation algorithm based on the error-state extended Kalman filtering framework (ESEKF). The major benefits of this method are its versatility, being applicable to different robotic systems without accurately modeling the robots’ specific dynamics, and the ability to estimate the robot’s (angular) acceleration, jerk, or higher order dynamic states with low delay. Mathematical analyses with numerical simulations are presented to show the properties of the statistical model-based estimation framework and reveal its connection to existing low-pass filters. Furthermore, a new paradigm is developed for robotic observability analysis by developing Lie derivatives and associated partial differentiation directly on manifolds. It is shown that this new paradigm is much simpler and more natural than existing methods based on quaternion parameterizations. It is also scalable to high-dimensional systems. A novel <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">thin</i> set concept is introduced to characterize the unobservable subset of the system states, providing the theoretical foundation to observability analysis of robotic systems operating on manifolds and in high dimension. Finally, extensive experiments, including full state estimation and extrinsic calibration (both POS-IMU and IMU-IMU) on a quadrotor unmanned aerial vehicle (UAV), a handheld platform, and a ground vehicle, are conducted. Comparisons with existing methods show that the proposed method can effectively estimate all extrinsic parameters, the robot’s translation/angular acceleration, and other state variables (e.g., position, velocity, and attitude) with high accuracy and low delay.

Accurate and Automatic Extrinsic Calibration for a Monocular Camera and Heterogenous 3D LiDARs

Nowadays, portable cameras and LiDARs are widely applied in robotic applications for environment perception, path planning, and high precision navigation. In such cases, accurate inter-sensor spatial transformation is critical to fuse these sensors seamlessly. However, the feasible and efficient extrinsic calibration of the LiDAR-camera sensor pair is still challenging as it is hard to establish the common feature correspondence between sparse LiDAR point clouds and monocular images. In this contribution, we design a novel calibration board with checkerboard grids and circular holes, through which the proper extrinsic parameters can be obtained automatically by matching the circular hole centers extracted from both images and heterogenous LiDAR scans. Profiting from the versatility and stability of circular hole extraction, the proposed calibration method is suitable for LiDARs with different precision and scanning modes, such as Velodyne LiDAR, Ouster LiDAR, and Livox LiDAR. Both simulation tests and real-world experiments were conducted to evaluate the effectiveness of the proposed calibration method. The simulated experiments indicate that the translation error is less than 0.5 cm, and the rotation error is well below 0.3 degrees. As for real-world experiments, the re-projection error can achieve sub-pixel level with different LiDARs.

A Novel and Simplified Extrinsic Calibration of 2D Laser Rangefinder and Depth Camera

It is too difficult to directly obtain the correspondence features between the two-dimensional (2D) laser-range-finder (LRF) scan point and camera depth point cloud, which leads to a cumbersome calibration process and low calibration accuracy. To address the problem, we propose a calibration method to construct point-line constraint relations between 2D LRF and depth camera observational features by using a specific calibration board. Through the observation of two different poses, we construct the hyperstatic equations group based on point-line constraints and solve the coordinate transformation parameters of 2D LRF and depth camera by the least square (LSQ) method. According to the calibration error and threshold, the number of observation and the observation pose are adjusted adaptively. After experimental verification and comparison with existing methods, the method proposed in this paper easily and efficiently solves the problem of the joint calibration of the 2D LRF and depth camera, and well meets the application requirements of multi-sensor fusion for mobile robots.

Map-based registration of multi-LiDAR infrastructure sensor setups for real-time applications

The use of infrastructure sensors has proven to be an effective method for recording large traffic datasets. In order to achieve a high data quality for such datasets, an accurate extrinsic calibration of the sensors is essential. While previous approaches are not fast enough or do not fit the use-case of infrastructure sensing, we present a method that is capable of registering LiDAR sensors to a digital map in real-time. We use a mobile infrastructure sensor setup consisting of two measurement units, which we position at different locations with real road traffic. Each measurement unit is equipped with two LiDARs at a mounting height of eight meters. Experiments have shown that our method can perform continuous registration of up to four 128-layer LiDARs. With a voxel size of 0.05 meters used, our method achieves a root-mean-square error (RMSE) of 0.13 meters at an average processing time of 0.05 seconds per frame.

Multi-Camera Extrinsic Calibration for Real-Time Tracking in Large Outdoor Environments

Calibrating intrinsic and extrinsic camera parameters is a fundamental problem that is a preliminary task for a wide variety of applications, from robotics to computer vision to surveillance and industrial tasks. With the advent of Internet of Things (IoT) technology and edge computing capabilities, the ability to track motion activities in large outdoor areas has become feasible. The proposed work presents a network of IoT camera nodes and a dissertation on two possible approaches for automatically estimating their poses. One approach follows the Structure from Motion (SfM) pipeline, while the other is marker-based. Both methods exploit the correspondence of features detected by cameras on synchronized frames. A preliminary indoor experiment was conducted to assess the performance of the two methods compared to ground truth measurements, employing a commercial tracking system of millimetric precision. Outdoor experiments directly compared the two approaches on a larger setup. The results show that the proposed SfM pipeline more accurately estimates the pose of the cameras. In addition, in the indoor setup, the same methods were used for a tracking application to show a practical use case.