Estimation Of Extrinsic Parameters Research Articles

DPCalib: Dual-Perspective View Network for LiDAR-Camera Joint Calibration

The precise calibration of a LiDAR-camera system is a crucial prerequisite for multimodal 3D information fusion in perception systems. The accuracy and robustness of existing traditional offline calibration methods are inferior to methods based on deep learning. Meanwhile, most parameter regression-based online calibration methods directly project LiDAR data onto a specific plane, leading to information loss and perceptual limitations. A novel network, DPCalib, a dual perspective view network that mitigates the aforementioned issue, is proposed in this paper. This paper proposes a novel neural network architecture to achieve the fusion and reuse of input information. We design a feature encoder that effectively extracts features from two orthogonal views using attention mechanisms. Furthermore, we propose an effective decoder that aggregates features from two views, thereby obtaining accurate extrinsic parameter estimation outputs. The experimental results demonstrate that our approach outperforms existing SOTA methods, and the ablation experiments validate the rationality and effectiveness of our work.

Experimental and simulation investigation of stereo-DIC via a deep learning algorithm based on initial speckle positioning technology.

For the deep-learning-based stereo-digital image correlation technique, the initial speckle position is crucial as it influences the accuracy of the generated dataset and deformation fields. To ensure measurement accuracy, an optimized extrinsic parameter estimation algorithm is proposed in this study to determine the rotation and translation matrix of the plane in which the speckle is located between the world coordinate system and the left camera coordinate system. First, the accuracy of different extrinsic parameter estimation algorithms was studied by simulations. Subsequently, the dataset of stereo speckle images was generated using the optimized extrinsic parameters. Finally, the improved dual-branch CNN deconvolution architecture was proposed to output displacements and strains simultaneously. Simulation results indicate that DAS-Net exhibits enhanced expressive capabilities, as evidenced by a reduction in displacement errors compared to previous research. The experimental results reveal that the mean absolute percentage error between the stereo-DIC results and the generated dataset is less than 2%, suggesting that the initial speckle positioning technology effectively minimizes the discrepancy between the images in the dataset and those obtained experimentally. Furthermore, the DAS-Net algorithm accurately measures the displacement and strain fields as well as their morphological characteristics.

A target-free vision-based method for out-of-plane vibration measurement using projection speckle and camera self-calibration technology

In recent decades, computer vision-based methods have been introduced into engineering structures to provide full-field, non-contact vibration measurement. However, the vision-based vibration measurement method is challenging to measure the targetless structures. This paper proposed a target-free out-of-plane vibration measurement method using projection speckle and camera self-calibration technology. In this method, the camera extrinsic parameter estimation result based on the direct method is used as the initial value in the proposed method. Then, a nonlinear inequality constraint on the extrinsic parameters is established regarding the epipolar condition to minimize the reprojection error. Furthermore, the projection relationship between the projection speckle displacement and the actual displacement of the structure is studied so that the deviation of the out-of-plane vibration displacement is corrected. In the experimental verification, 3D reconstruction and rigid translation experiments are designed, respectively. The results show that the self-calibration accuracy of the binocular camera proposed in this paper is notably improved compared with the existing methods. At the same time, the deviation correction effect of the vibration displacement is significant. Finally, the typical flat plate structure and composite wing structure are taken as vibration measurement cases to demonstrate that the proposed approach is reliable and accurate in measuring the dynamic responses of engineering structures.

Camera–LiDAR Calibration Using Iterative Random Sampling and Intersection Line-Based Quality Evaluation

This paper proposes a novel camera–LiDAR calibration method that utilizes an iterative random sampling and intersection line-based quality evaluation using a foldable plane pair. Firstly, this paper suggests using a calibration object consisting of two small planes with ChArUco patterns, which is easy to make and convenient to carry. Secondly, the proposed method adopts an iterative random sampling to make the calibration procedure robust against sensor data noise and incorrect object recognition. Lastly, this paper proposes a novel quality evaluation method based on the dissimilarity between two intersection lines of the plane pairs from the two sensors. Thus, the proposed method repeats random sampling of sensor data, extrinsic parameter estimation, and quality evaluation of the estimation result in order to determine the most appropriate calibration result. Furthermore, this method can also be used for the LiDAR–LiDAR calibration with a slight modification. In experiments, the proposed method was quantitively evaluated using simulation data and qualitatively assessed using real-world data. The experimental results show that the proposed method can successfully perform both camera–LiDAR and LiDAR–LiDAR calibrations while outperforming the previous approaches.

A novel method to calibrate the rotation axis of a line-structured light 3-dimensional measurement system

Rotation axis calibration is the key to line-structured light 3-dimensional (3D) measurement. The traditional calibration method based on circle fitting only uses the position information of the calibration target and ignores the orientation information, which is easily affected by noise. This paper proposes a new calibration method combining the orientation and position information to improve the calibration accuracy. Compared with the circle fitting method, the added orientation information makes the description of rotation more complete. On this basis, an improved extrinsic parameters estimation method is proposed to obtain more accurate position and orientation information of the target, which further improves the calibration accuracy. The experiment results show that, the radius error is reduced by about 90% and the standard deviation of radius error is reduced by about 40% compared with the traditional circle fitting method.

A Robust Parallel Initialization Method for Monocular Visual-Inertial SLAM.

In order to improve the initialization robustness of visual inertial SLAM, the complementarity of the optical flow method and the feature-based method can be used in vision data processing. The parallel initialization method is proposed, where the optical flow inertial initialization and the monocular feature-based initialization are carried out at the same time. After the initializations, the state estimation results are jointly optimized by bundle adjustment. The proposed method retains more mapping information, and correspondingly is more adaptable to the initialization scene. It is found that the initialization map constructed by the proposed method features a comparable accuracy to the one constructed by ORB-SLAM3 in monocular inertial mode. Since the online extrinsic parameter estimation can be realized by the proposed method, it is considered better than ORB-SLAM3 in the aspect of portability. By the experiments performed on the benchmark dataset EuRoC, the effectiveness and robustness of the proposed method are validated.

Estimation of Extrinsic Parameters With Trifocal Tensor for Intelligent Vehicle-Mounted Cameras

In this article, the extrinsic parameter estimation of a camera mounted on an intelligent vehicle is addressed. The trifocal tensor is utilized to construct vision dynamics which relates image coordinates, velocity signals, and extrinsic parameters. Artificial visual patterns such as chessboards and planar reference objects used in homography-based methods are no longer required. An auxiliary tensor decouples the rotational extrinsic parameters from the translational ones. A key frame strategy is adopted to deal with the field of view constraint and an unknown distance is eliminated from the vision dynamics to counter the scale change caused by key frame switching. The Lyapunov method is used to design nonlinear observers, which estimate the extrinsic parameters at each time step based all collected valid historical data. Performance of the proposed method is verified by both simulation and experimental results.

Calibration of Stereo Pairs Using Speckle Metrology.

The accuracy of 3D reconstruction for metrology applications using active stereo pairs depends on the quality of the calibration of the system. Active stereo pairs are generally composed of cameras mounted on tilt/pan mechanisms separated by a constant or variable baseline. This paper presents a calibration approach based on speckle metrology that allows the separation of translation and rotation in the estimation of extrinsic parameters. To achieve speckle-based calibration, a device called an Almost Punctual Speckle Source (APSS) is introduced. Using the APSS, a thorough method for the calibration of extrinsic parameters of stereo pairs is described. Experimental results obtained with a stereo system called the Agile Stereo Pair (ASP) demonstrate that speckle-based calibration achieves better reconstruction performance than methods using standard calibration procedures. Although the experiments were performed with a specific stereo pair, such as the ASP, which is described in the paper, the speckle-based calibration approach using the APSS can be transposed to other stereo setups.

A Novel Method for Intrinsic and Extrinsic Parameters Estimation by Solving Perspective-Three-Point Problem with Known Camera Position

The aim of the perspective-three-point (P3P) problem is to estimate extrinsic parameters of a camera from three 2D–3D point correspondences, including the orientation and position information. All the P3P solvers have a multi-solution phenomenon that is up to four solutions and needs a fully calibrated camera. In contrast, in this paper we propose a novel method for intrinsic and extrinsic parameter estimation based on three 2D–3D point correspondences with known camera position. Our core contribution is to build a new, virtual camera system whose frame and image plane are defined by the original 3D points, to build a new, intermediate world frame by the original image plane and the original 2D image points, and convert our problem to a P3P problem. Then, the intrinsic and extrinsic parameter estimation is to solve frame transformation and the P3P problem. Lastly, we solve the multi-solution problem by image resolution. Experimental results show its accuracy, numerical stability and uniqueness of the solution for intrinsic and extrinsic parameter estimation in synthetic data and real images.

Automatic Multiple LiDAR Calibration Based on the Plane Features of Structured Environments

The multiple light detection and ranging (LiDAR) system has been attached to numerous autonomous vehicles to reduce blind spot risks and increase measurement resolution. It is also important to accurately aligning multi-LiDAR data because misaligned sensor data can affect the localization and perception algorithms of LiDAR sensors. However, precise extrinsic parameter estimation is a challenging research area because the feature points in LiDAR data cannot be precisely aligned owing to the LiDAR resolution. In this paper, we proposed a targetless multi-LiDAR calibration method without any initial information. In the proposed approach, a random sample consensus (RANSAC)-based plane fitting technique was utilized for plane extraction, and cross-feature points were extracted from the polar view transform of segmented planes. To resolve the discontinuity problem of optimization, a nonlinear optimization method was utilized, and the experiments were performed within a structured environment. Overall, the experimental results showed that the proposed method yields comparable results with state-of-the-art methods.

Extrinsic LiDAR/Ground Calibration Method Using 3D Geometrical Plane-Based Estimation.

This paper details a new extrinsic calibration method for scanning laser rangefinder that is precisely focused on the geometrical ground plane-based estimation. This method is also efficient in the challenging experimental configuration of a high angle of inclination of the LiDAR. In this configuration, the calibration of the LiDAR sensor is a key problem that can be be found in various domains and in particular to guarantee the efficiency of ground surface object detection. The proposed extrinsic calibration method can be summarized by the following procedure steps: fitting ground plane, extrinsic parameters estimation (3D orientation angles and altitude), and extrinsic parameters optimization. Finally, the results are presented in terms of precision and robustness against the variation of LiDAR’s orientation and range accuracy, respectively, showing the stability and the accuracy of the proposed extrinsic calibration method, which was validated through numerical simulation and real data to prove the method performance.

An Extrinsic Calibration Method With Closed-Form Solution for Underwater Opti-Acoustic Imaging System

Forward-looking sonar (FLS) and camera are widely used sensors in multimodal systems for underwater environments. The extrinsic calibration is a precondition for this kind of multimodal systems to fuse optical and acoustic information. Due to the very different imaging formation mechanisms and models of 3-D-to-2-D projection for imaging sonar and optical camera, the multimodal correspondence and the sonar-pose recovery from a 2-D sonar image are two main issues in extrinsic calibration work. In this article, we propose an extrinsic calibration method for the opti-acoustic imaging system combining a camera and an FLS in underwater environment. The proposed method includes an estimation process yielding closed-form expressions to determine the sonar pose with respect to the calibration board, which enables the estimation of extrinsic parameters to be directly computed from sensor measurements and the knowledge of calibration-board structure. A calibration board with two sets of feature arrays is used to generate two 2-D-to-3-D correspondences, which circumvents the 2-D-to-2-D multimodal correspondence problem. An iterative optimization process is provided to further improve the performance, while the extra computation time is negligible. Both simulated and actual experiments show that the method outperforms the existing iterative method based on bundle adjustment.

Extrinsic Calibration and Odometry for Camera-LiDAR Systems

Most autonomous mobile robots are often equipped with monocular cameras and 3D LiDARs to perform vital tasks such as localization and mapping. In this paper, we present a two-stage extrinsic calibration method as well as a hybrid-residual-based odometry approach for such camera-LiDAR systems. Our extrinsic calibration method can estimate the relative transformation between the camera and the LiDAR with high accuracy, allowing us to better register the image and the point cloud data. After the calibration, our hybrid-residual-based odometry can be used to provide real-time, accurate odometry estimates. Our approach exploits both direct and indirect image features. The sensor motions are estimated by jointly minimizing reprojection residuals and photometric residuals in a nonlinear optimization procedure. Experiments are conducted to show the accuracy and robustness of our extrinsic calibration and odometry algorithms using both public and self-owned real-world datasets. The results suggest that our calibration method can provide accurate extrinsic parameters estimation without using initial values, and our odometry approach can achieve competitive estimation accuracy and robustness.

A Novel Feature-Based Approach for Indoor Monocular SLAM

Camera tracking and the construction of a robust and accurate map in unknown environments are still challenging tasks in computer vision and robotic applications. Visual Simultaneous Localization and Mapping (SLAM) along with Augmented Reality (AR) are two important applications, and their performance is entirely dependent on the accuracy of the camera tracking routine. This paper presents a novel feature-based approach for the monocular SLAM problem using a hand-held camera in room-sized workspaces with a maximum scene depth of 4–5 m. In the core of the proposed method, there is a Particle Filter (PF) responsible for the estimation of extrinsic parameters of the camera. In addition, contrary to key-frame based methods, the proposed system tracks the camera frame by frame and constructs a robust and accurate map incrementally. Moreover, the proposed algorithm initially constructs a metric sparse map. To this end, a chessboard pattern with a known cell size has been placed in front of the camera for a few frames. This enables the algorithm to accurately compute the pose of the camera and therefore, the depth of the primary detected natural feature points are easily calculated. Afterwards, camera pose estimation for each new incoming frame is carried out in a framework that is merely working with a set of visible natural landmarks. Moreover, to recover the depth of the newly detected landmarks, a delayed approach based on linear triangulation is used. The proposed method is applied to a realworld VGA quality video (640 × 480 pixels) where the translation error of the camera pose is less than 2 cm on average and the orientation error is less than 3 degrees, which indicates the effectiveness and accuracy of the developed algorithm.

An Optimized Adaptive Camera Calibration Algorithm

Camera calibration is a basic problem in computer vision. An optimized adaptive camera calibration method is proposed in this paper, which can efficiently and precisely calibrate cameras. Our calibration method is easy-to-use by using only one calibration board. In our method, we do lens distortion rectification and camera intrinsic and extrinsic parameters estimation separately, which increase the precision of lens distortion rectification. Meanwhile, dimensions are reduced in nonlinear optimization of camera parameters. By means of alternatively optimize camera’s intrinsic and extrinsic parameter in nonlinear optimization, our method reduces the calibration error by at most 63.14% compared to Zhang’s method [1].

Multicamera rig calibration by double‐sided thick checkerboard

A multi‐camera rig calibration algorithm based on a double sided planar target is proposed. Due to their inherently simple realisation, low cost and accuracy, planar calibration targets came out as one of the most largely adopted calibration tools both for intrinsic and extrinsic camera parameters. However, concerning the estimation of extrinsic parameters, one of the major drawbacks of these targets is their requirement for distinct target visibility from both cameras. This prevents many configurations from being adopted where, e.g. two cameras are facing each other. An inexpensive solution could be based on printing/pasting a planar pattern on both target sides, however, the relative misalignment between the patterns on the two sides and the target thickness could be unknown. The authors propose a solution where double‐sided target displacement error is estimated together with the extrinsic parameters allowing the reuse of all the available planar calibration tools in less constrained configurations. To assess their approach the authors tested the system in two scenarios, one using two professional 4K cameras and one using two smartphones.

Automatic calibration of camera and LRF based on morphological pattern and optimal angular back-projection error

Nowadays, integration systems based on cameras and LRFs have been applied in many researches on robotics, such as autonomous navigation vehicles, and intelligent transportation systems. The systems using multiple sensors usually require extracting the relative information of sensors posed, which support for processing. Therefore, the task of calibration of cameras and laser devices is very important. This paper presents an automatic calibration method for determining the relative position and direction of a camera with respect to a laser rangefinder. The calibration method makes use of depth discontinuities of calibration patterns, which emphasizes laser range-beams supporting easy automatic estimation of the occurred position of laser scans on the calibration pattern. The laser range scans are also used for estimating corresponding depth image points from the camera. Finally, the relative parameters between the camera and the laser device are discovered using the set of depth point correspondences of the LRF and the camera. This paper also presents new criterion for evaluating the error of extrinsic parameters estimation based on optimal error of angular deviation between the projection and back-projection of 3D corresponding points. The evaluation results demonstrate outperformance of this method.

Parameters Separated Calibration Based on Particle Swarm Optimization for a Camera and a Laser-Rangefinder Information Fusion

Heterogeneous sensors fusion of a camera and a laser-rangefinder can greatly improve the environment perception ability, and its primary problem is the calibration of depth scan and image information. At first, the mapping relationship among world coordinate system, camera coordinate system, and image plane is discussed, and then the calibration of camera intrinsic parameters is achieved. Moreover, the intrinsic and extrinsic parameters separated calibration is presented for a camera and a laser-rangefinder, and the characteristic identification is adopted by two intersection calibration boards with a certain angel for fusion characters extraction. Furthermore, the particle swarm optimization is proposed for the extrinsic parameters estimation with different objectives, and the Gaussian elimination is utilized for the initial particle swarm. The simulation and real experimental results show that the standard deviation of calibration error in the 21-group experiments is decreased by 10.175%, and it also proves the accuracy and effectiveness of our approaches.

Inside looking out camera pose estimation for virtual studio

This paper studies the inside looking out camera pose estimation for the virtual studio. The camera pose estimation process, the process of estimating a camera’s extrinsic parameters, is based on closed-form geometrical approaches which use the benefit of simple corner detection of 3D cubic-like virtual studio landmarks. We first look at the effective parameters of the camera pose estimation process for the virtual studio. Our studies include all characteristic landmark parameters like landmark lengths, landmark corner angles and their installation position errors and some camera parameters like lens focal length and CCD resolution. Through computer simulation we investigate and analyze all these parameters’ efficiency in camera extrinsic parameters, including camera rotation and position matrixes. Based on this work, we found that the camera translation vector is affected more than other camera extrinsic parameters because of the noise of effective camera pose estimation parameters. Therefore, we present a novel iterative geometrical noise cancellation method for the closed-form camera pose estimation process. This is based on the collinearity theory that reduces the estimation error of the camera translation vector, which plays a major role in camera extrinsic parameters estimation errors. To validate our method, we test it in a complete virtual studio simulation. Our simulation results show that they are in the same order as those of some commercial systems, such as the BBC and InterSense IS-1200 VisTracker.

Camera Motion Parameter Estimation Technique using 2D Homography and LM Method based on Invariant Features

In this paper, we propose a method to estimate camera motion parameter based on invariant point features. Typically, feature information of image has drawbacks, it is variable to camera viewpoint, and therefore information quantity increases after time. The LM(Levenberg-Marquardt) method using nonlinear minimum square evaluation for camera extrinsic parameter estimation also has a weak point, which has different iteration number for approaching the minimal point according to the initial values and convergence time increases if the process run into a local minimum. In order to complement these shortfalls, we, first propose constructing feature models using invariant vector of geometry. Secondly, we propose a two-stage calculation method to improve accuracy and convergence by using homography and LM method. In the experiment, we compare and analyze the proposed method with existing method to demonstrate the superiority of the proposed algorithms.