Multiple Lidar Research Articles

Three-dimensional spatiotemporal wind field reconstruction based on LiDAR and multi-scale PINN

In this numerical study, we use a multi-scale version of physics-informed neural network (PINN) for wind field reconstruction from LiDAR measurements. The reference velocity field is obtained from high-fidelity large-eddy simulation of neutral atmospheric boundary layer, and the LiDAR measurement strategy is restricted to that of a real LiDAR. The multi-scale PINN reconstructs the velocity field by minimizing a combination of the L2-error with respect to the LiDAR measurements and residuals of the governing equations. Our most significant finding is that adding a multi-scale layer to PINN enables us to: (i) capture wider range of scales, (ii) reconstruct the flow field outside the LiDAR scanning area, and (iii) reach faster convergence. We also find that for volumetric flow reconstruction and to deal with uncertainty in the wind direction, the reconstruction accuracy can be greatly improved by employing multiple LiDAR devices. Overall, our results show that the normalized errors in the reconstructed wind field are lower than 5% for all the experiments conducted in this study and that the errors do not increase even in the presence of measurement noise levels of typical LiDAR. These findings show the feasibility of three-dimensional dynamic wind field reconstruction across large wind farms using LiDAR devices.

Point Cloud Densification Algorithm for Multiple Cameras and Lidars Data Fusion.

Fusing data from many sources helps to achieve improved analysis and results. In this work, we present a new algorithm to fuse data from multiple cameras with data from multiple lidars. This algorithm was developed to increase the sensitivity and specificity of autonomous vehicle perception systems, where the most accurate sensors measuring the vehicle's surroundings are cameras and lidar devices. Perception systems based on data from one type of sensor do not use complete information and have lower quality. The camera provides two-dimensional images; lidar produces three-dimensional point clouds. We developed a method for matching pixels on a pair of stereoscopic images using dynamic programming inspired by an algorithm to match sequences of amino acids used in bioinformatics. We improve the quality of the basic algorithm using additional data from edge detectors. Furthermore, we also improve the algorithm performance by reducing the size of matched pixels determined by available car speeds. We perform point cloud densification in the final step of our method, fusing lidar output data with stereo vision output. We implemented our algorithm in C++ with Python API, and we provided the open-source library named Stereo PCD. This library very efficiently fuses data from multiple cameras and multiple lidars. In the article, we present the results of our approach to benchmark databases in terms of quality and performance. We compare our algorithm with other popular methods.

Place Recognition through Multiple LiDAR Scans Based on the Hidden Markov Model.

Autonomous driving systems for unmanned ground vehicles (UGV) operating in enclosed environments strongly rely on LiDAR localization with a prior map. Precise initial pose estimation is critical during system startup or when tracking is lost, ensuring safe UGV operation. Existing LiDAR-based place recognition methods often suffer from reduced accuracy due to only matching descriptors from individual LiDAR keyframes. This paper proposes a multi-frame descriptor-matching approach based on the hidden Markov model (HMM) to address this issue. This method enhances the place recognition accuracy and robustness by leveraging information from multiple frames. Experimental results from the KITTI dataset demonstrate that the proposed method significantly enhances the place recognition performance compared with the scan context-based single-frame descriptor-matching approach, with an average performance improvement of 5.8% and with a maximum improvement of 15.3%.

M-GCLO: Multiple Ground Constrained LiDAR Odometry

Abstract. Accurate LiDAR odometry results contribute directly to high-quality point cloud maps. However, traditional LiDAR odometry methods drift easily upward, leading to inaccuracies and inconsistencies in the point cloud maps. Considering abundant and reliable ground points in the Mobile Mapping System(MMS), ground points can be extracted, and constraints can be built to eliminate pose drifts. However, existing LiDAR-based odometry methods either do not use ground point cloud constraints or consider the ground plane as an infinite plane (i.e., single ground constraint), making pose estimation prone to errors. Therefore, this paper is dedicated to developing a Multiple Ground Constrained LiDAR Odometry(M-GCLO) method, which extracts multiple grounds and optimizes those plane parameters for better accuracy and robustness. M-GCLO includes three modules. Firstly, the original point clouds will be classified into the ground and non-ground points. Ground points are voxelized, and multiple ground planes are extracted, parameterized, and optimized to constrain the pose errors. All the non-ground point clouds are used for point-to-distribution matching by maintaining an NDT voxel map. Secondly, a novel method for weighting the residuals is proposed by considering the uncertainties of each point in a scan. Finally, the jacobians and residuals are given along with the weightings for estimating LiDAR states. Experimental results in KITTI and M2DGR datasets show that M-GCLO outperforms state-of-the-art LiDAR odometry methods in large-scale outdoor and indoor scenarios.

Online Calibration of Extrinsic Parameters for Solid-State LIDAR Systems.

This work addresses the challenge of calibrating multiple solid-state LIDAR systems. The study focuses on three different solid-state LIDAR sensors that implement different hardware designs, leading to distinct scanning patterns for each system. Consequently, detecting corresponding points between the point clouds generated by these LIDAR systems-as required for calibration-is a complex task. To overcome this challenge, this paper proposes a method that involves several steps. First, the measurement data are preprocessed to enhance its quality. Next, features are extracted from the acquired point clouds using the Fast Point Feature Histogram method, which categorizes important characteristics of the data. Finally, the extrinsic parameters are computed using the Fast Global Registration technique. The best set of parameters for the pipeline and the calibration success are evaluated using the normalized root mean square error. In a static real-world indoor scenario, a minimum root mean square error of 7 cm was achieved. Importantly, the paper demonstrates that the presented approach is suitable for online use, indicating its potential for real-time applications. By effectively calibrating the solid-state LIDAR systems and establishing point correspondences, this research contributes to the advancement of multi-LIDAR fusion and facilitates accurate perception and mapping in various fields such as autonomous driving, robotics, and environmental monitoring.

HYBRID ADJUSTMENT OF UAS-BASED LiDAR AND IMAGE DATA

Abstract. Several advancements are going with Unmanned Aerial Systems (UAS) with the addition of multiple sensors and simultaneous data acquisition to obtain detailed geo-data for various applications. However, simultaneous data acquisition with multiple sensors, namely camera, and LiDAR, will also result in possible discrepancies associated with them, and they need to be solved to use a reliable and accurate final product. Several errors can be associated with both camera and LiDAR datasets due to the different characteristics of the sensors and terrain conditions. This research paper aimed to minimize the errors between LiDAR and the image datasets simultaneously acquired with an Unmanned Aerial System (UAS) by implementing a hybrid adjustment approach with a criterion for the roughness and threshold angle between surface normals. The initial trajectory of the UAS, raw LiDAR measurements, and image observations were the inputs used for the hybrid adjustment. The hybrid adjustment workflow minimizes the discrepancies with a least-squares-based simultaneous adjustment for both LiDAR and image datasets. For the hybrid adjustment process, three types of correspondences were established, namely: between image pairs, overlapping LiDAR strips, and between Image tie points and LiDAR strips. For quality control, mean Cloud-to-Cloud distances (C2C) were compared between both LiDAR and camera point clouds before and after hybrid adjustment. The surface-level analysis of the results was also carried out to analyze the errors before and after hybrid adjustment at a surface level for different types of surfaces. The results showed that the alignment between the point clouds has significantly improved from the range of meters to a centimeter-level after implementing the hybrid adjustment process. The proposed hybrid adjustment workflow can be used in mapping applications where a centimeter-level accuracy is requested.

PMLIO: PANORAMIC TIGHTLY-COUPLED MULTI-LIDAR-INERTIAL ODOMETRY AND MAPPING

Abstract. The limited field of view (FoV) of single LiDAR poses challenges for robots to achieve comprehensive environmental perception. Incorporating multiple LiDAR sensors can effectively broaden the FoV of robots, providing abundant measurements to facilitate simultaneous localization and mapping (SLAM). In this paper, we propose a panoramic tightly-coupled multi-LiDAR-inertial odometry and mapping framework, which fully leverages the properties of solid-state LiDAR and spinning LiDAR. The key of the proposed framework lies in the effective completion of multi-LiDAR spatial-temporal fusion. Additionally, we employ the iterated extended Kalman filter to achieve tightly-coupled inertial odometry and mapping with IMU data. PMLIO showcases competitive performance on multiple scenarios data, compared with state-of-the-art single LiDAR-inertial SLAM algorithms, and reaches a noteworthy improvement of 27.1% and 12.9% in max and median of absolute pose error (APE) respectively.

Vertical profiles and regional transport of ozone in typical area of Yangtze-Huaihe River Basin during the autumn base on multiple lidars

In recent years, the Yangtze-Huaihe River Basin has become one of the key areas of air pollution control in China, however, there are some differences in the spatial and temporal characteristics of ozone in different cities. In order to explore the vertical structure characteristics of ozone in Yangtze-Huaihe River Basin and its influencing factors, this study analyzed the vertical and horizontal distribution of ozone in typical cities in the Yangtze-Huaihe River Basin in October 2021 by using differential absorption lidar(DIAL), temperature and humidity profile lidar and wind profile radar system, studied two typical ozone pollution processes, and the meteorological factors of ozone pollution at vertical height. The results showed that the ozone pollution was mainly produced locally from October 13 to 14, while the ozone pollution was affected by local production and transport from October 21 to 31, the average ozone concentration reached a maximum of 357.6 μg/m3. There is a significant residual layer at night. The temperature of 272.85–285.88 k and water vapor mixing ratio of 3.22–5.7 g/kg are favorable for ozone generation in Huaibei City. The ozone contribution of Henan, Shandong and Anhui to Huaibei City is above 200 μg/m3. The study on the vertical structure of ozone is helpful to fill the knowledge gap between the evolution of ozone events in Yangtze-Huaihe River Basin.

Placement Method of Multiple Lidars for Roadside Infrastructure in Urban Environments.

Sensors on autonomous vehicles have inherent physical constraints. To address these limitations, several studies have been conducted to enhance sensing capabilities by establishing wireless communication between infrastructure and autonomous vehicles. Various sensors are strategically positioned within the road infrastructure, providing essential sensory data to these vehicles. The primary challenge lies in sensor placement, as it necessitates identifying optimal locations that minimize blind spots while maximizing the sensor's coverage area. Therefore, to solve this problem, a method for positioning multiple sensor systems in road infrastructure is proposed. By introducing a voxel grid, the problem is formulated as an optimization challenge, and a genetic algorithm is employed to find a solution. Experimental findings using lidar sensors are presented to demonstrate the efficacy of this proposed approach.

Intensity-Modulated Direct-Detection Doppler LiDAR Using Pseudo-Random Code and Optical Direct Down-Conversion of Modulation Frequency for Range and Speed Measurement of a Hard Target

Intensity-modulated direct-detection Doppler LiDAR (IM-DDDL) with pseudo-random code is demonstrated for range and speed measurement of a hard target. This IM-DDDL has key features of (i) long-time coherent signal accumulation without requirement of narrow-linewidth laser source, (ii) high receiving sensitivity with optical direct down-conversion of modulation frequency, (iii) ranging using pseudo-random code, and (iv) Doppler shift detection at baseband frequency. These features are demonstrated in preliminary experiments with fiber-based circuit at 1550 nm wavelength which has eye-safety. The employment of fiber-based configuration contributes to compactness and reliability of the instrument. Ranging function using pseudo-random code is suitable for prevention from interference between multiple LiDARs. The condition for detecting Doppler frequency shift is also shown. A moving stroke of a target within a measurement time needs to be enough larger than the wavelength of modulation frequency. Process for future practical use and application to volumetric target sensing are additionally investigated.

A Machine-Learning Method to Integrate Arctic Supersite Observations and Diagnose Weather Element Occurrence

ABSTRACT The accurate detection and quantification of light precipitation is problematic, particularly in the Arctic region. Satellite and ground-based observations of light precipitation are frequently underestimated at high latitudes. Remote sensing and in-situ observations from the Iqaluit, NU supersite (64oN, 69oW) were integrated to train, develop, and validate a random forest (RF) model that can diagnose precipitation type and other weather element occurrences. Observations from multiple lidars, optical disdrometers, traditional precipitation gauges and meteorological aerodrome (METAR) reports from 2015–2020 were integrated and used in the RF model development. The model was trained at Iqaluit, validated over different time periods, and applied to another region (Whitehorse, YT; 61oN, 135oW). Results indicate the importance of accurate visibility observations to train the model. Overall, the RF model was capable of distinguishing precipitation types and demonstrated the potential to be used at all sites/networks where similar automated and cost-effective instruments are already deployed (e.g. radar sites, airports with ceilometers, etc.). This would reduce the dependency on METARs while improving weather element occurrence accuracy.

Multi-Lidar System Localization and Mapping with Online Calibration

Currently, the demand for automobiles is increasing, and daily travel is increasingly reliant on cars. However, accompanying this trend are escalating traffic safety issues. Surveys indicate that most traffic accidents stem from driver errors, both intentional and unintentional. Consequently, within the framework of vehicular intelligence, intelligent driving uses computer software to assist drivers, thereby reducing the likelihood of road safety incidents and traffic accidents. Lidar, an essential facet of perception technology, plays an important role in vehicle intelligent driving. In real-world driving scenarios, the detection range of a single laser radar is limited. Multiple laser radars can improve the detection range and point density, effectively mitigating state estimation degradation in unstructured environments. This, in turn, enhances the precision and accuracy of synchronous positioning and mapping. Nonetheless, the relationship governing pose transformation between multiple lidars is intricate. Over extended periods, perturbations arising from vibrations, temperature fluctuations, or collisions can compromise the initially converged external parameters. In view of these concerns, this paper introduces a system capable of concurrent multi-lidar positioning and mapping, as well as real-time online external parameter calibration. The method first preprocesses the original measurement data, extracts linear and planar features, and rectifies motion distortion. Subsequently, leveraging degradation factors, the convergence of the multi-lidar external parameters is detected in real time. When deterioration in external parameters is identified, the local map of the main laser radar and the feature point cloud of the auxiliary laser radar are associated to realize online calibration. This is succeeded by frame-to-frame matching according to the converged external parameters, culminating in laser odometer computation. Introducing ground constraints and loop closure detection constraints in the back-end optimization effectuates global estimated pose rectification. Concurrently, the feature point cloud is aligned with the global map, and map update is completed. Finally, experimental validation is conducted on data acquired from Chang’an University to substantiate the system’s online calibration and positioning mapping accuracy, robustness, and real-time performance.

An Extended Vector Polar Histogram Method Using Omni-Directional LiDAR Information

This study presents an extended vector polar histogram (EVPH) method for efficient robot navigation using omni-directional LiDAR data. Although the conventional vector polar histogram (VPH) method is a powerful technique suitable for LiDAR sensors, it is limited in its sensing range by the single LiDAR sensor to a semicircle. To address this limitation, the EVPH method incorporates multiple LiDAR sensor’s data for omni-directional sensing. First off, in the EVPH method, the LiDAR sensor coordinate systems are directly transformed into the robot coordinate system to obtain an omni-directional polar histogram. Several techniques are also employed in this process, such as minimum value selection and linear interpolation, to generate a uniform omni-directional polar histogram. The resulting histogram is modified to represent the robot as a single point. Subsequently, consecutive points in the histogram are grouped to construct a symbol function for excluding concave blocks and a threshold function for safety. These functions are combined to determine the maximum cost value that generates the robot’s next heading angle. Robot backward motion is made feasible based on the determined heading angle, enabling the calculation of the velocity vector for time-efficient and collision-free navigation. To assess the efficacy of the proposed EVPH method, experiments were carried out in two environments where humans and obstacles coexist. The results showed that, compared to the conventional method, the robot traveled safely and efficiently in terms of the accumulated amount of rotations, total traveling distance, and time using the EVPH method. In the future, our plan includes enhancing the robustness of the proposed method in congested environments by integrating parameter adaptation and dynamic object estimation methods.

Using spatiotemporal stacks for precise vehicle tracking from roadside 3D LiDAR data

This paper develops a non-model based vehicle tracking methodology for extracting road user trajectories as they pass through the field of view of a 3D LiDAR sensor mounted on the side of the road. To minimize the errors, our work breaks from conventional practice and postpones target segmentation until after collecting LiDAR returns over many scans. Specifically, our method excludes all non-vehicle returns in each scan and retains the ungrouped vehicle returns. These vehicle returns are stored over time in a spatiotemporal stack (ST stack) and we develop a vehicle motion estimation framework to cluster the returns from the ST stack into distinct vehicles and extract their trajectories. This processing includes removing the impacts of the target's changing orientation relative to the LiDAR sensor while separately taking care to preserve the crisp transition to/from a stop that would normally be washed out by conventional data smoothing or filtering. This proof of concept study develops the methodology using a single LiDAR sensor, thus, limiting the surveillance region to the effective range of the given sensor. It should be clear from the presentation that, provided sufficient georeferencing, the surveillance region can be extended indefinitely by deploying multiple LiDAR sensors with overlapping fields of view.

Learning Observation Model for Factor Graph Based-State Estimation Using Intrinsic Sensors

The navigation system of autonomous mobile robots has appeared challenging when using exteroceptive sensors such as cameras, LiDARs, and radars in textureless and structureless environments. This paper presents a robust state estimation system for holonomic mobile robots using intrinsic sensors based on adaptive factor graph optimization in the degradation scenarios. In particular, the neural networks are employed to learn the observation and noise model using only IMU sensor and wheel encoder data. Investigating the learning model for the holonomic mobile robot is discussed with various neural network architectures. We also explore the neural networks that are far more powerful and have cheaper computing power when using the inertial-wheel encoder sensors. Furthermore, we employ an industrial holonomic robot platform equipped with multiple LiDARs, cameras, IMU, and wheel encoders to conduct the experiments and create the ground truth without a bulky motion capture system. The collected datasets are then implemented to train the neural networks. Finally, the experimental evaluation presents that our solution provides better accuracy and real-time performance than other solutions. <i>Note to Practitioners</i>&#x2014;Autonomous mobile robots need to serve in challenging environments robustly that deny extrinsic sensors such as cameras, LiDARs, and radars. In order to operate in this situation, the navigation system shall rely on the intrinsic sensor as inertial sensor and wheel encoders. Existing conventional methods have combined the intrinsic sensors in the form of the recursive Bayesian filtering technique without adapting these models. Besides, deep learning-based solutions have adopted extensive networks like LSTM or CNN to handle the estimation problem. This work aims to develop a state estimation subsystem of the navigation system for the holonomic mobile robots that utilize the intrinsic sensors in adaptive factor graph optimization. In particular, we present how to join the factor graph efficiently with the learning observation model of IMU and wheel encoder factor. Moreover, the neural networks are introduced to learn the observation model with an IMU and wheel encoder data inputs. We recognize that lightweight neural networks can achieve more power than deep learning techniques using the IMU sensor and wheel encoders. Finally, the neural networks are embedded in a factor graph to handle the smoothing state estimation. The proposed system could operate with high accuracy in real time.

Multi-Modal 3D Object Detection in Autonomous Driving: A Survey and Taxonomy

Autonomous vehicles require constant environmental perception to obtain the distribution of obstacles to achieve safe driving. Specifically, 3D object detection is a vital functional module as it can simultaneously predict surrounding objects' categories, locations, and sizes. Generally, autonomous vehicles are equipped with multiple sensors, including cameras and LiDARs. The fact that single-modal methods suffer from unsatisfactory detection performance motivates utilizing multiple modalities as inputs to compensate for single sensor faults. Although many multi-modal fusion detection algorithms exist, there is still a lack of comprehensive and in-depth analysis of these methods to clarify how to fuse multi-modal data effectively. Therefore, this paper surveys recent advancements in fusion detection methods. First, we present the broad background of multi-modal 3D object detection and identify the characteristics of widely used datasets along with their evaluation metrics. Second, instead of the traditional classification method of early, middle, and late fusion, we categorize and analyze all fusion methods from three aspects: feature representation, alignment, and fusion, which reveals how these fusion methods are implemented in an essential way. Third, we provide an in-depth comparison of their pros and cons and compare their performance in mainstream datasets. Finally, we further summarize current challenges and research trends for realizing the full potential of multi-modal 3D object detection.

Observability-Aware Online Multi-Lidar Extrinsic Calibration

Accurate and robust extrinsic calibration is necessary for deploying autonomous systems which need multiple sensors for perception. In this paper, we present a robust system for real-time extrinsic calibration of multiple lidars in vehicle base frame without the need for any fiducial markers or features. We base our approach on matching absolute GNSS (Global Navigation Satellite System) and estimated lidar poses in real-time. Comparing rotation components allows us to improve the robustness of the solution than traditional least-square approach comparing translation components only. Additionally, instead of comparing all corresponding poses, we select poses comprising maximum mutual information based on our novel observability criteria. This allows us to identify a subset of the poses helpful for real-time calibration. We also provide stopping criteria for ensuring calibration completion. To validate our approach extensive tests were carried out on data collected using Scania test vehicles (7 sequences for a total of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\approx$</tex-math></inline-formula> 6.5 Km). The results presented in this paper show that our approach is able to accurately determine the extrinsic calibration for various combinations of sensor setups.

MuSLi: A multi sensor LiDAR detection for C-V2X networks

Obstacle detection is a tool adopted in vehicular safety applications, aiming to detect a moving obstacle in an area of interest with the highest accuracy. Different sensors are used for this aim, such as LiDAR devices mounted on board of a vehicle that capture images of the surrounding environment. Extending the number of LiDAR sensors can be useful to improve the obstacle detection accuracy, since multiple images are captured from different distances and directions, and this represents an interesting approach, specially in case of dense networks with cooperative nodes. In this paper we present MuSLi technique, aiming to (i) provide an accurate obstacle detection and (ii) forward alert messages to other cars in the network, in case of correct detection of a pedestrian crossing the street. MuSLi relies on the connected content islands scenario, where each vehicle defined as a content island subscribes to a service in order to receive and share published messages. Specifically, the road safety service allows the detection of an obstacle through multiple LiDAR sensors from neighboring cars. Furthermore, we investigate the fastest transmission mode among those defined in the C-V2X releases to alert the presence of a pedestrian to the other approaching cars. The proposed technique provides the distances by the crossroad in which is better to use V2V, V2I or V2N according to the environment, the scheduling technique and the measured interference.

A System Architecture of a Fusion System for Multiple LiDARs Image Processing

LiDAR sensors are extensively used in autonomous vehicles and their optimal use is a critical concern. In this paper, we propose an embedded software architecture for multiple LiDAR sensors, i.e., a fusion system that acts as an embedded system for processing data from multiple LiDAR sensors. The fusion system software comprises multiple clients and a single server. The client and server are connected through inter-process communication. Multiple clients create processes to process the data from each LiDAR sensor via a multiprocessing method. Our approach involves a scheduling method for efficient multiprocessing. The server uses multithreading to optimize the internal functions. For internal communication within the fusion system, multiple clients and a single server are connected using the socket method. In sequential processing, the response time increases in proportion to the number of connected LiDAR sensors. By contrast, in the proposed software architecture, the response time decreases in inverse proportion to the number of LiDAR sensors. As LiDAR sensors become increasingly popular in the field of autonomous driving, the results of this study can be expected to make a substantial contribution to technology development in this domain.

A Human Gait Tracking System Using Dual Foot-Mounted IMU and Multiple 2D LiDARs.

This paper proposes a human gait tracking system using a dual foot-mounted IMU and multiple 2D LiDARs. The combining system aims to overcome the disadvantages of each single sensor system (the short tracking range of the single 2D LiDAR and the drift errors of the IMU system). The LiDARs act as anchors to mitigate the errors of an inertial navigation algorithm. In our system, two 2D LiDARs are used. LiDAR 1 is placed around the starting point, and LiDAR 2 is placed at the ending point (in straight walking) or at the turning point (in rectangular path walking). Using the LiDAR 1, we can estimate the initial headings and positions of each IMU without any calibration process. We also propose a method to calibrate two LiDARs that are placed far apart. Then, the measurement from two LiDARs can be combined in a Kalman filter and the smoother algorithm to correct the two estimated feet trajectories. If straight walking is detected, we update the current stride heading and the foot position using the previous stride headings. Then, it is used as a measurement update in the Kalman filter. In the smoother algorithm, a step width constraint is used as a measurement update. We evaluate the stride length estimation through a straight walking experiment along a corridor. The root mean square errors compared with an optical tracking system are less than 3 cm. The performance of proposed method is also verified with a rectangular path walking experiment.