Lidar Sensors Research Articles

Learning Cross-modality Interaction for Robust Depth Perception of Autonomous Driving

As one of the fundamental tasks of autonomous driving, depth perception aims to perceive physical objects in three dimensions and to judge their distances away from the ego vehicle. Although great efforts have been made for depth perception, LiDAR-based and camera-based solutions have limitations with low accuracy and poor robustness for noise input. With the integration of monocular cameras and LiDAR sensors in autonomous vehicles, in this article, we introduce a two-stream architecture to learn the modality interaction representation under the guidance of an image reconstruction task to compensate for the deficiencies of each modality in a parallel manner. Specifically, in the two-stream architecture, the multi-scale cross-modality interactions are preserved via a cascading interaction network under the guidance of the reconstruction task. Next, the shared representation of modality interaction is integrated to infer the dense depth map due to the complementarity and heterogeneity of the two modalities. We evaluated the proposed solution on the KITTI dataset and CALAR synthetic dataset. Our experimental results show that learning the coupled interaction of modalities under the guidance of an auxiliary task can lead to significant performance improvements. Furthermore, our approach is competitive against the state-of-the-art models and robust against the noisy input. The source code is available at https://github.com/tonyFengye/Code/tree/master .

Enhancing Reliability of Advanced Driver-Assistance Systems through Predictive Maintenance and Data-Driven Insights

The advancement of Advanced Driver Assistance Systems (ADAS) marks a pivotal evolution in automotive technology, aiming to enhance road safety and driving efficiency through a wide array of functionalities like blind spot detection, emergency braking, and adaptive cruise control. This research paper delves into the operational integrity, performance metrics, and maintenance strategies of ADAS components, underpinned by a comprehensive methodology involving data collection, pre-processing, feature engineering, machine learning model development, and rigorous validation processes. Systematic inspection of ADAS components indicates their importance in vehicle safety and reliability. The visibility, distance, speed, and steering angle of front cameras, LiDAR, radar, and ultrasonic sensors are carefully evaluated. Maintenance logs show proactive error code management, boosting efficiency. SVM, Gradient Boosting, and Random Forest machine learning models predicted ADAS component failures during validation and testing. Random Forest scored 90% accuracy, 92% precision, 88% recall, and 90% F1. Gradient Boosting was the most accurate, with 93% accuracy, 94% precision, 91% recall, and 92% F1. SVM predicted ADAS component failures with 88% accuracy, 90% precision, 85% recall, and 87% F1 score. Machine learning helps shift from reactive to proactive maintenance. Modelling sensor signal quality, actuator reaction times, error code frequencies, and maintenance intervals enables predictive maintenance and failure detection. Feature engineering builds predictive models using maintenance logs and operational KPIs. The models predict ADAS component failures, boosting vehicle safety and dependability. Using external data improves predictive maintenance models. The maintenance model's adaptability and forecast accuracy are proved by ADAS operation after traffic, accident, and manufacturer upgrades. Predictive maintenance and machine learning improve ADAS dependability and safety, the study found. Advanced analytics and data-driven insights can reduce automotive system failures, improving safety and reliability.

A real-time digital twin for active safety in an aircraft hangar

The aerospace industry prioritises safety protocols to prevent accidents that can result in injuries, fatalities, or aircraft damage. One of the potential hazards that can occur while manoeuvring aircraft in and out of a hangar is collisions with other aircraft or buildings, which can lead to operational disruption and costly repairs. To tackle this issue, we have developed the Smart Hangar project, which aims to alert personnel of increased risks and prevent incidents from happening. The Smart Hangar project uses computer vision, LiDAR, and ultra-wideband sensors to track all objects and individuals within the hangar space. These data inputs are combined to form a real-time 3D Digital Twin (DT) of the hangar environment. The Active Safety system then uses the DT to perform real-time path planning, collision prediction, and safety alerts for tow truck drivers and hangar personnel. This paper provides a detailed overview of the system architecture, including the technologies used, and highlights the system’s performance. By implementing this system, we aim to reduce the risk of accidents in the aerospace industry and increase safety for all personnel involved. Additionally, we identify future research directions for the Smart Hangar project.

Global estimation of range resolved thermodynamic profiles from micropulse differential absorption lidar

We demonstrate thermodynamic profile estimation with data obtained using the MicroPulse DIAL such that the retrieval is entirely self contained. The only external input is surface meteorological variables obtained from a weather station installed on the instrument. The estimator provides products of temperature, absolute humidity and backscatter ratio such that cross dependencies between the lidar data products and raw observations are accounted for and the final products are self consistent. The method described here is applied to a combined oxygen DIAL, potassium HSRL, water vapor DIAL system operating at two pairs of wavelengths (nominally centered at 770 and 828 nm). We perform regularized maximum likelihood estimation through the Poisson Total Variation technique to suppress noise and improve the range of the observations. A comparison to 119 radiosondes indicates that this new processing method produces improved temperature retrievals, reducing total errors to less than 2 K below 3 km altitude and extending the maximum altitude of temperature retrievals to 5 km with less than 3 K error. The results of this work definitively demonstrates the potential for measuring temperature through the oxygen DIAL technique and furthermore that this can be accomplished with low-power semiconductor-based lidar sensors.

Revolutionizing Depth Sensing: A Review Study of Apple LiDAR Sensor for as-built Scanning Applications

Incorporating the LiDAR sensor in the most recent Apple devices represents a substantial development in 3D mapping technology. Meanwhile, Apple's Lidar is still a new sensor. Therefore, this article reviews the potential uses of the Apple Lidar sensor in various fields, including engineering and construction, focusing on indoor and outdoor as-built 3D mapping and cultural heritage conservation. The affordable cost and shorter observation times compared to traditional surveying and other remote sensing techniques make the Apple Lidar an attractive choice among scholars and professionals. This article highlights the need for continued research on the Apple LiDAR sensor technology while discussing its specifications and limitations. A comprehensive review found that the Apple LiDAR sensor has shown promise in capturing 3D point clouds of small to medium-sized objects with exceptional detail. This technology offers a cost-effective and accessible option to scan areas faster and analyze data more quickly and automatically for 3D mapping and modelling in indoor and outdoor environments, particularly in areas with restricted access when using other traditional techniques. It also opens the door for more sophisticated applications in future studies, including cultural heritage conservation, archaeological investigations and feature detection, building health monitoring and many more.

Tracking Multiple Vehicles with a Flexible Life Cycle Strategy Based on Roadside LiDAR Sensors

Tracking Multiple Vehicles with a Flexible Life Cycle Strategy Based on Roadside LiDAR Sensors

Investigation of collision estimation with vehicle and pedestrian using CARLA simulation software

The effectiveness of object detection systems in diverse driving environments is crucial in the growing field of automotive safety. The increasing frequency of traffic accidents, especially at busy intersections with heavy traffic and limited visibility, highlights the pressing requirement for advanced vehicle detection systems. Prior to implementing the real-time experiment, it is advisable first to conduct a simulation in order to gain a deeper understanding of the practical implementation in real-time scenarios. On the other hand, this approach has the potential to reduce both time and cost significantly. The system utilised a software-based solution by implementing the CARLA simulator. This study aims to analyse vehicle detection at T-junctions, cross-junctions, and roundabouts using image data obtained from the CARLA platform. Subsequent analysis differentiates between vehicles and non-vehicle objects in the dataset. The model concludes by proposing Python-based integrative solutions to enhance object detection systems for diverse roads and atmospheric situations. The significance of this study is evaluating the probability of accidents by tracking key factors like vehicle speed, distance, and density on various road types. In future research, it will be essential to investigate how different weather conditions, including rain, haze, and low-light scenarios, affect on sensor performance, specifically LiDAR sensors. Advanced machine learning techniques are proposed to evaluate the effectiveness of the vehicle detection system in collecting key parameters like vehicle count, speed, and distance in junction and roundabout scenarios. These findings have important implications for the advancement of more efficient, context-aware detection systems in the automotive sector.

Simulation of Mobile Robot Navigation System using Hector SLAM on ROS

The ability to move from one point to the destination point autonomously is very necessary in AMR robots, to be able to meet this, the robot must be able to detect the surrounding environment and know its location to the environment, the Hector SLAM algorithm is added using the LIDAR sensor, and to find out the ability of the LIDAR sensor with the Hector SLAM and computer specifications in order to process properly, a simulation of the HECTOR SLAM with the LIDAR sensor was made. Simulation is carried out by creating an environment map on the Gazebo. Then explore environmental mapping using Hokuyo LIDAR which has been added to the turtlebot3 model waffle_pi to the simulated environment map. In this study, a model of the second floor lobby environment and Brail of the Batam State Polytechnic was used which was made in the form of a simulation on the Gazebo, where robots that have used LIDAR will be controlled with a keyboard around the simulation environment, where simultaneously the mapping and localization process runs and the process can be seen on the Rviz in real-time, where LIDAR will send data in the form of distance readings that will be received by Hector SLAM. The results of this study are expected that Hector SLAM using LIDAR sensor simulation can produce environmental mapping and localization in the simulation environment and obtain a minimum computer specification to process data from the SLAM Hector process using LIDAR sensors.

Online Calibration of Extrinsic Parameters for Solid-State LIDAR Systems.

This work addresses the challenge of calibrating multiple solid-state LIDAR systems. The study focuses on three different solid-state LIDAR sensors that implement different hardware designs, leading to distinct scanning patterns for each system. Consequently, detecting corresponding points between the point clouds generated by these LIDAR systems-as required for calibration-is a complex task. To overcome this challenge, this paper proposes a method that involves several steps. First, the measurement data are preprocessed to enhance its quality. Next, features are extracted from the acquired point clouds using the Fast Point Feature Histogram method, which categorizes important characteristics of the data. Finally, the extrinsic parameters are computed using the Fast Global Registration technique. The best set of parameters for the pipeline and the calibration success are evaluated using the normalized root mean square error. In a static real-world indoor scenario, a minimum root mean square error of 7 cm was achieved. Importantly, the paper demonstrates that the presented approach is suitable for online use, indicating its potential for real-time applications. By effectively calibrating the solid-state LIDAR systems and establishing point correspondences, this research contributes to the advancement of multi-LIDAR fusion and facilitates accurate perception and mapping in various fields such as autonomous driving, robotics, and environmental monitoring.

The cross-disciplinary influence of aerial measurement techniques: Exploring archaeological studies through photogrammetry and LiDAR

This research aimed to evaluate the efficacy of contemporary digitization methods for archaeological sites, specifically focusing on aerial approaches. The study is concentrated on the examination of two primary methods: LiDAR sensor and photogrammetry. The chosen case study revolves around the Camp of the V Macedonica Legion, a pivotal feature of the former Roman city now known as Turda, a city in Cluj County, Transylvania, Romania. Through analysis and comparison, the paper revealed that each of these aerial 3D scanning techniques possesses unique strengths, which, when combined, offer a comprehensive approach to archaeological digitization. These complementary attributes must be carefully considered and integrated considering the specific requirements and objectives of the archaeological project at hand when selecting the appropriate method. Furthermore, the research underscores the pivotal role of presenting archaeology in 3D, emphasizing its significant impact on both public and academic audiences. Achieving this presentation necessitates the utilization of specialized software for modelling, rendering, and animating objects of interest, thus enhancing the accessibility and engagement of archaeological findings. The comprehensive findings of this study demonstrate the vast potential offered by aerial 3D scans in the field of archaeology. Moreover, it serves as a potential call for the meticulous selection of the analysis method, recognizing its crucial role as a valuable tool for researchers and archaeologists. By leveraging 3D technologies in their activities, professionals in the field can significantly enhance the accuracy, depth, and accessibility of their archaeological investigations, thereby enriching our understanding of the past.

Sparsity-Robust Feature Fusion for Vulnerable Road-User Detection with 4D Radar

Detecting vulnerable road users is a major challenge for autonomous vehicles due to their small size. Various sensor modalities have been investigated, including mono or stereo cameras and 3D LiDAR sensors, which are limited by environmental conditions and hardware costs. Radar sensors are a low-cost and robust option, with high-resolution 4D radar sensors being suitable for advanced detection tasks. However, they involve challenges such as few and irregularly distributed measurement points and disturbing artifacts. Learning-based approaches utilizing pillar-based networks show potential in overcoming these challenges. However, the severe sparsity of radar data makes detecting small objects with only a few points difficult. We extend a pillar network with our novel Sparsity-Robust Feature Fusion (SRFF) neck, which combines high- and low-level multi-resolution features through a lightweight attention mechanism. While low-level features aid in better localization, high-level features allow for better classification. As sparse input data are propagated through a network, the increasing effective receptive field leads to feature maps of different sparsities. The combination of features with different sparsities improves the robustness of the network for classes with few points.

RadarMOSEVE: A Spatial-Temporal Transformer Network for Radar-Only Moving Object Segmentation and Ego-Velocity Estimation

Moving object segmentation (MOS) and Ego velocity estimation (EVE) are vital capabilities for mobile systems to achieve full autonomy. Several approaches have attempted to achieve MOSEVE using a LiDAR sensor. However, LiDAR sensors are typically expensive and susceptible to adverse weather conditions. Instead, millimeter-wave radar (MWR) has gained popularity in robotics and autonomous driving for real applications due to its cost-effectiveness and resilience to bad weather. Nonetheless, publicly available MOSEVE datasets and approaches using radar data are limited. Some existing methods adopt point convolutional networks from LiDAR-based approaches, ignoring the specific artifacts and the valuable radial velocity information of radar measurements, leading to suboptimal performance. In this paper, we propose a novel transformer network that effectively addresses the sparsity and noise issues and leverages the radial velocity measurements of radar points using our devised radar self- and cross-attention mechanisms. Based on that, our method achieves accurate EVE of the robot and performs MOS using only radar data simultaneously. To thoroughly evaluate the MOSEVE performance of our method, we annotated the radar points in the public View-of-Delft (VoD) dataset and additionally constructed a new radar dataset in various environments. The experimental results demonstrate the superiority of our approach over existing state-of-the-art methods. The code is available at https://github.com/ORCAUboat/RadarMOSEVE.

MsLPCC: A Multimodal-Driven Scalable Framework for Deep LiDAR Point Cloud Compression

LiDAR sensors are widely used in autonomous driving, and the growing storage and transmission demands have made LiDAR point cloud compression (LPCC) a hot research topic. To address the challenges posed by the large-scale and uneven-distribution (spatial and categorical) of LiDAR point data, this paper presents a new multimodal-driven scalable LPCC framework. For the large-scale challenge, we decouple the original LiDAR data into multi-layer point subsets, compress and transmit each layer separately, so as to ensure the reconstruction quality requirement under different scenarios. For the uneven-distribution challenge, we extract, align, and fuse heterologous feature representations, including point modality with position information, depth modality with spatial distance information, and segmentation modality with category information. Extensive experimental results on the benchmark SemanticKITTI database validate that our method outperforms 14 recent representative LPCC methods.

SDAC: A Multimodal Synthetic Dataset for Anomaly and Corner Case Detection in Autonomous Driving

Nowadays, closed-set perception methods for autonomous driving perform well on datasets containing normal scenes. However, they still struggle to handle anomalies in the real world, such as unknown objects that have never been seen while training. The lack of public datasets to evaluate the model performance on anomaly and corner cases has hindered the development of reliable autonomous driving systems. Therefore, we propose a multimodal Synthetic Dataset for Anomaly and Corner case detection, called SDAC, which encompasses anomalies captured from multi-view cameras and the LiDAR sensor, providing a rich set of annotations for multiple mainstream perception tasks. SDAC is the first public dataset for autonomous driving that categorizes anomalies into object, scene, and scenario levels, allowing the evaluation under different anomalous conditions. Experiments show that closed-set models suffer significant performance drops on anomaly subsets in SDAC. Existing anomaly detection methods fail to achieve satisfactory performance, suggesting that anomaly detection remains a challenging problem. We anticipate that our SDAC dataset could foster the development of safe and reliable systems for autonomous driving.

DI-V2X: Learning Domain-Invariant Representation for Vehicle-Infrastructure Collaborative 3D Object Detection

Vehicle-to-Everything (V2X) collaborative perception has recently gained significant attention due to its capability to enhance scene understanding by integrating information from various agents, e.g., vehicles, and infrastructure. However, current works often treat the information from each agent equally, ignoring the inherent domain gap caused by the utilization of different LiDAR sensors of each agent, thus leading to suboptimal performance. In this paper, we propose DI-V2X, that aims to learn Domain-Invariant representations through a new distillation framework to mitigate the domain discrepancy in the context of V2X 3D object detection. DI-V2X comprises three essential components: a domain-mixing instance augmentation (DMA) module, a progressive domain-invariant distillation (PDD) module, and a domain-adaptive fusion (DAF) module. Specifically, DMA builds a domain-mixing 3D instance bank for the teacher and student models during training, resulting in aligned data representation. Next, PDD encourages the student models from different domains to gradually learn a domain-invariant feature representation towards the teacher, where the overlapping regions between agents are employed as guidance to facilitate the distillation process. Furthermore, DAF closes the domain gap between the students by incorporating calibration-aware domain-adaptive attention. Extensive experiments on the challenging DAIR-V2X and V2XSet benchmark datasets demonstrate DI-V2X achieves remarkable performance, outperforming all the previous V2X models. Code is available at https://github.com/Serenos/DI-V2X.

Effects of high-quality elevation data and explanatory variables on the accuracy of flood inundation mapping via Height Above Nearest Drainage

Abstract. Given the availability of high-quality and high-spatial-resolution digital elevation maps (DEMs) from the United States Geological Survey's 3D Elevation Program (3DEP), derived mostly from light detection and ranging (lidar) sensors, we examined the effects of these DEMs at various spatial resolutions on the quality of flood inundation map (FIM) extents derived from a terrain index known as Height Above Nearest Drainage (HAND). We found that using these DEMs improved the quality of resulting FIM extents at around 80 % of the catchments analyzed when compared to using DEMs from the National Hydrography Dataset Plus High Resolution (NHDPlusHR) program. Additionally, we varied the spatial resolution of the 3DEP DEMs at 3, 5, 10, 15, and 20 m (meters), and the results showed no significant overall effect on FIM extent quality across resolutions. However, further analysis at coarser resolutions of 60 and 90 m revealed a significant degradation in FIM skill, highlighting the limitations of using extremely coarse-resolution DEMs. Our experiments demonstrated a significant burden in terms of the computational time required to produce HAND and related data at finer resolutions. We fit a multiple linear regression model to help explain catchment-scale variations in the four metrics employed and found that the lack of reservoir flooding or inundation upstream of river retention systems was a significant factor in our analysis. For validation, we used Interagency Flood Risk Management (InFRM) Base Level Engineering (BLE)-produced FIM extents and streamflows at the 100- and 500-year event magnitudes in a sub-region in eastern Texas.

Combining a climate-permafrost model with fine resolution remote sensor products to quantify active-layer thickness at local scales

Quantification of active-layer thickness (ALT) over seasonally frozen terrains is critical to understand the impacts of climate warming on permafrost ecosystems in cold regions. Current large-scale process-based models cannot characterize the heterogeneous response of local landscapes to homogeneous climatic forcing. Here we linked a climate-permafrost model with a machine learning solution to indirectly quantify soil conditions reflected in the edaphic factor using high resolution remote sensor products, and then effectively estimated ALT across space and time down to local scales. Our nine-year field measurements during 2014–2022 and coincident high resolution airborne hyperspectral, lidar, and spaceborne sensor products provided a unique opportunity to test the developed protocol across two permafrost experiment stations in lowland terrains of Interior Alaska. Our developed model could explain over 60% of the variance of the field measured ALT for estimating the shallowest and deepest ALT in 2015 and 2019, suggesting the potential of the designed procedure for projecting local varying terrain response to long-term climate warming scenarios. This work will enhance the National Aeronautics and Space Administration’s Arctic-Boreal Vulnerability Experiment’s mission of combining field, airborne, and spaceborne sensor products to understand the coupling of permafrost ecosystems and climate change.

Navigating the future: A comprehensive survey of localization systems in autonomous vehicles

<p>This survey paper provides an in-depth analysis of localization systems for autonomous vehicles (AVs), a cornerstone technology crucial for the safe and efficient operation of AVs. The paper encompasses a comprehensive examination of various localization technologies, their comparative analysis, the challenges they face, and the emerging trends shaping their future. We begin by exploring the principal technologies employed in AV localization: GPS, LiDAR, radar, cameras, and ultrasonic sensors. Each technology is scrutinized to understand its strengths, weaknesses, and applicability in different environmental contexts. GPS offers broad geographical positioning but struggles with precision in dense urban areas. LiDAR provides high-resolution mapping but is hindered by adverse weather conditions. Radar ensures reliability in poor visibility but lacks the finer details provided by optical systems. Cameras offer rich visual data but are dependent on lighting conditions, and ultrasonic sensors, while effective for close-range detection, are limited to low-speed applications. The paper then presents a comparative analysis of these technologies against key performance metrics such as accuracy, reliability, latency, scalability, and cost-effectiveness. This analysis is critical in understanding the suitability of each technology for various driving scenarios, from congested urban streets to open highways and in diverse weather conditions. Challenges in AV localization are multifaceted, encompassing technological, environmental, and system integration issues. Technological challenges include sensor limitations and computational constraints, while environmental factors like weather and urban infrastructure significantly impact localization accuracy. System integration poses another significant challenge, necessitating seamless interaction between localization systems and other vehicle control systems for real-time decision-making. Emerging trends and potential solutions to these challenges are discussed, highlighting advancements in sensor technology, AI, machine learning, and 5G communications. These developments promise to overcome existing limitations and enhance the accuracy and reliability of AV localization. The integration of localization systems with smart city infrastructures and the implementation of collaborative technologies like V2X communication are speculated to further augment AV capabilities. In conclusion, the paper emphasizes that the future of AV localization is intrinsically linked to the continuous evolution of technologies and their integration. Addressing current challenges and harnessing emerging trends are pivotal for the advancement of AVs, steering us toward a future of autonomous transportation characterized by increased safety, efficiency, and reliability.<strong></strong></p>

Playing Flappy Bird Based on Motion Recognition Using a Transformer Model and LIDAR Sensor.

A transformer neural network is employed in the present study to predict Q-values in a simulated environment using reinforcement learning techniques. The goal is to teach an agent to navigate and excel in the Flappy Bird game, which became a popular model for control in machine learning approaches. Unlike most top existing approaches that use the game's rendered image as input, our main contribution lies in using sensory input from LIDAR, which is represented by the ray casting method. Specifically, we focus on understanding the temporal context of measurements from a ray casting perspective and optimizing potentially risky behavior by considering the degree of the approach to objects identified as obstacles. The agent learned to use the measurements from ray casting to avoid collisions with obstacles. Our model substantially outperforms related approaches. Going forward, we aim to apply this approach in real-world scenarios.

Change Detection in Point Clouds Using 3D Fractal Dimension

The management of large point clouds obtained by LiDAR sensors is an important topic in recent years due to the widespread use of this technology in a wide variety of applications and the increasing volume of data captured. One of the main applications of LIDAR systems is the study of the temporal evolution of the real environment. In open environments, it is important to know the evolution of erosive processes or landscape transformation. In the context of civil engineering and urban environments, it is useful for monitoring urban dynamics and growth, and changes during the construction of buildings or infrastructure facilities. The main problem with change detection (CD) methods is erroneous detection due to precision errors or the use of different capture devices at different times. This work presents a method to compare large point clouds, based on the study of the local fractal dimension of point clouds at multiple scales. Our method is robust in the presence of environmental and sensor factors that produce abnormal results with other methods. Furthermore, it is more stable than others in cases where there is no significant displacement of points but there is a local alteration of the structure of the point cloud. Furthermore, the precision can be adapted to the complexity and density of the point cloud. Finally, our solution is faster than other CD methods such as distance-based methods and can run at O(1) under some conditions, which is important when working with large datasets. All these improvements make the proposed method more suitable than the others to solve complex problems with LiDAR data, such as storage, time series data management, visualization, etc.