3D LiDAR Research Articles

Navigation system for orchard spraying robot based on 3D LiDAR SLAM with NDT_ICP point cloud registration

Autonomous navigation of orchard spraying robots can effectively improve operational efficiency and reduce workload. The conventional robot navigation solutions based on the Global Navigation Satellite System (GNSS) are susceptible to signal loss in orchards due to interference from the canopy of fruit trees. In this paper, an autonomous navigation scheme for orchard spraying robots based on multi-sensor devices and the three-dimensional (3D) LiDAR Simultaneous Localization and Mapping (SLAM) technology. The initial step of the scheme involved the construction of a 3D point cloud map of the orchard using the 3D LiDAR SLAM algorithm. Subsequently, a designed point cloud map transformation algorithm was used to convert the 3D point cloud map of the orchard into a two-dimensional (2D) grid map containing key feature information of the orchard. To enhance the mapping accuracy of the orchard point cloud, a point cloud registration algorithm combining Iterative Closet Point (ICP) and Normal Distributions Transform (NDT) was proposed. This algorithm initially utilized the NDT algorithm for coarse point cloud registration to obtain transformation parameters. Then, the ICP algorithm was applied in conjunction with the obtained parameters for fine registration. Regarding obstacle avoidance, a cooperative obstacle avoidance algorithm based on the Robot Operating System (ROS) multithreaded communication mechanism was introduced. Experimental trials conducted in a standardized spindle-shaped peach orchard validated the successful completion of the SLAM mapping trajectory, which was achieved through the utilization of the proposed NDT_ICP point cloud registration algorithm. The average absolute pose error between the mapped trajectory and the reference trajectory was 1.173 m, with a standard deviation of 0.498 m. The robot's positioning and navigation errors increased with higher speeds, with the largest positioning deviation observed at 1.2 m/s speed, with a maximum lateral positioning error of 7.04 cm. The robot's average lateral navigation error did not exceed 16 cm, and the average heading deviation was less than 8° at different speeds of 0.4, 0.8 and 1.2 m/s. Both the positioning and navigation accuracies of the robot were sufficient to meet the autonomous operational requirements of orchard spraying robots.

Simple modulation of Lissajous MEMS laser beam scanning with reconfigurable structured light patterns for 3D imaging

Structured light 3D imaging systems commonly employ panel-based projectors or 1-axis MEMS mirrors with beam expander lens to project multi-frame barcodes or dot clouds, addressing challenges posed by objects with multi-scale feature sizes. However, these methods often result in large system volumes due to the required projection multi-lens modules, high hardware costs, or limited light pattern generation capabilities that hindering measurement precision enhancement. This paper introduces an innovative approach to reconfigurable spatial light pattern projection using a single bi-axial MEMS mirror with Lissajous scanning. In contrast to the pixel-by-pixel pre-defined image patterns encoding of conventional 2D laser beam scanning, the proposed method simply aligns the MEMS bi-axial resonance frequencies with laser pulse modulation, enabling the projection of diverse structured light patterns such as stripes, lines, dot matrices, and random dot clouds, which can adapt to different 3D imaging algorithms demands. It eliminates the need for multi-frame encoding and streamlines data caching, simplifies digital logic hardware. A prototype 3D imaging system was developed to demonstrate the mathematical model for laser modulation and the technical feasibility based on the proposed principle. Beyond its lens-free essence, the system supports focal-free optics and a compact projection form factor, which accommodates to a broad range of projection distances and field-of-views based on object’s location. 3D depth map of polynomial surface and blocks objects are extracted through single-frame pattern projection with a relative high accuracy. The presented modulation theory for diverse structured light pattern generation opens avenues for versatile and compact 3D imaging applications of LiDAR and robotic 3D vision.

Sparsity-Robust Feature Fusion for Vulnerable Road-User Detection with 4D Radar

Detecting vulnerable road users is a major challenge for autonomous vehicles due to their small size. Various sensor modalities have been investigated, including mono or stereo cameras and 3D LiDAR sensors, which are limited by environmental conditions and hardware costs. Radar sensors are a low-cost and robust option, with high-resolution 4D radar sensors being suitable for advanced detection tasks. However, they involve challenges such as few and irregularly distributed measurement points and disturbing artifacts. Learning-based approaches utilizing pillar-based networks show potential in overcoming these challenges. However, the severe sparsity of radar data makes detecting small objects with only a few points difficult. We extend a pillar network with our novel Sparsity-Robust Feature Fusion (SRFF) neck, which combines high- and low-level multi-resolution features through a lightweight attention mechanism. While low-level features aid in better localization, high-level features allow for better classification. As sparse input data are propagated through a network, the increasing effective receptive field leads to feature maps of different sparsities. The combination of features with different sparsities improves the robustness of the network for classes with few points.

Localization of hotspots via a lightweight system combining Compton imaging with a 3D lidar camera

Efficient and secure decommissioning of nuclear facilities demands advanced technologies. In this context, gamma-ray detection and imaging are crucial in identifying radioactive hotspots and monitoring radiation levels. Our study is dedicated to developing a gamma-ray detection system tailored for integration into robotic platforms for nuclear decommissioning, offering a safe and automated solution for this intricate task and ensuring the safety of human operators by mitigating radiation exposure and streamlining hotspot localization.Our approach integrates a Compton camera based 3D reconstruction algorithm with a single Timepix3 detector. This eliminates the need for a second detector and significantly reduces system weight and cost. Additionally, combining a 3D camera with the setup enhances hotspot visualization and interpretation, rendering it an ideal solution for practical nuclear decommissioning applications.In a proof-of-concept measurement utilizing a 137Cs source, our system accurately localized and visualized the source in 3D with an angular error of 1° and estimated the activity with a 3% relative error. This promising result underscores the system's potential for deployment in real-world decommissioning settings. Future endeavors will expand the technology's applications in authentic decommissioning scenarios and optimize its integration with robotic platforms.The outcomes of our study contribute to heightened safety and accuracy for nuclear decommissioning works through the advancement of cost-effective and efficient gamma-ray detection systems.

Regulating Intermediate 3D Features for Vision-Centric Autonomous Driving

Multi-camera perception tasks have gained significant attention in the field of autonomous driving. However, existing frameworks based on Lift-Splat-Shoot (LSS) in the multi-camera setting cannot produce suitable dense 3D features due to the projection nature and uncontrollable densification process. To resolve this problem, we propose to regulate intermediate dense 3D features with the help of volume rendering. Specifically, we employ volume rendering to process the dense 3D features to obtain corresponding 2D features (e.g., depth maps, semantic maps), which are supervised by associated labels in the training. This manner regulates the generation of dense 3D features on the feature level, providing appropriate dense and unified features for multiple perception tasks. Therefore, our approach is termed Vampire, stands for ``Volume rendering As Multi-camera Perception Intermediate feature REgulator''. Experimental results on the Occ3D and nuScenes datasets demonstrate that Vampire facilitates fine-grained and appropriate extraction of dense 3D features, and is competitive with existing SOTA methods across diverse downstream perception tasks like 3D occupancy prediction, LiDAR segmentation and 3D objection detection, while utilizing moderate GPU resources. We provide a video demonstration in the supplementary materials and Codes are available at github.com/cskkxjk/Vampire.

SwiftPillars: High-Efficiency Pillar Encoder for Lidar-Based 3D Detection

Lidar-based 3D Detection is one of the significant components of Autonomous Driving. However, current methods over-focus on improving the performance of 3D Lidar perception, which causes the architecture of networks becoming complicated and hard to deploy. Thus, the methods are difficult to apply in Autonomous Driving for real-time processing. In this paper, we propose a high-efficiency network, SwiftPillars, which includes Swift Pillar Encoder (SPE) and Multi-scale Aggregation Decoder (MAD). The SPE is constructed by a concise Dual-attention Module with lightweight operators. The Dual-attention Module utilizes feature pooling, matrix multiplication, etc. to speed up point-wise and channel-wise attention extraction and fusion. The MAD interconnects multiple scale features extracted by SPE with minimal computational cost to leverage performance. In our experiments, our proposal accomplishes 61.3% NDS and 53.2% mAP in nuScenes dataset. In addition, we evaluate inference time on several platforms (P4, T4, A2, MLU370, RTX3080), where SwiftPillars achieves up to 13.3ms (75FPS) on NVIDIA Tesla T4. Compared with PointPillars, SwiftPillars is on average 26.58% faster in inference speed with equivalent GPUs and a higher mAP of approximately 3.2% in the nuScenes dataset.

3D parallel pulsed chaos LiDAR system.

We propose and experimentally demonstrate a parallel pulsed chaos light detection and ranging (LiDAR) system with a high peak power, parallelism, and anti-interference. The system generates chaotic microcombs based on a chip-scale Si3N4 microresonator. After passing through an acousto-optic modulator, the continuous-wave chaotic microcomb can be transformed into a pulsed chaotic microcomb, in which each comb line provides pulsed chaos. Thus, a parallel pulsed chaos signal is generated. Using the parallel pulsed chaos as the transmission signal of LiDAR, we successfully realize a 4-m three-dimensional imaging experiment using a microelectromechanical mirror for laser scanning. The experimental results indicate that the parallel pulsed chaos LiDAR can detect twice as many pixels as direct detection continuous wave parallel chaos LiDAR under a transmission power of -6 dBm, a duty cycle of 25%, and a pulse repetition frequency of 100 kHz. By further increasing the transmission power to 10 dBm, we acquire an 11 cm × 10 cm image of a target scene with a resolution of 30 × 50 pixels. Finally, the anti-jamming ability of the system is evaluated, and the results show that the system can withstand interferences of at least 15 dB.

New Eldercare Robot with Path-Planning and Fall-Detection Capabilities

The rapid growth of the elderly population has led to an increased demand for effective and personalized eldercare solutions. In this paper, the design and development of an eldercare robot is presented. This robot is specifically tailored to meet the two specific challenges faced by the elderly. The first is the continuous indoor tracking of the elder, while the second is the fall detection. A comprehensive overview of the hardware and software components, as well as the control architecture of the robot is presented. The hardware design of the robot incorporates a range of features, including a perception system comprising a 2D Lidar, IMU, and camera for environment mapping, localization, and fall detection. The software stack of the robot is explained as consisting of layers for perception, mapping, and localization. The robot is tested experimentally to validate its path planning capability by using Hector SLAM and the RRT* technique. Experimental path planning has shown a positioning accuracy of 93.8% on average. Elderly fall detection is achieved by using the YOLOv7 algorithm at a percentage of 96%. Experimental results have been discussed and evaluated.

Fusion of Dense Airborne LiDAR and Multispectral Sentinel-2 and Pleiades Satellite Imagery for Mapping Riparian Forest Species Biodiversity at Tree Level.

Multispectral and 3D LiDAR remote sensing data sources are valuable tools for characterizing the 3D vegetation structure and thus understanding the relationship between forest structure, biodiversity, and microclimate. This study focuses on mapping riparian forest species in the canopy strata using a fusion of Airborne LiDAR data and multispectral multi-source and multi-resolution satellite imagery: Sentinel-2 and Pleiades at tree level. The idea is to assess the contribution of each data source in the tree species classification at the considered level. The data fusion was processed at the feature level and the decision level. At the feature level, LiDAR 2D attributes were derived and combined with multispectral imagery vegetation indices. At the decision level, LiDAR data were used for 3D tree crown delimitation, providing unique trees or groups of trees. The segmented tree crowns were used as a support for an object-based species classification at tree level. Data augmentation techniques were used to improve the training process, and classification was carried out with a random forest classifier. The workflow was entirely automated using a Python script, which allowed the assessment of four different fusion configurations. The best results were obtained by the fusion of Sentinel-2 time series and LiDAR data with a kappa of 0.66, thanks to red edge-based indices that better discriminate vegetation species and the temporal resolution of Sentinel-2 images that allows monitoring the phenological stages, helping to discriminate the species.

Adapting projection-based LiDAR semantic segmentation to natural domains

In this paper, an approach to the semantic segmentation of 3D LiDAR point clouds obtained from natural scenes is introduced. Using a state-of-the-art projection-based semantic segmentation model as the core segmentation network, several recent advances in projection-based 3D semantic segmentation methods are aggregated into a single model. These adaptions include: scan unfolding, soft-kNN post-processing, and multi-projection fusion. A novel Naïve Bayesian approach to multi-projection fusion which weights class probabilities based on the outputs of the base classifiers is proposed to further increase robustness.Quantitative and qualitative evaluations on several datasets, including scenes from both urban and natural environments; show that aggregating these adaptions into a single model can further improve the accuracy of state-of-the-art projection-based approaches. Finally, it is demonstrated that the novel Naïve Bayesian approach to multi-projection fusion addresses a number of the challenges inherent to natural data while also improving results on urban data.

LinK3D: Linear Keypoints Representation for 3D LiDAR Point Cloud

LinK3D: Linear Keypoints Representation for 3D LiDAR Point Cloud

A Staged Real-Time Ground Segmentation Algorithm of 3D LiDAR Point Cloud

Ground segmentation is a crucial task in the field of 3D LiDAR perception for autonomous driving. It is commonly used as a preprocessing step for tasks such as object detection and road extraction. However, the existing ground segmentation algorithms often struggle to meet the requirements of robustness and real-time performance due to significant variations in ground slopes and flatness across different scenes, as well as the influence of objects such as grass, flowerbeds, and trees in the environment. To address these challenges, this paper proposes a staged real-time ground segmentation algorithm. The proposed algorithm not only achieves high real-time performance but also exhibits improved robustness. Based on a concentric zone model, the algorithm filters out reflected noise points and vertical non-ground points in the first stage, improving the validity of the fitted ground plane. In the second stage, the algorithm effectively addresses the issue of undersegmentation of ground points through three steps: ground plane fitting, ground plane validity judgment, and ground plane repair. The experimental results on the SemanticKITTI dataset demonstrate that the proposed algorithm outperforms the existing methods in terms of segmentation results.

Robotic system for indoor illuminance map generation

The paper describes Romulux, a mobile robot equipped with 3D LiDAR carrying an illuminance meter and dedicated to measure indoor lighting quantity. The aim is to check compliance with lighting standards in terms of level and uniformity. The general architecture of the robot using the Robot Operating System (ROS) is presented, and technical choices are explained according to the different constraints. The space localization of illuminance data is computed with Synchronous Localization And Mapping (SLAM) algorithms. Then measurements are compared to standards requirements and interpolated into a user-defined grid to generate a dense illuminance map. Two experiments, one in a sport hall and one in an ice rink, are presented and results are exposed. The paper will follow the workflow from designing a robot to a functional system able to generate illuminance maps, compare with standards and save energy in case of over lighting.

Developing a Flying Explorer for Autonomous Digital Modelling in Wild Unknowns

Digital modelling stands as a pivotal step in the realm of Digital Twinning. The future trend of Digital Twinning involves automated exploration and environmental modelling in complex scenes. In our study, we propose an innovative solution for robot odometry, path planning, and exploration in unknown outdoor environments, with a focus on Digital modelling. The approach uses a minimum cost formulation with pseudo-randomly generated objectives, integrating multi-path planning and evaluation, with emphasis on full coverage of unknown maps based on feasible boundaries of interest. The approach allows for dynamic changes to expected targets and behaviours. The evaluation is conducted on a robotic platform with a lightweight 3D LiDAR sensor model. The robustness of different types of odometry is compared, and the impact of parameters on motion planning is explored. The consistency and efficiency of exploring completely unknown areas are assessed in both indoor and outdoor scenarios. The experiment shows that the method proposed in this article can complete autonomous exploration and environmental modelling tasks in complex indoor and outdoor scenes. Finally, the study concludes by summarizing the reasons for exploration failures and outlining future focuses in this domain.

Analysis of Beach Cusp Formation and Evolution Using High‐Frequency 3D Lidar Scans

AbstractIn this study, beach cusp characteristics were explored using 15 months of 3D lidar scans collected hourly at the Field Research Facility in Duck, NC. Fourier analyses were performed on lidar‐derived beach elevation contours to generate spatial cusp spectra. Active cusp events were identified on the basis of the location and magnitude of each spectrum's peak and used to evaluate conditions during cusp formation and evolution. Cusps primarily developed during times with normally‐incident, long period, low energy wave conditions with low frequency spread, and reflective beach conditions. The upper and lower beaches often exhibited different behavior and morphologies, with persistent upper‐beach cusps lasting days to months and dynamic lower‐beach cusps evolving over individual tidal cycles. At times, beaches exhibiting multiple cusp systems reverted to a single cusp system extending over the entire beach when the high‐tide waterline reached the upper‐beach cusps, with the location and spacing of the resulting lower‐beach cusps controlled by the upper‐beach cusps. These observations are consistent with a “morphological coupling” hypothesis proposing that hydrodynamic‐morphodynamic feedbacks between the swash and upper‐beach cusps can result in the formation of lower‐beach cusps with a related wavelength as the tide falls. However, there were also times when the high‐tide waterline reached the upper‐beach cusps that did not result in a unified beach state. These results suggest that while morphological coupling is often an important factor in controlling the development of new lower‐beach cusps, this coupling cannot initiate cusp formation in hydrodynamic conditions outside those favorable for cusp activity.

An Indoor 2-D LiDAR SLAM and Localization Method Based on Artificial Landmark Assistance

An Indoor 2-D LiDAR SLAM and Localization Method Based on Artificial Landmark Assistance

Moving object detection algorithm and motion capture based on 3D LiDAR

Moving object detection algorithm and motion capture based on 3D LiDAR

A Smart Cane Based on 2D LiDAR and RGB-D Camera Sensor-Realizing Navigation and Obstacle Recognition

A Smart Cane Based on 2D LiDAR and RGB-D Camera Sensor-Realizing Navigation and Obstacle Recognition

Transformer based 3D semantic segmentation of urban bicycle infrastructure

ABSTRACT This work investigates Point Cloud Semantic Segmentation (PCSS) to detect bicycle paths from 3D point cloud data of bicycle-mounted LiDARs. For the task of PCSS, an existing Convolutional Neural Network architecture (CNN) is first pre-trained on the Semantic KITTI data set, an important data set for PCSS in LiDAR scans of driving situations. Furthermore, a new, semantically-labelled 3D LiDAR data set, the Salzburg Bicycle LiDAR Data Set (SBLD), is presented, which consists of 16,008 point clouds recorded by a ROS2-enabled sensor bicycle with five 3D LiDARs covering every direction. After fine-tuning the CNN on the SBLD train set, the segmentation performance is evaluated on the SBLD test set. The CNN shows promising results in recognising bike paths, vegetation, terrain and buildings in the SBLD. To further push the segmentation performance, the existing CNN is enhanced with self-attention blocks. The evaluation of this enhanced architecture shows a performance gain of 2.79% points compared to the original architecture. The proposed segmentation approach shows practical potential for detecting bike paths. The SBLD is provided to the scientific community to further drive research in the field of sensor bicycles.

Self-adaptive learning particle swarm optimization-based path planning of mobile robot using 2D Lidar environment

AbstractThe loading and unloading operations of smart logistic application robots depend largely on their perception system. However, there is a paucity of study on the evaluation of Lidar maps and their SLAM algorithms in complex environment navigation system. In the proposed work, the Lidar information is finetuned using binary occupancy grid approach and implemented Improved Self-Adaptive Learning Particle Swarm Optimization (ISALPSO) algorithm for path prediction. The approach makes use of 2D Lidar mapping to determine the most efficient route for a mobile robot in logistical applications. The Hector SLAM method is used in the Robot Operating System (ROS) platform to implement mobile robot real-time location and map building, which is subsequently transformed into a binary occupancy grid. To show the path navigation findings of the proposed methodologies, a navigational model has been created in the MATLAB 2D virtual environment using 2D Lidar mapping point data. The ISALPSO algorithm adapts its parameters inertia weight, acceleration coefficients, learning coefficients, mutation factor, and swarm size, based on the performance of the generated path. In comparison to the other five PSO variants, the ISALPSO algorithm has a considerably shorter path, a quick convergence rate, and requires less time to compute the distance between the locations of transporting and unloading environments, based on the simulation results that was generated and its validation using a 2D Lidar environment. The efficiency and effectiveness of path planning for mobile robots in logistic applications are validated using Quanser hardware interfaced with 2D Lidar and operated in environment 3 using proposed algorithm for production of optimal path.