Fusion Of Light Detection And Ranging Research Articles

Dynamic Occupancy Grid Map with Semantic Information Using Deep Learning-Based BEVFusion Method with Camera and LiDAR Fusion.

In the field of robotics and autonomous driving, dynamic occupancy grid maps (DOGMs) are typically used to represent the position and velocity information of objects. Although three-dimensional light detection and ranging (LiDAR) sensor-based DOGMs have been actively researched, they have limitations, as they cannot classify types of objects. Therefore, in this study, a deep learning-based camera-LiDAR sensor fusion technique is employed as input to DOGMs. Consequently, not only the position and velocity information of objects but also their class information can be updated, expanding the application areas of DOGMs. Moreover, unclassified LiDAR point measurements contribute to the formation of a map of the surrounding environment, improving the reliability of perception by registering objects that were not classified by deep learning. To achieve this, we developed update rules on the basis of the Dempster-Shafer evidence theory, incorporating class information and the uncertainty of objects occupying grid cells. Furthermore, we analyzed the accuracy of the velocity estimation using two update models. One assigns the occupancy probability only to the edges of the oriented bounding box, whereas the other assigns the occupancy probability to the entire area of the box. The performance of the developed perception technique is evaluated using the public nuScenes dataset. The developed DOGM with object class information will help autonomous vehicles to navigate in complex urban driving environments by providing them with rich information, such as the class and velocity of nearby obstacles.

Semi-supervised cross-domain feature fusion classification network for coastal wetland classification with hyperspectral and LiDAR data

Multi-source remote sensing monitoring plays a crucial part in the ecological protection and restoration of coastal wetlands. However, due to the inaccessible of wetlands environment, lacking of labeled samples is a challenge in wetland classification. In this article, an unsupervised cross-domain feature fusion and supervised classification network (UF2SCN) is proposed for coastal wetland classification, which fuses hyperspectral image (HSI) and light detection and ranging (LiDAR) data. First, an unsupervised single branch end to end network is developed to get HSI and LiDAR fusion feature, in which a feature extraction model with spectral attention is deployed to obtain the average distribution characteristics of all samples, and the HSI and LiDAR data is utilized to guide the whole process. Second, a supervised classification network with spatial attention is applied to used fusion feature for classification, which uses the limited samples. Finally, a two stages training strategy is proposed to improve the ability of feature fusion. Experiments conducted on two coastal wetland datasets created by ourselves prove the validity of the proposed method on HSI and LiDAR classification for coastal wetland.

Distinguishing forest types in restored tropical landscapes with UAV-borne LIDAR

Forest landscape restoration is a global priority to mitigate negative effects of climate change, conserve biodiversity, and ensure future sustainability of forests, with international pledges concentrated in tropical forest regions. To hold restoration efforts accountable and monitor their outcomes, traditional strategies for monitoring tree cover increase by field surveys are falling short, because they are labor-intensive and costly. Meanwhile remote sensing approaches have not been able to distinguish different forest types that result from utilizing different restoration approaches (conservation versus production focus). Unoccupied Aerial Vehicles (UAV) with light detection and ranging (LiDAR) sensors can observe forests` vertical and horizontal structural variation, which has the potential to distinguish forest types. In this study, we explored this potential of UAV-borne LiDAR to distinguish forest types in landscapes under restoration in southeastern Brazil by using a supervised classification method. The study area encompassed 150 forest plots with six forest types divided in two forest groups: conservation (remnant forests, natural regrowth, and active restoration plantings) and production (monoculture, mixed, and abandoned plantations) forests. UAV-borne LiDAR data was used to extract several Canopy Height Model (CHM), voxel, and point cloud statistic based metrics at a high resolution for analysis. Using a random forest classification model we could successfully classify conservation and production forests (90% accuracy). Classification of the entire set of six types was less accurate (62%) and the confusion matrix showed a divide between conservation and production types. Understory Leaf Area Index (LAI) and the variation in vegetation density in the upper half of the canopy were the most important classification metrics. In particular, LAI understory showed the most variation, and may help advance ecological understanding in restoration. The difference in classification success underlines the difficulty of distinguishing individual forest types that are very similar in management, regeneration dynamics, and structure. In a restoration context, we showed the ability of UAV-borne LiDAR to identify complex forest structures at a plot scale and identify groups and types widely distributed across different restored landscapes with medium to high accuracy. Future research may explore a fusion of UAV-borne LiDAR with optical sensors, and include successional stages in the analyses to further characterize and distinguish forest types and their contributions to landscape restoration.

Automatic Obstacle Detection Method for the Train Based on Deep Learning

Automatic obstacle detection is of great significance for improving the safety of train operation. However, the existing autonomous operation of trains mainly depends on the signaling control system and lacks the extra equipment to perceive the environment. To further enhance the efficiency and safety of the widely deployed fully automatic operation (FAO) systems of the train, this study proposes an intelligent obstacle detection system based on deep learning. It collects perceptual information from industrial cameras and light detection and ranging (LiDAR), and mainly implements the functionality including rail region detection, obstacle detection, and visual–LiDAR fusion. Specifically, the first two parts adopt deep convolutional neural network (CNN) algorithms for semantic segmentation and object detection to pixel-wisely identify the rail track area ahead and detect the potential obstacles on the rail track, respectively. The visual–LiDAR fusion part integrates the visual data with the LiDAR data to achieve environmental perception for all weather conditions. It can also determine the geometric relationship between the rail track and obstacles to decide whether to trigger a warning alarm. Experimental results show that the system proposed in this study has strong performance and robustness. The system perception rate (precision) is 99.994% and the recall rate reaches 100%. The system, applied to the metro Hong Kong Tsuen Wan line, effectively improves the safety of urban rail train operation.

RI-Fusion: 3D Object Detection Using Enhanced Point Features With Range-Image Fusion for Autonomous Driving

The 3D object detection is becoming indispensable for environmental perception in autonomous driving. Light detection and ranging (LiDAR) point clouds often fail to distinguish objects with similar structures and are quite sparse for distant or small objects, thereby introducing false and missed detections. To address these issues, LiDAR is often fused with cameras due to the rich textural information provided by images. However, current fusion methods suffer the inefficient data representation and inaccurate alignment of heterogeneous features, leading to poor precision and low efficiency. To this end, we propose a plug-and-play module termed range-image fusion (RI-Fusion) to achieve an effective fusion of LiDAR and camera data, designed to be easily accessible by existing mainstream LiDAR-based algorithms. In this process, we design an image and point cloud alignment method by converting a point cloud into a compact range-view representation through a spherical coordinate transformation. The range image is then integrated with a corresponding camera image utilizing an attention mechanism. The original range image is then concatenated with fusion features to retain point cloud information, and the results are projected onto a spatial point cloud. Finally, the feature-enhanced point cloud can be input into a LiDAR-based 3D object detector. The results of validation experiments involving the KITTI 3D object detection benchmark showed that our proposed fusion method significantly enhanced multiple mainstream LiDAR-based 3D object detectors, PointPillars, SECOND, and Part <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{A}{^{2}}$ </tex-math></inline-formula> , improving the 3D mAP (mean Average Precision) by 3.61%, 2.98%, and 1.27%, respectively, particularly for small objects such as pedestrians and cyclists.

Estimation of ship berthing parameters based on Multi-LiDAR and MMW radar data fusion

Achieving autonomous and safe berthing can reflect the intelligence level of unmanned ships, which is also an important manifestation to ensure the safe navigation of ships. As high-precision sensors, LiDAR (Light Detection and Ranging) and MMW Radar (millimeter wave radar) can provide environmental data for berthing. However, the use of one of them has certain shortcomings, and the fusion of the data from both sensors can provide more accurate and ample berthing information. Therefore, a novel data fusing method is proposed for assisting berthing. This method combines multi-solid-state LiDAR point cloud and MMW Radar data to obtain berthing parameters. Firstly, the collected data are pre-processed according to the characteristics of asynchronous sensors separately, and then the berthing parameters under the two sensors are obtained by the relative spatial position between the ship and the berth; In addition, the data-level fusion of multiple LiDARs point cloud is achieved by the point cloud registration-based 3D-NDT (Normal Distributions Transform) algorithm, and the decision-level fusion of berthing parameters acquired from the multi-LiDAR point cloud and MMW Radar data is performed by the weighted data fusion algorithm. Two sensors were used as perceptual devices to effectively monitor the berthing process of the experimental Ro-Ro ship at Pikou Port. The results demonstrate that the proposed method can make the data of the two sensors complement each other and effectively estimate model-free berthing parameters, which is important to assist in achieving efficient and safe berthing of ships.

3D SLAM based on NDT matching and ground constraints for ground robots in complex environments

PurposeIn large-scale environments and unstructured scenarios, the accuracy and robustness of traditional light detection and ranging (LiDAR) simultaneous localization and mapping (SLAM) algorithms are reduced, and the algorithms might even be completely ineffective. To overcome these problems, this study aims to propose a 3D LiDAR SLAM method for ground-based mobile robots, which uses a 3D LiDAR fusion inertial measurement unit (IMU) to establish an environment map and realize real-time localization.Design/methodology/approachFirst, we use a normal distributions transform (NDT) algorithm based on a local map with a corresponding motion prediction model for point cloud registration in the front-end. Next, point cloud features are tightly coupled with IMU angle constraints, ground constraints and gravity constraints for graph-based optimization in the back-end. Subsequently, the cumulative error is reduced by adding loop closure detection.FindingsThe algorithm is tested using a public data set containing indoor and outdoor scenarios. The results confirm that the proposed algorithm has high accuracy and robustness.Originality/valueTo improve the accuracy and robustness of SLAM, this method proposed in the paper introduced the NDT algorithm in the front-end and designed ground constraints and gravity constraints in the back-end. The proposed method has a satisfactory performance when applied to ground-based mobile robots in complex environments experiments.

3D point cloud fusion from UAV and TLS to assess temperate managed forest structures

Light detection and ranging (LiDAR) technology has become one of the most dominant acquisition methods for modeling forest attributes, such as very accurate tree structure information, which is necessary for qualitative forest management and research activities. In this study, we evaluated the efficacy of standalone unmanned aerial vehicle-laser scanning (UAV-LS) and terrestrial laser scanning (TLS) data to accurately estimate forest tree metrics under differing management types. Furthermore, we combined the UAV-LS and TLS data to test whether fusion can improve the mapping of the three-dimensional (3D) structure of individual trees to favor accurate estimates of tree metrics. We initially calculated the percentage of point density per square meter aboveground in each height class at intervals of 1 m for the UAV-LS, TLS, and fusion datasets. This helped illustrate the vertical point density distribution that reflects the structural complexity between broadleaf and conifer trees. We then used tree-level point clouds to assess several tree metrics, such as diameter at breast height (DBH), total tree height (HT), crown projection area (PAC), crown width (WC), crown length (LC), 3D crown surface (SC), and 3D crown volume (VC). Our results indicated that LiDAR fusion can increase the estimation accuracy of DBH and HT, especially in broadleaves (97.8% accuracy). In addition, fusion significantly reshaped the modeled crown structures in both plots, which led to improved estimates for all crown metrics. The results show empirical evidence that LiDAR fusion can also have a determining role in supporting ecosystem services. For example, detailed 3D mapping of tree crowns can be used to assess the mitigation of rainfall`s kinetic energy by tree crowns concerning soil erosion and sedimentation near habitable zones.

Fusion of UAV Hyperspectral Imaging and LiDAR for the Early Detection of EAB Stress in Ash and a New EAB Detection Index—NDVI(776,678)

Beijing’s One Million Mu Plain Afforestation Project involves planting large areas with the exotic North American tree species Fraxinus pennsylvanica Marsh (ash). As an exotic tree species, ash is very vulnerable to infestations by the emerald ash borer (EAB), a native Chinese wood borer pest. In the early stage of an EAB infestation, attacked trees show no obvious sign. Once the stand has reached the late damage stage, death occurs rapidly. Therefore, there is a need for efficient early detection methods of EAB stress over large areas. The combination of unmanned aerial vehicle (UAV)-based hyperspectral imaging (HI) with light detection and ranging (LiDAR) is a promising practical approach for monitoring insect disturbance. In this study, we identified the most useful narrow-band spectral HI data and 3D LiDAR data for the early detection of EAB stress in ash. UAV-HI data of different infested stages (healthy, light, moderate and severe) of EAB in the 400–1000 nm range were collected from ash canopies and were processed by Partial Least Squares–Variable Importance in Projection (PLS-VIP) to identify the maximally sensitive bands. Band R678 nm had the highest PLS-VIP scores and the most robust classification ability. We combined this band with band R776 nm to develop an innovative normalized difference vegetation index (NDVI(776,678)) to estimate EAB stress. LiDAR data were used to segment individual trees and supplement the HI data. The new NDVI(776,678) identified different stages of EAB stress, with a producer’s accuracy of 90% for healthy trees, 76.25% for light infestation, 58.33% for moderate infestation, and 100% for severe infestation, with an overall accuracy of 82.90% when combined with UAV-HI and LiDAR.

Disparity estimation based on fusion of vision and LiDAR

Advanced driver assistance systems improve driving safety and comfort by applying onboard sensors to collect environmental data, analyze environmental data and decision making. Therefore, advanced driver assistance systems have high requirements for distance perception of the environment. Perceptual sensors commonly used in traditional solutions include stereo vision sensors and the Light Detection and Ranging (LiDAR) sensors. This paper proposes a multi-sensing sensor fusion method for disparity estimation, which combines the perceptual data density characteristics of stereo vision sensors and the measurement accuracy characteristics of LiDAR sensors. The method enhances the sensing accuracy by ensuring high-density sense, which is suitable for distance sensing tasks in complex environments. This paper demonstrates with experimental results on real data that our proposed disparity estimation method performs well and is robust in different scenarios.

Traffic Sign Based Point Cloud Data Registration with Roadside LiDARs in Complex Traffic Environments

The intelligent road is an important component of the intelligent vehicle infrastructure cooperative system, the latest development of intelligent transportation systems. As an advanced sensor, Light Detection and Ranging (LiDAR) has gradually been used to collect high-resolution micro-traffic data on the roadside of intelligent roads. Furthermore, a fusion of multiple LiDARs has become a current hot spot to extend the data collection range and improve detection accuracy. This paper focuses on point cloud registration in a complex traffic environment and proposes a three-dimensional (3D) registration method based on traffic signs and prior knowledge of traffic scenes. Traffic signs with their reflective films are used as reference targets to register 3D point cloud data from roadside LiDARs. The proposed method consists of a vertical registration and a horizontal registration. For the vertical registration, we propose a panel rotation algorithm to rotate the initial point cloud to register it vertically, converting the 3D point cloud registration into a two-dimensional (2D) rigid body transformation. For the vertical registration, our system registers traffic signs from different LiDARs. Our method has been verified in some actual scenarios. Compared with previous methods, the proposed method is automatic and does not need to search reference targets manually. Furthermore, it is suitable for actual engineering use and can be applied to sparse point cloud data from LiDAR with few beams, realizing point cloud registration of large disparity.

On the Integration of Enabling Wireless Technologies and Sensor Fusion for Next-Generation Connected and Autonomous Vehicles

The automotive industry is transitioning towards intelligent, connected, and autonomous vehicles to avoid traffic congestion, conflicts, and collisions with increased driver safety. Connected and autonomous vehicles (CAV) must be aware of their surroundings and act as per their environment. Communication infrastructure can be vital in transmitting necessary information to peers and receiving critical information for timely decisions. This article provides a comprehensive review of the topic, covering the aspects of enabling wireless technologies and sensor fusion. The article reviews data acquisition using various sensing devices such as RADAR (Radio Detection and Ranging), LiDAR (Light Detection and Ranging), cameras, and multi-modal sensor fusion of the acquired data after signal processing. Thereafter, it reviews the communication and networking infrastructure for intra- and inter-vehicle communication and related technologies. For each of these themes, research challenges and future directions have been identified.

Vegetation structure parameters determine high burn severity likelihood in different ecosystem types: A case study in a burned Mediterranean landscape

The design and implementation of pre-fire management strategies in heterogeneous landscapes requires the identification of the ecological conditions contributing to the most adverse effects of wildfires. This study evaluates which features of pre-fire vegetation structure, estimated through broadband land surface albedo and Light Detection and Ranging (LiDAR) data fusion, promote high wildfire damage across several fire-prone ecosystems dominated by either shrub (gorse, heath and broom) or tree species (Pyrenean oak and Scots pine). Topography features were also considered since they can assist in the identification of priority areas where vegetation structure needs to be managed. The case study was conducted within the scar of a mixed-severity wildfire that occurred in the Western Mediterranean Basin. Burn severity was estimated using the differenced Normalized Burn Ratio index computed from Sentinel-2 multispectral instrument (MSI) Level 2 A at 10 m of spatial resolution and validated in the field using the Composite Burn Index (CBI). Ordinal regression models were implemented to evaluate high burn severity outcome based on three groups of predictors: topography, pre-fire broadband land surface albedo computed from Sentinel-2 and pre-fire LiDAR metrics. Models were validated both by 10-fold cross-validation and external validation. High burn severity was largely ecosystem-dependent. In oak and pine forest ecosystems, severe damage was promoted by a high canopy volume (model accuracy = 79%) and a low canopy base height (accuracy = 82%), respectively. Land surface albedo, which is directly related to aboveground biomass and vegetation cover, outperformed LiDAR metrics to predict high burn severity in ecosystems with sparse vegetation. This is the case of gorse and broom shrub ecosystems (accuracy of 80% and 77%, respectively). The effect of topography was overwhelmed by that of the vegetation structure portion of the fire triangle behavior, except for heathlands, in which warm and steep slopes played a key role in high burn severity outcome together with horizontal and vertical fuel continuity (accuracy = 71%). The findings of this study support the fusion of LiDAR and satellite albedo data to assist forest managers in the development of ecosystem-specific management actions aimed at reducing wildfire damage and promote ecosystem resilience.

Multisource Remote Sensing Data Classification With Graph Fusion Network

The land cover classification has been an important task in remote sensing. With the development of various sensors technologies, carrying out classification work with multisource remote sensing (MSRS) data has shown an advantage over using a single type of data. Hyperspectral images (HSIs) are able to represent the spectral properties of land cover, which is quite common for land cover understanding. Light detection and ranging (LiDAR) images contain altitude information of the ground, which is greatly helpful with urban scene analysis. Current HSI and LiDAR fusion methods perform feature extraction and feature fusion separately, which cannot well exploit the correlation of data sources. In order to make full use of the correlation of multisource data, an unsupervised feature extraction-fusion network for HSI and LiDAR, which utilizes feature fusion to guide the feature extraction procedure, is proposed in this article. More specifically, the network takes multisource data as input and directly output the unified fused feature. A multimodal graph is constructed for feature fusion, and graph-based loss functions including Laplacian loss and t-distributed stochastic neighbor embedding (t-SNE) loss are utilized to constrain the feature extraction network. Experimental results on several data sets demonstrate the proposed network can achieve more excellent classification performance than some state-of-the-art methods.

Automatic 3D Landmark Extraction System Based on an Encoder–Decoder Using Fusion of Vision and LiDAR

To provide a realistic environment for remote sensing applications, point clouds are used to realize a three-dimensional (3D) digital world for the user. Motion recognition of objects, e.g., humans, is required to provide realistic experiences in the 3D digital world. To recognize a user’s motions, 3D landmarks are provided by analyzing a 3D point cloud collected through a light detection and ranging (LiDAR) system or a red green blue (RGB) image collected visually. However, manual supervision is required to extract 3D landmarks as to whether they originate from the RGB image or the 3D point cloud. Thus, there is a need for a method for extracting 3D landmarks without manual supervision. Herein, an RGB image and a 3D point cloud are used to extract 3D landmarks. The 3D point cloud is utilized as the relative distance between a LiDAR and a user. Because it cannot contain all information the user’s entire body due to disparities, it cannot generate a dense depth image that provides the boundary of user’s body. Therefore, up-sampling is performed to increase the density of the depth image generated based on the 3D point cloud; the density depends on the 3D point cloud. This paper proposes a system for extracting 3D landmarks using 3D point clouds and RGB images without manual supervision. A depth image provides the boundary of a user’s motion and is generated by using 3D point cloud and RGB image collected by a LiDAR and an RGB camera, respectively. To extract 3D landmarks automatically, an encoder–decoder model is trained with the generated depth images, and the RGB images and 3D landmarks are extracted from these images with the trained encoder model. The method of extracting 3D landmarks using RGB depth (RGBD) images was verified experimentally, and 3D landmarks were extracted to evaluate the user’s motions with RGBD images. In this manner, landmarks could be extracted according to the user’s motions, rather than by extracting them using the RGB images. The depth images generated by the proposed method were 1.832 times denser than the up-sampling-based depth images generated with bilateral filtering.

Real-Time Vehicle Detection Framework Based on the Fusion of LiDAR and Camera

Vehicle detection is essential for driverless systems. However, the current single sensor detection mode is no longer sufficient in complex and changing traffic environments. Therefore, this paper combines camera and light detection and ranging (LiDAR) to build a vehicle-detection framework that has the characteristics of multi adaptability, high real-time capacity, and robustness. First, a multi-adaptive high-precision depth-completion method was proposed to convert the 2D LiDAR sparse depth map into a dense depth map, so that the two sensors are aligned with each other at the data level. Then, the You Only Look Once Version 3 (YOLOv3) real-time object detection model was used to detect the color image and the dense depth map. Finally, a decision-level fusion method based on bounding box fusion and improved Dempster–Shafer (D–S) evidence theory was proposed to merge the two results of the previous step and obtain the final vehicle position and distance information, which not only improves the detection accuracy but also improves the robustness of the whole framework. We evaluated our method using the KITTI dataset and the Waymo Open Dataset, and the results show the effectiveness of the proposed depth completion method and multi-sensor fusion strategy.

Fusion of Multispectral LiDAR, Hyperspectral, and RGB Data for Urban Land Cover Classification

With the increasing importance of monitoring urban areas, the question arises which sensors are best suited to solve the corresponding challenges. This letter proposes novel node tests within the random forest (RF) framework, which allows them to apply them to optical RGB images, hyperspectral images, and light detection and ranging (LiDAR) data, either individually or in combination. This does not only allow to derive accurate classification results for many relevant urban classes without preprocessing or feature extraction but also provides insights into which sensor offers the most meaningful data to solve the given classification task. The achieved results on a public benchmark data set are superior to results obtained by deep learning approaches despite being based on only a fraction of training samples.

LiDAR and Camera Fusion Approach for Object Distance Estimation in Self-Driving Vehicles

The fusion of light detection and ranging (LiDAR) and camera data in real-time is known to be a crucial process in many applications, such as in autonomous driving, industrial automation, and robotics. Especially in the case of autonomous vehicles, the efficient fusion of data from these two types of sensors is important to enabling the depth of objects as well as the detection of objects at short and long distances. As both the sensors are capable of capturing the different attributes of the environment simultaneously, the integration of those attributes with an efficient fusion approach greatly benefits the reliable and consistent perception of the environment. This paper presents a method to estimate the distance (depth) between a self-driving car and other vehicles, objects, and signboards on its path using the accurate fusion approach. Based on the geometrical transformation and projection, low-level sensor fusion was performed between a camera and LiDAR using a 3D marker. Further, the fusion information is utilized to estimate the distance of objects detected by the RefineDet detector. Finally, the accuracy and performance of the sensor fusion and distance estimation approach were evaluated in terms of quantitative and qualitative analysis by considering real road and simulation environment scenarios. Thus the proposed low-level sensor fusion, based on the computational geometric transformation and projection for object distance estimation proves to be a promising solution for enabling reliable and consistent environment perception ability for autonomous vehicles.

On-Ground Vineyard Reconstruction Using a LiDAR-Based Automated System.

Crop 3D modeling allows site-specific management at different crop stages. In recent years, light detection and ranging (LiDAR) sensors have been widely used for gathering information about plant architecture to extract biophysical parameters for decision-making programs. The study reconstructed vineyard crops using light detection and ranging (LiDAR) technology. Its accuracy and performance were assessed for vineyard crop characterization using distance measurements, aiming to obtain a 3D reconstruction. A LiDAR sensor was installed on-board a mobile platform equipped with an RTK-GNSS receiver for crop 2D scanning. The LiDAR system consisted of a 2D time-of-flight sensor, a gimbal connecting the device to the structure, and an RTK-GPS to record the sensor data position. The LiDAR sensor was facing downwards installed on-board an electric platform. It scans in planes perpendicular to the travel direction. Measurements of distance between the LiDAR and the vineyards had a high spatial resolution, providing high-density 3D point clouds. The 3D point cloud was obtained containing all the points where the laser beam impacted. The fusion of LiDAR impacts and the positions of each associated to the RTK-GPS allowed the creation of the 3D structure. Although point clouds were already filtered, discarding points out of the study area, the branch volume cannot be directly calculated, since it turns into a 3D solid cluster that encloses a volume. To obtain the 3D object surface, and therefore to be able to calculate the volume enclosed by this surface, a suitable alpha shape was generated as an outline that envelops the outer points of the point cloud. The 3D scenes were obtained during the winter season when only branches were present and defoliated. The models were used to extract information related to height and branch volume. These models might be used for automatic pruning or relating this parameter to evaluate the future yield at each location. The 3D map was correlated with ground truth, which was manually determined, pruning the remaining weight. The number of scans by LiDAR influenced the relationship with the actual biomass measurements and had a significant effect on the treatments. A positive linear fit was obtained for the comparison between actual dry biomass and LiDAR volume. The influence of individual treatments was of low significance. The results showed strong correlations with actual values of biomass and volume with R2 = 0.75, and when comparing LiDAR scans with weight, the R2 rose up to 0.85. The obtained values show that this LiDAR technique is also valid for branch reconstruction with great advantages over other types of non-contact ranging sensors, regarding a high sampling resolution and high sampling rates. Even narrow branches were properly detected, which demonstrates the accuracy of the system working on difficult scenarios such as defoliated crops.

Research on SLAM Algorithm of Mobile Robot Based on the Fusion of 2D LiDAR and Depth Camera

This paper proposes a new Simultaneous Localization and Mapping (SLAM) method on the basis of graph-based optimization through the combination of the Light Detection and Ranging (LiDAR), RGB-D camera, encoder and Inertial Measurement Unit (IMU). It can conduct joint positioning of four sensors by taking advantaging of the unscented Kalman filter (UKF) to design the related strategy of the 2D LiDAR point cloud and RGB-D camera point cloud. 3D LiDAR point cloud information generated by the RGB-D camera under the 2D LiDAR has been added into the new SLAM method in the sequential registration stage, and it can match the 2D LiDAR point cloud and the 3D RGB-D point cloud by using the method of the Correlation Scan Matching (CSM); In the loop closure detection stage, this method can further verify the accuracy of the loop closure after the 2D LiDAR matching by describing 3D point cloud. Additionally, this new SLAM method has been verified feasibility and availability through the processes of theoretical derivation, simulation experiment and physical verification. As a result, the experiment shows that the multi-sensor SLAM framework designed has a good mapping effect, high precision and accuracy.