Multispectral LiDAR Research Articles

Individual tree species classification using low-density airborne multispectral LiDAR data via attribute-aware cross-branch transformer

Traditional forest inventory supplies essential data for forest monitoring and management, including tree species, but obtaining individual tree-level information is increasingly crucial. Airborne Light Detection and Ranging (LiDAR) with multispectral observation offers rich information for improved forest inventory mapping with reliable individual tree attributes. Although deep learning techniques have shown promise in tree species classification, they are not sufficiently explored for individual tree-level classification using low-density (less than 30 point/m2) Airborne Multispectral LiDAR (AML) data. This study therefore explores the feasibility of using a deep learning (DL) framework for processing low-density AML point clouds to enhance tree species classification in challenging forest environments. A point-based deep learning network with a dual-branch mechanism combined Cross-Branch Attention modules named Attribute-Aware Cross-Branch (AACB) Transformer is designed for AML data to better differentiate tree species from delineated individual trees. In addition, a channel merging approach is introduced, which is suited to prepare the training samples of deep learning networks and reduces the computational costs. This study was tested with an average 9 points/m2 AML point cloud for 6 tree species including Populus tremuloides, Larix laricina, Acer saccharum, Picea abies, Pinus resinosa, and Pinus strobus from a Canadian mixed forest. The overall accuracies achieved 83.1 %, 85.8 %, and 95.3 % at species, genus, and leaf-type levels, respectively. The comparison between the proposed method and other widely used tree species classification methods demonstrates the effectiveness of the proposed approach in enhancing tree species classification accuracy. We discuss potentials and remaining challenges, and our findings allow to further improve tree species classification of low-density AML point clouds by DL technology.

Research Progress of Spectral Imaging Techniques in Plant Phenotype Studies.

Spectral imaging technique has been widely applied in plant phenotype analysis to improve plant trait selection and genetic advantages. The latest developments and applications of various optical imaging techniques in plant phenotypes were reviewed, and their advantages and applicability were compared. X-ray computed tomography (X-ray CT) and light detection and ranging (LiDAR) are more suitable for the three-dimensional reconstruction of plant surfaces, tissues, and organs. Chlorophyll fluorescence imaging (ChlF) and thermal imaging (TI) can be used to measure the physiological phenotype characteristics of plants. Specific symptoms caused by nutrient deficiency can be detected by hyperspectral and multispectral imaging, LiDAR, and ChlF. Future plant phenotype research based on spectral imaging can be more closely integrated with plant physiological processes. It can more effectively support the research in related disciplines, such as metabolomics and genomics, and focus on micro-scale activities, such as oxygen transport and intercellular chlorophyll transmission.

Large-Scale Mapping of Complex Forest Typologies Using Multispectral Imagery and Low-Density Airborne LiDAR: A Case Study in Pinsapo Fir Forests

Climate change increases the vulnerability of relict forests. To address this problem, regional Forest Services require silvicultural and conservation actions to designate specific forest management alternatives. In this context, the main objective of this study was to develop a methodology to map complex Abies pinsapo forest typologies using multispectral and low-density airborne LiDAR data and machine learning. Stand density, species composition and cover were used to identify seven forest typologies. Random forest resulted as the more accurate model (OA = 0.62; Kappa = 0.43) to classify those types based on multispectral and LiDAR data, although showing a moderate model performance. Classification performance showed great differences between forest types with better results for the uneven-aged stands compared to the even-aged and two-aged stands. The developed typology was applied to supply local forest managers with more accurate forest maps that can be used to improve forest management plans. The typology proposed is easy to apply in forest management practices since it only uses as input the diameter at breast height, tree density and specific composition. The study demonstrated the potential of low-density LiDAR data combined with spectral information from high-resolution orthophotos to predict the structural characteristics of complex forest typologies.

Fusing Multispectral and LiDAR Data for CNN-Based Semantic Segmentation in Semi-Arid Mediterranean Environments: Land Cover Classification and Analysis

Spectral confusion among land cover classes is quite common, let alone in a complex and heterogenous system like the semi-arid Mediterranean environment; thus, employing new developments in remote sensing, such as multispectral imagery (MSI) captured by unmanned aerial vehicles (UAVs) and airborne light detection and ranging (LiDAR) techniques, with deep learning (DL) algorithms for land cover classification can help to address this problem. Therefore, we propose an image-based land cover classification methodology based on fusing multispectral and airborne LiDAR data by adopting CNN-based semantic segmentation in a semi-arid Mediterranean area of northeastern Aegean, Greece. The methodology consists of three stages: (i) data pre-processing, (ii) semantic segmentation, and (iii) accuracy assessment. The multispectral bands were stacked with the calculated Normalized Difference Vegetation Index (NDVI) and the LiDAR-based attributes height, intensity, and number of returns converted into two-dimensional (2D) images. Then, a hyper-parameter analysis was performed to investigate the impact on the classification accuracy and training time of the U-Net architecture by varying the input tile size and the patch size for prediction, including the learning rate and algorithm optimizer. Finally, comparative experiments were conducted by altering the input data type to test our hypothesis, and the CNN model performance was analyzed by using accuracy assessment metrics and visually comparing the segmentation maps. The findings of this investigation showed that fusing multispectral and LiDAR data improves the classification accuracy of the U-Net, as it yielded the highest overall accuracy of 79.34% and a kappa coefficient of 0.6966, compared to using multispectral (OA: 76.03%; K: 0.6538) or LiDAR (OA: 37.79%; K: 0.0840) data separately. Although some confusion still exists among the seven land cover classes observed, the U-Net delivered a detailed and quite accurate segmentation map.

Automatic Building Extraction from Multispectral LiDAR Using Novel 3D Spatial Indices and Deep Learning Point CNN

Automatic Building Extraction from Multispectral LiDAR Using Novel 3D Spatial Indices and Deep Learning Point CNN

Time division multiplexing based multi-spectral semantic camera for LiDAR applications

The recent progress in the development of measurement systems for autonomous recognition had a substantial impact on emerging technology in numerous fields, especially robotics and automotive applications. In particular, time-of-flight (TOF) based light detection and ranging (LiDAR) systems enable to map the surrounding environmental information over long distances and with high accuracy. The combination of advanced LiDAR with an artificial intelligence platform allows enhanced object recognition and classification, which however still suffers from limitations of inaccuracy and misidentification. Recently, multi-spectral LiDAR systems have been employed to increase the object recognition performance by additionally providing material information in the short-wave infrared (SWIR) range where the reflection spectrum characteristics are typically very sensitive to material properties. However, previous multi-spectral LiDAR systems utilized band-pass filters or complex dispersive optical systems and even required multiple photodetectors, adding complexity and cost. In this work, we propose a time-division-multiplexing (TDM) based multi-spectral LiDAR system for semantic object inference by the simultaneous acquisition of spatial and spectral information. By utilizing the TDM method, we enable the simultaneous acquisition of spatial and spectral information as well as a TOF based distance map with minimized optical loss using only a single photodetector. Our LiDAR system utilizes nanosecond pulses of five different wavelengths in the SWIR range to acquire sufficient material information in addition to 3D spatial information. To demonstrate the recognition performance, we map the multi-spectral image from a human hand, a mannequin hand, a fabric gloved hand, a nitrile gloved hand, and a printed human hand onto an RGB-color encoded image, which clearly visualizes spectral differences as RGB color depending on the material while having a similar shape. Additionally, the classification performance of the multi-spectral image is demonstrated with a convolution neural network (CNN) model using the full multi-spectral data set. Our work presents a compact novel spectroscopic LiDAR system, which provides increased recognition performance and thus a great potential to improve safety and reliability in autonomous driving.

MKANet: Multiscale Kernel Aggregation Network for Multispectral LiDAR 3-D Point Cloud Scene Recognition

MKANet: Multiscale Kernel Aggregation Network for Multispectral LiDAR 3-D Point Cloud Scene Recognition

Data quality analysis after hyperspectral LiDAR sequentially mapping trees

Abstract. Light detection and ranging (LiDAR), as an innovative remote sensing tool, not only captures target reflectance but also provides its morphological parameters. Traditional single/multi-band LiDAR and multispectral LiDAR (MSL) are presently employed in applications such as 3D modeling and plant biochemical parameter inversion albeit with effectiveness limited. Moreover, hyperspectral LiDAR (HSL) distinguished by its expanded array of spectral detection channels and enhanced spectral resolution, has proven more effective in meeting these requirements and also exhibits superior capabilities in both feature and land cover classification tasks. Nevertheless, point clouds acquired through HSL frequently exhibit quality deficiencies, including uneven density and excessive noise. Meanwhile, there exists a notable absence of technical specifications and operational standards governing the measurement protocols for HSL systems globally. To address this gap, this study constructed a systematic analysis framework of data quality in hyperspectral point clouds and endeavors to qualitatively analyse 30 tree point clouds continuously scanned with Finnish Geospatial Research Institute (FGI) 8-band hyperspectral laser scanner. Furthermore, this research validated the theoretical feasibility of employing the 8-band HSL system for inversion processes aimed at quantifying chlorophyll leaf content. Apart from detecting the time-varying patterns of reflectance within birch canopy point clouds, the results of this study also effectively pinpointed the band exhibiting heightened noise level of the HSL system, demonstrating the efficacy of our proposed quality analysis methodology. The endeavor presented in this study can serve as a cornerstone for advancing hyperspectral LiDAR across a diverse array of related remote sensing and earth observation applications.

A feature selection method for multimodal multispectral LiDAR sensing

Optical remote sensing techniques can indicate the properties of objects by observing different modalities (physical quantities) of the backscattered light at different optical wavelengths. Established examples are reflectance, fluorescence, Raman, or depolarization spectroscopy. LiDAR sensing, on the other hand, allows acquiring the geometry of objects by measuring the propagation delay of optical probing signals. Multimodal multispectral (MM) LiDAR combines these capabilities and extends conventional monochromatic LiDAR in both spectral and modal dimensions within a single instrument, thus enriching point cloud data with non-geometric information. The potentially high dimension of MM LiDAR data, however, poses significant challenges for instrumental design, data acquisition, and data processing. MM LiDAR data are structured as several or all modalities are available in each of the spectral channels. The above challenges can thus be mitigated by feature selection (FS), if the structure of the features is taken into account, i.e., if entire spectral channels or entire modalities are selected or omitted. Herein, we focus on the feature selection method for MM LiDAR and propose a multiclass group feature selection algorithm (MGSVM FS) consisting of a structural sparsity-based embedded feature selection method with an all-in-one support vector machine (SVM). It tackles jointly the challenges arising from the high dimension of the MM data and the need for a multiclass classification task while exploiting the structure of the MM data. In addition, we introduce a complete workflow for evaluating the feature selection and for decision-making. We apply the framework to selecting an optimum spectral and modal configuration for remote material classification using an experimental MM LiDAR system that provides reflectance, distance, and degree of linear polarization in 28 spectral channels of 10 nm width. For the experimental investigation, we use MM LiDAR data obtained in a controlled lab environment from thirty specimens of four material classes relevant for construction. Using all three modalities, we find a configuration with only 3 spectral channels that achieves a classification mean-F1 score of 100% within this small dataset. Similar classification performance can also be achieved with only two modalities when using more spectral channels. MGSVM FS improves the classification mean-F1 score by up to 25% as compared to random selection and outperforms two other commonly used filter and embedded feature selection methods, in this application example. The proposed group feature selection algorithm and decision-making are useful for MM LiDAR, providing a link between instrumental design, data acquisition, and data processing. However, they are also transferable to other application fields related to multiclass classification, regression, and knowledge discovery, with features structured in groups. The collected MM feature dataset, the MGSVM FS algorithm, and the evaluation pipeline are accessible online.11https://github.com/yuhan-yhyh/Dataset_Code_MGSVM-FS-MM-LiDAR.git.

Wildfire response of forest species from multispectral LiDAR data. A deep learning approach with synthetic data

Forests play a crucial role as the lungs and life-support system of our planet, harbouring 80% of the Earth's biodiversity. However, we are witnessing an average loss of 480 ha of forest every hour because of destructive wildfires spreading across the globe. To effectively mitigate the threat of wildfires, it is crucial to devise precise and dependable approaches for forecasting fire dynamics and formulating efficient fire management strategies, such as the utilisation of fuel models.The objective of this study was to enhance forest fuel classification that considers only structural information, such as the Prometheus model, by integrating data on the fire responses of various tree species and other vegetation elements, such as ground litter and shrubs. This distinction can be achieved using multispectral (MS) Light Detection and Ranging (LiDAR) data in mixed forests. The methodology involves a novel approach in semantic classifications of forests by generating synthetic data with semantic labels regarding fire responses and reflectance information at different spectral bands, as a real MS scanner device would detect. Forests, which are highly intricate environments, present challenges in accurately classifying point clouds. To address this complexity, a deep learning (DL) model for semantic classification was trained on synthetic point clouds in different formats to achieve the best performance when leveraging MS data.Forest plots in the study region were scanned using different Terrestrial Laser Scanning sensors at wavelengths of 905 and 1550 nm. Subsequently, an interpolation process was applied to generate the MS point clouds of each plot, and the trained DL model was applied to classify them. These classifications surpassed the average thresholds of 90% and 75% for accuracy and intersection over union, respectively, resulting in a more precise categorisation of fuel models based on the distinct responses of forest elements to fire. The results of this study reveal the potential of MS LiDAR data and DL classification models for improving fuel model retrieval in forest ecosystems and enhancing wildfire management efforts.

Dynamic clustering transformer network for point cloud segmentation

Point cloud segmentation is one of the most important tasks in LiDAR remote sensing with widespread scientific, industrial, and commercial applications. The research thereof has resulted in many breakthroughs in 3D object and scene understanding. Existing methods typically utilize hierarchical architectures for feature representation. However, the commonly used sampling and grouping methods in hierarchical networks are not only time-consuming but also limited to point-wise 3D coordinates, ignoring the local semantic homogeneity of point clusters. To address these issues, we propose a novel 3D point cloud representation network, called Dynamic Clustering Transformer Network (DCTNet). It has an encoder–decoder architecture, allowing for both local and global feature learning. Specifically, the encoder consists of a series of dynamic clustering-based Local Feature Aggregating (LFA) blocks and Transformer-based Global Feature Learning (GFL) blocks. In the LFA block, we propose novel semantic feature-based dynamic sampling and clustering methods, which enable the model to be aware of local semantic homogeneity for local feature aggregation. Furthermore, instead of traditional interpolation approaches, we propose a new semantic feature-guided upsampling method in the decoder for dense prediction. To our knowledge, DCTNet is the first work to introduce semantic information-based dynamic clustering into 3D Transformers. Extensive experiments on an object-based dataset (ShapeNet), and an airborne multispectral LiDAR dataset demonstrate the State-of-the-Art (SOTA) segmentation performance of DCTNet in terms of both accuracy and efficiency. Our code will be made publicly available.

Fusion of Dense Airborne LiDAR and Multispectral Sentinel-2 and Pleiades Satellite Imagery for Mapping Riparian Forest Species Biodiversity at Tree Level.

Multispectral and 3D LiDAR remote sensing data sources are valuable tools for characterizing the 3D vegetation structure and thus understanding the relationship between forest structure, biodiversity, and microclimate. This study focuses on mapping riparian forest species in the canopy strata using a fusion of Airborne LiDAR data and multispectral multi-source and multi-resolution satellite imagery: Sentinel-2 and Pleiades at tree level. The idea is to assess the contribution of each data source in the tree species classification at the considered level. The data fusion was processed at the feature level and the decision level. At the feature level, LiDAR 2D attributes were derived and combined with multispectral imagery vegetation indices. At the decision level, LiDAR data were used for 3D tree crown delimitation, providing unique trees or groups of trees. The segmented tree crowns were used as a support for an object-based species classification at tree level. Data augmentation techniques were used to improve the training process, and classification was carried out with a random forest classifier. The workflow was entirely automated using a Python script, which allowed the assessment of four different fusion configurations. The best results were obtained by the fusion of Sentinel-2 time series and LiDAR data with a kappa of 0.66, thanks to red edge-based indices that better discriminate vegetation species and the temporal resolution of Sentinel-2 images that allows monitoring the phenological stages, helping to discriminate the species.

Multispectral Light Detection and Ranging Technology and Applications: A Review.

Light Detection and Ranging (LiDAR) is a well-established active technology for the direct acquisition of 3D data. In recent years, the geometric information collected by LiDAR sensors has been widely combined with optical images to provide supplementary spectral information to achieve more precise results in diverse remote sensing applications. The emergence of active Multispectral LiDAR (MSL) systems, which operate on different wavelengths, has recently been revolutionizing the simultaneous acquisition of height and intensity information. So far, MSL technology has been successfully applied for fine-scale mapping in various domains. However, a comprehensive review of this modern technology is currently lacking. Hence, this study presents an exhaustive overview of the current state-of-the-art in MSL systems by reviewing the latest technologies for MSL data acquisition. Moreover, the paper reports an in-depth analysis of the diverse applications of MSL, spanning across fields of "ecology and forestry", "objects and Land Use Land Cover (LULC) classification", "change detection", "bathymetry", "topographic mapping", "archaeology and geology", and "navigation". Our systematic review uncovers the potentials, opportunities, and challenges of the recently emerged MSL systems, which integrate spatial-spectral data and unlock the capability for precise multi-dimensional (nD) mapping using only a single-data source.

Identification of Larch Caterpillar Infestation Severity Based on Unmanned Aerial Vehicle Multispectral and LiDAR Features

Utilizing UAV remote sensing technology to acquire information on forest pests is a crucial technical method for determining the health of forest trees. Achieving efficient and precise pest identification has been a major research focus in this field. In this study, Dendrolimus superans (Butler) was used as the research object to acquire UAV multispectral, LiDAR, and ground-measured data for extracting sensitive features using ANOVA and constructing a severity-recognizing model with the help of random forest (RF) and support vector machine (SVM) models. Sixteen sensitive feature sets (including multispectral vegetation indices and LiDAR features) were selected for training the recognizing model, of which the normalized differential greenness index (NDGI) and 25% height percentile were the most sensitive and could be used as important features for recognizing larch caterpillar infestations. The model results show that the highest accuracy is SVMVI+LIDAR (OA = 95.8%), followed by SVMVI, and the worst accuracy is RFLIDAR. For identifying healthy, mild, and severely infested canopies, the SVMVI+LIDAR model achieved 90%–100% for both PA and UA. The optimal model chosen to map the spatial distribution of severity at the single-plant scale in the experimental area demonstrated that the severity intensified with decreasing elevation, especially from 748–758 m. This study demonstrates a high-precision identification method of larch caterpillar infestation severity and provides an efficient and accurate data reference for intelligent forest management.

An Improved Method for Individual Tree Segmentation in Complex Urban Scenes Based on Using Multispectral LiDAR by Deep Learning

An Improved Method for Individual Tree Segmentation in Complex Urban Scenes Based on Using Multispectral LiDAR by Deep Learning

Multichannel Weak Signal Extraction Based on Multispectral LiDAR

Multichannel Weak Signal Extraction Based on Multispectral LiDAR

Multispectral LiDAR Data Classification Method Based on an Improved PointNet++ Model

Multispectral LiDAR Data Classification Method Based on an Improved PointNet++ Model

Local Enhanced Transformer Networks for Land Cover Classification With Airborne Multispectral LiDAR Data

Local Enhanced Transformer Networks for Land Cover Classification With Airborne Multispectral LiDAR Data

Multiscale Adjacency Matrix CNN: Learning on Multispectral LiDAR Point Cloud via Multiscale Local Graph Convolution

Multiscale Adjacency Matrix CNN: Learning on Multispectral LiDAR Point Cloud via Multiscale Local Graph Convolution

Evaluation of Forestry Component Survival in Plots of the Program “Sembrando Vida” (Sowing Life) Using Drones

Reforestation is one of the main actions undertaken to mitigate the effects of climate change. In Mexico, the Federal Government program “Sembrando Vida” (Sowing Life) is currently the most important reforestation effort. It aims to recoup forest cover and achieve food self-sufficiency through the establishment of agroforestry systems. The evaluation of tree survival in reforested areas helps to identify achievements and failures, as well as aspects of the program that require improvement. However, given the magnitude of this program, evaluation using traditional methodologies is labor-intensive and costly. In this context, drones equipped with high-resolution cameras are a promising tool. The objective of this study was to evaluate the feasibility of using drones to monitor tree survival in reforested areas. This study was conducted in 12 randomly chosen plots, benefited by the “Sembrando Vida” program, located on the Purépecha Plateau in the state of Michoacán, in central–western Mexico. Field surveys with GPS were conducted to record the total number of live and dead forest-tree seedlings. Simultaneously, high-resolution images were captured using a DJI Phantom 4 Pro drone equipped with an RGB camera for subsequent visual interpretation in a geographic information system to determine the status of each seedling and calculate the rates of survival. ANOVA was performed to compare the survival calculated using the drone images compared to that recorded in the field. No significant difference was found between survival estimated using the drone and that recorded directly in the field in any of the study plots, although the drone overestimated survival by an average of 6%, mostly due to the presence of dead seedlings that had already lost their foliage and were thus missed when scoring the RGB image. It is therefore concluded that the estimation of survival using drones is a reliable method. For future research, it is recommended to evaluate machine-learning algorithms in terms of detecting both living and dead trees in reforested sites. It is also recommended to use multispectral thermal cameras and LiDAR technology to broaden the knowledge of the different levels of vigor/stress present in the vegetation.