Fusion Of LiDAR Data Research Articles

Coarse-to-Fine Registration of Airborne LiDAR Data and Optical Imagery on Urban Scenes

Applications based on synergistic integration of optical imagery and LiDAR data are receiving a growing interest from the remote sensing community. However, a misaligned integration between these datasets may fail to fully profit the potential of both sensors. In this regard, an optimum fusion of optical imagery and LiDAR data requires an accurate registration. This is a complex problem since a versatile solution is still missing, especially when considering the context where data are collected at different times, from different platforms, under different acquisition configurations. This paper presents a coarse-to-fine registration method of aerial/satellite optical imagery with airborne LiDAR data acquired in such context. Firstly, a coarse registration involves extracting and matching of buildings from LiDAR data and optical imagery. Then, a Mutual Information-based fine registration is carried out. It involves a super-resolution approach applied to LiDAR data, and a local approach of transformation model estimation. The proposed method succeeds at overcoming the challenges associated with the aforementioned difficult context. Considering the experimented airborne LiDAR (2011) and orthorectified aerial imagery (2016) datasets, their spatial shift is reduced by 48.15% after the proposed coarse registration. Moreover, the incompatibility of size and spatial resolution is addressed by the mentioned super-resolution. Finally, a high accuracy of dataset alignment is also achieved, highlighted by a 40-cm error based on a check-point assessment and a 64-cm error based on a check-pair-line assessment. These promising results enable further research for a complete versatile fusion methodology between airborne LiDAR and optical imagery data in this challenging context.

Estimation of maize above-ground biomass based on stem-leaf separation strategy integrated with LiDAR and optical remote sensing data.

Above-ground biomass (AGB) is an important indicator for effectively assessing crop growth and yield and, in addition, is an important ecological indicator for assessing the efficiency with which crops use light and store carbon in ecosystems. However, most existing methods using optical remote sensing to estimate AGB cannot observe structures below the maize canopy, which may lead to poor estimation accuracy. This paper proposes to use the stem-leaf separation strategy integrated with unmanned aerial vehicle LiDAR and multispectral image data to estimate the AGB in maize. First, the correlation matrix was used to screen optimal the LiDAR structural parameters (LSPs) and the spectral vegetation indices (SVIs). According to the screened indicators, the SVIs and the LSPs were subjected to multivariable linear regression (MLR) with the above-ground leaf biomass (AGLB) and above-ground stem biomass (AGSB), respectively. At the same time, all SVIs derived from multispectral data and all LSPs derived from LiDAR data were subjected to partial least squares regression (PLSR) with the AGLB and AGSB, respectively. Finally, the AGB was computed by adding the AGLB and the AGSB, and each was estimated by using the MLR and the PLSR methods, respectively. The results indicate a strong correlation between the estimated and field-observed AGB using the MLR method (R2 = 0.82, RMSE = 79.80 g/m2, NRMSE = 11.12%) and the PLSR method (R2 = 0.86, RMSE = 72.28 g/m2, NRMSE = 10.07%). The results indicate that PLSR more accurately estimates AGB than MLR, with R2 increasing by 0.04, root mean square error (RMSE) decreasing by 7.52 g/m2, and normalized root mean square error (NRMSE) decreasing by 1.05%. In addition, the AGB is more accurately estimated by combining LiDAR with multispectral data than LiDAR and multispectral data alone, with R2 increasing by 0.13 and 0.30, respectively, RMSE decreasing by 22.89 and 54.92 g/m2, respectively, and NRMSE decreasing by 4.46% and 7.65%, respectively. This study improves the prediction accuracy of AGB and provides a new guideline for monitoring based on the fusion of multispectral and LiDAR data.

Urban Tree Species Classification Using a WorldView-2/3 and LiDAR Data Fusion Approach and Deep Learning.

Urban areas feature complex and heterogeneous land covers which create challenging issues for tree species classification. The increased availability of high spatial resolution multispectral satellite imagery and LiDAR datasets combined with the recent evolution of deep learning within remote sensing for object detection and scene classification, provide promising opportunities to map individual tree species with greater accuracy and resolution. However, there are knowledge gaps that are related to the contribution of Worldview-3 SWIR bands, very high resolution PAN band and LiDAR data in detailed tree species mapping. Additionally, contemporary deep learning methods are hampered by lack of training samples and difficulties of preparing training data. The objective of this study was to examine the potential of a novel deep learning method, Dense Convolutional Network (DenseNet), to identify dominant individual tree species in a complex urban environment within a fused image of WorldView-2 VNIR, Worldview-3 SWIR and LiDAR datasets. DenseNet results were compared against two popular machine classifiers in remote sensing image analysis, Random Forest (RF) and Support Vector Machine (SVM). Our results demonstrated that: (1) utilizing a data fusion approach beginning with VNIR and adding SWIR, LiDAR, and panchromatic (PAN) bands increased the overall accuracy of the DenseNet classifier from 75.9% to 76.8%, 81.1% and 82.6%, respectively. (2) DenseNet significantly outperformed RF and SVM for the classification of eight dominant tree species with an overall accuracy of 82.6%, compared to 51.8% and 52% for SVM and RF classifiers, respectively. (3) DenseNet maintained superior performance over RF and SVM classifiers under restricted training sample quantities which is a major limiting factor for deep learning techniques. Overall, the study reveals that DenseNet is more effective for urban tree species classification as it outperforms the popular RF and SVM techniques when working with highly complex image scenes regardless of training sample size.

Behavioral Pedestrian Tracking Using a Camera and LiDAR Sensors on a Moving Vehicle.

In this paper, we present a novel 2D–3D pedestrian tracker designed for applications in autonomous vehicles. The system operates on a tracking by detection principle and can track multiple pedestrians in complex urban traffic situations. By using a behavioral motion model and a non-parametric distribution as state model, we are able to accurately track unpredictable pedestrian motion in the presence of heavy occlusion. Tracking is performed independently, on the image and ground plane, in global, motion compensated coordinates. We employ Camera and LiDAR data fusion to solve the association problem where the optimal solution is found by matching 2D and 3D detections to tracks using a joint log-likelihood observation model. Each 2D–3D particle filter then updates their state from associated observations and a behavioral motion model. Each particle moves independently following the pedestrian motion parameters which we learned offline from an annotated training dataset. Temporal stability of the state variables is achieved by modeling each track as a Markov Decision Process with probabilistic state transition properties. A novel track management system then handles high level actions such as track creation, deletion and interaction. Using a probabilistic track score the track manager can cull false and ambiguous detections while updating tracks with detections from actual pedestrians. Our system is implemented on a GPU and exploits the massively parallelizable nature of particle filters. Due to the Markovian nature of our track representation, the system achieves real-time performance operating with a minimal memory footprint. Exhaustive and independent evaluation of our tracker was performed by the KITTI benchmark server, where it was tested against a wide variety of unknown pedestrian tracking situations. On this realistic benchmark, we outperform all published pedestrian trackers in a multitude of tracking metrics.

Evaluation on Spaceborne Multispectral Images, Airborne Hyperspectral, and LiDAR Data for Extracting Spatial Distribution and Estimating Aboveground Biomass of Wetland Vegetation Suaeda salsa

Suaeda salsa ( S. salsa ) has a significant protective effect on salt marshes in coastal wetlands. In this study, the abilities of airborne multispectral images, spaceborne hyperspectral images, and LiDAR data in spatial distribution extraction and aboveground biomass (AB) estimation of S. salsa were explored for mapping the spatial distribution of S. salsa AB. Results showed that the increasing spectral and structural features were conducive to improving the classification accuracy of wetland vegetation and the AB estimation accuracy of S. salsa . The fusion of hyperspectral and LiDAR data provided the highest accuracies for wetlands classification and AB estimation of S. salsa in the study. Multispectral images alone provided relatively high user's and producer's accuracies of S. salsa classification (87.04% and 88.28%, respectively). Compared to multispectral images, hyperspectral data with more spectral features slightly improved the Kappa coefficient and overall accuracy. The AB estimation reached a relatively reliable accuracy based only on hyperspectral data ( R 2 of 0.812, root-mean-square error of 0.295, estimation error of 24.56%, residual predictive deviation of 2.033, and the sums of squares ratio of 1.049). The addition of LiDAR data produced a limited improvement in the process of extraction and AB estimation of S. salsa . The spatial distribution of mapped S. salsa AB was consistent with the field survey results. This study provided an important reference for the effective information extraction and AB estimation of wetland vegetation S. salsa .

Urban classification by multi-feature fusion of hyperspectral image and LiDAR data

Urban classification by multi-feature fusion of hyperspectral image and LiDAR data

Improved forest height estimation by fusion of simulated GEDI Lidar data and TanDEM-X InSAR data

Interferometric Synthetic Aperture Radar (InSAR) and lidar are increasingly used active remote sensing techniques for forest structure observation. The TanDEM-X (TDX) InSAR mission of German Aerospace Center (DLR) and the upcoming Global Ecosystem Dynamics Investigation (GEDI) of National Aeronautics and Space Administration (NASA) together may provide more accurate estimates of global forest structure and biomass via their synergic use. In this paper, we explored the efficacy of simulated GEDI data in improving height estimates from TDX InSAR data. Our study sites span three major forest types: a temperate forest, a mountainous conifer forest, and a tropical rainforest. The GEDI lidar coverage was simulated for the full nominal two-year mission duration, under both cloud-free and 50%-cloud conditions. We then used these GEDI data to parameterize the Random Volume over Ground (RVoG) model driven by TDX imagery. In particular, we explored the following three strategies for forest structure estimation: 1) TDX data alone; 2) TDX + GEDI-derived digital terrain model (DTM); and 3) TDX + GEDI DTM + GEDI canopy height. We then validated the retrieved forest heights against wall-to-wall airborne lidar measurements. We found relatively large biases at 90 [m] spatial resolution, from 4.2–11.9 [m], and root mean square errors (RMSEs), from 7.9–12.7 [m] when using TDX data alone under constrained RVoG assumptions of a fixed extinction coefficient (σ) and a zero ground-to-volume amplitude ratio (μ = 0). Results improved significantly with the aid of a DTM derived from GEDI data which enabled estimation of spatially-varying σ values (vs. fixed extinction) under a μ = 0 assumption, with biases reduced to 1.7–4.2 [m] and RMSEs to 4.9–8.6 [m] across cloudy and cloud-free cases. The best agreement was achieved in the third strategy by also incorporating information of GEDI-derived canopy height to further enhance the RVoG parameters. The improved model, when still assuming μ = 0, reduced biases to less than or close to 1 m and further reduced RMSEs to 4.0–6.7 [m]. Finally, we used GEDI data to estimate spatially-varying μ in the RVoG model. We found biases of between −0.7–0.9 [m] and RMSEs in the range from 2.6–7.1 [m] over the three sites. Our results suggest that use of GEDI data improves height inversion from TDX, providing heights at more accuracy than can be achieved by TDX alone, and enabling wall-to-wall height estimation at much finer spatial resolution than can be achieved by GEDI alone.

Fusion of hyperspectral and LiDAR data: A case study for refined crop classification in agricultural region of Zhangye Oasis in the middle reaches of Heihe River

Fusion of hyperspectral and LiDAR data: A case study for refined crop classification in agricultural region of Zhangye Oasis in the middle reaches of Heihe River

Roof Shape Classification from LiDAR and Satellite Image Data Fusion Using Supervised Learning.

Geographic information systems (GIS) provide accurate maps of terrain, roads, waterways, and building footprints and heights. Aircraft, particularly small unmanned aircraft systems (UAS), can exploit this and additional information such as building roof structure to improve navigation accuracy and safely perform contingency landings particularly in urban regions. However, building roof structure is not fully provided in maps. This paper proposes a method to automatically label building roof shape from publicly available GIS data. Satellite imagery and airborne LiDAR data are processed and manually labeled to create a diverse annotated roof image dataset for small to large urban cities. Multiple convolutional neural network (CNN) architectures are trained and tested, with the best performing networks providing a condensed feature set for support vector machine and decision tree classifiers. Satellite image and LiDAR data fusion is shown to provide greater classification accuracy than using either data type alone. Model confidence thresholds are adjusted leading to significant increases in models precision. Networks trained from roof data in Witten, Germany and Manhattan (New York City) are evaluated on independent data from these cities and Ann Arbor, Michigan.

A Machine-Learning Approach to PolInSAR and LiDAR Data Fusion for Improved Tropical Forest Canopy Height Estimation Using NASA AfriSAR Campaign Data

This paper investigates the benefits of integrating multibaseline polarimetric interferometric SAR (PolInSAR) data with LiDAR measurements using a machine-learning approach in order to obtain improved forest canopy height estimates. Multiple interferometric baselines are required to ensure consistent height retrieval performance across a broad range of tree heights. Previous studies have proposed multibaseline merging strategies using metrics extracted from PolInSAR measurements. Here, we introduce the multibaseline merging using a support vector machine trained by sparse LiDAR samples. The novelty of this method lies in the new way of combining the two datasets. Its advantage is that it does not require a complete LiDAR coverage, but only sparse LiDAR samples distributed over the PolInSAR image. LiDAR samples are not used to obtain the best height among a set of height stacks, but rather to train the retrieval algorithm in selecting the best height using the variables derived through PolInSAR processing. This enables more accurate height estimation for a wider scene covered by the SAR with only partial LiDAR coverage. We test our approach on NASA AfriSAR data acquired over tropical forests by the L-band UAVSAR and the LVIS LiDAR instruments. The estimated height from this approach has a higher accuracy (r <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> = 0.81, RMSE = 7.1 m) than previously introduced multibaseline merging approach (r <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> = 0.67, RMSE = 9.2 m). This method is beneficial to future spaceborne missions, such as GEDI and BIOMASS, which will provide a wealth of near-contemporaneous LiDAR samples and PolInSAR measurements for mapping forest structure at global scale.

Forest Height Estimation Using Multibaseline PolInSAR and Sparse Lidar Data Fusion

We demonstrate a method using lidar data fusion to improve the forest height estimation accuracy of multibaseline polarimetric synthetic aperture radar interferometry (PolInSAR). Compared to single-baseline PolInSAR, multibaseline PolInSAR allows forest canopy height to be estimated more accurately across a wider range of height values. However, to arrive at a single forest height estimate, the estimates from the multiple baselines must be selected or weighted. A number of approaches to selecting between baselines have been proposed in the literature, but they are generally based on simple metrics of the PolInSAR data and do not necessarily capture the full range of characteristics that make one baseline produce more accurate forest height estimates than another. We solve this problem by treating baseline selection as a supervised classification problem that can be trained using a small amount of sparse lidar data located within the PolInSAR coverage area. We train a support vector machine classifier using a variety of coarse lidar sample spacings of 250 m and greater, to demonstrate that data from future spaceborne lidar missions will be sufficient for this purpose. We demonstrate results for multiple study areas in the country of Gabon using data collected by NASA's uninhabited aerial vehicle synthetic aperture radar and land, vegetation, and ice sensor lidar. The use of lidar fusion for PolInSAR baseline selection yields improved results compared to standard baseline selection methods, and further demonstrates the strong potential of PolInSAR and lidar fusion for remote sensing of forests.

INTEGRATION OF TREE DATABASE DERIVED FROM SATELLITE IMAGERY AND LIDAR POINT CLOUD DATA

Abstract. 3D tree database provides essential information of tree species abundance, spatial distribution and tree height for forest mapping, sustainable urban planning and 3D city modelling. Fusion of passive optical satellite imagery and active Lidar data can potentially be exploited for operational forest inventory. However, such fusion requires very high geometric accuracy for both data sets. This paper proposes an approach for 3D tree information extracted from passive and active data integrating into existing tree database by effectively using geometric information of satellite camera model and laser scanner scanning geometry. The paper also presents the individual methods for tree crown identification and delineation from satellite images and lidar point cloud data respectively, the geometric correction of tree position from tree top to tree base. The ground truth accuracy assessment for the tree extracted is also present.

An Improved Method for Impervious Surface Mapping Incorporating LiDAR Data and High-Resolution Imagery at Different Acquisition Times

Impervious surface mapping incorporating high-resolution remote sensing imagery has continued to attract increasing interest, as it can provide detailed information about urban structure and distribution. Previous studies have suggested that the combination of LiDAR data and high-resolution imagery for impervious surface mapping yields better performance than the use of high-resolution imagery alone. However, due to LiDAR data’s high cost of acquisition, it is difficult to obtain LiDAR data that was acquired at the same time as the high-resolution imagery in order to conduct impervious surface mapping by multi-sensor remote sensing data. Consequently, the occurrence of real landscape changes between multi-sensor remote sensing data sets with different acquisition times results in misclassification errors in impervious surface mapping. This issue has generally been neglected in previous works. Furthermore, observation differences that were generated from multi-sensor data—including the problems of misregistration, missing data in LiDAR data, and shadow in high-resolution images—also present obstacles to achieving the final mapping result in the fusion of LiDAR data and high-resolution images. In order to resolve these issues, we propose an improved impervious surface-mapping method incorporating both LiDAR data and high-resolution imagery with different acquisition times that consider real landscape changes and observation differences. In the proposed method, multi-sensor change detection by supervised multivariate alteration detection (MAD) is employed to identify the changed areas and mis-registered areas. The no-data areas in the LiDAR data and the shadow areas in the high-resolution image are extracted via independent classification based on the corresponding single-sensor data. Finally, an object-based post-classification fusion is proposed that takes advantage of both independent classification results while using single-sensor data and the joint classification result using stacked multi-sensor data. The impervious surface map is subsequently obtained by combining the landscape classes in the accurate classification map. Experiments covering the study site in Buffalo, NY, USA demonstrate that our method can accurately detect landscape changes and unambiguously improve the performance of impervious surface mapping.

Estimation of Vegetation Indices and Biomass by the Fusion of LiDAR Data and Landsat Images in Mudumalai Forests of the Western Ghats, India

Estimation of Vegetation Indices and Biomass by the Fusion of LiDAR Data and Landsat Images in Mudumalai Forests of the Western Ghats, India

A Symmetry-Based Method for LiDAR Point Registration

LiDAR point registration is a key procedure for the acquisition of complete point cloud datasets. It has great significance for the fusion of multisource LiDAR data. In general, the widely used methods for LiDAR point registration can be categorized into three types: auxiliary methods, direct methods, and feature methods. However, for the registration of complex objects (e.g., stadium and tower), such methods may face varying degrees of technical problems owing to the unavailability of auxiliary data or targets, requirement of sufficient overlapping areas, and difficulty in feature extraction and matching. In the real world, numerous objects with extremely complicated geometric shapes have the characteristic of symmetry. This study focuses on complex objects with symmetry and tries to exploit their intrinsic symmetry characteristic in order to facilitate their point cloud registration. A symmetry-based method for LiDAR point registration is proposed, in which the general idea is to derive 3-D central axes from multisource point clouds, based on the symmetry of objects. The proposed method consists of six main steps: detection of rotational symmetry, adaptive point cloud slicing, central point extraction, central axis fitting, central axis matching, and orientation and positioning. Comparative experiments and quantitative evaluations are conducted. The experimental results indicate that the proposed framework can achieve satisfactory registration of objects with rotational symmetry.

Beyond RGB: Very high resolution urban remote sensing with multimodal deep networks

In this work, we investigate various methods to deal with semantic labeling of very high resolution multi-modal remote sensing data. Especially, we study how deep fully convolutional networks can be adapted to deal with multi-modal and multi-scale remote sensing data for semantic labeling. Our contributions are threefold: (a) we present an efficient multi-scale approach to leverage both a large spatial context and the high resolution data, (b) we investigate early and late fusion of Lidar and multispectral data, (c) we validate our methods on two public datasets with state-of-the-art results. Our results indicate that late fusion make it possible to recover errors steaming from ambiguous data, while early fusion allows for better joint-feature learning but at the cost of higher sensitivity to missing data.

Road detection based on the fusion of Lidar and image data

In this article, we propose a road detection method based on the fusion of Lidar and image data under the framework of conditional random field. Firstly, Lidar point clouds are projected into the m...

A Novel Automatic Method for the Fusion of ALS and TLS LiDAR Data for Robust Assessment of Tree Crown Structure

Tree crown structural parameters are key inputs to studies spanning forest fire propagation, invasive species dynamics, avian habitat provision, and so on, but these parameters consistently are difficult to measure. While airborne laser scanning (ALS) provides uniform data and a consistent nadir perspective necessary for crown segmentation, the data characteristics of terrestrial laser scanning (TLS) make such crown segmentation efforts much more challenging. We present a data fusion approach to extract crown structure from TLS, by exploiting the complementary perspective of ALS. Multiple TLS point clouds are automatically registered to a single ALS point cloud by maximizing the normalized cross correlation between the global ALS canopy height model (CHM) and each of the local TLS CHMs through parameter optimization of a planar Euclidean transform. Per-tree canopy segmentation boundaries, which are reliably obtained from ALS, can then be adapted onto the more irregular TLS data. This is repeated for each TLS scan; the combined segmentation results from each registered TLS scan and the ALS data are fused into a single per-tree point cloud, from which canopy-level structural parameters readily can be extracted.

Automatic building extraction from LiDAR data fusion of point and grid-based features

This paper proposes a method for extracting buildings from LiDAR point cloud data by combining point-based and grid-based features. To accurately discriminate buildings from vegetation, a point feature based on the variance of normal vectors is proposed. For a robust building extraction, a graph cuts algorithm is employed to combine the used features and consider the neighbor contexture information. As grid feature computing and a graph cuts algorithm are performed on a grid structure, a feature-retained DSM interpolation method is proposed in this paper. The proposed method is validated by the benchmark ISPRS Test Project on Urban Classification and 3D Building Reconstruction and compared to the state-art-of-the methods. The evaluation shows that the proposed method can obtain a promising result both at area-level and at object-level. The method is further applied to the entire ISPRS dataset and to a real dataset of the Wuhan City. The results show a completeness of 94.9% and a correctness of 92.2% at the per-area level for the former dataset and a completeness of 94.4% and a correctness of 95.8% for the latter one. The proposed method has a good potential for large-size LiDAR data.

Mapping Spartina alterniflora Biomass Using LiDAR and Hyperspectral Data

Large-scale coastal reclamation has caused significant changes in Spartina alterniflora (S. alterniflora) distribution in coastal regions of China. However, few studies have focused on estimation of the wetland vegetation biomass, especially of S. alterniflora, in coastal regions using LiDAR and hyperspectral data. In this study, the applicability of LiDAR and hypersectral data for estimating S. alterniflora biomass and mapping its distribution in coastal regions of China was explored to attempt problems of wetland vegetation biomass estimation caused by different vegetation types and different canopy height. Results showed that the highest correlation coefficient with S. alterniflora biomass was vegetation canopy height (0.817), followed by Normalized Difference Vegetation Index (NDVI) (0.635), Atmospherically Resistant Vegetation Index (ARVI) (0.631), Visible Atmospherically Resistant Index (VARI) (0.599), and Ratio Vegetation Index (RVI) (0.520). A multivariate linear estimation model of S. alterniflora biomass using a variable backward elimination method was developed with R squared coefficient of 0.902 and the residual predictive deviation (RPD) of 2.62. The model accuracy of S. alterniflora biomass was higher than that of wetland vegetation for mixed vegetation types because it improved the estimation accuracy caused by differences in spectral features and canopy heights of different kinds of wetland vegetation. The result indicated that estimated S. alterniflora biomass was in agreement with the field survey result. Owing to its basis in the fusion of LiDAR data and hyperspectral data, the proposed method provides an advantage for S. alterniflora mapping. The integration of high spatial resolution hyperspectral imagery and LiDAR data derived canopy height had significantly improved the accuracy of mapping S. alterniflora biomass.