Fusion Of Light Detection And Ranging Data Research Articles

Automatic registration framework for multi-platform point cloud data in natural forests

ABSTRACT The use of light detection and ranging (LiDAR) for investigating forest parameters has gained attention in recent years. However, the occlusion of trees in natural forests makes it difficult for LiDAR on a single platform to capture complete point clouds of trees. In order to solve this problem, it is crucial to combine multi-platform LiDAR data. Because of the complexity of natural forests and the small difference between the geometric characteristics of trees, current multi-platform LiDAR data fusion remains an ongoing challenge in natural forests. In this paper, an automatic registration framework for multi-platform point cloud data in natural forests based on tree distribution pattern was proposed. It consists of five steps, namely segmenting trees, generating feature descriptors, matching trees and registering coarsely and finely. The proposed registration framework can determine the same and accurate location information of matching trees from multi-platform LiDAR data, and a large number of correct matching trees can be obtained through two rounds of a single tree matching process. The proposed framework was validated by fusing airborne laser scanner (ALS) and backpack laser scanner (BLS) data in natural forest. According to experimental results, the proposed framework has a high registration accuracy (root-mean-square error (RMSE) = 0.133 m, mean absolute error (MAE) = 0.126 m). In addition, when the single tree segmentation accuracy exceeds 0.85, the proposed framework is less affected by segmentation errors. In natural forests, the proposed framework can effectively improve the accuracy and efficiency of multi-platform LiDAR data registration.

Using aerial LiDAR to assess regional availability of potential habitat for a conservation dependent forest bird

Remotely-sensed data can inform conservation efforts that target forest wildlife, however, few spatial data products are able to quantify fine-scale aspects of structural variation within forests. Increased availability of Light Detection and Ranging (LiDAR) datasets that cover broad spatial extents and ownership types (e.g., entire states) provide useful information regarding canopy and understory structure within forested landscapes. The fusion of LiDAR data with field-based species surveys can advance our understanding of species-habitat relationships and improve the effectiveness of conservation programs to recover habitat-limited species. The Golden-winged Warbler (Vermivora chrysoptera) is a forest-dependent songbird that nests in structurally-complex young forest across eastern North America. As with many early-successional obligates, this species has been declining for decades due, in part, to the steady loss of young forest/shrubland nesting habitat. Although conservation programs have begun restoring Golden-winged Warbler habitat, these efforts are currently limited by the inability to identify existing habitat across large spatial extents and diverse ownership patterns. Recent availability of state-wide LiDAR data for Pennsylvania provides an opportunity to overcome this limitation. From 2019 to 20, we surveyed for Golden-winged Warblers and structural vegetation at 837 sites across six forest blocks in eastern Pennsylvania. We combined these data with LiDAR derived forest structural metrics to develop statistical models to predict patterns of occupancy. Golden-winged Warbler occupancy probability was explained by models containing several LiDAR-derived structural metrics (e.g., percentage of first returns between 1 and 5 m in height, structural complexity, etc.). Moreover, models fit with LiDAR-derived covariates predicted occupancy much better than those using only field-measured vegetation covariates (ΔAICc = 53.27). Mapped predictions of Golden-winged Warbler occupancy revealed potential habitat (especially regenerating timber harvests) on both private and public lands. These results demonstrate the efficacy of LiDAR for modeling forest bird habitat associations, and how such data sources can provide a valuable tool for conservation planning.

Remote Sensing in Field Crop Monitoring: A Comprehensive Review of Sensor Systems, Data Analyses and Recent Advances

The key elements that underpin food security require the adaptation of agricultural systems to support productivity increases while minimizing inputs and the adverse effects of climate change. The advances in precision agriculture over the past few years have substantially enhanced the efficiency of applying spatially variable agronomic inputs for irrigation, such as fertilizers, pesticides, seeds, and water, and we can attribute them to the increasing number of innovations that utilize new technologies that are capable of monitoring field crops for varying spatial and temporal changes. Remote sensing technology is the primary driver of success in precision agriculture, along with other technologies, such as the Internet of Things (IoT), robotic systems, weather forecasting technology, and global positioning systems (GPSs). More specifically, multispectral imaging (MSI) and hyperspectral imaging (HSI) have made the monitoring of the field crop health to aid decision making and the application of spatially and temporally variable agronomic inputs possible. Furthermore, the fusion of remotely sensed multisource data—for instance, HSI and LiDAR (light detection and ranging) data fusion—has even made it possible to monitor the changes in different parts of an individual plant. To the best of our knowledge, in most reviews on this topic, the authors focus on specific methods and/or technologies, with few or no comprehensive reviews that expose researchers, and especially students, to the vast possible range of remote sensing technologies used in agriculture. In this article, we describe/evaluate the remote sensing (RS) technologies for field crop monitoring using spectral imaging, and we provide a thorough and discipline-specific starting point for researchers of different levels by supplying sufficient details and references. We also high light strengths and drawbacks of each technology, which will help readers select the most appropriate method for their intended uses.

RI-Fusion: 3D Object Detection Using Enhanced Point Features With Range-Image Fusion for Autonomous Driving

The 3D object detection is becoming indispensable for environmental perception in autonomous driving. Light detection and ranging (LiDAR) point clouds often fail to distinguish objects with similar structures and are quite sparse for distant or small objects, thereby introducing false and missed detections. To address these issues, LiDAR is often fused with cameras due to the rich textural information provided by images. However, current fusion methods suffer the inefficient data representation and inaccurate alignment of heterogeneous features, leading to poor precision and low efficiency. To this end, we propose a plug-and-play module termed range-image fusion (RI-Fusion) to achieve an effective fusion of LiDAR and camera data, designed to be easily accessible by existing mainstream LiDAR-based algorithms. In this process, we design an image and point cloud alignment method by converting a point cloud into a compact range-view representation through a spherical coordinate transformation. The range image is then integrated with a corresponding camera image utilizing an attention mechanism. The original range image is then concatenated with fusion features to retain point cloud information, and the results are projected onto a spatial point cloud. Finally, the feature-enhanced point cloud can be input into a LiDAR-based 3D object detector. The results of validation experiments involving the KITTI 3D object detection benchmark showed that our proposed fusion method significantly enhanced multiple mainstream LiDAR-based 3D object detectors, PointPillars, SECOND, and Part <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{A}{^{2}}$ </tex-math></inline-formula> , improving the 3D mAP (mean Average Precision) by 3.61%, 2.98%, and 1.27%, respectively, particularly for small objects such as pedestrians and cyclists.

Identification of tree species based on the fusion of UAV hyperspectral image and LiDAR data in a coniferous and broad-leaved mixed forest in Northeast China.

Rapid and accurate identification of tree species via remote sensing technology has become one of the important means for forest inventory. This paper is to develop an accurate tree species identification framework that integrates unmanned airborne vehicle (UAV)-based hyperspectral image and light detection and ranging (LiDAR) data under the complex condition of natural coniferous and broad-leaved mixed forests. First, the UAV-based hyperspectral image and LiDAR data were obtained from a natural coniferous and broad-leaved mixed forest in the Maoer Mountain area of Northeast China. The preprocessed LiDAR data was segmented using a distance-based point cloud clustering algorithm to obtain the point cloud of individual trees; the hyperspectral image was segmented using the projection outlines of individual tree point clouds to obtain the hyperspectral data of individual trees. Then, different hyperspectral and LiDAR features were extracted, respectively, and the importance of the features was analyzed by a random forest (RF) algorithm in order to select appropriate features for the single-source and multi-source data. Finally, tree species identification in the study area were conducted by using a support vector machine (SVM) algorithm together with hyperspectral features, LiDAR features and fused features, respectively. Results showed that the total accuracy for individual tree segmentation was 84.62%, and the fused features achieved the best accuracy for identification of the tree species (total accuracy = 89.20%), followed by the hyperspectral features (total accuracy = 86.08%) and LiDAR features (total accuracy = 76.42%). The optimal features for tree species identification based on fusion of the hyperspectral and LiDAR data included the vegetation indices that were sensitive to the chlorophyll, anthocyanin and carotene contents in the leaves, the partial components of the transformed independent component analysis (ICA), minimum noise fraction (MNF) and principal component analysis (PCA), and the intensity features of the LiDAR echo, respectively. It was concluded that the framework developed in this study was effective in tree species identification under the complex conditions of natural coniferous and broad-leaved mixed forest and the fusion of UAV-based hyperspectral image and LiDAR data can achieve enhanced accuracy compared the single-source UAV-based remote sensing data.

Image-Aided LiDAR Mapping Platform and Data Processing Strategy for Stockpile Volume Estimation

Stockpile quantity monitoring is vital for agencies and businesses to maintain inventory of bulk material such as salt, sand, aggregate, lime, and many other materials commonly used in agriculture, highways, and industrial applications. Traditional approaches for volumetric assessment of bulk material stockpiles, e.g., truckload counting, are inaccurate and prone to cumulative errors over long time. Modern aerial and terrestrial remote sensing platforms equipped with camera and/or light detection and ranging (LiDAR) units have been increasingly popular for conducting high-fidelity geometric measurements. Current use of these sensing technologies for stockpile volume estimation is impacted by environmental conditions such as lack of global navigation satellite system (GNSS) signals, poor lighting, and/or featureless surfaces. This study addresses these limitations through a new mapping platform denoted as Stockpile Monitoring and Reporting Technology (SMART), which is designed and integrated as a time-efficient, cost-effective stockpile monitoring solution. The novel mapping framework is realized through camera and LiDAR data-fusion that facilitates stockpile volume estimation in challenging environmental conditions. LiDAR point clouds are derived through a sequence of data collections from different scans. In order to handle the sparse nature of the collected data at a given scan, an automated image-aided LiDAR coarse registration technique is developed followed by a new segmentation approach to derive features, which are used for fine registration. The resulting 3D point cloud is subsequently used for accurate volume estimation. Field surveys were conducted on stockpiles of varying size and shape complexity. Independent assessment of stockpile volume using terrestrial laser scanners (TLS) shows that the developed framework had close to 1% relative error.

Ground-based/UAV-LiDAR data fusion for quantitative structure modeling and tree parameter retrieval in subtropical planted forest

Light detection and ranging (LiDAR) has contributed immensely to forest mapping and 3D tree modelling. From the perspective of data acquisition, the integration of LiDAR data from different platforms would enrich forest information at the tree and plot levels. This research develops a general framework to integrate ground-based and UAV-LiDAR (ULS) data to better estimate tree parameters based on quantitative structure modelling (QSM). This is accomplished in three sequential steps. First, the ground-based/ULS LiDAR data were co-registered based on the local density peaks of the clustered canopy. Next, redundancy and noise were removed for the ground-based/ULS LiDAR data fusion. Finally, tree modeling and biophysical parameter retrieval were based on QSM. Experiments were performed for Backpack/Handheld/UAV-based multi-platform mobile LiDAR data of a subtropical forest, including poplar and dawn redwood species. Generally, ground-based/ULS LiDAR data fusion outperforms ground-based LiDAR with respect to tree parameter estimation compared to field data. The fusion-derived tree height, tree volume, and crown volume significantly improved by up to 9.01%, 5.28%, and 18.61%, respectively, in terms of r RMSE. By contrast, the diameter at breast height (DBH) is the parameter that has the least benefits from fusion, and r RMSE remains approximately the same, because stems are already well sampled from ground data. Additionally, particularly for dense forests, the fusion-derived tree parameters were improved compared to those derived from ground-based LiDAR. Ground-based LiDAR can potentially be used to estimate tree parameters in low-stand-density forests, whereby the improvement owing to fusion is not significant. • Mitigation of the limitations of tree localization in case of ground-based/ULS LiDAR data co-registration using virtual keypoints. • Fusion of ground-based/ULS LiDAR data based on removal of redundant and noisy points. • Investigation and evaluation of the utility of ground-based/ULS LiDAR data fusion for tree parameter estimation. • Ground-based/ULS LiDAR data fusion improved estimation of tree height, tree volume, and crown volume. • Ground-based/ULS LiDAR data fusion has strong potential in QSM of trees particularly in high-stand-density forest.

Global–Local Transformer Network for HSI and LiDAR Data Joint Classification

Hyperspectral images (HSI) contain rich spatial and spectral detail information, while light detection and ranging (LiDAR) data can provide the elevation information. Thus, the fusion of HSI and LiDAR data can help for more accurate image classification, which becomes a hot research topic. However, it is difficult to capture complex local and global spatial-spectral associations, meanwhile, how to build an effective interaction between multi-modal data is another important issue. To this end, a novel global-local transformer network (GLT-Net) is proposed for the joint classification of HSI and LiDAR data, in this paper. The main idea is to fully exploit the advantage of the convolution operator in characterizing locally correlated features and the promising capability of transformer architecture in learning long-range dependencies. Moreover, multi-scale feature fusion and probabilistic decision fusion strategies are also designed in one framework, in order to further improve classification performance. Here, the proposed GLT-Net mainly consists of multi-scale local spatial feature learning, global spectral feature learning, and global-local feature fusion classification. In specific, multi-modal image cubes of different sizes are firstly extracted and sent into convolutional neural networks (CNNs) to learn local spatial features, which is followed by multi-modal information propagation and spatial-attention guided multi-scale feature fusion. Afterwards, by considering spectral feature channels from a sequential perspective, vision transformers are introduced to model the global spectral dependencies. Finally, multiple class estimations based on local and global features are integrated via a probabilistic decision fusion strategy. In this way, complementary information of multi-modal data as well as local/global spectral-spatial information can be fully mined and jointly utilized. Extensive experiments on three popular HSI and LiDAR datasets demonstrate that the proposed method performs superiority over state-of-the-art methods. The source code of the proposed method will be made publicly available at https://github.com/Ding-Kexin/GLT-Net.

Vegetation structure parameters determine high burn severity likelihood in different ecosystem types: A case study in a burned Mediterranean landscape

The design and implementation of pre-fire management strategies in heterogeneous landscapes requires the identification of the ecological conditions contributing to the most adverse effects of wildfires. This study evaluates which features of pre-fire vegetation structure, estimated through broadband land surface albedo and Light Detection and Ranging (LiDAR) data fusion, promote high wildfire damage across several fire-prone ecosystems dominated by either shrub (gorse, heath and broom) or tree species (Pyrenean oak and Scots pine). Topography features were also considered since they can assist in the identification of priority areas where vegetation structure needs to be managed. The case study was conducted within the scar of a mixed-severity wildfire that occurred in the Western Mediterranean Basin. Burn severity was estimated using the differenced Normalized Burn Ratio index computed from Sentinel-2 multispectral instrument (MSI) Level 2 A at 10 m of spatial resolution and validated in the field using the Composite Burn Index (CBI). Ordinal regression models were implemented to evaluate high burn severity outcome based on three groups of predictors: topography, pre-fire broadband land surface albedo computed from Sentinel-2 and pre-fire LiDAR metrics. Models were validated both by 10-fold cross-validation and external validation. High burn severity was largely ecosystem-dependent. In oak and pine forest ecosystems, severe damage was promoted by a high canopy volume (model accuracy = 79%) and a low canopy base height (accuracy = 82%), respectively. Land surface albedo, which is directly related to aboveground biomass and vegetation cover, outperformed LiDAR metrics to predict high burn severity in ecosystems with sparse vegetation. This is the case of gorse and broom shrub ecosystems (accuracy of 80% and 77%, respectively). The effect of topography was overwhelmed by that of the vegetation structure portion of the fire triangle behavior, except for heathlands, in which warm and steep slopes played a key role in high burn severity outcome together with horizontal and vertical fuel continuity (accuracy = 71%). The findings of this study support the fusion of LiDAR and satellite albedo data to assist forest managers in the development of ecosystem-specific management actions aimed at reducing wildfire damage and promote ecosystem resilience.

LiDAR-INS/GNSS-Based Real-Time Ground Removal, Segmentation, and Georeferencing Framework for Smart Transportation

Light detection and ranging (LiDAR) sensor is attracting significant attention in the field of smart transportation because of its ability to give depth information. It is already extensively used for obstacle detection. The inertial navigation system (INS) with a global navigation satellite system (GNSS) gives the global position, orientation, and velocity of a given object in real time. In autonomous driving, LiDAR point clouds are segmented to find out the positions of various objects in the surrounding of the vehicle of interest. However, the obstacle position is determined in local east–north–up (ENU) coordinates. In this article, an end-to-end framework has been presented, which takes input from LiDAR and INS/GNSS systems and gives the obstacle position in universal coordinates (latitude, longitude) in real time. The proposed framework includes ground point removal from raw LiDAR data, obstacle segmentation, and LiDAR data fusion with INS/GNSS data for georeferencing. For LiDAR data processing, a novel method for removing ground points has been presented, in which the entire point cloud is divided into square grids in the horizontal plane. Based on the statistics of the vertical distribution of the points in each grid, ground points are identified. Two ground point datasets were also created for testing the proposed algorithm. These datasets contain point clouds of ground with varying inclination from various multichannel LiDARs. The proposed ground removal method was tested on the Paris-Lille-3D dataset and the dataset created. An F1-score of greater than 0.99 and 0.98 was achieved on our dataset and the Paris Lille-3D dataset, respectively. The framework is tested on different hardware configurations and found to be suitable for real-time applications.

Fusion of Hyperspectral CASI and Airborne LiDAR Data for Ground Object Classification through Residual Network.

Modern satellite and aerial imagery outcomes exhibit increasingly complex types of ground objects with continuous developments and changes in land resources. Single remote-sensing modality is not sufficient for the accurate and satisfactory extraction and classification of ground objects. Hyperspectral imaging has been widely used in the classification of ground objects because of its high resolution, multiple bands, and abundant spatial and spectral information. Moreover, the airborne light detection and ranging (LiDAR) point-cloud data contains unique high-precision three-dimensional (3D) spatial information, which can enrich ground object classifiers with height features that hyperspectral images do not have. Therefore, the fusion of hyperspectral image data with airborne LiDAR point-cloud data is an effective approach for ground object classification. In this paper, the effectiveness of such a fusion scheme is investigated and confirmed on an observation area in the middle parts of the Heihe River in China. By combining the characteristics of hyperspectral compact airborne spectrographic imager (CASI) data and airborne LiDAR data, we extracted a variety of features for data fusion and ground object classification. Firstly, we used the minimum noise fraction transform to reduce the dimensionality of hyperspectral CASI images. Then, spatio-spectral and textural features of these images were extracted based on the normalized vegetation index and the gray-level co-occurrence matrices. Further, canopy height features were extracted from airborne LiDAR data. Finally, a hierarchical fusion scheme was applied to the hyperspectral CASI and airborne LiDAR features, and the fused features were used to train a residual network for high-accuracy ground object classification. The experimental results showed that the overall classification accuracy was based on the proposed hierarchical-fusion multiscale dilated residual network (M-DRN), which reached an accuracy of 97.89%. This result was found to be 10.13% and 5.68% higher than those of the convolutional neural network (CNN) and the dilated residual network (DRN), respectively. Spatio-spectral and textural features of hyperspectral CASI images can complement the canopy height features of airborne LiDAR data. These complementary features can provide richer and more accurate information than individual features for ground object classification and can thus outperform features based on a single remote-sensing modality.

Fusion of multispectral imagery and LiDAR data for roofing materials and roofing surface conditions assessment

ABSTRACT Assessment of rooftop rainwater harvesting (RRWH) quality and suitability requires detail and reliable information on roofs. Characterization of roof surface conditions affects the quality of harvested rainwater. Nevertheless, the implementation of the system requires improvement in terms of the roof detection techniques to ensure the roof of the building is selected appropriately. Thus, the classification techniques need to be optimized to detect roof materials and roof surface conditions (new or old) with high accuracy. This study aimed to produce high precision detailed roof materials and roof surface conditions map with using high-resolution remote sensing imagery, WorldView-3 (WV3) and light detection and ranging (LiDAR) data. Three different fusion methods; layer stacking (LS), Gram-Schmidt (GS) and principal components spectral sharpening (PCSS) were explored and their performances were compared to improve the spatial and spectral richness of the image. Subsequently, the roof materials and roof surface conditions classes which include old concrete, new concrete, old metal, new metal, old asbestos and new asbestos had been discriminated by employing support vector machine (SVM) and the rule-based technique known as a decision tree (DT). Generally, generated rule-sets present a higher overall accuracy with 87%, 72% and 66% for LS, GS and PCSS, respectively. For SVM classifier, the maximum accuracy recorded for LS, PCSS and GS were 70%, 63% and 43% respectively. Therefore, rule-based classification via LS fusion technique was utilized to identify suitable rooftops for the development of harvested rainwater system in the urban area. Findings indicate that the degradation status of a roof in heterogenous urban environments could be determined from satellite observation and the quality of roof-based harvested rainwater affected by roofing materials and roofing surface conditions can be analysed effectively.

Fusion of Multispectral LiDAR, Hyperspectral, and RGB Data for Urban Land Cover Classification

With the increasing importance of monitoring urban areas, the question arises which sensors are best suited to solve the corresponding challenges. This letter proposes novel node tests within the random forest (RF) framework, which allows them to apply them to optical RGB images, hyperspectral images, and light detection and ranging (LiDAR) data, either individually or in combination. This does not only allow to derive accurate classification results for many relevant urban classes without preprocessing or feature extraction but also provides insights into which sensor offers the most meaningful data to solve the given classification task. The achieved results on a public benchmark data set are superior to results obtained by deep learning approaches despite being based on only a fraction of training samples.

A Novel Framework to Automatically Fuse Multiplatform LiDAR Data in Forest Environments Based on Tree Locations

The emerging near-surface light detection and ranging (LiDAR) platforms [e.g., terrestrial, backpack, mobile, and unmanned aerial vehicle (UAV)] have shown great potential for forest inventory. However, different LiDAR platforms have limitations either in data coverage or in capturing undercanopy information. The fusion of multiplatform LiDAR data is a potential solution to this problem. Because of the complexity and irregularity of forests and the inaccurate positioning information under forest canopies, current multiplatform data fusion still involves substantial manual efforts. In this article, we proposed an automatic multiplatform LiDAR data registration framework based on the assumption that each forest has a unique tree distribution pattern. Five steps are included in the proposed framework, i.e., individual tree segmentation, triangulated irregular network (TIN) generation, TIN matching, coarse registration, and fine registration. TIN matching, as the essential step to find the corresponding tree pairs from multiplatform LiDAR data, uses a voting strategy based on the similarity of triangles composed of individual tree locations. The proposed framework was validated by fusing backpack and UAV LiDAR data and fusing multiscan terrestrial LiDAR data in coniferous forests. The results showed that both registration experiments could reach a satisfying data registration accuracy (horizontal root-mean-square error (RMSE) < 30 cm and vertical RMSE < 20 cm). Moreover, the proposed framework was insensitive to individual tree segmentation errors, when the individual tree segmentation accuracy was higher than 80%. We believe that the proposed framework has the potential to increase the efficiency of accurately registering multiplatform LiDAR data in forest environments.

An Approach to Tree Detection Based on the Fusion of Multitemporal LiDAR Data

The repetitive acquisition of airborne light detection and ranging (LiDAR) data for forest surveys is rapidly increasing, thus making possible the forest dynamic analysis. Moreover, the availability of multitemporal data enables the possibility to improve the forest attribute estimates performed at single date, especially when one LiDAR acquisition has a lower pulse density with respect to the other. This letter presents a novel approach that exploits the bitemporal data information to: 1) improve the tree detection at both dates and 2) identify forest changes at single tree level. This is done by using a novel compound approach to the detection of trees in bitemporal data based on the Bayes rule for minimum error. Significant geometric features are extracted for each candidate tree-top and are used to estimate statistical terms employed in the compound approach. The multitemporal information is considered by estimating (in an iterative way) the probabilities of transition, which takes into account the temporal dependence between the LiDAR acquisitions. The proposed approach is evaluated on multitemporal LiDAR data acquired in a coniferous forest located in the Southern Italian Alps. Experimental results confirm the effectiveness of the compound detection that increases the overall accuracy (OA) up to 8.6% with respect to the single-date detection.

Pixels and 3-D Points Alignment Method for the Fusion of Camera and LiDAR Data

The fusion of light detection and ranging (LiDAR) and camera data is a promising approach to improve the environmental perception and recognition for intelligent vehicles because of the combination of depth and color information. One of the difficulties in achieving the fusion is the accurate alignment of the 3-D points with the image pixels. Current methods of data alignment involve the steps of estimating the camera intrinsic parameters and developing a transformation matrix between the camera and LiDAR frame. The drawback of these methods is the accumulation of errors during the calculation of the camera intrinsic parameters and the transformation matrix. In order to improve the data alignment accuracy, we propose a novel algorithm that directly calculates the alignment between the 3-D points and the pixels without the need for camera parameters and calibration of the coordinate transformation matrix. We call the proposed method the pixel and 3-D point alignment (PPA) method. The alignment procedure is achieved by using the extracted corresponding points. First, we calculate a linear alignment matrix without considering the image distortion; and second, we optimize the parameters using the maximum likelihood estimation to consider the camera distortion. Simulation and experimental results indicate that the PPA method is able to align the 3-D points in LiDAR frame with the pixels in image frame with higher accuracy and increased robustness against noise in calibration process than comparable state-of-the-art methods.

Coupled Higher-Order Tensor Factorization for Hyperspectral and LiDAR Data Fusion and Classification

Hyperspectral and light detection and ranging (LiDAR) data fusion and classification has been an active research topic, and intensive studies have been made based on mathematical morphology. However, matrix-based concatenation of morphological features may not be so distinctive, compact, and optimal for classification. In this work, we propose a novel Coupled Higher-Order Tensor Factorization (CHOTF) model for hyperspectral and LiDAR data classification. The innovative contributions of our work are that we model different features as multiple third-order tensors, and we formulate a CHOTF model to jointly factorize those tensors. Firstly, third-order tensors are built based on spectral-spatial features extracted via attribute profiles (APs). Secondly, the CHOTF model is defined to jointly factorize the multiple higher-order tensors. Then, the latent features are generated by mode-n tensor-matrix product based on the shared and unshared factors. Lastly, classification is conducted by using sparse multinomial logistic regression (SMLR). Experimental results, conducted with two popular hyperspectral and LiDAR data sets collected over the University of Houston and the city of Trento, respectively, indicate that the proposed framework outperforms the other methods, i.e., different dimensionality-reduction-based methods, independent third-order tensor factorization based methods, and some recently proposed hyperspectral and LiDAR data fusion and classification methods.

Hyperspectral and LiDAR Data Classification Using Kernel Collaborative Representation Based Residual Fusion

A new framework is proposed for the fusion of hyperspectral and light detection and ranging (LiDAR) data based on the extinction profiles (EPs), local binary pattern (LBP), and kernel collaborative representation classification. Specifically, EP and LBP features are extracted from both sources. Then, the derived features of each source are classified by collaborative representation-based classifier with Tikhonov regularization (KCRT). Reconstruction residuals are fused to produce the final label assignment. Experimental results demonstrate that the proposed method outperforms the existing methods in hyperspectral and LiDAR data fusion.

Extraction of Urban Objects in Cloud Shadows on the basis of Fusion of Airborne LiDAR and Hyperspectral Data

Feature extraction in cloud shadows is a difficult problem in the field of optical remote sensing. The key to solving this problem is to improve the accuracy of classification algorithms by fusing multi-source remotely sensed data. Hyperspectral data have rich spectral information but highly suffer from cloud shadows, whereas light detection and ranging (LiDAR) data can be acquired from beneath clouds to provide accurate height information. In this study, fused airborne LiDAR and hyperspectral data were used to extract urban objects in cloud shadows using the following steps: (1) a series of LiDAR and hyperspectral metrics were extracted and selected; (2) cloud shadows were extracted; (3) the new proposed approach was used by combining a pixel-based support vector machine (SVM) and object-based classifiers to extract urban objects in cloud shadows; (4) a pixel-based SVM classifier was used for the classification of the whole study area with the selected metrics; (5) a decision-fusion strategy was employed to get the final results for the whole study area; (6) accuracy assessment was conducted. Compared with the SVM classification results, the decision-fusion results of the combined SVM and object-based classifiers show that the overall classification accuracy is improved by 5.00% (from 87.30% to 92.30%). The experimental results confirm that the proposed method is very effective for urban object extraction in cloud shadows and thus improve urban applications such as urban green land management, land use analysis, and impervious surface assessment.

Hyperspectral and LiDAR Data Fusion Classification Using Superpixel Segmentation-Based Local Pixel Neighborhood Preserving Embedding

A new method of superpixel segmentation-based local pixel neighborhood preserving embedding (SSLPNPE) is proposed for the fusion of hyperspectral and light detection and ranging (LiDAR) data based on the extinction profiles (EPs), superpixel segmentation and local pixel neighborhood preserving embedding (LPNPE). A new workflow is proposed to calibrate the Goddard’s LiDAR, hyperspectral and thermal (G-LiHT) data, which allows our method to be applied to actual data. Specifically, EP features are extracted from both sources. Then, the derived features of each source are fused by the SSLPNPE. Using the labeled samples, the final label assignment is produced by a classifier. For the open standard experimental data and the actual data, experimental results prove that the proposed method is fast and effective in hyperspectral and LiDAR data fusion.