Classification Of Light Detection And Ranging Data Research Articles

Joint Classification of Hyperspectral and LiDAR Data via Multiprobability Decision Fusion Method

With the development of sensor technology, the sources of remotely sensed image data for the same region are becoming increasingly diverse. Unlike single-source remote sensing image data, multisource remote sensing image data can provide complementary information for the same feature, promoting its recognition. The effective utilization of remote sensing image data from various sources can enhance the extraction of image features and improve the accuracy of feature recognition. Hyperspectral remote sensing (HSI) data and light detection and ranging (LiDAR) data can provide complementary information from different perspectives and are frequently combined in feature identification tasks. However, the process of joint use suffers from data redundancy, low classification accuracy and high time complexity. To address the aforementioned issues and improve feature recognition in classification tasks, this paper introduces a multiprobability decision fusion (PRDRMF) method for the combined classification of HSI and LiDAR data. First, the original HSI data and LiDAR data are downscaled via the principal component–relative total variation (PRTV) method to remove redundant information. In the multifeature extraction module, the local texture features and spatial features of the image are extracted to consider the local texture and spatial structure of the image data. This is achieved by utilizing the local binary pattern (LBP) and extended multiattribute profile (EMAP) for the two types of data after dimensionality reduction. The four extracted features are subsequently input into the corresponding kernel–extreme learning machine (KELM), which has a simple structure and good classification performance, to obtain four classification probability matrices (CPMs). Finally, the four CPMs are fused via a multiprobability decision fusion method to obtain the optimal classification results. Comparison experiments on four classical HSI and LiDAR datasets demonstrate that the method proposed in this paper achieves high classification performance while reducing the overall time complexity of the method.

A Hierarchical Coarse–Fine Adaptive Fusion Network for the Joint Classification of Hyperspectral and LiDAR Data

Hyperspectral image (HSI) and light detection and ranging (LiDAR) data have gained significant attention due to their excellent complementarity, which can be combined to improve the accuracy of land cover classification. However, there are still many challenges, such as unrelated physical characteristics, different data structures, and a lack of labeled samples. Many methods fail to leverage the full potential of multi-source data, particularly hierarchical complementary information. To address these problems, a hierarchical coarse–fine adaptive (HCFA) fusion network with dynamic convolution and a transformer is proposed for multi-source remote sensing land cover classification. Although fusing hierarchical information can improve the classification accuracy of the model, improper hierarchical feature selection and optimization may negatively affect the classification results. Therefore, a coarse–fine mutual learning strategy is proposed to dynamically fuse hierarchical information. Additionally, the disparity between multi-source data continues to prevent the realization of effective fusion. To tackle this challenge, cross-tokenization and cross-token attention are implemented to enhance information interaction. Furthermore, to improve the model representation with limited computational cost, we combine the advantages of dynamic convolution with a transformer. Validation on three standard datasets demonstrates that HCFA achieves high accuracy with just 1% of the training set while maintaining low computational costs.

Interactive transformer and CNN network for fusion classification of hyperspectral and LiDAR data

ABSTRACT The Transformer has become pivotal for the integrated analysis of multi-source remote-sensing (RS) data in Earth observation, particularly in applications such as the fusion classification of hyperspectral images (HSI) and Light Detection and Ranging (LiDAR) data. However, Transformers are often employed as effective feature extractors by adopting similar processing blocks for different modalities from multi-source sensors, overlooking differences in imaging principles and data characteristics. Moreover, in the feature extraction process across different sensor data, there is a lack of necessary cross-modal information interaction, leading to insufficient utilization of complementary information between different sensors and resulting in suboptimal fusion outcomes. In this paper, we propose an interactive Transformer and CNN network for the fusion classification of HSI and LiDAR data. Specifically, a heterogeneous three-branch network architecture is designed for HSI and LiDAR data, where Transformers and CNNs encapsulate global contextual spatial and spectral information for HSI and capture geometric elevation patterns for LiDAR data, respectively. Elevation-Spatial Interaction (ESI) and Spectral-Spatial Interaction (SSI) modules are then introduced for multi-stage feature interaction. ESI enables the CNN-Transformer network to focus on essential local elevation details while simultaneously modelling global contextual spatial information. SSI facilitates the Transformer-Transformer network to cyclically intertwine spectral and spatial information for long-range spectral-spatial feature fusion. Finally, the interacted elevation, spatial, and spectral features undergo the Gated Fusion module to achieve hierarchical fusion adaptively, resulting in an elevation-spatial-spectral representation. Experiments conducted on three benchmark HSI-LiDAR datasets demonstrate the effectiveness of our proposed approach.

Classification of hyperspectral and LiDAR data by transformer-based enhancement

ABSTRACT The integration of multi-modal data allows for a more accurate representation of the ground characteristics. For a comprehensive interpretation of remote sensing data, existing multi-modal data fusion research mainly focuses on the joint utilization of 3D Light Detection and Ranging (LiDAR) and 2D Hyperspectral Image (HSI) data. However, existing algorithms do not pay much attention to the interaction of high-level semantic information between different modal data before fusion. This paper proposes a novel multi-modal data fusion deep learning network with the Cross-Modal Self-Attentive Feature Fusion Transformer (SAFFT). The framework employs a multi-head self-attention layer to fuse various attention information from multiple heads, effectively enhancing advanced feature information from different modalities for comprehensive integration. Experimental results on the Houston 2013 dataset demonstrate the effectiveness of the proposed method, which achieves an overall accuracy (OA) of 94.3757% in classifying 15 semantic classes.

A cross-modal feature aggregation and enhancement network for hyperspectral and LiDAR joint classification

Advancements in Earth observation technologies have greatly enhanced the potential of integrating hyperspectral (HS) images with Light Detection and Ranging (LiDAR) data for land use and land cover classification. Despite this, most existing methods primarily focus on employing deep network layers to extract features from two heterogeneous data modalities, often overlooking a gradual modeling data representation approach from shallow to deep layers. Furthermore, excessive network layers can result in the deterioration of modality-specific features, therefore lowering the classification performance. The paper proposed a novel cross-modal feature aggregation and enhancement network for the joint classification of HSI and LiDAR data. Initially, a cross-modal feature fusion module is developed to exploit spatial scale consistency to complete the interchange and fusion of feature embedding at the pixel level, preserving the original information from the two heterogeneous modalities to a certain degree. Then two straightforward strategies (i.e., addition and concatenation) are employed in the shallow network layers before being sent to the transformer encoder. The former facilitates the model’s ability to discern more subtle distinctions and refine spatial location details. The latter ensures the preservation of information integrity, effectively mitigating the risk of feature loss. Moreover, invertible neural networks and a feature enhancement module are introduced, leveraging the complementary information of HSI and LiDAR data to enhance the detail and texture information extracted in deeper layers. Extensive experiments on Houston2013, Trento, and MUUFL datasets demonstrate that the proposed method outperforms several state-of-the-art models in three evaluation metrics, achieving an accuracy improvement of up to 2%. The proposed model brings new inspirations for HSI and LiDAR classification, which is critical for accurate environmental monitoring, urban planning, and precision agriculture. The source code is publicly accessible at https://github.com/zhangyiyan001/CMFAEN.

Multi-level interactive fusion network based on adversarial learning for fusion classification of hyperspectral and LiDAR data

With the growing awareness of the limitations of unimodal data, research on joint classification using multimodal remote sensing data has garnered increasing attention. However, existing methods still have certain deficiencies in dealing with feature extraction and feature fusion of multimodal remote sensing data. In order to obtain more accurate classification results, it is crucial to fully exploit and utilize the complementary information in multimodal remote sensing data. In this paper, a multi-level interactive fusion network based on adversarial learning (MLIF-AL) is proposed for hyperspectral image (HSI) and light detection and ranging (LiDAR) data fusion classification. First, the feature extraction part based on generative adversarial network (GAN) utilizes adversarial training between the generator and the discriminator to guide the network to learn more discriminative and distinguished feature representations. Meanwhile, a cross-modal interactive information extraction (CMIIE) module is also designed for the generator encoding part to promote the interaction between multimodal data from a global perspective and realize the full utilization of multimodal complementary information. In addition, in the multi-level feature fusion classification (MLFFC) part, the complementarities between the features learned by the neural networks at each layer are comprehensively utilized to obtain higher-level multimodal fusion semantic information. Finally, the adversarial loss and classification loss are combined for network training. Extensive experiments on three commonly used HSI and LiDAR datasets and comparisons with state-of-the-art methods demonstrate the effectiveness and superiority of the model.

A novel graph-attention based multimodal fusion network for joint classification of hyperspectral image and LiDAR data

The joint classification of hyperspectral image (HSI) and Light Detection and Ranging (LiDAR) data can provide complementary information for each other, which has become a prominent topic in the field of remote sensing. Nevertheless, the common CNN-based fusion techniques still suffer from the following drawbacks. (1) Most of these models omit the correlation and complementarity between different data sources and always fail to model the long-distance dependencies of spectral information well. (2) Simply splicing the multi-source feature embeddings overlooks the deep semantic relationships among them. To tackle these issues, we propose a novel graph-attention based multimodal fusion network (GAMF). Specifically, it employs three major components, including an HSI–LiDAR feature extractor, a graph-attention based fusion module and a classification module. In the feature extraction module, we consider the correlation and complementarity between multi-sensor data by parameter sharing and employ Gaussian tokenization for feature transformation additionally. To address the problem of long-distance dependencies, the deep fusion module utilizes modality-specific tokens to construct an undirected weighted graph, which is essentially a heterogeneous graph. And the deep semantic relationships between them are exploited utilizing a graph-attention based fusion framework. At the end, two fully connected layers classify the fused embeddings. Experiment evaluations on several benchmark HSI–LiDAR datasets (Trento, University of Houston 2013 and MUUFL) show that GAMF achieves more accurate prediction results than some state-of-the-art baselines. The code is available at https://github.com/tyust-dayu/GAMF.

Joint Classification of Hyperspectral and LiDAR Data Using Binary-Tree Transformer Network

The joint utilization of multi-source data is of great significance in geospatial observation applications, such as urban planning, disaster assessment, and military applications. However, this approach is confronted with challenges including inconsistent data structures, irrelevant physical properties, scarce training data, insufficient utilization of information and an imperfect feature fusion method. Therefore, this paper proposes a novel binary-tree Transformer network (BTRF-Net), which is used to fuse heterogeneous information and utilize complementarity among multi-source remote sensing data to enhance the joint classification performance of hyperspectral image (HSI) and light detection and ranging (LiDAR) data. Firstly, a hyperspectral network (HSI-Net) is employed to extract spectral and spatial features of hyperspectral images, while the elevation information of LiDAR data is extracted using the LiDAR network (LiDAR-Net). Secondly, a multi-source transformer complementor (MSTC) is designed that utilizes the complementarity and cooperation among multi-modal feature information in remote sensing images to better capture their correlation. The multi-head complementarity attention mechanism (MHCA) within this complementor can effectively capture global features and local texture information of images, hence achieving full feature fusion. Then, to fully obtain feature information of multi-source remote sensing images, this paper designs a complete binary tree structure, binary feature search tree (BFST), which fuses multi-modal features at different network levels to obtain multiple image features with stronger representation abilities, effectively enhancing the stability and robustness of the network. Finally, several groups of experiments are designed to compare and analyze the proposed BTRF-Net with traditional methods and several advanced deep learning networks using two datasets: Houston and Trento. The results show that the proposed network outperforms other state-of-the-art methods even with small training samples.

Hyperspectral and LiDAR Data Classification Based on Structural Optimization Transmission.

With the development of the sensor technology, complementary data of different sources can be easily obtained for various applications. Despite the availability of adequate multisource observation data, for example, hyperspectral image (HSI) and light detection and ranging (LiDAR) data, existing methods may lack effective processing on structural information transmission and physical properties alignment, weakening the complementary ability of multiple sources in the collaborative classification task. The complementary information collaboration manner and the redundancy exclusion operator need to be redesigned for strengthening the semantic relatedness of multisources. As a remedy, we propose a structural optimization transmission framework, namely, structural optimization transmission network (SOT-Net), for collaborative land-cover classification of HSI and LiDAR data. Specifically, the SOT-Net is developed with three key modules: 1) cross-attention module; 2) dual-modes propagation module; and 3) dynamic structure optimization module. Based on above designs, SOT-Net can take full advantage of the reflectance-specific information of HSI and the detailed edge (structure) representations of multisource data. The inferred transmission plan, which integrates a self-alignment regularizer into the classification task, enhances the robustness of the feature extraction and classification process. Experiments show consistent outperformance of SOT-Net over baselines across three benchmark remote sensing datasets, and the results also demonstrate that the proposed framework can yield satisfying classification result even with small-size training samples.

A Multi-Scale Pseudo-Siamese Network with an Attention Mechanism for Classification of Hyperspectral and LiDAR Data

For the remote sensing classification task, the ability of a single data source to identify the ground objects remains limited due to the lack of feature diversity. As the typical remote sensing data sources, hyperspectral imagery (HSI) and light detection and ranging (LiDAR) data can provide complementary spectral features and elevation information, respectively. To enhance classification ability, a multi-scale Pseudo-Siamese Network with attention mechanism (MA-PSNet) is proposed by fusing HSI and LiDAR data. In the network, two sub-branch networks are designed for extracting the features from HSI and LiDAR, respectively, and the connection is further established between these two branches. Specifically, a multi-scale feature learning module is incorporated, enabling the image features to be fully extracted at different scales. Similarly, a convolutional attention module is also embedded to highlight the saliency information of the objects, which makes the network training can be more targeted, thereby eventually improving the model performance for classification. The evaluation experiments of the proposed model are carried out on an urban dataset from Houston, USA, and a rural dataset from Trento, Italy. The overall accuracy (OA) of the model can reach 95.03% on the Houston data and 99.16% on the Trento data. The experimental results fully demonstrate that the proposed model has competitive performance compared with several state-of-the-art methods.

Local Information Interaction Transformer for Hyperspectral and LiDAR Data Classification

The multisource remote sensing classification task has two main challenges. 1) How to capture hyperspectral image (HSI) and light detection and ranging (LiDAR) features cooperatively to fully mine the complementary information between data. 2) How to adaptively fuse multisource features, which should not only overcome the imbalance between HSI and LiDAR data but also avoid the generation of redundant information. The local information interaction transformer (LIIT) model proposed herein can effectively address these above issues. Specifically, multibranch feature embedding is first performed to help in the fine-grained serialization of multisource features; subsequently, a local-based multisource feature interactor (L-MSFI) is designed to explore HSI and LiDAR features together. This structure provides an information transmission environment for multibranch features and further alleviates the homogenization processing mode of the self-attention process. More importantly, a multisource feature selection module (MSTSM) is developed to dynamically fuse HSI and LiDAR features to solve the problem of insufficient fusion. Experiments were carried out on three multisource remote-sensing classification datasets, the results of which show that LIIT has more performance advantages than the state-of-the-art CNN and transformer methods.

Collaborative Contrastive Learning for Hyperspectral and LiDAR Classification

Using single-source remote sensing (RS) data for classification of ground objects has certain limitations, however, multi-modal RS data contain different types of features, such as spectral features and spatial features of hyperspectral image (HSI) and elevation information of light detection and ranging (LiDAR) data, which can be used to extract and fuse high-quality features to improve the classification accuracy. Nevertheless, the existing fusion techniques are mostly limited by the number of labeled samples due to the difficulty of label collection in the multi-modal RS data. In this article, a fusion method of collaborative contrastive learning (CCL) is proposed to tackle the abovementioned issues for HSI and LiDAR data classification. The proposed CCL approach includes two stages of pre-training (CCL-PT) and fine-tuning (CCL-FT). In the CCL-PT stage, a collaborative strategy is introduced into contrastive learning (CL), which can extract features from HSI and LiDAR data separately, and achieve the coordinated feature representation and matching between the two-modal RS data without labeled samples. In the CCL-FT stage, a multi-level fusion network is designed to optimize and fuse the unsupervised collaborative features which are extracted in the CCL-PT stage for the classification tasks. Experimental results on three real-world data sets show that the developed CCL approach can perform excellently on the small sample classification tasks and CL is feasible for the fusion of multi-modal RS data.

Nearest Neighbor-Based Contrastive Learning for Hyperspectral and LiDAR Data Classification

The joint hyperspectral image (HSI) and light detection and ranging (LiDAR) data classification aims to interpret ground objects at more detailed and precise level. Although deep learning methods have shown remarkable success in the multisource data classification task, self-supervised learning has rarely been explored. It is commonly nontrivial to build a robust self-supervised learning model for multisource data classification, due to the fact that the semantic similarities of neighborhood regions are not exploited in the existing contrastive learning framework. Furthermore, the heterogeneous gap induced by the inconsistent distribution of multisource data impedes the classification performance. To overcome these disadvantages, we propose a nearest neighbor-based contrastive learning network (NNCNet), which takes full advantage of large amounts of unlabeled data to learn discriminative feature representations. Specifically, we propose a nearest neighbor-based data augmentation scheme to use enhanced semantic relationships among nearby regions. The intermodal semantic alignments can be captured more accurately. In addition, we design a bilinear attention module to exploit the second-order and even high-order feature interactions between the HSI and LiDAR data. Extensive experiments on four public datasets demonstrate the superiority of our NNCNet over state-of-the-art methods. The source codes are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/summitgao/NNCNet</uri> .

Multimodal Attention-Aware Convolutional Neural Networks for Classification of Hyperspectral and LiDAR Data

The attention mechanism is one of the most influential ideas in the deep learning community, which has shown excellent efficiency in various computer vision tasks. Thus, this article proposes the convolution neural network method with the attention mechanism to enhance the feature extraction of light detection and ranging (LiDAR) data. Meanwhile, our elaborately designed cascaded block contains a short path architecture beneficial for multistage information exchange. With the full exploitation of elevation information from LiDAR data and efficient utilization of the spatial-spectral information underlying hyperspectral data, our method provides a novel solution for multimodal feature fusion. Experiments are conducted on the LiDAR and hyperspectral dataset provided by the 2013 IEEE GRSS Data Fusion Contest and multisource Trento dataset to demonstrate the effectiveness of the proposed method. The experimental results have shown the superior results of the proposed method on both LiDAR and multimodality remote sensing data in comparison with several popular baselines.

Joint Classification of Hyperspectral and LiDAR Data Using a Hierarchical CNN and Transformer

The joint use of multisource remote-sensing (RS) data for Earth observation missions has drawn much attention. Although the fusion of several data sources can improve the accuracy of land-cover identification, many technical obstacles, such as disparate data structures, irrelevant physical characteristics, and a lack of training data, exist. In this article, a novel dual-branch method, consisting of a hierarchical convolutional neural network (CNN) and a transformer network, is proposed for fusing multisource heterogeneous information and improving joint classification performance. First, by combining the CNN with a transformer, the proposed dual-branch network can significantly capture and learn spectral–spatial features from hyperspectral image (HSI) data and elevation features from light detection and ranging (LiDAR) data. Then, to fuse these two sets of data features, a cross-token attention (CTA) fusion encoder is designed in a specialty. The well-designed deep hierarchical architecture takes full advantage of the powerful spatial context information extraction ability of the CNN and the strong long-range dependency modeling ability of the transformer network based on the self-attention (SA) mechanism. Four standard datasets are used in experiments to verify the effectiveness of the approach. The experimental results reveal that the proposed framework can perform noticeably better than state-of-the-art methods. The source code of the proposed method will be available publicly at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/zgr6010/Fusion_HCT.git</uri> .

Coupled adversarial learning for fusion classification of hyperspectral and LiDAR data

Hyperspectral image (HSI) provides rich spectral–spatial information and the light detection and ranging (LiDAR) data reflect the elevation information, which can be jointly exploited for better land-cover classification. However, due to different imaging mechanisms, HSI and LiDAR data always present significant image difference, current pixel-wise feature fusion classification methods relying on concatenation or weighted fusion are not effective. To achieve accurate classification result, it is important to extract and fuse similar high-order semantic information and complementary discriminative information contained in multimodal data. In this paper, we propose a novel coupled adversarial learning based classification (CALC) method for fusion classification of HSI and LiDAR data. In specific, a coupled adversarial feature learning (CAFL) sub-network is first trained, to effectively learn the high-order semantic features from HSI and LiDAR data in an unsupervised manner. On one hand, the proposed CAFL sub-network establishes an adversarial game between dual generators and discriminators, so that the learnt features can preserve detail information in HSI and LiDAR data, respectively. On the other hand, by designing weight-sharing and linear fusion structure in the dual generators, we can simultaneously extract similar high-order semantic information and modal-specific complementary information. Meanwhile, a supervised multi-level feature fusion classification (MFFC) sub-network is trained, to further improve the classification performance via adaptive probability fusion strategy. In brief, the low-level, mid-level and high-level features learnt by the CAFL sub-network lead to multiple class estimation probabilities, which are then adaptively combined to generate a final accurate classification result. Both the CAFL and MFFC sub-networks are collaboratively trained by optimizing a designed joint loss function, which consists of unsupervised adversarial loss and supervised classification loss. Overall, by optimizing the joint loss function, the proposed CALC network is pushed to learn highly discriminative fusion features from multimodal data, leading to higher classification accuracies. Extensive experiments on three well-known HSI and LiDAR data sets demonstrate the superior classification performance by the proposed CALC method than several state-of-the-art methods. The source code of the proposed method will be made publicly available at https://github.com/Ding-Kexin/CALC.

Hyperspectral and LiDAR Data Classification Using Joint CNNs and Morphological Feature Learning

Convolutional Neural Networks (CNNs) have been extensively utilized for Hyperspectral (HSI) as well as Light Detection and Ranging (LiDAR) data Classification. However, CNNs have not been much explored for joint HSI and LiDAR image classification. Therefore, this article proposes a joint feature learning (HSI and LiDAR) and fusion mechanism using CNN and Spatial Morphological blocks which generates highly accurate land-cover maps. The CNN model comprises three Conv3D layers and is directly applied to the HSIs for extracting discriminative spectral-spatial feature representation. On the contrary, the spatial morphological block is able to capture the information relevant to the height or shape of the different land-cover regions from LiDAR data. The LiDAR features are extracted using morphological dilation and erosion layers which increase the robustness of the proposed model by considering elevation information as an additional feature. Finally, both the obtained features from CNNs and spatial morphological blocks are combined using an additive operation prior to the classification. Extensive experiments are shown with widely used HSIs and LiDAR datasets, i.e., University of Houston (UH), Trento, and MUUFL Gulfport scene. The reported results show that the proposed model significantly outperforms traditional methods and other state-of-the-art deep learning models. The source code for the proposed model will be made available publicly at https://github.com/AnkurDeria/HSI+LiDAR.

Multiview Hierarchical Network for Hyperspectral and LiDAR Data Classification

In recent times, multisource remote sensing technology [e.g., hyperspectral image (HSI) and light detection and ranging (LiDAR) data] has been widely used in urban land-use recognition owing to its high classification effectiveness compared to using only single-source data. In this study, a multiview hierarchical network (MVHN) technique is developed for HSI and LiDAR data classification, which conducts the following execution procedures. First, based on the a preset band step length, the original HSI is sampled and divided into multiple groups with exactly the same number of bands to obtain spectral features. Then, principal components analysis is performed on the raw HSI to extract the first principal components (PCs) that meet the size of the LiDAR image. The Gabor filters are applied to the PCs and LiDAR to capture spatial details (i.e., textural features) of scenes. Specifically, a stacking mechanism is employed to generate fusion features once the above features are available. Next, a three-dimensional ResNet-like deep CNN is designed to extract the spectral–spatial information of the fusion feature. Finally, majority-voting is introduced into the classification results of the network trained using each fusion feature to achieve high-confidence final results. Experiments on three well-known HSI and LiDAR datasets (i.e., Houston, MUUFL, and Trento datasets) demonstrate the effectiveness of the proposed MVHN method compared to state-of-the-art comparable classification methods.

MSLAENet: Multiscale Learning and Attention Enhancement Network for Fusion Classification of Hyperspectral and LiDAR Data

The effective use of multimodal data to obtain accurate land cover information has become an interesting and challenging research topic in the field of remote sensing. In this paper, we propose a new method, multi-scale learning and attention enhancement network (MSLAENet), to implement Hyperspectral image (HSI) and light detection and ranging (LiDAR) data fusion classification in an end-to-end manner. Specifically, our model consists of three main modules. First, we design the composite attention (CA) module, which adopts self-attention to enhance the feature representations of HSI and LiDAR data, respectively, and cross-attention to achieve cross-modal information enhancement. Second, the proposed multi-scale learning (MSL) module combines self-calibrated convolutions and hierarchical residual structure to extract different scales of information to further improve the representation capability of the model. Finally, the attention-based feature fusion (FF) module fully considers the complementary information properties between different modalities and adaptively fuses heterogeneous features from different modalities. To test the performance of MSLAENet, we conduct experiments on three multimodal remote sensing datasets and compare them with the state-of-the-art fusion model, which demonstrates the effectiveness and superiority of the model.

A Triplet Semisupervised Deep Network for Fusion Classification of Hyperspectral and LiDAR Data

Data fusion of hyperspectral and light detection and ranging (LiDAR) is conducive to obtain more comprehensive surface information and thereby achieve better classification result in Earth Monitoring Systems. However, lack of labeled samples usually limits the performance of supervised classifiers, and the heterogeneity of multi-source data also brings great challenges to data fusion. Aiming to address these issues, we propose a triplet semi-supervised deep convolutional neural network (TSDN) for fusion classification of hyperspectral and LiDAR. Specifically, we utilize three basic pathways to extract deep learning features: 1D-CNN for spectral features in hyperspectral, 2D-CNN for spatial features in hyperspectral and Cascade Net for elevation features in LiDAR data. Furthermore, a novel label calibration module (LCM) is proposed to generate effective pseudo labels with high confidence based on the superpixel segmentation by comparing the multi-view classification results for assisting semi-supervised model training. In addition, we design a novel 3D-Cross Attention Block to enhance the complementary spatial features of multi-source data. Experiments on three public HSI-LiDAR benchmarks: Houston, Trento, and MUUFL Gulfport have demonstrated the effectiveness and superiority of our proposed method.