Local Information Interaction Transformer for Hyperspectral and LiDAR Data Classification

Abstract

The multisource remote sensing classification task has two main challenges. 1) How to capture hyperspectral image (HSI) and light detection and ranging (LiDAR) features cooperatively to fully mine the complementary information between data. 2) How to adaptively fuse multisource features, which should not only overcome the imbalance between HSI and LiDAR data but also avoid the generation of redundant information. The local information interaction transformer (LIIT) model proposed herein can effectively address these above issues. Specifically, multibranch feature embedding is first performed to help in the fine-grained serialization of multisource features; subsequently, a local-based multisource feature interactor (L-MSFI) is designed to explore HSI and LiDAR features together. This structure provides an information transmission environment for multibranch features and further alleviates the homogenization processing mode of the self-attention process. More importantly, a multisource feature selection module (MSTSM) is developed to dynamically fuse HSI and LiDAR features to solve the problem of insufficient fusion. Experiments were carried out on three multisource remote-sensing classification datasets, the results of which show that LIIT has more performance advantages than the state-of-the-art CNN and transformer methods.