A Collaborative Correlation-Matching Network for Multimodality Remote Sensing Image Classification

Abstract

Recently, with the increasing availability of the high-quality panchromatic (PAN) and multi-spectral (MS) remote sensing images, the inherent complementarity between PAN and MS images provides a wide development prospect for the multi-modality remote sensing image classification task. However, how to cleverly relieve the modal differences and effectively integrate the single-modality PAN and MS features is still a challenge. In this paper, we design a Collaborative Correlation-Matching Network (CCM-Net) for multi-modality remote sensing image classification. Concretely, we first propose a Bi-directional Dominant Feature Supervision (Bi-DFS) learning, it utilizes single-modality dominant features as supplementary supervision information to establish the joint optimization loss function, thereby adaptively narrowing the differences between modalities before the feature extraction. In the feature extraction stage, the Interactive Correlation Feature Matching (ICFM) learning, composing the Spa-FM and Spe-FM strategies, is proposed to establish interactive matching and enhancement between multi-modality strong correlation features from the perspective of spatial and spectral, respectively, thereby effectively alleviating the semantic deviation of multi-modality features. Finally, we aggregate finer multi-level multi-modality features to obtain top-level features with high discrimination. The effectiveness of the proposed algorithm has been verified on multiple data sets. Our code is available at: https://github.com/Momuli/CCM-Net.git.