A dual attention driven multiscale-multilevel feature fusion approach for hyperspectral image classification

Abstract

ABSTRACT Deep learning has achieved promising results for hyperspectral image (HSI) classification in recent years due to its hierarchical structure and automatic feature extraction ability from raw data. The HSI has continuous spectral information, allowing for the precise identification of materials by capturing minute spectral differences. Convolutional neural networks (CNNs) have proven to be effective feature extractors for HSI classification. However, inherent network limitations prevent them from adequately mining and representing the sequence attributes of spectral signatures and learning critical and valuable features from both spectral and spatial dimensions simultaneously. This paper proposes a deep learning-based framework called a novel dual attention-based multiscale-multilevel ConvLSTM3D (DAMCL) to address these challenges. In this work, our contribution is threefold; firstly, a dual attention mechanism is proposed, effectively learning critical and valuable features from spectral and spatial dimensions. Secondly, multiscale ConvLSTM3D blocks can learn the discriminative features alongside handling long-range dependencies of spectral data. Thirdly, these features are combined by a multilevel feature fusion approach to maximize the impact of features learned at different levels. To assess the performance of the proposed method, extensive experiments are carried out on five different benchmark datasets containing complex and challenging land cover classes. The results confirm that the proposed method outperforms state-of-the-art techniques with a small number of training samples in terms of overall accuracy (OA), average accuracy (AA), and Kappa . The overall accuracy of 98.88%, 99.42%, 99.20%, 95.37%, and 92.57% is achieved over the Indian Pines, Salinas Valley, University of Pavia, Houston 2013, and Houston 2018 datasets, respectively.