Hybrid network model based on 3D convolutional neural network and scalable graph convolutional network for hyperspectral image classification

Abstract

Hyperspectral images (HSIs) contain hundreds of continuous spectral bands and are rich in spectral-spatial information. In terms of HSIs’ classification, traditional convolutional neural networks (CNNs) extract features based on HSI's spectral-spatial information through 2D convolution. However, 2D convolution extracts features in 2D plane without considering the relationships between spectral bands, which inevitably leads to insufficient feature extraction. 3D convolutional neural networks (3DCNNs) take account of the correlations among spectral bands and outperform 2D convolutional networks in feature extraction, but the computational cost is rather expensive. To address the above problem, a light-weight three-layer 3D convolutional network Module (3D-M) for HSIs’ spectral-spatial feature extraction is proposed. Another challenge is that neither 2D convolution nor 3D convolution utilizes the structural information inherent in the data. Graph convolution networks (GCNs) can model and utilize such information through the similarity matrix, also known as adjacency matrix. However, traditional GCNs cannot handle large-scale data because they construct adjacency matrix on all data, which results in high computational complexity and large storage requirement. To conquer this challenge, this article proposes a batch-graph strategy on which a scalable GCN is developed. Finally, a hybrid network model (HNM) based on the proposed light-weight 3D-M and scalable GCN is presented. HNM extracts spectral-spatial features of HSIs with low computational complexity through the light-weight 3D convolution network and leverages the structural information in data via the scalable GCN. The experimental results on three public datasets with different sizes demonstrate that the proposed HNM produces better classification results than other state-of-the-art hyperspectral images classification models in terms of overall accuracy (OA), average accuracy (AA) and kappa coefficient (Kappa).