Dual-stream spectral-spatial convolutional neural network for hyperspectral image classification and optimal band selection

Abstract

Along with the high spectral rich information it provides, one of the difficulties in processing a hyperspectral image is the need for expert knowledge and high-spec hardware to process very high-dimensional data. 3D convolutional neural network(3D CNN), which uses spectral and spatial features together, enables a powerful solution for HSI classification. This study proposes an efficient dual-stream 3D CNN for accurate HSI classification.The proposed method offers effective classification using spectral-spatial features without relying on pre-processing or post-processing. A comparative study of how CNN classification performance is affected by hyperspectral band selection based on deep reinforcement learning (DRL) is presented. Using the most relevant bands in the hyperspectral image is decisive in deep CNN networks without losing information and accuracy. The proposed method was compared with 3D CNN, 3D + 1D CNN, Multiscale 3D deep convolutional neural network (M3D-DCNN), and InceptionV3 algorithms using Indian Pines (IP), Salinas, Pavia Center (PaviaC), Houston 2013 and QUH-Tangdaowan datasets. It achieved 92.43 % overall accuracy (OA) in IP, 95.06 % OA in Salinas dataset, 99.00 % OA in PaviaC dataset, 91.25 % OA in Houston 2013 and 94.87 % OA in QUH-Tangdaowan. Codes are released at: https://github.com/lapistlazuli/DS-3DCNN.