A multi-scale 3D convolution neural network for spectral-spatial classification of hyperspectral imagery

Abstract

Along with the remarkable achievements of various neural networks in the field of computer vision, deep learning methods have gradually been applied to hyperspectral image (HSI) classification. Traditional classification methods have several outstanding issues, including the insufficient hand-craft features and time-consuming and laborious feature extraction. We therefore propose a multi-scale, 3D convolutional neural network (CNN) framework (MSCNN), trained in an end-to-end manner for hyperspectral image classification. Using the original hyperspectral image as an input, the MSCNN framework leverages two branches of a 3D residual neural network to extract deep and abstract HSI features. Then, the HSI spectral-spatial features of different scales are fused and fed into the SoftMax layer to achieve an end-to-end hyperspectral image classification. Besides, data augmentation, dynamic learning rate, and regularization methods are used to promote the rapid convergence of MSCNN and avoid model over-fitting. Three well-known hyperspectral datasets (Indian Pines, University of Pavia and Pavia Center) were used to evaluate the classification performance of the proposed MSCNN method. The results indicate that, compared with the existing deep learning methods, the proposed MSCNN method achieves the best classification performance within the 20 training epochs. The overall accuracy (OA), average accuracy (AA), and kappa statistic (K) of MSCNN are 97.16%, 98.69% and 0.9665, respectively, for Indian Pines; 99.17%, 98.97% and 0.9888 for University of Pavia; and 99.87%, 99.70% and 0.9981 for Pavia Center.