Beyond the Patchwise Classification: Spectral-Spatial Fully Convolutional Networks for Hyperspectral Image Classification

Abstract

In recent years, patchwise classification methods are commonly adopted when dealing with the hyperspectral image (HSI) classification. Despite their promising results from the perspective of accuracy, the efficiency of these methods can hardly be ensured since there are redundant computations between adjacent patches. In this paper, we propose a spectral-spatial fully convolutional network for HSI classification with an end-to-end, pixel-to-pixel architecture. Compared with patchwise methods, the proposed framework can avoid the patch extraction and is more efficient. Since the training samples in HSIs are highly sparse, the training strategy in original fully convolutional networks is no longer feasible for HSIs. To solve this problem, we propose a novel mask matrix to assist the back-propagation in the training stage. Considering the importance of spectral and spatial features may vary for different objects and scenes, we combine both features with two weighting factors which can be adaptively learned during the network training. Besides, the dense conditional random field (CRF) is introduced into the framework to further balance the local and global information. Experiments on three benchmark HSI data sets demonstrate that the proposed method can yield competitive results with less time costs compared with patchwise methods.