Deformable Convolutional Neural Networks for Hyperspectral Image Classification

Abstract

Convolutional neural networks (CNNs) have recently been demonstrated to be a powerful tool for hyperspectral image (HSI) classification, since they adopt deep convolutional layers whose kernels can effectively extract high-level spatial–spectral features. However, sampling locations of traditional convolutional kernels are fixed and cannot be changed according to complex spatial structures in HSIs. In addition, the typical pooling layers (e.g., average or maximum operations) in CNNs are also fixed and cannot be learned for feature downsampling in an adaptive manner. In this letter, a novel deformable CNN-based HSI classification method is proposed, which is called deformable HSI classification networks (DHCNet). The proposed network, DHCNet, introduces the deformable convolutional sampling locations, whose size and shape can be adaptively adjusted according to HSIs’ complex spatial contexts. Specifically, to create the deformable sampling locations, 2-D offsets are first calculated for each pixel of input images. The sampling locations of each pixel with calculated offsets can cover the locations of other neighboring pixels with similar characteristics. With the deformable sampling locations, deformable feature images are then created by compressing neighboring similar structural information of each pixel into fixed grids. Therefore, applying the regular convolutions on the deformable feature images can reflect complex structures more effectively. Moreover, instead of adopting the pooling layers, the strided convolution is further introduced on the feature images, which can be learned for feature downsampling according to spatial contexts. Experimental results on two real HSI data sets demonstrate that DHCNet can obtain better classification performance than can several well-known classification methods.