S3Net: Spectral–Spatial Siamese Network for Few-Shot Hyperspectral Image Classification

Abstract

Deep learning (DL) has shown great potentials for hyperspectral image (HSI) classification due to its powerful ability of nonlinear modeling and end-to-end optimization. However, DL models are easily get trapped into overfitting due to limited training labels since the labeling process is time-consuming and laborious in real classification scenario. To overcome this issue, we propose a novel spectral-spatial siamese network (S3Net) for few-shot HSI classification. Firstly, a lightweight spectral-spatial network (SSN) composed of 1-D and 2-D convolution is proposed to extract spectral-spatial features. Secondly, S3Net is constructed by two SSNs in dual branches, which can augment training set by feeding sample pairs into each branch, and thus enhancing the model separability. To provide more features for the model, differentiated patches are fed into each branch, where negative samples are random selected to avoid redundancy. Finally, a weighted contrastive loss is designed to promote the model to fit in the right direction by focusing on sample pairs that are hardly to be identified. Moreover, another adaptive cross entropy loss is conceived to learn the fusion ratio of the two branches. Experiments based on three commonly used HSI data sets demonstrate that S3Net outperforms traditional and state-of-the-art DL-based HSI classification methods under few-shot training scenario. In addition, the weighted contrastive loss and the adaptive cross entropy loss jointly improve the discrimination power of the model.