Lightweight omni-dimensional dynamic convolution with spatial-spectral self-attention network for hyperspectral image classification

Abstract

ABSTRACT Omni-dimensional dynamic convolution can adjust spatial sizes, the amount of input/output channels, and kernel size based on the characteristics of the input, thus improving the capacity to extract characteristics through convolution, resulting in increased computation and model complexity. As a result, it is challenging to deploy these models on resource-limited airborne high-spectral testing systems. This paper proposes a lightweight omni-dimensional dynamic convolution network as a possible solution to this issue. In this network, the LS2CM module is used for spatial-spectral information extraction, and the pyramidal residual network is the basic architecture for extracting features from shallow to deep layers, and lightweight omni-dimensional dynamic convolution is adopted in the convolution module. Overcoming the pyramidal structure can only extract local information, we proposed a new simple spatial-spectral self-attention mechanism to extract global information from hyperspectral. Besides, the Huber function as an activation function. The IP, WHU-Hi-LongKou, SA, and PU datasets were used to verify the accuracy of the suggested network model. The results show that the proposed lightweight network maintains an appropriate equilibrium among model complexity as well as accuracy.