Dual flow transformer network for multispectral image segmentation of wheat yellow rust

Abstract

Yellow rust is widely regarded as one of the most significant hazards faced by wheat. Traditional prevention methods are mainly based on pesticide application, which has high prevention costs and is harmful to the environment. Early disease detection and management can significantly reduce the cost of disease prevention and the impact on the environment. Therefore, a stable and reliable yellow rust detection method has become a direction for researchers. Spectral technology and deep learning methods are being introduced by more and more people to detect wheat yellow rust. In the recently years, Convolutional Neural Network (CNN) has got specific achievements of remote sensing image (RSI) processing. However, although CNN has achieved excellent performance, due to the locality of the convolution operation, it cannot learn the global and long-range semantic information interaction well. In this paper, we propose a dual flow transformer network (DF-Transformer), a combination of transformer and convolution, for multispectral image segmentation of wheat yellow rust. Tokenized image patches are delivered to a transformer-based encoder-decoder architecture with Unet blocks for global and long-range semantic feature learning. In particular, it uses a shifted window as an encoder, a maximum pooling-based U-Net block, and a hierarchical Swin transformer that extracts context functions. Next, an upconvolution-based decoder is designed to recover the spatial resolution of the feature maps. Experiments on multispectral images segmentation of wheat yellow rust demonstrate that the dual flow transformer network outperforms those methods with full-convolution.