A two-stream network with complementary feature fusion for pest image classification

Abstract

Pests are diverse and the available datasets often contain an uneven number of examples for different pests (a.k.a., the long-tail distribution). This poses a great challenge to learning-based classification methods, especially deep networks, and often leads to degraded performance, especially for the minority (tail) classes. This paper presents a deep learning integration architecture based on decoupling training and fusion learning, which integrates different models with complementary performance on pest datasets with a long-tailed distribution to improve the overall classification performance of pests. A deep neural network is designed that fuses two complementary deep learning models at the feature level, which consists of a convolution neural network (ConvNeXt) and a Swin Transformer model for decoupling training. Experiments are conducted using three datasets (d0, insect, and IP102), and evaluation on accuracy, recall, and F1-Score is reported. For the large-scale pest dataset with long-tailed distribution IP102, the accuracy achieves 76.1%, which outperforms the state-of-the-art methods. In addition, the accuracy for d0 and insect datasets are 98.5% and 92.3%, respectively.