A novel multi-label pest image classifier using the modified Swin Transformer and soft binary cross entropy loss

Abstract

As pests can cause heavy crop losses, integrated pest management is a vital aspect of agriculture. In general, pest recognition is essential to the integrated pest management. Many studies have explored how to achieve automatic pest recognition using computer vision and artificial intelligence techniques. However, most existing methods did not consider the class ambiguity problem. That is, a pest image may belong to multiple possibly true categories, but only one possible class label is assigned to the pest image. To close the above gap, this study converted the conventional one-label pest classification task into a multi-label one. In detail, the state-of-the-art deep network, Swin Transformer, was first modified to enable the predicted scores of possible classes to approximate one simultaneously by replacing the fully connected soft-max layer with a sigmoid activation layer. Then, a two-stage supervised learning algorithm using the binary cross entropy loss and the novel soft binary cross entropy loss was designed to train the Swin-Transformer-based multi-label classification model with single-label images. Experiments on the IP102 image dataset showed that the proposed method obtained the highest F1-score value of 60.83%. It outperformed the second-best one by a margin of 7.52%. In conclusion, the proposed method can tackle the pest class ambiguity problem on IP102 better.