Transformer-based ripeness segmentation for tomatoes

Abstract

With the recent development of computer vision technology, various computer vision techniques have been applied to agriculture. Recently, the Transformer network has been introduced to image recognition, which allows a different approach to extracting features from images compared to convolutional neural networks (CNNs). In this study, we will verify whether Transformer can correctly perform feature extraction for instance segmentation, an important task for understanding the location and condition of agricultural crops. Instance segmentation based on three different ripeness was performed on the tomato dataset by varying image sizes and data augmentation processes. We compared the performances of CNN-based ResNet (ResNet-50 and ResNet-101) and Transformer-based Swin Transformer (Swin-T and Swin-S) for the backbone of Mask R-CNN. The results show that Transformer-based models are sufficiently competitive compared to CNN models in the instance segmentation task for crops. Moreover, Swin Transformer is effective for larger image sizes. Also, by adding a data augmentation process, Swin Transformer achieved better results than ResNet. It can be said that Transformer-based models are useful in feature extraction for agricultural crops.