Semantic Segmentation of Strawberry Plants Using DeepLabV3+ for Small Agricultural Robot

Abstract

This study presents a method for identifying strawberry plants (fruits, flowers, calyxes, and trusses) with the aim of developing a multi-functional agricultural work assist robot for small-scale facilities. We focus on semantic segmentation by deep learning. Some pre-trained CNNs (ResNet-18, ResNet-50, Xception and MobileNetV2) are compared as the initial value of weights for feature extraction in DeepLabV3+. In this study, ResNet-50 is used as the backbone network of DeepLabV3+. In addition, we propose a method of applying post-processing based on the shape characteristics of plants to the results of semantic segmentation. The proposed method is evaluated using the images obtained in the actual strawberry farm, and its accuracy is evaluated by mean IoU. The effectiveness of the proposed method is shown by comparing with and without post-processing. The maximum was 0.731 for fruits and the minimum was 0.294 for trusses (0.643 and 0.199 respectively without post-processing). We discuss the validity of the evaluation method IoU, and evaluate the results using boundary F1 score, which takes contours more carefully into account. If boundary F1 score is sufficient even if IoU is not sufficient, it is possible that the semantic segmentation results are valid depending on the task of the robot.