MTD-YOLO: Multi-task deep convolutional neural network for cherry tomato fruit bunch maturity detection

Abstract

In recent years, the escalating labor costs in agricultural production have emerged as a major concern. The use of inspection robots to achieve automated inspection of fruit and fruit bunches for ripeness not only enhances production efficiency and cost savings, but also simplifies the tasks for workers. To address this issue, an improved YOLOv7-based multi-task deep convolutional neural network (DCNN) detection model, called MTD-YOLOv7, is proposed in this paper. Initially, the dataset labels were expanded to meet the requirements of multi-task classification. Two additional decoders were then added on the basis of YOLOv7 to detect tomato fruit clusters, fruit maturity and cluster maturity. Subsequently, the loss function was designed based on the characteristics of multi-task and the Scale-Sensitive Intersection over Union (SIoU) was used instead of Complete Intersection over Union (CIoU) to improve the model’s recognition accuracy. Finally, to verify the effectiveness of the algorithm, tests were conducted on the cherry tomato dataset, and comparisons were made with common target detection algorithms, classification models, and cascade models. The experimental findings reveal that MTD-YOLOv7 achieved an overall score of 86.6% in multi-task learning, with an average inference time of 4.9 ms (RTX3080). It excels in simultaneous detection of cherry tomato fruits and bunches, fruit maturity, and bunch maturity, offering exceptional precision, rapid detection, and robust generalization capabilities. Its suitability extends to various applications, notably in inspection tasks.