Smart solutions for capsicum Harvesting: Unleashing the power of YOLO for Detection, Segmentation, growth stage Classification, Counting, and real-time mobile identification

Abstract

This research paper explores a comprehensive approach to advancing capsicum harvesting by integrating cutting-edge technologies. The study addresses key objectives, including capsicum detection using various YOLO algorithms, peduncle detection through YOLO segmentation models in a proposed robotic harvester, laboratory testing of cutting target point coordinates using the Real Sense D455 RGB-D camera, growth stage determination, and capsicum counting/tracking with a supervision algorithm. The investigation highlights the YOLOv8s model as the most successful for capsicum detection, achieving a remarkable mean Average Precision (mAP) of 0.967 at a 0.5 Intersection over Union (IOU) threshold. As part of the growth stage determination task, YOLOv8s achieved a satisfactory mAP of 0.614 at the same IOU threshold. Additionally, the YOLOv8s-seg model demonstrated superior performance in peduncle detection, attaining a box mAP of 0.790 and a mask mAP of 0.771. The YOLOv8s-seg model excels in peduncle detection with a box mAP of 0.790 and a mask mAP of 0.771. Laboratory experiments using the Real Sense D455 RGB-D camera showcased its capability to localize the target point with a maximum error of 8 mm longitudinally, 9 mm vertically, and 12 mm laterally. The developed tracking and counting algorithm achieve a notable counting accuracy of 94.1 % during the third harvesting cycle in the greenhouse. The Android application developed demonstrated robust performance, achieving high accuracy (Mean IoU: 0.92), precise localization (Mean Euclidean Distance: 5 pixels), responsive user interface (Touch Response Time: 150 ms), and broad compatibility across Android versions and device types, with effective error handling (Success Rate: 95 %). The study not only advances capsicum harvesting techniques but also presents practical insights for the integration of advanced technologies, paving the way for efficient robotic harvesting systems in agriculture.