Fast and stable pedicel detection for robust visual servoing to harvest shaking fruits

Abstract

This study introduces a fast and stable pedicel detection method for robust visual servoing (RVS), supplemented with video stabilization (ViS) and fast pedicel detection, to realize automated harvesting of shaking fruits. By incorporating ViS, the system effectively mitigates the effects of motion blur, thereby ensuring consistent and precise object detection. In addition, the Fourier spectrum-based band-stop filter (FSBF) is used to improve clarity. The proposed approach also leverages the fast point feature histogram (FPFH) for fast pedicel detection, achieving real-time detection rates of 15–37 fps. Furthermore, it incorporates 6D pose estimation, culminating in the implementation of a 6D pose-based robust visual servoing (6DRVS) system. The performance of this system is evaluated using standard metrics such as perception accuracy, approach accuracy, precision, recall, accuracy, and F1-score in both preliminary tests and on-site experiments at two cucumber farms in Korea. The 6DRVS, supplemented with fast pedicel detection and ViS, exhibited improvements across all evaluation metrics. It recorded 90.00% perception accuracy, 82.22% approach accuracy, 0.957 precision, 0.938 recall, 0.900 accuracy, and 0.947 F1-score, highlighting its essential role in ensuring precise and efficient harvesting.