NDMFCS: An automatic fruit counting system in modern apple orchard using abatement of abnormal fruit detection

Abstract

Automatic fruit counting is an important task for growers to estimate yield and manage orchards. Although many deep-learning-based fruit detection algorithms have been developed to improve performance of automatic fruit counting systems, abnormal fruit detection has often been caused by these algorithms detecting non-target fruits that have similar growth characteristics to target fruits. For abnormal fruit detection, detected fruits in the back row of the tree were defined as DFBRT, while detected fruits on the ground were defined as DFG. Both of them would result in a higher number of fruits counting than the ground truth. This study proposes an automatic fruit counting system called NDMFCS (Normal Detection Matched Fruit Counting System) to solve this problem for improving fruit counting accuracy in modern apple orchard. NDMFCS consists of three sub-systems, i.e. object detection based on You Only Look Once Version 4-tiny (YOLOv4-tiny), abatement of abnormal fruit detection based on threshold, and fruit counting based on trunk tracking and identity document (ID) assignment. YOLOv4-tiny was selected to implement detection of fruits and trunks, whose output is confidence and pixel coordinates of detected object. The DFBRT and DFG were abated by thresholds to improve detection performance of fruit. This meant that detected fruits were removed when their distance from camera is further than a distance threshold or the confidence of fruit detection is less than a confidence threshold. Finally, fruit counting was implemented by trunk tracking and ID assignment, where each fruit was assigned a unique tracking ID. Results on 10 sets of original videos indicated that average fruit detection precision was improved from 89.1% to 93.3% after abatement of abnormal fruit detection. Also, Multiple Object Tracking Accuracy and Multiple Object Tracking Precision were improved on average by 4.2% and 3.3%, respectively, while average ID Switch Rate was decreased on average by 1.1%. And average fruit counting accuracy was improved to 95.0% by 4.2%. Coefficient of determination (R2) was 0.97, which indicated the number of fruits counted by NDMFCS was near to the ground truth. These results demonstrate that the abatement of abnormal fruit detection can improve performance of apple counting, which has the potential to provide a technical support for estimating fruit yield in modern apple orchards.