Complete and accurate holly fruits counting using YOLOX object detection

Abstract

Fruits counting is important in management of orchard and plantation since better decision for labor and logistic can be made based on complete and accurate counting of fruits. Computer vision-based fruits counting has been research focus as it’s an automatic way for recognition of dense fruit on the branch. However, complete fruits counting of a whole tree hasn’t hardly been studied. And there is a lack of robust and accurate fruits counting method in complex orchard scenarios, like covering, shadow, clustering in image. In this paper, a panoramic method for fruit complete yield counting based on deep learning object detection is proposed, and was validated on a holly tree with dense fruits. Firstly, images were taken surrounding the fruit trees using UAV, and SIFT based images matching were performed to form a complete panoramic unfolding map of the fruit tree surface. Then, a YOLOX object detection network was built and trained with novel samples augmentation and composition strategies. Finally, fruits counting YOLOX was performed on the panorama to count the whole plant fruits number. The accuracy and effectiveness of this method were tested at different scales and scenarios. The results show that: (1) high-quality panoramic images can be built for an accurate fruit number counting. (2) The Statistical Rate (SR) between detected number and actual number is as high as SR > 96% when the ring shot parameter of Holly tree is R ≤ 1.2 m, SR > 95% when R ≤ 1.6 m. The Detection Rate between detected number and captured number in the panorama image is over 99% when R ≤ 1.2 m and over 97% when R ≤ 2.0 m. The result is superior to previous researches. (3) it has good robustness against shading, covering, incomplete contour. Comparisons between the proposed method and other methods has been done and the result show the proposed method is the most effective in fruits counting. Moreover, we proposed and verified the positive effects of Gaussian convolution kernel and γ-component control on fruit detection rate. The YOLOX-based fruit counting method can be extended to a wide range of fruits, like apples, lychee and so. Moreover, YOLOX has excellent inferencing efficiency which makes it a good potential for real-time application in orchard and plantation management.