Apple fruit detection and counting based on deep learning and trunk tracking

Abstract

<b><sc>Abstract.</sc></b> Accurate estimation of apple yield is important for producers to make informed decisions when managing their orchards and be competitive when selling their products. Yield estimation is a key for predicting stock volumes, allocating needed labor, and planning harvesting operations. Manual visual yield estimation has traditionally been employed, resulting in inaccurate and misleading information. Although some studies have developed related algorithms based on machine vision to count fruits in tree images, they are difficult to apply in large orchards. The main goal of this study is to develop an automated video processing methodology to detect and count the apple fruits in modern vertical fruiting-wall architecture using deep learning techniques and object tracking. Existing researches using tracking technology to count fruits based on videos have all put the tracking target on the tracking fruit target itself. However, the small size and large number of video fruit targets have brought great obstacles to the realization of tracking technology. Therefore, according to the fact that all relatively stationary targets in the video have the same displacement trajectory, this study select tree trunk as the tracking target to obtain the displacement trajectories of all detected fruits. A detection model of apples and tree trunks was trained used YOLOv4-tiny network, which integrated with the tracking algorithm to develop a counting algorithm for video fruits. Combining the detected position information of the apple and the trunk with the displacement track obtained by tracking can establish the connection between the fruits in the video frame, thereby realizing the counting of fruits. The developed counting method was used to detect and count the images and videos in testing dataset, respectively. An AP of 99.35% for apple and trunk detection was obtained in this study, and over 90% apples were counted correctly using proposed method compared with manual counting results. These promising results demonstrate the potential of the present method to provide yield estimates for apple fruits or even other types of fruit.