Detection and counting of banana bunches by integrating deep learning and classic image-processing algorithms

Abstract

Robots must first detect the number of banana bunches when making judgements on sterile bud removal and estimating weight for harvest in the field environment. Banana bunches are complex in shape, arranged in a nonlinear helical curve along the stalk, and have different growth states in different periods, with bunches widely spaced in the early period and densely arranged in the harvest period. Deep learning nor classical image-processing algorithms alone can detect and count bunches in both periods. Therefore, these algorithms were combined to calculate the number of bunches in the two periods. For counting bunches in the debudding period, the convolutional neural network Deeplab V3 + model and classic image-processing algorithm were combined to finely segment bunches and calculate bunch numbers, providing intelligent decision-making for judgment on the timing for debudding. To count bunches during harvest, based on deep learning to identify the overall banana fruit cluster, the edge detection algorithm was employed to extract the centroid points of fruit fingers, and the clustering algorithm was used to determine the optimal number of bunches on the visual detection surface. An estimation model for the total number of bunches, including hidden ones, was created based on their helical curve arrangement. The results indicated a target segmentation MIoU of 0.878 during the debudding period, a mean pixel precision of 0.936, and a final bunch detection accuracy rate of 86%. Bunch detection was highly challenging during the harvest period, with a detection accuracy rate of 76% and a final overall bunch counting accuracy rate of 93.2%. Software was designed to estimate banana fruit weight during the harvest period. This research method provided a theoretical basis and experimental data support for automatic sterile bud removal and weight estimation for bananas.