Detection and segmentation of overlapped fruits based on optimized mask R-CNN application in apple harvesting robot

Abstract

In order to better apply the good performance of feature extraction and target detection used in deep learning to fruit detection in orchards, a model of harvesting robot vision detector based on Mask Region Convolutional Neural Network (Mask R-CNN) is proposed. The model was improved to make it more suitable for the recognition and segmentation of overlapped apples. Residual Network (ResNet) combined with Densely Connected Convolutional Networks (DenseNet) can greatly reduce input parameters and is used as a backbone network for feature extraction. Feature maps are input to the Region Proposal Network (RPN) for end-to-end training to generate the region of interest (RoI), and finally the mask is generated by the full convolution network (FCN) to get the region where the apple is located. The method is tested by a random test set with 120 images, and the Precision Rate has reached 97.31%, and the Recall Rate has reached 95.70%. And the recognition speed is faster, which can meet the requirements of the apple harvesting robot’s vision system.