Research on Spatial Positioning System of Fruits to be Picked in Field Based on Binocular Vision and SSD Model

Abstract

The accurate fruit recognition in the field was one of the key technologies of fruit picking agricultural robots. An improved Single Shot Multi-Box Detector (SSD) model based on the color and morphological characteristics of fruit was proposed in this paper when aimed at the large collection workload and low secondary transfer efficiency of fruit such as palm fruit, durian, pineapple and other fruits grown in a complex field environment. A binocular depth camera RealSense D435i was used to collect images of the fruit to be picked in the field. Meanwhile, the MobileNet was replaced with the VGG16 basic network based on the Tensor-flow deep learning framework to reduce the amount of convolution operations for extracting image features in the SSD model, and a spatial positioning system for pineapple fruit was designed. Furtherly, experiments showed that the improved SSD depth detection model had a smaller size and it was more convenient to be deployed on the mobile end of agricultural robots, which the model had a high accuracy in the effective recognition of the fruits to be picked under the weed occlusion and overlapping scenes. The frame rate of the video reading and detection for the binocular depth camera reached 16.74 Frames Per Second (FPS), which had good robustness and real-time, and a good solution for the automatic picking of agricultural picking robots could be provided in the field.