Recognition of sweet peppers and planning the robotic picking sequence in high-density orchards

Abstract

• A YOLO-V4-CBAM method for the accurate recognition of green pepper in a dense orchard environment was developed. • A DPC algorithm for automatic classification of picking clusters is proposed. • A principle of anti-collision picking within picking clusters is proposed based on the "empirical expert method", Gaussian distance weighting, and "winner-take-all" optical neural filter competition method. To improve the operational efficiency of and to prevent possible collision damage in the near-neighbor multi-target picking of sweet peppers by robots in densely planted complex orchards, this study proposes an algorithm for recognizing sweet peppers and planning a picking sequence called AYDY. First, the convolutional block attention module is embedded into the you only look once model (YOLO-V4), and this combined model is used to recognize and localize sweet peppers. Then, the clustering algorithm for the fast search-and-find of density peaks is improved based on the inflection points and gaps of a decision graph. Sweet peppers with multiple near-neighbor targets are automatically partitioned into picking clusters. An anti-collision picking sequence for a picking cluster is determined based on the experience of experts. The algorithm combines Gaussian distance weights with the winner-takes-all approach as an optic neural filter. In tests, the F 1 -score of this method for sweet peppers in a densely planted environment was 91.84%, which is a 9.14% improvement compared to YOLO-V4. The average localization accuracy and collision-free harvesting success rate were 89.55% and 90.04%, respectively. The recognition and localization time for a single image was 0.3033 s. The time to plan a picking sequence for a single image was 0.283 s. When the robotic arm harvested 22 and 24 sweet peppers, compared to sequential and stochastic planning, the proposed method had higher collision-free picking rates by 18.18, 18.18, 16.67, and 25 percentage points, respectively. This method can accurately detect sweet peppers, reduce collision damage, and improve picking efficiency in high-density orchard environments. This study may provide technical support for anti-collision picking of sweet peppers by robots.