Detection of Persimmon Posture by a Convolutional Neural Network for Fully Automating the Peeling Process

Abstract

Highlights Robots have been developed for fully automated peeling, which is the main operation in the production of dried persimmons. Vision sensors were used to pick and place persimmons that were originally placed in random orientations. A convolutional neural network was used for fast and robust detection. The roots of peduncles and positions of fruits were detected with high accuracy. Abstract. The manufacturing process of dried persimmons includes peeling, drying, and finishing. Although a considerable part of this process is mechanized, manual labor is still required. The purpose of this study is to develop a robot machine vision system for placing persimmons onto a suction cup in a peeling machine to fully automate the peeling process. This article describes a method for detecting the position of the center of an entire persimmon fruit and the root of the peduncle in a two-dimensional image as a preliminary step to estimating the three-dimensional position and orientation of the persimmon, which is required for pick-and-place operations by a robot manipulator. A convolutional neural network (CNN) was used to ensure robust detection regardless of variations in the persimmon shape and surface color. The position of the fruit and the position of the peduncle root were detected with accuracies of 1.0 mm and 2.0 mm, respectively, when the calyx of the persimmon was facing sideways at the time of picking, and the directional accuracy was 3.2°. When placing the persimmon with the calyx facing up, the accuracies of the fruit position and the position of the peduncle root were 0.8 mm and 0.8 mm, respectively. The peduncle root position was detectable even if the peduncle was hidden by the calyx. The main contribution of this article is to confirm that a CNN can detect the center of a persimmon and the root of the peduncle with sufficient speed and accuracy for practical use, which is difficult to achieve with conventional image processing methods due to the variable shape and color of persimmons. Keywords: Automatic peeling machine, Calyx, Dried persimmon, Peduncle, Serial robot, YOLOv3.