In-field citrus detection and localisation based on RGB-D image analysis

Abstract

In-field citrus detection and localisation are highly challenging tasks due to varying illumination conditions, partial occlusion of citrus, and the colour variation of citrus at different stages of maturity. A reliable algorithm based on red-green-blue-depth (RGB-D) images was developed to detect and locate citrus in real, outdoor orchard environments for robotic harvesting. A depth filter and a Bayes-classifier-based image segmentation method were first developed to exclude as many backgrounds as possible. A density clustering method was then used to group adjacent points in the filtered RGB-D images into clusters, where each cluster represents a possible citrus. A colour, gradient, and geometry feature-based support vector machine classifier was trained to remove false positives. To test the method, a dataset with 506 RGB-D images was acquired in a citrus orchard on sunny and cloudy days. Results showed that the proposed algorithm was robust with an F1 score of 0.9197; the positioning errors in the x, y and z directions were 7.0 ± 2.5 mm, −4.0 ± 3.0 mm and 13.0 ± 3.0 mm, respectively, and the sizing error was −1.0 ± 4.0 mm. These excellent performance values demonstrate that the proposed method could be used to guide a citrus-harvesting robot.