Fruit Visibility Analysis for Robotic Citrus Harvesting

Abstract

Automating orange harvesting in Florida could support the citrus industry in the face of a decreasing labor force and global market competition. Fruit recognition is the first critical operation in robotic harvesting, and fruit visibility in the tree canopy poses a challenge to fruit detection. Fruit trees such as oranges have a dense canopy, which can often result in partial or complete occlusion of fruit. Fruit visibility and approaches to increase visibility of oranges by viewing the canopy with different camera perspectives were investigated. Fruit visibility was defined as the ratio of the number of fruits visible to a human observer to the total number of fruits inside a region of interest (ROI), which was a volume of tree canopy enclosed by a 0.125 m3 bounding cube. Multiple images of ROIs were acquired using two methods: orthographic viewing and multiple-perspective viewing. Orthographic viewing involved taking the six orthographic views perpendicular to the ROI faces, while multiple-perspective viewing acquired nine different perspectives at combinations of 45° angles to the ROI's front face. Sets of orthographic and multiple-perspective images were obtained from a commercial orange grove located in Florida. Combining visible fruit from multiple-perspective images yielded a fruit visibility of 0.91 compared to 0.82 from combined orthographic images. In addition, an image processing fruit recognition algorithm detected 0.87 of the visible fruits in the ROI using the multiple-perspective images. These fruit visibility levels show a substantial improvement compared to results from previous literature, which reported 0.40 to 0.70 fruit visibility for citrus trees. Integrating the multiple-perspective viewing approach into a fruit exploration function of a harvesting robot could improve overall harvesting efficiency.