Detecting occluded Y-shaped fruit tree segments using automated iterative training with minimal labeling effort

Abstract

• Introduction and application of Automating-the-loop algorithm for semi-supervised semantic segmentation of y-shaped tree segments. • Introduction of a new metric, Complete Grid Scan. Training of convolutional neural networks for semantic segmentation of fruit tree branches requires accurate pixel-wise labeling which requires large amounts of human effort. The human-in-the-loop method, where a human annotator corrects the outputs of a neural network, reduces labeling effort; however, it requires human intervention for each image. This paper describes an iterative training methodology for semantic segmentation, Automating-the-Loop. This aims to replicate the manual adjustments of the human-in-the-loop method with an automated process, hence, drastically reducing labeling effort. Using the application of detecting partially occluded apple tree segmentation, we compare manually labeled annotations, self-training, human-in-the-loop, and Automating-the-Loop methods in both the quality of the trained convolutional neural networks, and the effort needed to create them. The convolutional neural network (U-Net) performance is analyzed using traditional metrics and a new metric, Complete Grid Scan. It is shown that in our application, the new Automating-the-Loop method greatly reduces the labeling effort while producing comparable performance to both human-in-the-loop and complete manual labeling methods.