Faster R-CNN With Classifier Fusion for Automatic Detection of Small Fruits

Abstract

Fruit detection is a fundamental task for automatic yield estimation. The goal is to detect all the fruits in images. The-state of the art of fruit detection algorithm, Faster R-CNN, shows a lack of detection advantage on small fruits. One of the reasons is only that single-level features and a classifier are used for localization of proposal candidates. In this article, we propose to incorporate a multiple classifier fusion strategy into a Faster R-CNN network for small fruit detection. We utilize features from three different levels to learn three classifiers for objectness classification in the stage of proposal localization. Probabilities from classifiers are combined by a simple convolutional layer to generate final objectness classification for proposal candidates. During training, in order to train a model with strong generalization capability, we propose to use correlation coefficients to measure the diversity of multiple classifiers. A novel loss function with classifier correlation is introduced to train the region proposal network. We evaluate the proposed model on two data sets of small fruits. Extensive experiments show that the proposed model outperforms the state-of-the-art detectors for fruit detection. Note to Practitioners —This article was motivated by the problem of the detection of small fruits for yield estimation but it also applies to the detection of other small objects. Existing approaches to fruit detection experience difficulty in detecting small fruits and the overall detection accuracy suffers as a result. This article suggests a novel approach to detect fruits of all sizes by using a multiple classifier fusion strategy in the stage of fruit localization. In the learning of the classifiers, multilevel features of fruits are extracted. In addition, we achieve classifier diversity through the analysis of the correlation of classification probabilities. The diversity of classifiers also helps improve the performance of fruit localization. Using experiments on real fruit tree images, we show that the suggested approach is feasible on orange detection of various sites in different seasons. As a future work, we will integrate the suggested algorithm into a fruit yield estimation system and test it online.