A deep neural network approach towards real-time on-branch fruit recognition for precision horticulture

Abstract

Real-time and accurate on-branch fruit recognition in an uncontrolled/unstructured environment of orchards could facilitate Precision Horticulture (PH) practices. These practices which are based on the site-specific or variety-specific treatment of an orchard include applications like remote recognition of tree species, variety-specific orchard agrochemical applications, orchard yield mapping, robotic fruit picking, fruit tree disease treatment, etc. For this purpose, in the current work a Convolutional Neural Network (CNN) was developed and optimized for fruit recognition based on RGB images. The images from six classes of on-branch fruits i.e. green apples, nectarine, apricot, peach, sour cherry, and amber-colored plums were captured from local orchards at Semnan province, Iran. To avoid over-fitting, the dataset was then augmented to create more training data from existing samples. The model consisted of multiple convolutional, max-pooling, Global Average Pooling (GAP), and fully connected layers. Using GAP instead of the Flatten layer improved the performance of the model by increasing the accuracy as well as significantly reducing the trainable parameters (about 65 times reduction). The optimization phase of the model development was performed by testing four optimizers (RMSprop, SGD, Adam, and Nadam) with three batch sizes (16, 32, and 64) each with 50 epochs. Accordingly, Nadam optimizer with batch size = 32 demonstrated the best results. The best configuration achieved the accuracy of 99.8% and the cross-entropy loss of 0.019 for the test dataset. This result shows that the model is well developed and has good generalization. This reflects the potential of the method for the remote recognition and classification of different varieties of fruits in an orchard regardless of the environmental effects like complex background, variable light, overlaps, and occlusions with other plant parts, etc. The proposed network was also compared with popular structures like VGG11, ResNet50, ResNet152, and YOLOv3. The processing time of this model was about 8 ms per image while it was 351 ms for ResNet152, proving that the proposed network is much better for real-time applications. Consequently, this study presents a robust method for fulfilling the requirements of a PH practice in a high-tech horticulture system.