An intelligent framework for estimating grade and quantity of tropical fruits in a multi-modal latent representation network

Abstract

This research addresses the crucial aspects of ensuring the quality and origin of fruits in the supply chain, as well as estimating their quantity measures. A novel interwoven deep neural framework is proposed, incorporating an Object Detection Network (ODN) and a Multi-modal Regressive Convolution Neural Network (MRCNN). Two distinct datasets, Annotated FruitNet and FruitBox, were compiled to support the framework’s tasks. The FruitNet360 dataset was re-clustered based on geographical origin, enabling robust fruit quality detection and localization with a mean Average Precision (mAP) score of 95.70% using YOLOv7. Leveraging the representation learning capability of Residual Networks, the framework accurately predicted quantitative weight measures of fruit boxes, achieving a marginal Mean Squared Error (MSE) rate of 0.034. Furthermore, the origin of fruits was identified with an impressive accuracy of 98.67%. The proposed framework, with its combined regression and classification model, captured the latent representations of source data effectively, surpassing the limitations of conventional approaches and reducing manual system overhead. The automation framework holds potential for integration into smart devices, offering valuable assistance to both vendors and consumers in fruit analysis and selection.