A Vision-Based Method Utilizing Deep Convolutional Neural Networks for Fruit Variety Classification in Uncertainty Conditions of Retail Sales

Katarzyna Katarzyna,Paweł Paweł

doi:10.3390/app9193971

Katarzyna Katarzyna, Paweł Paweł

Open Access

https://doi.org/10.3390/app9193971

Copy DOI

Journal: Applied sciences	Publication Date: Sep 22, 2019
Citations: 31	License type: CC BY 4.0

Affiliation: Opole University of Technology

Abstract

This study proposes a double-track method for the classification of fruit varieties for application in retail sales. The method uses two nine-layer Convolutional Neural Networks (CNNs) with the same architecture, but different weight matrices. The first network classifies fruits according to images of fruits with a background, and the second network classifies based on images with the ROI (Region Of Interest, a single fruit). The results are aggregated with the proposed values of weights (importance). Consequently, the method returns the predicted class membership with the Certainty Factor (CF). The use of the certainty factor associated with prediction results from the original images and cropped ROIs is the main contribution of this paper. It has been shown that CFs indicate the correctness of the classification result and represent a more reliable measure compared to the probabilities on the CNN outputs. The method is tested with a dataset containing images of six apple varieties. The overall image classification accuracy for this testing dataset is excellent (99.78%). In conclusion, the proposed method is highly successful at recognizing unambiguous, ambiguous, and uncertain classifications, and it can be used in a vision-based sales systems in uncertain conditions and unplanned situations.

Highlights

Recognizing different kinds of fruits and vegetables is perhaps the most difficult task in supermarkets and fruit shops [1]
Our goal was to determine the image types necessary to estimate appropriate values of weights in the Convolutional Neural Networks (CNNs) model to classify the varieties of the fruit correctly
The comparisons can relate to each method separately or the whole proposed method, which calculated the Certainty Factor (CF) of object classes

Summary

Introduction

Recognizing different kinds of fruits and vegetables is perhaps the most difficult task in supermarkets and fruit shops [1]. Retail sales systems based on bar code identification require the seller (cashier) to enter the unique code of the given fruit or vegetable because they are individually sold by weight. This procedure often leads to mistakes because the seller must correctly recognize every type of vegetable and fruit; a significant challenge even for highly-trained employees. A partial solution to this problem is the introduction of an inventory with photos and codes This requires the cashier to browse the catalog during check-out, extending the time of the transaction. In the case of self-service sales, the species (types) and varieties of fruits must be specified by the buyer. The likelihood of an incorrect assessment increases when different fresh products are mixed up

Objectives

Methods

Results

Conclusion