A Sample Weight and AdaBoost CNN-Based Coarse to Fine Classification of Fruit and Vegetables at a Supermarket Self-Checkout

Abstract

The physical features of fruit and vegetables make the task of vision-based classification of fruit and vegetables challenging. The classification of fruit and vegetables at a supermarket self-checkout poses even more challenges due to variable lighting conditions and human factors arising from customer interactions with the system along with the challenges associated with the colour, texture, shape, and size of a fruit or vegetable. Considering this complex application, we have proposed a progressive coarse to fine classification technique to classify fruit and vegetables at supermarket checkouts. The image and weight of fruit and vegetables have been obtained using a prototype designed to simulate the supermarket environment, including the lighting conditions. The weight information is used to change the coarse classification of 15 classes down to three, which are further used in AdaBoost-based Convolutional Neural Network (CNN) optimisation for fine classification. The training samples for each coarse class are weighted based on AdaBoost optimisation, which are updated on each iteration of a training phase. Multi-class likelihood distribution obtained by the fine classification stage is used to estimate a final classification with a softmax classifier. GoogleNet, MobileNet, and a custom CNN have been used for AdaBoost optimisation, with promising classification results.