Recognizing the sweet and sour taste of pineapple fruits using residual networks and green-relative color transformation attached with Mask R-CNN

Abstract

The aromas and tastes produced by different chemical compounds in fruits are important drivers of consumption, but they are difficult to discern from visual appearance alone. This paper presents a novel three-stage deep-learning-based model framework that first localizes a pineapple and then identifies its taste. The first stage extracts the pixel-wise region of interest using a segmentation model. The segmented object is preprocessed and transformed into green-relative color space (YCbCr) that highlights various pineapple elements, such as pineapple buds and their surrounding areas regardless of lighting conditions. An experienced specialist characterizes pineapple taste by a similar process. Finally, the segmented and processed image is fed into residual networks for taste classification. After generalization via image augmentation with contrast adjustment, the model framework was shown to be highly robust with an F1-score, precision, and recall of 0.9025, 0.9026, and 0.9025, respectively. The proposed framework correlates the visual appearance of a fruit with its corresponding taste. Therefore, it can be a valuable tool in fruit export or local consumption applications.