Food Ingredients Identification from Dish Images by Deep Learning

Abstract

Diet plays an important role in people’s daily life with its strong correlation to health and chronic diseases. Meanwhile, deep based food computing emerges to provide lots of works which including food recognition, food retrieval, and food recommendation, and so on. This work focuses on the food recognition, specially, the ingredients identification from food images. The paper proposes two types of ways for ingredient identification. Type1 method involves the combination of salient ingredients classifier with salient ingredient identifiers. Type 2 method introduces the segment-based classifier. Furthermore, this work chooses 35 kinds of ingredients in the daily life as identification categories, and constructs three kinds of novel datasets for establishing the ingredient identification models. All of the classifiers and identifiers are trained on Resnet50 by transfer learning. Many experiments are conducted to analyze the effectiveness of proposed methods. As the results, Salient ingredients classifier predict one ingredient and achieves 91.97% on test set of salient ingredients dataset and 82.48% on test dish image dataset. Salient ingredients identifiers predict remained ingredients and achieve mean accuracy of 85.96% on test dish image dataset. Furthermore, Segment-based classifier achieves 94.81% on test set of segment-based ingredients dataset.