Instance-Aware Semantic Segmentation for Food Calorie Estimation using Mask R-CNN

Abstract

Knowing the number of calorie content in the food we consume can help in maintaining body health. By fulfilling the basic calorie need well, it will produce many positive effects to the body, including controlling the ideal body weight and becoming an adequate source of energy for physical activity. Conversely, people who do not care about their calorie needs will face various health problems, including obesity and worsening degenerative diseases such as diabetes or high blood pressure. Calculating the actual number of calories digitally from food requires the parameters of area, volume, and mass of the food. Some previous studies in the field of computer vision have been carried out to get a constant number of calories based on food types and not based on actual food volume measurements. In this research, a system will be developed using a computer vision approach that can be used to calculate the number of food calories automatically based on the size of the food volume using the Deep Learning Mask Region-based Convolutional Neural Network (R-CNN) algorithm. The segmentation technique uses the instance-aware semantic segmentation approach, which is to identify each pixel from instance of objects for each object found in a food image. This work uses the concept of instance-aware data labeling or segmentation detection that distinguishes each instances in a similar class, where this model will be used to recognize each different food object instantaneously in one class so that the number of calories of each food object can be obtained precisely. The expected benefit of the results of this research is to help someone get information about the size of food calories according to the calorie needs of the body with the mean Average Precision (mAP) level obtained at 89.4% and the percentage accuracy in calories calculated at 97.48%.