An Exploratory Approach to Deriving Nutrition Information of Restaurant Food from Crowdsourced Food Images: Case of Hartford.

Xiang Chen,Caiwen Ding,Aditya Kulkarni,Evelyn Johnson,Ran Xu,Natalie Ranelli,Yanyan Chen

doi:10.3390/nu13114132

Abstract

Deep learning models can recognize the food item in an image and derive their nutrition information, including calories, macronutrients (carbohydrates, fats, and proteins), and micronutrients (vitamins and minerals). This technology has yet to be implemented for the nutrition assessment of restaurant food. In this paper, we crowdsource 15,908 food images of 470 restaurants in the Greater Hartford region on Tripadvisor and Google Place. These food images are loaded into a proprietary deep learning model (Calorie Mama) for nutrition assessment. We employ manual coding to validate the model accuracy based on the Food and Nutrient Database for Dietary Studies. The derived nutrition information is visualized at both the restaurant level and the census tract level. The deep learning model achieves 75.1% accuracy when compared with manual coding. It has more accurate labels for ethnic foods but cannot identify portion sizes, certain food items (e.g., specialty burgers and salads), and multiple food items in an image. The restaurant nutrition (RN) index is further proposed based on the derived nutrition information. By identifying the nutrition information of restaurant food through crowdsourced food images and a deep learning model, the study provides a pilot approach for large-scale nutrition assessment of the community food environment.

Highlights

Americans’ eating habits have been through a drastic change—they are spending more on eating out rather than cooking at home [1,2]
Out of the 281 coded images, we found that the deep learning model correctly identified 211 food images, reaching an accuracy level of 75.1%
We explore a new deep learning approach for the nutrition assessment of restaurant food

Summary

Introduction

Americans’ eating habits have been through a drastic change—they are spending more on eating out rather than cooking at home [1,2]. $389,677 million, while the total expenditure for FAFH exceeded $418,933 million [3]. The largest portion of the FAFH (i.e., 36.8% based on the 2019 Food Expenditure Series [3]) was consumed at a limited-service restaurant, which is generally known as a fast-food restaurant. Compared to FAH, FAFH is relatively calorie-dense and nutrient-poor, as it contains more saturated fat, sodium, and cholesterol but less dietary fiber [4]. The change in dietary behaviors has posed risks for FAFH consumers to develop obesity and obesity-related chronic diseases (e.g., Type II diabetes and cardiovascular diseases). Recent literature identified a strong association between FAFH consumption and calorific intake among children, which strengthened the evidence of the health adversities as a result of FAFH consumption [5,6]

Methods

Results

Discussion

Conclusion