DeepTrayMeal: Automatic dietary assessment for Chinese tray meals based on deep learning

Abstract

Tray meal is a popular way of eating in China, and tray-based automatic dietary assessment is important for public health. Relevant research is lacking because public tray meal datasets and suitable methods are unavailable. In this study, we established and published the first Chinese tray meal dataset, the ChinaLunchTray-99. We collected real-world 1185 tray meal images, covering 99 dish categories with corresponding manually annotated bounding box and category-level labels. We developed a new framework for automatic dietary assessment, which consists of dish image recognition, volume estimation and nutrition mapping. First, we demonstrated a tray meal detection model considering feature extraction, anchor scales, and loss function, resulting in a high mean Average Precision of 92.13%. Second, we proposed an automatic method to estimate volume via detection results and tray’s information. Finally, nutrients were mapped from the estimated volume. Our research can promote applications of automatic dietary assessment for Chinese tray meals.