Smart Diet Management through Food Image and Cooking Recipe Analysis

Abstract

Food monitoring has become an indispensable practice for personal health management in increasingly growing populations. To facilitate this process, advanced image processing and AI technologies have been applied to empower automated recognition of food item and nutrient using food images taken by smart mobile devices. However, existing tools suffered from unsatisfactory precision and compromised convenience, which has hindered broad application of food logging tools. In this study we explore new solutions to image-based food recognition for improved performance, with a particular focus on domestic cooking applications. Particularly, we leverage advanced machine learning and nature language processing techniques, in conjunction with comprehensive food nutrient profiles in the knowledge base and the contextual ingredient information parsed from publicly available recipes, in developing a new food recognition system. Our optimized models were proved to be effective in ingredient recognition based on food images and is under the integration into an Android app named FoodInsight, available at https://github.con zhuxinyishcn/FoodInSight.