The Deep Features and Attention Mechanism-Based Method to Dish Healthcare Under Social IoT Systems: An Empirical Study With a Hand-Deep Local–Global Net

Abstract

Many mobile apps of social Internet of Things (sIOT) systems can help us record and share daily events, such as health and sport events. In fact, healthy diet recognition is an important and challenging problem in dish health assessment. Via the collection and monitoring of data pertaining to our daily diet, we can work in collaborative ways to achieve dish image annotation based on sIOT systems to enhance deep features. To this end, this article proposes a deep feature and attention mechanism-based method for dish health assessment, which aims to apply a hand-deep local–global net (HDLGN) for dish image recognition. Then, food taste is used as health guidance for people who want to lose weight or follow doctors’ advice. First, the local attention mechanism is introduced to identify key areas of the dish image. Second, ingredient and handcrafted color features are extracted to learn deep features. Subsequently, we combine local and global attention mechanisms to return the dish taste as the recognition result. Finally, experiments show that our proposed method can effectively improve the accuracy of taste recognition.