Fusing image feature and title semantic regularization with adversarial networks for cross-modal retrieval

Abstract

With the improvement of living standard, people pay more and more attention to their health. Food is the foundation of human life, modern society places more emphasis on a balanced diet, which controls fat, protein, carbohydrate and vitamin intake. So food computing is more and more important. Food retrieval is one of the important research directions in food computing. On the basis of food retrieval, we can predict ingredients and instructions in each dish, according to the ingredients and instructions to speculate on the visual effect of cooking, which can guide human reasonable diet, analyse human diet structure and diet culture and so on. In this paper, we focus on cross-modal retrieval between food image and recipe. Firstly, we analyze the problems existing in the present method. Based on the problems existing in the existing method, a fusion image feature and title regularization with adversarial network is proposed, which uses the idea of generative adversarial to align the modes, fuses the local features and global features of the image, and adds the semantic regularity of title to improve the accuracy of the retrieval.