Food Image Classification and Data Extraction Using Convolutional Neural Network and Web Crawlers

Abstract

Food image recognition is one among the various propitious applications in the area of computer vision. An application with the ability to identify all kinds of food images along with its nutritious value will help people in maintaining a balanced diet. The proposed convolutional neural network (CNN) model can be used for the identification and classification of food images. A pre-trained Inception v3 CNN model is employed via transfer learning to galvanize the original custom-made CNN framework. With the aid of this pre-trained model, the learning process is boosted and is hence more efficient. Data augmentation is performed on the training set as it improves the robustness of the model and as it also helps avoid overfitting. The predicted food label generated by the model is forwarded to the web crawlers for information retrieval. The web crawlers are deployed on the browser through automation for the retrieval of relevant information such as the food item's origin, nutritional details, its recipe, and even the nearby restaurants that serve the dish. The crawlers are built using Python Scrapy and the process of scraping data from the websites is automated through Selenium. The model achieved an accuracy of 97.00% for 20 classes. This model can be further enhanced in terms of scalability by building a deeper more advanced neural network, collecting more images per class for each of the respective food items and by fine-tuning the model hyperparameters.