Image caption extraction to aid visual learning

Abstract

An image caption generator is essential for social media enthusiasts or visually impaired individuals. It can be used as a plugin in popular social media platforms to recommend suitable captions or to assist visually impaired people in comprehending the image content on the web, thereby eliminating ambiguity in image meaning and ensuring accurate knowledge acquisition. This research describes an image caption generator that utilizes a Convolutional Neural Network (CNN) and a Long Short-Term Memory (LSTM) model to generate natural language descriptions of images. The CNN was employed to extract features from the input image, which were then fed into the LSTM to generate the corresponding caption. The model is trained on a large dataset of image-caption pairs, using a combination of supervised and reinforcement learning techniques. The model's performance is evaluated using several metrics, and the results demonstrate that the proposed CNN LSTM model outperforms existing state-of-the-art approaches in generating accurate and diverse image captions. This model has the potential to be used in various applications, including image retrieval, content-based image search, and assisting visually impaired individuals to understanding their surroundings. It also discusses about the structure and functions of the various neural networks involved.