Abstract
Image caption generation is attractive research which focuses on generating natural language sentences to describe the visual content of a given image. It is an interdisciplinary subject combining computer vision (CV) and natural language processing (NLP). The existing image captioning methods are mainly focused on generating the final image caption directly, which may lose significant identification information of objects contained in the raw image. Therefore, we propose a new middle-level attribute-based language retouching (MLALR) method to solve this problem. Our proposed MLALR method uses the middle-level attributes predicted from the object regions to retouch the intermediate image description, which is generated by our language generation model. The advantage of our MLALR method is that it can correct descriptive errors in the intermediate image description and make the final image caption more accurate. Moreover, evaluation using benchmark datasets—MSCOCO, Flickr8K, and Flickr30K—validated the impressive performance of our MLALR method with evaluation metrics—BLEU, METEOR, ROUGE-L, CIDEr, and SPICE.
Highlights
Image caption generation is attractive research in the field of artificial intelligence and has emerged in recent years
We mainly focus on predicting the middle-level attribute labels from object regions, which can be used to retouch the intermediate image description generated by our language generation model
Our main assumption is that the global image feature provides an overview of the raw image, which is significant for the machine to understand the visual content in a rough manner
Summary
Image caption generation is attractive research in the field of artificial intelligence and has emerged in recent years. It is an interdisciplinary subject combining computer vision (CV) and natural language processing (NLP). It focuses on generating a readable sentence to depict the visual content of a given image. Researchers have used convolution neural networks (CNNs) [1] to extract the visual information of a given image, and to generate the corresponding language description by a decoder, such as a recurrent neural network (RNN) [2]. Most methods of image caption generation are based on an encoder-decoder framework. RNN-based models are commonly used encoders, such as Gated Recurrent Unit (GRU) [5], Bidirectional RNNs (Bi-RNNs) [6], Long Short-Term Memory cell (LSTM) [7], etc
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
