Deep image captioning: A review of methods, trends and future challenges

Abstract

Image captioning, also called report generation in medical field, aims to describe visual content of images in human language, which requires to model semantic relationship between visual and textual elements and generate corresponding descriptions that conform to human language cognition. Image captioning is significant for promoting human–computer interaction in all fields and particularly, for computer-aided diagnosis in medical field. Currently, with the rapid development of deep learning technologies, image caption has attracted increasing attention of many researchers in artificial intelligence-related fields. To this end, this study attempts to provide readers with systematic and comprehensive research about different deep image captioning methods in natural and medical fields. We first introduce workflow of image captioning from perspective of simulating human process of describing images, including seeing, focusing and telling, which is respectively behavioralized into feature representation, visual encoding and language generation. Within it, we present common-used feature representation, visual encoding and language generation models. Then, we review datasets, evaluations and basic losses used in image captioning, and summarize typical caption methods which are generally divided into that with or without using reinforcement learning. Besides, we describe advantages and disadvantages of existing methods, and conclusion and challenges are finally presented.