Abstract
News image captioning is a popular and advanced task requiring the generation of image captions that reflect the relevant information contained in the news articles. The existing approaches usually align objects in images with corresponding news content and focus on entity-based approaches. However, their limitations are threefold: (a) semantic discourses of news contexts are largely unexplored, (b) the caption style has not yet been used, and (c) key entities are not ensured to be generated during caption generation. In this paper, we propose a novel framework for generating news image captions which are semantically informative, well-styled and entity-aware controllable. Specifically, the semantics of news articles are preserved by separating them into elementary discourse units (EDUs), which are fine-grained basic semantic units. Moreover, we introduce a Contrastive Style Reward to ensure that the style of captions and corresponding news content are coherent. Furthermore, we introduce a controllable mechanism to ensure that key entities are generated during the generation process. Extensive experiments on two large-scale news datasets demonstrate the effectiveness of the proposed framework.
Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
