ON-AFN: Generating Image Caption based on the Fusion of Residual Attention and Ordered Memory Module

Abstract

In this paper, we propose a novel and effective Fused Network, which is based on the residual attention mechanism and ordered memory module, a framework for image captioning that enables computer to produce the more accurate natural language description. Inspired by the recent advanced studies, our approach creatively divided our caption survey into two subtasks, one is how to efficiently extract visual features in the image, and the other is how to accurately express the semantic information in the features, and then propose excellent solutions for these tasks. In other words, Ordered Memory Attention Fusion Network is a comprehensive captioning model with the support of residual attention network and ordered memory module. Specially in this model, we innovatively employ the residual attention learning mechanism to optimize the extraction of features, and generate a more accurate and comprehensive natural language caption by applying the ordered memory module. We validated our model through extensive experiments on COCO Captioning benchmark, experiments show that it is conducive to generate the caption with higher accuracy and sets the new state-of-the-art by a significant margin.