Towards a semantics representation framework for narrative images

Abstract

PurposeThe purpose of this paper is to explore a semantics representation framework for narrative images, conforming to the image-interpretation process.Design/methodology/approachThis paper explores the essential features of semantics evolution in the process of narrative images interpretation. It proposes a novel semantics representation framework, ESImage (evolution semantics of image) for narrative images. ESImage adopts a hierarchical architecture to progressively organize the semantic information in images, enabling the evolutionary interpretation under the support of a graph-based semantics data model. Also, the study shows the feasibility of this framework by addressing the issues of typical semantics representation with the scenario of the Dunhuang fresco.FindingsThe process of image interpretation mainly concerns three issues: bottom-up description, the multi-faceted semantics representation and the top-down semantics complementation. ESImage can provide a comprehensive solution for narrative image semantics representation by addressing the major issues based on the semantics evolution mechanisms of the graph-based semantics data model.Research limitations/implicationsESImage needs to be combined with machine learning to meet the requirements of automatic annotation and semantics interpretation of large-scale image resources.Originality/valueThis paper sorts out the characteristics of the gradual interpretation of narrative images and has discussed the major issues in its semantics representation. Also, it proposes the semantic framework ESImage which deploys a flexible and sound mechanism to represent the semantic information of narrative images.