Anchor-based knowledge embedding for image aesthetics assessment

Abstract

Deep neural networks have shown their advantage in image aesthetics assessment (IAA). However, the current deep IAA models largely work in a data-driven manner, but the ambiguity of aesthetics poses huge challenge. When judging image aesthetics, people usually take advantage of commonsense knowledge. Further, people are good at making relative comparison instead of absolute scoring. Motivated by the above facts, this paper presents a new ANchor-based Knowledge Embedding (ANKE) approach for generic image aesthetics assessment, which makes predictions based on a universal aesthetic knowledge base. First, the knowledge base is built by extracting aesthetic features from anchor images with diversified visual contents and aesthetic levels, which can provide rich reference information for aesthetics assessment. Then, given an image, the model is trained to dynamically pick up the most informative anchors from the knowledge base and adaptively weight the difference features to produce the final aesthetic prediction. Experimental results demonstrate that, with a universally built aesthetic knowledge base, the proposed ANKE model achieves the state-of-the-art performance on three public IAA databases.