Learning with privileged information for photo aesthetic assessment

Abstract

Privileged information (PI), known as teacher providing students helpful comments, comparisons, and explanations to improve students’ performance, has been widely applied in various machine learning tasks, resulting in great success. Existing approaches utilizing attributes either fail to leveraging the attributes information thoroughly, or suffer from the complex network structures for automatically attributes learning. Therefore, we propose a new Deep Convolutional Neural Network with Privileged Information (PI-DCNN) for photo aesthetic assessment by utilizing the prior knowledge of photo and photographic elements as privileged information. This paper is the first to systematically summarize all the attributes (i.e., photo and photographic attributes) related to aesthetics assessment. Specifically, we first explore the privileged information of photo and photography attributes, which is available at the training stage but it is not available for the test set. After that, we transfer the probabilistic dependency relations as constraints, and formulate photo aesthetics assessment in a deep convolutional neural network. Lastly, we propose a new pair-wise ranking loss that can exploit the relationship of photo aesthetic quality within a pair of photos. Experimental results on two widely benchmark databases of aesthetic assessment, AADB and AVA, demonstrate the effectiveness of the proposed PI-DCNN method on photo aesthetic assessment task.