A composition-oriented aesthetic view recommendation network supervised by the simplified golden ratio theory

Abstract

Composition is the most vital element in photography. Previous research on automatic image cropping or view recommendation merely studied the aesthetic value of the cropping image without explicitly regarded the rules of composition. This work proposes a composition-oriented aesthetic view recommendation network supervised by the simplified golden ratio theory (CAVR-Net), compared to other methods that output only aesthetic scores, which outputs a sequence of local views with aesthetic rank and composition category for an image and helps to enhance the interpretation of aesthetic perception. We adopt a model distillation technique (teacher–student framework) to train the student model CAVR-S supervised by two teacher models including a composition prediction network (CPN) and an aesthetic evaluation network (AEN). In addition, for the challenging candidate crops selection problem in the image cropping task, we propose a candidate crops extraction scheme based on the simplified golden ratio theory, which reduces the training cost and improves the model performance. The CAVR-Net achieves state-of-the-art performance on two benchmark datasets in the image cropping task and a real-time recommendation efficiency of 80+ fps in the view recommendation task. Our method has essential application value in intelligent photography guidance system or intelligent image analysis system based on image composition, such as photography guide, automatic post-processing, automatic extraction of video frames.