Robust Image Cropping by Filtering Composition Irrelevant Factors

Abstract

Numerous factors can impact the aesthetic quality of images: composition, resolution, exposure, color saturation and so on. Image cropping is to improve the aesthetic quality by recomposing the images. When the only consideration of an image cropping system is composition, the automatic image cropping algorithm should be single input single output system (SISOS). However, most of the existing approaches are multiple input multiple output systems (MIMOS) which consider image composition and image aesthetics synchronously. In these MIMOSs, cropping result may change when composition irrelevant factors (e.g., resolution, exposure, color saturation) varies, which is undesirable to users. Based on this observation, we try to discriminate image composition and aesthetics to get a SISOS based on composition by the saliency map. From our observation, although the saliency map is robust to the composition irrelevant factors, it is a less informative data format for composition. Hence, it is transformed to the salient cluster that is similar to point cloud. The salient points in salient cluster can directly describe the spatial structure of an image, so the salient cluster can be treated as an expression of composition and serves as the only input of the proposed model. Our model is designed based on PointNet and made up of content screening module (CSM) and composition regression module (CRM). CSM extracts the points of interest and CRM outputs a cropping box. The experimental results on public datasets shows that, compared with prior arts, our network is more robust to composition irrelevant factor with a comparable or better performance.