Content-Based Prediction of Movie Style, Aesthetics, and Affect: Data Set and Baseline Experiments

Abstract

The affective content of a movie is often considered to be largely determined by its style and aesthetics. Recently, studies have attempted to estimate affective movie content with computational features, but results have been mixed, one of the main reasons being a lack of data on perceptual stylistic and aesthetic attributes of film, which would provide a ground truth for the features. The distinctions between energetic and tense arousal as well as perceived and felt affect are also often neglected. In this study, we present a data set of ratings by 73 viewers of 83 stylistic, aesthetic, and affective attributes for a selection of movie clips containing complete scenes taken from mainstream movies. The affective attributes include the temporal progression of perceived and felt valence and arousal within the clips. The data set is aimed to be used to train algorithms that predict viewer assessments based on low-level computational features. With this data set, we performed a baseline study modeling the relation between a large selection of low-level computational features (i.e., visual, auditory, and temporal) and perceptual stylistic, aesthetic, and affective attributes of movie clips. Two algorithms were compared in a realistic prediction scenario: linear regression and the neural-network-based Extreme Learning Machine (ELM). Felt and perceived affect as well as stylistic attributes were shown to be equally easy to predict, whereas the prediction of aesthetic attributes failed. The performance of the ELM predictor was overall found to be slightly better than the linear regression. A feature selection experiment illustrated that features from all low-level computational modalities, visual, auditory and temporal, contribute to the prediction of the affect assessments. We have made our assessment data and extracted computational features publicly available.