Enhancing image sentiment analysis: A user-centered approach through user emotions and visual features

Abstract

To bridge the gap between user understanding of emotions and image representation, this paper amalgamated a variety of user features to formulate an image sentiment analysis model. Through controlled user experiments, participants were engaged in discerning sentiment from abstract painting images. User visual features were derived from eye-tracking data collected during the experiment. This paper employed the FFM (Five Factor Model) and POMS (Profile of Mood States) methodologies to gather personality and mood data, which were utilized as user emotional features. Additionally, fundamental components of abstract painting were captured as basic image features. By synergizing features across these three dimensions, the study introduced the "User-Feature-Enhanced-Image-Sentiment-Model" (UFE-ISM) for image sentiment recognition. The research findings underscore the model's superior performance in sentiment classification on abstract painting. Notably, both basic image features and user visual features exhibited significant contributions to the image sentiment classification model.