Exploring visual relationship for social media popularity prediction

Abstract

Social media popularity prediction is an important channel to explore content sharing and communication on social networks. It aims to capture informative cues by analyzing multi-type data (such as images, user profiles, and text) to decide the popularity of a specified post. Intuitively, given an image, humans can volitionally focus on salient objects and relationships that are associated with their interests. For example, when we see the image including the relationship “elephant-attack-van”, it is more natural to increase our interest than the image with “elephant-near-van”. Therefore, exploiting such structural relationships is expected to help the prediction model search for evidence in support of the popularity of posts. However, most current works only focus on the global representation or the isolated objects, while ignoring the structure knowledge contained in images. To address this problem, we propose the relationship-aware social media popularity predictor. First, we extract inter-object relationships via a pre-trained scene graph generator. Then, we design a content-based filtering module to filter redundant relationships and capture the key 〈subject–predicate–object〉 information. Finally, we integrate relationship information with multi-type heterogeneous data and feed them into the CatBoost model for regression. Moreover, our predictor is capable of generating more intuitive interpretations by analyzing visual relationships in images to reasonably infer popularity scores. Extensive experiments conducted on the Social Media Prediction Dataset demonstrate that the proposed method can outperform other state-of-the-art models. Additional ablation studies and visualizations further validate the effectiveness and interpretability.