Personality Trait Analysis in Social Networks Based on Weakly Supervised Learning of Shared Images

Pau Rodríguez,F Xavier Roca,Diego Velazquez,Jordi Gonzàlez,Seiichi Ozawa,Josep M Gonfaus,Guillem Cucurull

doi:10.3390/app10228170

Pau Rodríguez, F Xavier Roca + Show 5 more

Open Access

https://doi.org/10.3390/app10228170

Copy DOI

Abstract

Social networks have attracted the attention of psychologists, as the behavior of users can be used to assess personality traits, and to detect sentiments and critical mental situations such as depression or suicidal tendencies. Recently, the increasing amount of image uploads to social networks has shifted the focus from text to image-based personality assessment. However, obtaining the ground-truth requires giving personality questionnaires to the users, making the process very costly and slow, and hindering research on large populations. In this paper, we demonstrate that it is possible to predict which images are most associated with each personality trait of the OCEAN personality model, without requiring ground-truth personality labels. Namely, we present a weakly supervised framework which shows that the personality scores obtained using specific images textually associated with particular personality traits are highly correlated with scores obtained using standard text-based personality questionnaires. We trained an OCEAN trait model based on Convolutional Neural Networks (CNNs), learned from 120K pictures posted with specific textual hashtags, to infer whether the personality scores from the images uploaded by users are consistent with those scores obtained from text. In order to validate our claims, we performed a personality test on a heterogeneous group of 280 human subjects, showing that our model successfully predicts which kind of image will match a person with a given level of a trait. Looking at the results, we obtained evidence that personality is not only correlated with text, but with image content too. Interestingly, different visual patterns emerged from those images most liked by persons with a particular personality trait: for instance, pictures most associated with high conscientiousness usually contained healthy food, while low conscientiousness pictures contained injuries, guns, and alcohol. These findings could pave the way to complement text-based personality questionnaires with image-based questions.

Highlights

Social media, as a major platform for communication and information exchange, constitutes a rich repository of the opinions and sentiments of 3.6 billion users regarding a vast spectrum of topics.In addition, image sharing on social networks has increased exponentially in the past years; officiallyAppl
Current machine learning tools for trends discovery and post analysis are still based mainly on texts, likes behaviour and browsing logs [1,2,3,4], though the automatic analysis of such a vast amount of images and videos is becoming a strong feature thanks to the recent advances in deep learning, with the use of Convolutional Neural Networks (CNNs) for image analysis [5]. Such machine learning frameworks have been applied to psychological assessment [6], first by finding correlations between text and personality [7,8], and later by demonstrating that some image features are related to the personality of users in social networks [9]
We proposed a weakly supervised learning framework for obtaining image-based personality data without the use of personality questionnaires, providing new tools to deal with the ground-truth annotation bottleneck

Summary

Introduction

As a major platform for communication and information exchange, constitutes a rich repository of the opinions and sentiments of 3.6 billion users regarding a vast spectrum of topics.In addition, image sharing on social networks has increased exponentially in the past years; officiallyAppl. Current machine learning tools for trends discovery and post analysis are still based mainly on texts, likes behaviour and browsing logs [1,2,3,4], though the automatic analysis of such a vast amount of images and videos is becoming a strong feature thanks to the recent advances in deep learning, with the use of Convolutional Neural Networks (CNNs) for image analysis [5] Such machine learning frameworks have been applied to psychological assessment [6], first by finding correlations between text and personality [7,8], and later by demonstrating that some image features are related to the personality of users in social networks [9]. A recent overview of the most popular approaches to automated personality detection, various computational datasets, its industrial applications, and state-of-the-art machine learning models for personality detection with specific focus on multimodal approaches vcan be found in [10]

Objectives

Methods

Results

Conclusion