Self-reported willingness to share political news articles in online surveys correlates with actual sharing on Twitter.

Mohsen Mosleh,Gordon Pennycook,David G Rand,Jarosław Jankowski

doi:10.1371/journal.pone.0228882

Mohsen Mosleh, Gordon Pennycook + Show 2 more

Open Access

https://doi.org/10.1371/journal.pone.0228882

Copy DOI

Abstract

There is an increasing imperative for psychologists and other behavioral scientists to understand how people behave on social media. However, it is often very difficult to execute experimental research on actual social media platforms, or to link survey responses to online behavior in order to perform correlational analyses. Thus, there is a natural desire to use self-reported behavioral intentions in standard survey studies to gain insight into online behavior. But are such hypothetical responses hopelessly disconnected from actual sharing decisions? Or are online survey samples via sources such as Amazon Mechanical Turk (MTurk) so different from the average social media user that the survey responses of one group give little insight into the on-platform behavior of the other? Here we investigate these issues by examining 67 pieces of political news content. We evaluate whether there is a meaningful relationship between (i) the level of sharing (tweets and retweets) of a given piece of content on Twitter, and (ii) the extent to which individuals (total N = 993) in online surveys on MTurk reported being willing to share that same piece of content. We found that the same news headlines that were more likely to be hypothetically shared on MTurk were also shared more frequently by Twitter users, r = .44. For example, across the observed range of MTurk sharing fractions, a 20 percentage point increase in the fraction of MTurk participants who reported being willing to share a news headline on social media was associated with 10x as many actual shares on Twitter. We also found that the correlation between sharing and various features of the headline was similar using both MTurk and Twitter data. These findings suggest that self-reported sharing intentions collected in online surveys are likely to provide some meaningful insight into what content would actually be shared on social media.

Highlights

Social media platforms such as Facebook and Twitter are frequented by billions of people worldwide
For a set of political news headlines, we compare the fraction of Mechanical Turk (MTurk) workers who say they would consider sharing the headline on social media with the number of times the headline was shared on Twitter
To assess MTurk sharing intentions, we analyzed sharing data from the control conditions of the two previous experiments of ours in which participants indicated their willingness to share a series of actual news headlines from social media related to American politics [12, 13]

Summary

Introduction

Social media platforms such as Facebook and Twitter are frequented by billions of people worldwide. Studies using social media sharing data are typically observational in nature This makes it hard to draw clear causal inferences about the relationships observed. It largely excludes an entire range of studies aimed at exploring the impact of experimentally manipulating various features of the content or environment–most notably, studies trying to develop interventions to address problems on social media such as misinformation and hate speech ( it is possible to conduct field experiments on social media platforms [5,6,7] they are technically challenging and can raise ethical issues.)

Methods

Results

Conclusion