Spot the bot: Investigating user's detection cues for social bots and their willingness to verify Twitter profiles

Abstract

Detecting social bots is important for users to assess the credibility and trustworthiness of information on social media. In this work, we therefore investigate how users become suspicious of social bots and users' willingness to verify Twitter profiles. Focusing on political social bots, we first explored which cues users apply to detect social bots in a qualitative online study (N = 30). Content analysis revealed three cue categories: content and form, behavior, profile characteristics. In a subsequent online experiment (N = 221), we examined which cues evoke users’ willingness to verify profiles. Extending prior literature on partisan-motivated reasoning, we further investigated the effects of type of profile (bot, ambiguous, human) and opinion-congruency, i.e., whether a profile shares the same opinion or not, on the willingness to verify a Twitter profile. Our analysis showed that homogeneity in behavior and content and form was most important to users. Confirming our hypothesis, participants were more willing to verify opinion-incongruent profiles than congruent ones. Bot profiles were most likely to be verified. Our main conclusion is that users apply profile verification tools to confirm their perception of a social media profile instead of alleviating their uncertainties about it. Partisan-motivated reasoning drives profile verification for bot and human profiles.