Detecting Effective Tobacco Control Messages via Linguistic Analysis and Item Response Theory

Abstract

Abstract Current health communication research on tobacco control often examines the persuasion effects of messages in an experimental setting, which produces small effect sizes and limits the external validity of how well the tested messages will perform in real word settings. In the new media landscape, (anti)smoking and (anti)vaping messages are posted, shared, and disseminated across media platforms by a broader set of users. Tracking the persuasion effects of each message can be challenging with the experimental approaches. Novel analytic methods are needed to evaluate the effective strategies for tobacco control communication. We apply linguistic analysis and item-response theory to antismoking/antivaping messages to detect their persuasion effects on cognitive and emotional outcomes at an individual message level. Methods: We recruited respondents from social media and crowdsourcing platforms (n = 6,566). The eligible current smokers were randomly assigned to one of the 80 antismoking and antivaping messages, or 16 food advertisements (the control group). We measured attitudes toward vaping/smoking, emotional arousal, and intention to quit smoking or vape. We mined text data from each stimulus message to examine their linguistic characteristics and its links to the persuasion outcomes. Results: Majority of participants were White (86%) and female (55%). Antismoking messages were more likely to reveal anxiety (P = 0.01) and dangers or concerns (P < 0.01), compared to the control messages; and express high expertise and confidence compared to antivaping messages (P = 0.04). We report linguistic properties of each message and their relationship to antismoking attitudes, emotional arousal, intention to quit cigarettes, and intention to vape. Visualized patterns of message effectiveness demonstrate text/image-based antivaping messages, compared to video-based antivaping messages, unintentionally increased participants' favorable attitudes toward e-cigarettes. Conclusions: These message-level analyses can help identify best candidate messages that generate positive persuasion outcomes, which, in turn, can inform the selection of messages for health education and campaigns.