Anecdotes Ushered in Marijuana Legalization: A Machine Learning-aided Big Data Analysis of Reddit Discourse (2008-2019)

Abstract

At least two fundamental types of evidence feature in attempts to persuade: Anecdotal and generalized (Baesler & Burgoon, 1994). Experimental research has found anecdotal evidence more effective at changing attitudes in issues marked by personal significance and health-relevance (Freling et al., 2020). These apply to marijuana legalization, where a massive shift in American attitudes (from 35% to 67% in favor during 2008-2019; Pew Research Center, 2019) was followed by rapid legalization. However, no research to date has examined whether the movement benefited from anecdotal framing. Since the attitude shift coincided with the rise of social media, we developed the largest corpus of marijuana legalization discussions from Reddit to address this gap (more than 3M comments from 2008-2019, comprising more than 300M words). The dataset is the first to separate marijuana legalization discourse from general mentions of cannabis (e.g., product advertisements) across an entire popular platform. We then developed neural network models to distinguish anecdotal from generalized text in the dataset based on three clause-level features derived from linguistic theory: Whether a clause is about a generic kind rather than specific instances, whether it presents a reliable state or an event, and whether events are bounded in time. Principal Components Analysis provided a reliable composite score of the three features, treated as a measure of the degree to which major themes of discourse are anecdotal versus generalized. We combined topic modeling (Latent Dirichlet Allocation; Blei et al., 2003) with hierarchical clustering and smoothed polynomial regressions to track themes’ prominence over time and bin them into broader categories. Anecdotal themes were less prevalent but present in most comments. We trained separate neural networks on human annotations of attitude and persuasion attempt. Within non-argumentative discourse, anecdotes became more prominent only later in time, presumably as a consequence of softening societal attitudes. But they played a more prominent role throughout in arguments favoring legalization, suggesting that they were actively used to persuade others. Were such anecdotal arguments timed in a way that benefitted legalization ballot initiatives? To answer, we inferred user locations and compared the rate of anecdotal themes before and after legalization in comments from pioneering states. Despite the experimental evidence favoring anecdotal argumentation, we found that the 2012 and 2016 legal milestones followed short-term increases in generalized arguments instead. The particular content, however, varied between the two periods. Character judgments were prominent in 2012, while crime and politics took center-stage in 2016. The generalized precedents of legalization in leading states were argumentative and moralistic but had distinctive clause-level profiles. Meanwhile, legal and medical arguments were sidelined, meaning the novel consensus was not informed by much of the relevant information, anecdotal or otherwise. Together, our results show that while the emerging consensus probably benefited from anecdotal argumentation, the legalization movement’s success happened despite its reliance on less effective generalized discussions with less concrete information content. Addressing this discrepancy between experimental research and the direction of societal discourse may help bring about more informed discussions while better enabling the changing of attitudes.