American National Election Studies Research Articles

Synthetic Replacements for Human Survey Data? The Perils of Large Language Models

Abstract Large language models (LLMs) offer new research possibilities for social scientists, but their potential as “synthetic data” is still largely unknown. In this paper, we investigate how accurately the popular LLM ChatGPT can recover public opinion, prompting the LLM to adopt different “personas” and then provide feeling thermometer scores for 11 sociopolitical groups. The average scores generated by ChatGPT correspond closely to the averages in our baseline survey, the 2016–2020 American National Election Study (ANES). Nevertheless, sampling by ChatGPT is not reliable for statistical inference: there is less variation in responses than in the real surveys, and regression coefficients often differ significantly from equivalent estimates obtained using ANES data. We also document how the distribution of synthetic responses varies with minor changes in prompt wording, and we show how the same prompt yields significantly different results over a 3-month period. Altogether, our findings raise serious concerns about the quality, reliability, and reproducibility of synthetic survey data generated by LLMs.

Younger Americans are less politically polarized than older Americans about climate policies (but not about other policy domains).

Political polarization of Americans' support for climate policies often impedes the adoption of new, urgently needed climate solutions. However, recent polls suggest that younger conservatives favor adopting pro-climate policies to a greater degree than older conservatives, resulting in less political polarization among younger Americans relative to older Americans. To better understand these patterns, we analyzed Americans' support for various climate policies from 1982-2020, across 16 waves of historical, nationally representative survey data from the American National Election Studies (total N = 29,467). Regression models consistently show that, since 2012, younger Americans have been less politically polarized than older Americans on support for climate policies. Before 2012 and on non-climate policy topics, we did not find consistent statistical evidence for political polarization varying with age. These findings can inform policy debates about climate change and offer hope to environmentalists and policymakers who seek to build broad consensus for climate action at the policy level.

The personality of a personality cult? Personality characteristics of Donald Trump's most loyal supporters

AbstractThe unusually loyal supporters of Donald Trump are often described as a cult. How can we understand this extreme phenomenon in U.S. politics? We develop theoretical expectations and use the Big Five personality dimensions to investigate whether Trump's most loyal supporters share personality characteristics that might make them inclined to cult‐like support. We find that (1) Trump's supporters share high levels of Conscientiousness; (2) this is substantively and statistically distinguishable from the commonly identified association between Conscientiousness and Conservatism; and (3) the association is highly robust to a range of sensitivity tests. Our main findings are robust across two surveys. Study 1 is an online survey conducted in 2021 designed specifically for this project. Study 2 is the 2016 American National Election Study (ANES).

Altruistic concern: how racial ideologies explain racial differences in concerns for Black victims of crime and the police

We explore how concerned Americans are about the risk of violence faced by Black Americans. Using a 2022 American National Election Studies (ANES) survey, we compare concern for Black Americans as the victims of crime versus the police. We find White Americans to be the least concerned about both kinds of victimization, Black Americans the most concerned, and Latinx Americans in between. Interestingly, White Americans expressed significantly less concern about Black Americans being victimized by the police than by civilians. People’s racial ideologies play the largest role in explaining the average differences in concern between people with different racial identifications.

Trump Voters' social position in U.S. Society: Uniqueness and radical‐right support

Trump Voters' social position in U.S. Society: Uniqueness and radical‐right support

Institutional knowledge and perceived rural representation

Place-based identities could potentially shape how the public evaluates political information, especially when cleavages between rural and non-rural become more central to politics. Rural voters may particularly respond distinctly to knowledge about the United States political system, which through several political institutions, notably the federal Senate, gives rural areas a share of votes exceeding their share of the population. Observational survey responses from the 2020 American National Election Study show that although most respondents feel that government institutions give less political influence to rural areas, knowledge about American politics reduces that feeling. The relationship between this knowledge and beliefs about rural political influence, however, occurs only in those with suburban and, especially, urban identities; rural and small-town identifiers seemingly shrug off their knowledge in assessing rural political influence.

Validating the “Genuine Pipeline” to Limit Social Desirability Bias in Survey Estimates of Voter Turnout

Abstract It is well documented that survey overreporting of voter turnout due to social desirability bias threatens inference about political behavior. This paper reports four studies that contained question wording experiments to test questions designed to minimize that bias using a “pipeline” approach. The “pipeline” informs survey participants that researchers can perform vote validation to verify turnout self-reports. This approach reduced self-reported turnout by 5.7 points in the 2020 American National Election Study, which represents a majority of the estimated overreporting bias. It reduced reported turnout by 4 points in two nonprobability samples. No effect was found in a third nonprobability study with Amazon Mechanical Turk workers. Validated vote data also confirm that the pipeline approach reduced overreporting. We tested heterogeneous effects for sophistication and several other variables, but results were inconclusive. The pipeline approach reduces overreporting of voter turnout and produces more accurate estimates of voters’ characteristics.

Open Questions Self-Administered on the Web versus Interviewer-Administered in Person: The 2016 American National Election Study Mode Comparison

Abstract Relatively little is known about the performance of open questions in self-administered questionnaires compared to interviewer-administered ones. We examined this issue using the responses to the 2016 American National Election Study’s Web self-administered questionnaires and face-to-face computer-assisted personal interviews (CAPI) from independently drawn samples that asked three sets of open questions: most important problem(s) facing the country; candidate for and against reasons; and political party likes and dislikes. Although Web respondents gave shorter answers with fewer codeable ideas than face-to-face respondents, the two modes produced similar distributions of substantive answers. Despite our expectation that open answers to interviewers would be more civil than those expressed on the Web, there was no difference in civility between the modes. However, the construct validity of the four candidate for and against items (predicting vote choice of Clinton vs. Trump) was higher on the Web than in the interviewer-administered CAPI. These findings call into question the conventional wisdom that open questions are not well suited for self-administration and suggest that researchers should consider more frequent use of open questions in Web surveys.

Political Alienation and the Trump Vote in the 2016 and 2020 US Presidential Elections

Abstract Following Donald Trump’s surprising victory in the 2016 US presidential election, some popular and scholarly sources suggested that Trump’s candidacy may have been bolstered, in part, by the mobilization of “politically alienated” voters. This argument is puzzling, however, as certain forms of political alienation are often negatively related to political participation, making it unclear whether or how alienation may have been related to turnout and to support for Trump at the ballot box. I shed light on this puzzle using data from the American National Election Studies, which contain measures of two dimensions of political alienation: inefficacy and cynicism. With these data I examine how either dimension relates to turnout and to vote choice in 2016 and in 2020. Cynicism emerges as a positive predictor of both turnout and the Trump vote in 2016, but not in 2020. Inefficacy, however, does not positively predict turnout or the Trump vote in either election. I offer a potential explanation for the diminished relationship between cynicism and mobilization in the 2020 elections by applying a Structural Topic Model to open-ended survey responses about Trump, which reveals a substantial decrease in the salience of Trump’s “political outsider” qualities during his reelection bid.

Social Mobility through Immigrant Resentment: Explaining Latinx Support for Restrictive Immigration Policies and Anti-immigrant Candidates

Abstract Various polls suggest that Donald Trump has enjoyed the support of a sizable minority of the Latinx electorate despite his racially offensive rhetoric and support for some of the most restrictive immigration policies in recent memory. Building on Social Identity Theory and Self-Categorization Theory, we contend that some Latinxs harbor negative stereotypes about immigrants, blame them for the status devaluation of the Latinx community, and cognitively distinguish themselves from Latinx immigrants. Rather than viewing anti-immigrant policies, rhetoric, and politicians as a direct status threat, those exhibiting this “Latinx Immigrant Resentment (LIR)” may regard them as a means to enhance the status and interests of “prototypical” Latinxs by signaling their distinction from “atypical” Latinxs. To evaluate this theory, we use the 2020 American National Election Study (ANES) and 2016 Collaborative MultiRacial Post-Election Survey (CMPS) as a proof-of-concept to first confirm that negative immigrant stereotypes and cognitive intragroup distinctions are associated with increased support for Donald Trump and restrictive immigration policies. We then introduce a more refined measure of LIR by fielding online surveys of US Latinxs administered through Lucid in 2020–2021 (N = 1,164) and 2021/22 (N = 1,017). We demonstrate the validity of this measure and its predictive power for attitudes toward Donald Trump, Ron DeSantis, and restrictive immigration policies after accounting for a range of rival explanations.

Political efficacy and the perceived influence of urban and rural residents

AbstractWhile political scientists have long studied citizens' political efficacy as an important indicator of attitudes toward government, less attention has been devoted to the efficacy of rural or urban residents, which is important given the intensifying rural–urban divide in American society. This study fills this gap by analyzing the 2020 American National Election Studies. Using ordered logistic regression, this study finds that (1) city residents tend to believe that small towns and rural areas have too much influence on government; (2) residents of small towns and rural areas demonstrate lower levels of external efficacy than city residents; and (3) people who believe that small towns and rural areas have too much influence tend to demonstrate high external and internal efficacies, a tendency that is clearer in cities than in other community types. These findings reflect mutual in‐group bias and place‐based resentment between rural and urban residents in American society.Related ArticlesPeterson, Holly L., Mark K. McBeth, and Michael D. Jones. 2020. “Policy Process Theory for Rural Studies: Navigating Context and Generalization in Rural Policy.” Politics & Policy 48(4): 576–617. https://doi.org/10.1111/polp.12366.Shortall, Sally, and Margaret Alston. 2016. “To Rural Proof or Not to Rural Proof: A Comparative Analysis.” Politics & Policy 44(1): 35–55. https://doi‐org.libproxy.usouthal.edu/10.1111/polp.12144.Smith‐Walter, Aaron, Holly L. Peterson, Michael D. Jones, and Ashley Nicole Reynolds Marshall. 2016. “Gun Stories: How Evidence Shapes Firearm Policy in the United States.” Politics & Policy 44(6): 1053–88. https://doi‐org.libproxy.usouthal.edu/10.1111/polp.12187.

Growing Up in a Polarized Party System: Ideological Divergence and Partisan Sorting Across Generations

AbstractAmerican political elites have increasingly polarized over the past decades, which has inspired much research into mass polarization. We study whether there is a generational component to mass polarization by disentangling period, age, and cohort differences while distinguishing two forms of mass polarization: partisan sorting and ideological divergence. Drawing from General Social Survey and American National Election Studies data, we find that partisan sorting has increased across long-standing and emerging issues, while ideological divergence has not. Contrary to expectations, over-time increases in sorting are clearly driven by changes within generations rather than by generational replacement. On several issues, newer generations turn out to be less sorted than those they replace. This tentatively suggests that, partially as a consequence of demographic changes, generational replacement will gradually lead to less polarization in American public opinion as it converges toward more liberal positions.

Racial Bias and U.S. Presidential Candidate Preference

Abstract Much research has suggested that Republican candidates in U.S. presidential elections benefit from voter bias against non-White groups. The present study supplements this research by including in the analysis bias against Whites. Estimates from the American National Election Studies 2020 Time Series Study indicated that a nontrivial percentage of the U.S. population has a bias that disfavors Whites, with this bias more prevalent among Democrats than Republicans. Further analyses estimated the extent to which the type of voter racial bias that favors Republican presidential candidates offsets the type of voter racial bias that favors Democratic presidential candidates. Estimates for the 2020 U.S. presidential election indicated that the two-party vote share for Donald Trump was higher among voters who rated Whites, Blacks, Hispanics, and Asians/Asian-Americans equal to each other on 0-to-100 feeling thermometers than among the full population of two-party voters, suggesting that Donald Trump was disadvantaged on net by the electorate including racially biased voters. These results call into question research that has used an unrepresentative set of racial attitudes to suggest that, in recent U.S. presidential elections, only Republican candidates have benefitted from racial bias among the electorate.

Testing the Robustness of the ANES Feeling Thermometer Indicators of Affective Polarization

Affective polarization (AP)—the tendency of political partisans to view their opponents as a stigmatized “out group”—is now a major field of research. Relevant evidence in the United States derives primarily from a single source, the American National Election Studies (ANES) feeling thermometer time series. We investigate whether the design of the ANES produces overestimates of AP. We consider four mechanisms: overrepresentation of strong partisans, selection bias conditional on strong identification, priming effects of partisan content, and survey mode variation. Our analysis uses the first-ever collaboration between ANES and the General Social Survey and a novel experiment that manipulates the amount of political content in surveys. Our tests show that variation in survey mode has caused an artificial increase in the mixed-mode ANES time series, but the general increase in out-party animus is nonetheless real and not merely an artifact of selection bias or priming effects.

Bridging prediction and theory: Introducing the Bayesian Partially-Protected Lasso

This work examines the application of regularization methods in political science, both theoretically and practically. The focus of the study is on the use of regularization to specify models for predicting political outcomes. Specifically, what is the predictive price for protecting theoretically important variables from shrinkage? While many machine learning applications prioritize maximizing predictive accuracy, political science relies on reliable prior findings as a theoretical foundation. The literature shows that regularization methods can shrink crucial theoretically-driven variables to insignificance. To address this, we propose a protected Lasso approach in a Bayesian framework that safeguards these variables, balancing theoretical robustness and predictive power. Our analyses, applied to the American National Election Study, demonstrates the effectiveness of this approach.

Party Affiliation in the Southern Electorate

This article chronicles and analyzes the changing state of party affiliation in the American South from the 1990s to the present. Party identification (PID) is examined with data from three survey sources: the American National Election Study (ANES), Cooperative Election Study (CES), and Exit Polls (national and state-level). Emphasis is placed on the changing PID patterns of white, Black, and Hispanic voters South-wide and in individual states. The findings strongly suggest that the current position of Republican electoral dominance will persist for some time to come, though there are multiple reasons why, in certain parts of the South, Democratic affiliation will likely increase.

The Democratic Majority and the 2016 American Presidential Election: Feminist Political Behavior Across Multiple Axes of Identity

Scholarship on women voters using an intersectional lens has shown feminists and women of color within the Democratic Party to be a powerful force in electoral politics—specifically, Black and Latina women—are the most reliable voters. Combined, they represent the majority of women voters in the Democratic Party, and they have shown themselves to be a sizeable voting bloc in the last two American presidential election cycles. Using data from the 2016 American National Election Studies (ANES), we demonstrate important differences in support for the Democratic candidate (Hillary Clinton) between feminists and women of color—Black and Latina—underscoring the need to study gender, race, and ethnicity simultaneously (versus independently) when they are co-constitutive identity categories. By examining women and the groups the Clinton campaign appealed most to—feminists, Black, and Latina women—we add to the scholarship in political science that followed this unique election and demonstrate the importance of feminist identity for vote choice and various modes of political behavior for women voters. Knowing and understanding how women respond through affect and how this affect can result in political behaviors will help determine what it takes for future historic first candidates to emerge successful.

Health Behavior Model (HBM) and American’s perceived health risks of the COVID-19 vaccine

ABSTRACT We assessed the perceived risks of COVID vaccines using a Health Belief Model (HBM) among the American public during the vaccination roll-out in the USA. We employed a retrospective study of 1,149 adults from the 2020 American National Election Studies surveys to assess the public’s belief about the COVID-19 vaccine during the pandemic. The logistic regression analysis found several predictors of perceived health risks of the COVID vaccine. For urban/suburban residents, having higher education, being older, being White, and having higher income emerged as protective factors. People with these characteristics are less likely to perceive the COVID vaccine as risky. Also, higher use of Twitter emerged as a protective factor for residents from urban/suburban regions. However, residents from rural/small towns who use Facebook often are more likely to perceive the COVID vaccine as risky. Perceived risks of vaccination are high among Facebook users of residents from rural/small towns and low among Twitter users among residents from urban/suburban areas. The authors identified several protective factors: being educated, older, economically well-off, and White. The authors propose interventions to reduce perceived risks and vaccine hesitancy among vulnerable populations.

Populist Hyperpartisans?: The Interaction Between Partisan Media Exposure and Populism in the 2020 US Presidential Election

ABSTRACT Aspects of our political world brought to the forefront in recent years include the impact of declining media trust and a surge of populist politicians around the globe. Highlighting those aspects, this study utilizes representative data from the American National Election Studies (ANES) collected during the 2020 US Presidential Election. We examine the relationship between use of different media (mainstream liberal, mainstream conservative, and hyper-conservative) and media trust as well as the moderating role of populist beliefs. Results found that hyper-conservative media use and anti-elitist populist beliefs are negatively associated with media trust while liberal media use is positively associated with media trust; mainstream conservative media use is unrelated to media trust when controlling for other media types and populist beliefs. This lack of a main effect for conservative media is clarified by examining the interaction with anti-elitist populism, which shows increased use of conservative media for individuals with low levels of populist beliefs is associated with increased media trust while the opposite is true for those high in populist beliefs.

ARE CATHOLICS UNIQUELY CROSS-PRESSURED? POLICY BELIEFS AND VOTING BEHAVIOR BY RELIGIOUS TRADITION IN RECENT U.S. ELECTIONS

The Catholic Church’s pro-life, pro-social justice policy agenda takes the sides of both major US political parties. This potentially cross-pressures Catholic voters’ choice between those parties, but could alternatively legitimate a Catholic voter’s personal partisan preference. This paper examines whether Catholic voters who share the Church’s core policy positions are more or less likely than comparably cross-pressured non-Catholic voters to exhibit political behaviors associated with cross-pressures: avoiding identification with a major party, avoiding voting or a major-party vote choice, defecting from one’s party in voting, and selectively misperceiving candidate issue positions. Analyzing data from the 2016-2018 Cooperative Congressional Election Studies and the 1992-2016 American National Election Studies, I find little evidence that Catholics are uniquely, strongly cross-pressured. If anything, cross-pressured (and other) Catholics are more likely than comparable non-Catholics–even those in faith traditions that are more clearly aligned with a single party–to embrace partisan politics. In some cases, partisan differences between Catholics in their responsiveness to cross-pressures exceed differences between Catholics and non-Catholics.