Large-scale Survey Experiment Research Articles

Social norms, political polarization, and vaccination attitudes: Evidence from a survey experiment in Turkey

This paper examines vaccination as a descriptive social norm in the context of the Covid-19 pandemic. Using a large-scale survey experiment in Turkey, we first elicit respondents’ vaccination attitudes and show that political affiliation is a strong predictor of it. We then use economic games to measure the extent of outgroup discrimination induced by respondents’ attitudes towards vaccination. We find that while both pro- and anti-vaxxers discriminate against each other substantially, the pro-vaxxers discriminate more than the anti-vaxxers do. This polarization intensifies when pro- and anti-vaxxers perceive a political difference between them. Using randomized informational treatments, we show that a reminder or priming of external threats, appealing to a broadly shared social identity, might mitigate such outgroup discrimination.

Using a novel multiple-source indicator to investigate the effect of scale format on careless and insufficient effort responding in a large-scale survey experiment

Common indicator-based approaches to identifying careless and insufficient effort responding (C/IER) in survey data scan response vectors or timing data for aberrances, such as patterns signaling straight lining, multivariate outliers, or signals that respondents rushed through the administered items. Each of these approaches is susceptible to unique types of misidentifications. We developed a C/IER indicator that requires agreement on C/IER identification from multiple behavioral sources, thereby alleviating the effect of each source’s standalone C/IER misidentifications and increasing the robustness of C/IER identification. To this end, we combined a response-pattern-based multiple-hurdle approach with a recently developed screen-time-based mixture decomposition approach. In an application of the proposed multiple-source indicator to PISA 2022 field trial data we (a) showcase how the indicator hedges against (presumed) C/IER overidentification of its constituting components, (b) replicate associations with commonly reported external correlates of C/IER, namely agreement with self-reported effort and C/IER position effects, and (c) employ the indicator to study the effects of changes of scale characteristics on C/IER occurrence. To this end, we leverage a large-scale survey experiment implemented in the PISA 2022 field trial and investigate the effects of using frequency instead of agreement scales as well as approximate instead of abstract frequency scale labels. We conclude that neither scale format manipulation has the potential to curb C/IER occurrence.

Social mobility perceptions and inequality acceptance

This paper examines how the perceived importance of family background affects distributional preferences using two large-scale survey experiments. In the first experiment, we randomly inform respondents about the relationship between parental income and economic success later in life, making their social mobility perceptions more pessimistic. However, this changes neither revealed distributional preferences nor pro-social behavior toward the rich and poor. The second experiment shows that respondents do not account for parental influence on economic success when making (re-)distribution decisions, suggesting that people view parental influence as a legitimate reason to justify some inequality. This can explain why distributional preferences are immune to changes in perceptions of social mobility.

Automation anxiety, fairness perceptions, and redistribution: Past experiences condition the response to future job loss

Do beliefs about fairness interact with past experiences of labor market shocks to condition redistributive preferences? In a large-scale survey experiment, we investigate the effect of informing individuals that growth in automation could disrupt labor markets in ways that are (possibly) viewed as unfair. We then exploit the COVID-19-induced labor market shock to test for an interaction between treatments and shock exposure. We find that beliefs interact with shock exposure to increase redistributive preferences and commitments to donate potential prize winnings. Our findings suggest this may be motivated by respondents' concerns about their vulnerability to future labor market shocks.

Message-sidedness in performance information disclosure and citizens’ perceived accountability

ABSTRACT Governments disclose performance information to citizens on the assumption that doing so helps improve public accountability. The previous literature implies that citizens’ interpretations of such information are shaped by the information’s presentation types, however, it is not known whether one-sided messages (only positive information) and two-sided messages (adding negative attributes to the one-sided messages) have different effects on citizens’ perceptions regarding government information disclosure. Drawing on attribution-correspondence theory and utilizing a large-scale survey experiment conducted in Korea, this study demonstrates that two-sided performance message is more effective in improving the citizens’ perceptions of governmental transparency and accountability than one-sided message.

Understanding Public Support for Policies Aimed at Gender Parity in Politics: A Cross-National Experimental Study

Abstract Across the globe, women are underrepresented in elected politics. The study's case countries of Australia (ranked 33), Canada (61) and the United States (66) rank poorly for women's political representation. Drawing on role strain and gender-mainstreaming theories and applying large-scale survey experiments, we examine public opinion on non-quota mechanisms to bolster women's political participation. The experimental design manipulates the politician's gender and level of government (federal/local) before asking about non-quota supports to help the politician. We find public support for policies aimed at lessening work–family role strain is higher for a woman politician; these include a pay raise, childcare subsidies and housework allowances. This support is amplified among women who are presented with a woman politician in our experiment, providing evidence of a gender-affinity effect. The study's findings contribute to scholarship on gender equality and point to gender-mainstreaming mechanisms to help mitigate the gender gap in politics.

Temperature-dependent trophic associations modulate soil bacterial communities along latitudinal gradients.

Understanding the environmental and biological mechanisms shaping latitudinal patterns in microbial diversity is challenging in the field of ecology. Although multiple hypotheses have been proposed to explain these patterns, a consensus has rarely been reached. Here, we conducted a large-scale field survey and microcosm experiments to investigate how environmental heterogeneity and putative trophic interactions (exerted by protist-bacteria associations and T4-like virus-bacteria associations) affect soil bacterial communities along a latitudinal gradient. We found that the microbial latitudinal diversity was kingdom dependent, showing decreasing, clumped, and increasing trends in bacteria, protists, and T4-like viruses, respectively. Climatic and edaphic drivers played predominant roles in structuring the bacterial communities; the intensity of the climatic effect increased sharply from 30°N to 32°N, whereas the intensity of the edaphic effect remained stable. Biotic associations were also essential in shaping the bacterial communities, with protist-bacteria associations showing a quadratic distribution, whereas virus-bacteria associations were significant only at high latitudes. The microcosm experiments further revealed that the temperature component, which is affiliated with climate conditions, is the primary regulator of trophic associations along the latitudinal gradient. Overall, our study highlights a previously underestimated mechanism of how the putative biotic interactions influence bacterial communities and their response to environmental gradients.

How Scary is the Risk of Automation? Evidence from a Large Scale Survey Experiment

How Scary is the Risk of Automation? Evidence from a Large Scale Survey Experiment

The journalistic preference for extreme exemplars: educational socialization, psychological biases, or editorial policy?

Abstract Exemplars are central in news reporting. However, extreme negative exemplars can bias citizens’ factual perceptions and attributions of political responsibility. Nonetheless, our knowledge of the factors shaping journalistic preferences for including exemplars in news stories is limited. We investigate the extent to which educational socialization, psychological biases, and editorial policy shape journalistic preferences for extreme negative exemplars. We field large-scale survey experiments to a population sample of journalism students, a nationally representative sample of citizens, and a representative sample of “young people” and obtain evaluations of news value, newsworthiness, and behavioral measures of the actual write-up of news articles. We find significant support for the role of editorial policy and limited support for the role of educational socialization and psychological biases. In a time where economic pressures and the proliferation of digital media potentially lead editors to prioritize clickbait, these findings suggest that structural biases in news coverage may be aggravated.

Gone, but not forgotten? The German federal election 2021 and the effect of an incumbent who did not run

The German federal election 2021 was remarkable partly because the incumbent chancellor Angela Merkel, albeit still popular, was not running for re-election and did not figure prominently in the campaign. Drawing on the literatures on incumbency advantage, cost of ruling, and candidate-centered voting, we analyze whether an outgoing incumbent might still have an effect on voters’ attitudes and behavior. We distinguish two phenomena: First, we expect the effects of outgoing incumbents on electoral behavior to wash off during campaigns, as voters get more informed and adjust to new candidates. Second, if the incumbent’s successor as front-runner of their party manages to link themselves to their predecessor, the incumbent’s reputation might rub off on the new candidate, possibly indirectly enhancing electoral fortunes of the party. We examine the dynamics of these phenomena using data from the panel and RCS components of the German Longitudinal Election Study and a large-scale survey experiment.

When Do Citizens Consider Political Parties Legitimate?

AbstractResearch on negative partisanship and affective polarization shows that wholesale rejections of individual parties are a common and growing phenomenon. This article offers a novel perspective on assessments of parties by considering citizens' legitimacy perceptions of political parties as institutional players. Combining research on political parties and public opinion, I develop a theoretical framework that explains how parties' characteristics shape their perception as legitimate institutional players. I argue that governing experience, age, ideology, and democratic behaviour provide informational cues to citizens about how democratically dangerous a party is. To test my argument, I fielded a cross-sectional survey in seven West European countries and a large-scale survey experiment. The results consistently show that citizens use party-level cues such as ideological moderation and democratic behaviour to form party legitimacy perceptions. The findings have important public opinion implications for political parties and their institutional role in democracies.

High-status individuals are held to higher ethical standards

Although there is evidence for the generosity of high-status individuals, there seems to be a strong perception that the elites are selfish and contribute little to others’ welfare, and even less so than poorer people. We argue that this perception may derive from a gap between normative and empirical expectations regarding the behavior of the elites. Using large-scale survey experiments, we show that high-status individuals are held to higher ethical standards in both the US and China, and that there is a strong income gradient in normatively expected generosity. We also present evidence for a gap between people’s normative expectations of how the rich should behave, and their empirical expectations of how they actually do: empirical expectations are generally lower than both normative expectations and actual giving.

Regulators as Guardians of Trust? The Contingent and Modest Positive Effect of Targeted Transparency on Citizen Trust in Regulated Sectors

Abstract Targeted transparency has become an essential tool for regulation. Through information disclosure, regulatory agencies try to get regulated companies to improve their practices and comply with regulations. In the past, regulation was associated with distrust in regulated sectors. Recent research suggests that regulation, especially targeted transparency, may also increase citizen trust in regulated sectors. However, empirical evidence on whether transparency as a regulatory tool undermines or decreases trust in a sector is lacking. We contribute to this debate by investigating the effect of targeted transparency on citizen trust through a large-scale representative survey experiment (n = 5,303). We used 12 transparency frames in three regulated domains in the Netherlands (consumer rights, healthcare safety, and nuclear plant safety). Our findings suggest that, in general, targeted transparency does not undermine trust, but has a positive effect on trust in regulated sectors. However, this effect is small and contextual, depending on the regulatory domain and type of transparency frame.

Setting limits to tolerance: An experimental investigation of individual reactions to extremism and violence

Tolerating others' opinions, even if disliked, is a cornerstone of liberal democracy. At the same time, there are limits to political tolerance as tolerating extremists and groups who use violence would threaten the foundations of tolerance itself. We study people's willingness to set limits to tolerance in case of violence and extremism (scope of tolerance)—under different conditions regarding ideological groups (left-wing, right-wing, religious) and offline/online contexts of free speech. Using data from a large-scale survey experiment conducted in Germany, we show that citizens clearly set limits to tolerance of different groups, especially if the latter have violent intentions, and that people tend to be more tolerant online than offline. Moreover, we find that citizens are more tolerant toward groups that are closer to their own ideological stance. However, violence disrupts such an ideological bias as respondents across the ideological spectrum exhibit low levels of tolerance toward violent groups—irrespectively of their political stance. Our findings highlight the importance of situational factors as foundations of judgments on the limits to tolerance.

Empathy-mediated altruism in intergroup contexts: The roles of posttraumatic stress and posttraumatic growth.

Exposure to war-related violence is associated with greater prosocial behavior. Although researchers point to empathy and individual differences in posttraumatic growth to explain this relationship, there is no direct empirical evidence of the psychological process by which exposure to wartime violence leads to prosociality. In this investigation, we propose and test a comprehensive model of empathy-mediated altruism that addresses both how and when exposure to violence may be associated with prosociality. Results from a large-scale survey experiment conducted in a naturalistic field setting (1,660 refugees from the wars in Syria and Iraq residing in Turkey) indicate that participants reported greater empathy and altruism toward ingroup versus outgroup targets, and that posttraumatic stress predicted less and posttraumatic growth predicted more empathy and altruism. Further, empathy mediated ingroup biases in altruism (i.e., allocation of resources to the self and others); this indirect effect was stronger for those reporting greater posttraumatic growth and posttraumatic stress. These results support our proposed model of empathy-mediated altruism that incorporates individual differences in response to war violence and ingroup preferences. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Divided by Europe: affective polarisation in the context of European elections

As European integration has become politicised over the last several decades, scholars have paid keen attention to the role of identity in shaping political conflicts and contestation in Europe. This article investigates the microfoundation of the political divide over European integration by building on and extending theories of social identity and recent studies on affective polarisation. Large-scale survey experiment data from 25 European Union member states are analysed to explore the extent of divides over European integration. This analysis shows a statistically significant and substantively large magnitude of the divide based on European identity. Moreover, results show that this divide over European integration deserves particular attention as it is largely driven by out-group animosity, rather than in-group favouritism. Lastly, results show that electoral context, such as electoral salience, involvement in elections, elite polarisation, and the strength of Eurosceptic parties, influences the intensity of the European divide.

Ringing true? The persuasiveness of Russian strategic narratives

International Relations (IR) scholars have theorized the significance of communication and messaging across state borders, using notions such as soft power, sharp power, propaganda and illiberal communication. This study contributes to this body of research by investigating narrative persuasiveness by way of a large-scale experimental exploration of narrative reception. The projection of strategic narratives has become a central feature of modern influencing across borders. Despite the existence of a growing literature on the potentially harmful effects of such narratives, however, their persuasiveness remains under-researched. This article seeks to help fill this gap by asking what might induce people in Sweden to side with strategic narratives projected by Sputnik, the Russian state-funded news media platform. The article puts a central component of Walter Fisher’s classic narrative paradigm to the test: the notions of narrative probability (consistency and coherence) and fidelity (previous life experience). In a rare large-scale survey experiment ( N = 2,032), three narratives from Sputnik were presented to respondents to establish the potential perceived narrative probability and fidelity. Contrary to Fisher’s argument and some previous works on strategic narratives, the results show that people can be persuaded by a narrative without having personal experience of the topic, and despite regarding the text as incoherent. This indicates that information influence projected through strategic narratives can be effective regardless of the form of the message and even when introducing unfamiliar ideas. This is an interesting addition to findings in previous studies that source awareness does not negatively affect the effectiveness of strategic narratives. The article ends by highlighting contributions to previous research on persuasion and by suggesting avenues ahead.

Lessons from an experiment with values-based messaging to support watershed conservation.

Conservation professionals use language related to instrumental, intrinsic, and relational values when communicating about the importance of conservation, frequently in connection with ecosystem services. However, few researchers have examined whether messages that emphasize values associated with ecosystem services result in different policy-support or behavior-change outcomes among different audiences. We conducted a large-scale survey experiment with participants (n = 815) who resided in the United States and were recruited online via the survey platform Qualtrics. The experiment tested whether messages about watershed protection that emphasize instrumental, intrinsic, or relational values (as opposed to the information-only control message) resulted in differing support for policies or behavioral intentions related to watershed conservation. Respondents' personal characteristics had a stronger effect on conservation beliefs than the way values were framed (i.e., than treatments in the experiment). For example, income positively predicted policy support (β = 0.07, 95% CI 0.02-0.12, p = 0.01, corrected p = 0.03). Instrumental messages decreased (SSG, tense) policy support among people who identified as politically liberal (β = -0.75, 95% CI -1.19 to -0.30, p = 0.001, corrected p = 0.003). Over 40% of respondents selected relational values over other value types as the main reason to protect watersheds. Our results demonstrated that political orientation interacts with how the importance of conservation is framed in complex ways and that conservation practitioners might improve the effectiveness of their communications by incorporating relational values and tailoring messages to different audiences.

What’s in a name? Experimental evidence of the coup taboo

ABSTRACT Leaders of military coups routinely deny that their actions amount to a coup, often labelling them as revolutions or even constitutional successions. These attempts to muddy the waters occasionally succeed in prompting discussions over whether the military’s actions truly amount to a coup. But does the label matter? Does public support for military intervention decrease when it is labelled a coup? If so, how large is this “coup taboo”? In this article, we provide the first empirical evidence of the coup taboo across three large-scale survey experiments in Egypt, Algeria, and Tunisia. Across all three countries, we find that the coup taboo is substantial, with support for the military hypothetically removing the president falling by 15–50 percentage points when labelled as a coup. These results underscore the importance of labels, and suggest that anti-coup norms may be superficial, decreasing support for military interventions only when labelled as coups.

Cognitive bias in citizens’ perceptions of government performance in response to COVID-19: Evidence from a large-scale survey experiment in China

This article explores the cognitive bias in citizens’ perceptions of government performance in response to the COVID-19 pandemic. Data were drawn from a survey experiment by manipulating the performance information frame (positive, neutral, and negative) and source (government, scientists, and unofficial sources) with 4,102 Chinese citizens. Findings show that the effect of information frame and source on citizens’ perceptions of government performance is very weak. Additionally, the heterogeneity tests indicate a gender difference in framing effect, and source effect emerges for those located far away from the pandemic epicenter. Frame and source show an interaction effect in the lower education group. The results call for further research on citizens’ cognitive bias in a crisis context.