Survey Experiment Research Articles

The Role of Salience and Memory in Fertility Decisions: Experimental Evidence

Public policies use communication campaigns to affect individual behavior. We analyze how providing women with information on the beneficial effects of using formal childcare may affect their realized fertility. We argue that cues in the messages are particularly salient for mothers and women with fertility intensions, since they activate these women’s past memories. Hence, cues induce these women to create mental representations of future actions, such as realized fertility. We exploit a randomized survey experiment run in 2011, which provides information on the positive effects that attending daycare may have on the children’ future cognitive development. Using a follow-up survey run six-year later we show that the treatment increases realized fertility among mothers and women with declared fertility intentions, for whom the communication was more salient. Yet, the treatment did not affect the individual knowledge nor recall of the information provided in the message. Our results carry important policy implications: persuading individuals is difficult, but communication can be effective if salient.

Do Foreign Military Deployments Provide Assurance? Unpacking the Micro-Mechanisms of Burden Sharing in Alliances

Abstract How do US foreign military deployments impact the defense policies of host states? Dominant scholarship holds that these deployments play a pivotal role in assuring allies that their security is guaranteed, which in turn leads host countries to neglect their national defense contributions. In this research note, we examine the micro-foundations of this conventional wisdom, investigating how nuclear and conventional troop deployments impact attitudes toward defense policies in (potential) host states. We highlight that the presumed linkage between assurance and free-riding critically implies that foreign military deployments must positively affect perceptions of security among host nations. We test this core logic, alongside some alternative pathways, at the micro level through two survey experiments that randomize hypothetical withdrawals (Germany) and deployments (Czech Republic). Although we find some evidence that foreign military deployments can decrease citizens’ subjective need for defense, the survey experiments also reveal that citizens hardly feel protected by these deployments. Thus, our results cast doubt on the core logic underlying the theory of free-riding in alliances.

Does Citizen Feedback Influence Public Service Motivation? Evidence From a Survey Experiment in the Context of Urban Service Provisions

Despite the growing interest of research on public service motivation (PSM), limited studies have considered the interaction between street-level bureaucrats and citizens. This study examined the impact of citizen feedback on PSM among street-level bureaucrats. Anchoring on the social exchange theory, this research posits that citizen feedback plays a pivotal role in influencing PSM. Using a vignette–survey experiment in the context of urban service provisions in Magelang city, Indonesia, this study empirically tested the causal relationship between citizen feedback and PSM. The finding reveals empirical evidence that positive citizen feedback enhances PSM, while, conversely, negative feedback diminishes PSM. These results underscore the importance of citizen feedback as an alternative managerial tool to motivate street-level bureaucrats who directly engage in service delivery. Public managers can also use these findings as a starting point to design a citizen feedback handling system in public service delivery.

Who on Earth Wants a World Government, What Kind, and Why? An International Survey Experiment

Abstract Amidst multiple transnational crises, global governance has retaken center stage in academic and public debates. While previous generations of thinkers and citizens vigorously discussed the perennial idea of a world government, such proposals are nowadays often discarded quickly among scholars and practitioners. However, we know little about citizens’ present-day attitudes toward world government proposals. In a survey experiment on more than 42,000 citizens in 17 countries in the global South, North, East, and West, we find that the idea is only rejected by international majorities if it remains unspecified and if we weight countries equally. Specifications as democratic and/or focused on global issues like climate change significantly increase public support and lead overwhelming majorities worldwide to favor a global government. Support is even stronger in more populous, less free, less powerful, and/or less developed countries. The only exception is the United States, where no global government specification receives majoritarian public approval. Overall, our findings show significant international support for fundamental transformations of global governance, and thus indicate to activists and policymakers that relevant reform efforts can build on widespread public endorsement. En medio de múltiples crisis transnacionales, la gobernanza global ha vuelto a ocupar un lugar importante en los debates, tanto académicos como públicos. Si bien las generaciones anteriores de pensadores y ciudadanos debatieron firmemente la idea perenne de un gobierno mundial, hoy en día los académicos y los profesionales descartan, con frecuencia, rápidamente tales propuestas. Sin embargo, tenemos poca información referente a las actitudes de los ciudadanos hacia las propuestas de un gobierno mundial. Llevamos a cabo un experimento de encuesta con más de 42.000 ciudadanos en 17 países del Sur, Norte, Este y Occidente global, y concluimos que la idea solo es rechazada por las mayorías internacionales si esta idea no es concreta (y solo cuando se ponderan los países por igual). El hecho de incluir especificaciones como la democracia y/o centrarse en temas globales como el cambio climático aumentan significativamente el apoyo público y llevan a mayorías abrumadoras en todo el mundo a favor de un gobierno mundial. Por lo general, el apoyo es más fuerte en los países más poblados, menos libres, menos poderosos y/o menos desarrollados. La única excepción es Estados Unidos, donde ninguna de estas especificaciones gubernamentales globales recibe una aprobación pública mayoritaria. En general, nuestras conclusiones muestran un importante apoyo público internacional a las transformaciones fundamentales de la gobernanza mundial y, por lo tanto, indican a los activistas y a los responsables políticos que los esfuerzos de reforma pertinentes pueden partir de un amplio respaldo público. Face aux nombreuses crises transnationales, la gouvernance mondiale revient sur le devant de la scène dans les débats académiques et publics. Alors que les générations antérieures de penseurs et de citoyens débattaient vivement de l’idée toujours d'actualité de gouvernement mondial, de telles propositions sont de nos jours souvent rapidement rejetées par les chercheurs et les professionnels. Cependant, nous en savons peu sur l'attitude des citoyens par rapport aux propositions de gouvernement mondial. Dans une expérience de sondage sur plus de 42 000 citoyens de 17 pays du Sud, du Nord, de l'Est et de l'Ouest, nous constatons que l'idée n'est rejetée par les majorités internationales que si elle reste vague (et seulement si l'on considère les pays à parts égales). Lorsque l'on précise l'idée en qualifiant ce gouvernement mondial de démocratique et/ou de centré sur des problématiques mondiales comme le changement climatique, le soutien public augmente drastiquement et des majorités écrasantes dans le monde entier en viennent à soutenir un gouvernement mondial. Généralement, le soutien est supérieur chez les pays plus peuplés, moins libres, moins puissants et/ou moins développés. La seule exception reste les États-Unis: aucune précision concernant le gouvernement mondial ne reçoit d'approbation d'une majorité du public. Dans l'ensemble, nos résultats montrent qu'il existe un soutien international important du public vis-à-vis de transformations fondamentales de la gouvernance mondiale, et donc signalent aux militants et aux décideurs qu'ils peuvent s'appuyer sur un large soutien populaire pour promouvoir des réformes pertinentes.

Unpacking Legitimacy Perceptions of Investment Dispute Settlement: Effects of Outcome and Procedure

Abstract How do citizens perceive different avenues for resolving investor–state disputes in case of an unfavorable decision? In this article, I argue that the interplay between procedure- and outcome-related considerations influences citizens’ legitimacy beliefs toward investor-state dispute settlement (ISDS) systems. First, an unfavorable ISDS ruling should decrease public legitimacy beliefs in the country that got sued. Second, the type of dispute settlement mechanism should influence legitimacy beliefs, leading to different assessments of the legal proceedings. I fielded a survey experiment in France, Germany, and the United States to test these theoretical expectations. The findings confirm that an unfavorable ISDS ruling has a strong negative effect on public perceptions of the legal proceeding, but further show that appropriate design choices can boost public legitimacy beliefs. These results have important implications for the debate about ISDS reform and the scholarship on legitimacy in global governance. The impact of unfavorable decisions on public legitimacy beliefs has been widely overlooked.

Audience Costs and the Credibility of Public versus Private Threats in International Crises

Abstract The credibility of public versus private threats in international crisis bargaining has received scholarly attention. While public threats are often believed to be more credible than private ones because of audience costs, others contend that private threats should be equally credible or even more. Also, the effect of public threats can be conditioned by other domestic factors such as domestic hawkishness and a leader’s unpopularity. To test these competing expectations, I conducted a conjoint survey experiment that examines how observers assess resolve to fight in an international crisis with a sample of US citizens (N = 1,203). Three major findings stand out. First, evidence suggests that both types of threats increase the perceived likelihood that a country will stand firm. Second, though their effect size is small, public threats are more effective than private ones. Lastly, I find support for the conditional effect of public threats, indicating that the underlying mechanism of the credibility of public threats is domestic audience costs.

Developing-Country Representation and Public Attitudes toward International Organizations: The Case of IMF Governance Reform

Abstract Several prominent international organizations (IOs) maintain decision-making structures that under-represent developing countries. This paper argues that individuals in developing countries are more supportive of engaging with IOs that give a greater voice to fellow developing countries. We posit that the balance of decision-making power influences support for IOs by improving perceptions of both input legitimacy and output legitimacy. Empirically, we focus on the International Monetary Fund (IMF) and draw on original survey experiments in four developing countries: Argentina, China, South Africa, and Turkey. Results reveal that increased representation of developing countries increases public support for IMF participation. We also find consistent evidence that this effect works through an input legitimacy mechanism, specifically by improving perceptions of procedural fairness. These findings suggest that public support for IOs is affected by the balance of decision-making power within these organizations.

Allegations of Democratic Election Fraud and Support for Political Violence Among Republicans

Are partisans more likely to endorse political violence when politicians accuse their rivals of election improprieties? I theorize that for Republican partisans in the United States, the answer to this question is yes. Republican partisans are primed to believe allegations of cheating by Democrats and view election improprieties through the lens of racial and xenophobic resentments. Allegations of Democratic election fraud prompt them to eschew nonviolent norms of political behavior and endorse political violence. I test these propositions using an original, online survey experiment involving 140 self-identified Republican subjects. I find that exposure to allegations by politicians that Democrats engage in election fraud prompts Republican partisans to increase their support for political violence. Furthermore, using mediation tests, I find that exposing Republicans to allegations of electoral fraud by Democrats reduces their trust of people of different races and religions which, in turn, increases their support for political violence.

The effects of communicating climate change threat: mobilizing anger and authoritarian affect displacement

ABSTRACT Research on climate change communication has made important contributions to our understanding of how to effectively inform and mobilize the public for climate action and yet open questions remain, not least because existing results are at times inconclusive or incomplete. The climate crisis poses an existential threat. But what effect does communicating that threat have? To what degree can communicating the threat invoke anger and how does anger interact with individual values predispositions? In this study, we conducted a survey experiment (N = 570) to better understand effects of communicating climate change threat. We find that exposure to climate change threat makes people angrier, and anger makes them more willing to act on climate change mitigation, confirming previous findings. However, our study adds a novel insight, the findings do not apply to people with authoritarian predispositions. They display lower levels of anger and willingness to act, while exposure to climate change threat results in increased authoritarian attitudes. This suggests that climate change threat can lead to affect displacement in people with authoritarian predispositions, where anger (typically directed at those in power, who have failed to tackle climate change) is redirected at non-conforming groups.

Exploring the clothing overconsumption of young adults: An experimental study with communication interventions

The increase in clothing consumption per capita in the last decades presents substantial environmental and societal challenges. Young adults, heavily influenced by advertisements and new trends via social media, emerge as substantial contributors to the escalating issue of clothing overconsumption. This research aims at better understanding the drivers of clothing consumption for young adults and the policy interventions that can be designed to change clothing overconsumption behaviour. This study employs a survey experiment with communication interventions using different framing strategies. In doing so, the study focuses on the potential impact of communication interventions on the clothing consumption rates of young adults. The study showed that a communication intervention can motivate young adults to purchase less clothing and gives an initial insight into how to implement this type of intervention. Moreover, it provides initial evidence that intervention strategies inspired by sufficiency can be effective. This research calls for more transformative policies to stimulate sustainable consumption that go beyond promoting sustainable alternatives.

Can Appeals to Religious Values Mitigate Anti-Refugee Sentiment? Experimental Evidence From Lebanon

This paper examines whether—and if so, to what extent—religious scripture and values can promote inclusionary attitudes towards refugees. Sacred texts across faiths call on believers to embrace foreigners, suggesting that religious discourses can be harnessed to promote refugees’ inclusion in host societies. To test this possibility, I implement a survey experiment in which citizens of Lebanon—a religiously diverse country that has received the most Syrian refugees per capita globally—are exposed to various forms of pro-refugee religious messaging. I find that inclusionary religious messaging can shift refugee-related attitudes towards tolerance, particularly for members of sectarian communities with the highest baseline anti-refugee animosity. The study's findings have important implications for understanding religion’s consequences for both the dynamics of refugee integration and intercommunal relations more generally, especially in the Middle East and North Africa and the broader developing world.

Does the public want the police to stop, stopping? An experimental look at the impact of outcome data on public perceptions of police discretionary traffic stops

Abstract Objectives To assess whether data on traffic stop outcomes causally impacts public approval of discretionary traffic stops as a crime control strategy. Methods We distributed an original online survey experiment randomly assigning respondents (N = 4740) into either the (1) contraband condition, (2) disparity condition, or (3) the control condition. Results In comparison to the control condition, the contraband condition significantly increased respondent support of discretionary traffic stops as a crime-fighting tool. Black respondents assigned the contraband treatment were significantly more likely to support the use of discretionary traffic stops compared to Black respondents assigned the control condition. Conclusions Although scrutiny exists regarding the efficacy of discretionary traffic stops, public opinion may be shifted if they are provided with information on the outcomes of such stops. Police agencies should consider coupling evidence-based strategies with data on the outcomes of crime control strategies, which may also address community desire for more transparency. Minority civilians may support crime control strategies if presented with data on such strategies’ effectiveness.

Validation of the Very Efficient Short‐Form Need for Cognition Scale in Chinese: Detecting the effect of item orientation

AbstractUsing a large‐scale randomized survey experiment, this study examined the dimensionality, item functioning, instrument properties and associations with validity‐related variables of the Very Efficient Short‐Form Need for Cognition Scale (NCS‐6) in Chinese and investigated whether its psychometric properties were affected by item orientation. For dimensionality, results indicated that a two‐factor model demonstrated a superior fit to the data compared to the one‐factor model when the NCS‐6 contained mixed‐worded items. The one‐factor model showed excellent model fit when the NCS‐6 contained only nonreversed or reversed items. For item functioning, factor loadings were invariant across experimental conditions, but item thresholds were not. For instrument properties, although the internal consistency reliabilities were excellent in each experimental condition, mean scores of the NCS‐6 showed a decreasing pattern as the number of reversed items increased. For associations with validity‐related variables, the structural relationships between NCS‐6 and validity‐related variables were not significantly different across experimental conditions. Overall, the dimensionality and instrument properties of the Chinese version NCS‐6 were affected when it contained mixed‐worded items.

The worthy and the fallible: Framing the issue of state party funding

The party politics literature has shown that parties being mostly reliant on state funding (as in most of Europe) is more conducive to equal representation than them being mostly reliant on big donors (as in the US and UK). But what do citizens think? Relatively little attention has been paid to public opinion on party funding. We only know that citizens tend to instinctively oppose both ways of funding parties. Yet, party funding is not a salient issue except in times of scandal, meaning it is an issue on which citizens are ill-informed. In this paper, I test what happens when citizens in the UK and Denmark are exposed to arguments in support of state party funding, through survey experiments. I find that highlighting state funding’s ability to reduce party reliance on big donors substantially increases support for the policy, especially among low trusting Britons.

Online Abuse of Politicians: Experimental Evidence on Politicians’ Own Perceptions

AbstractCriticism from dissatisfied citizens is an inevitable part of being an elected politician in a democracy. However, when negative feedback from citizens turns into outright abuse, this can have pernicious personal and democratic consequences. In this study, we therefore investigate how politicians perceive criticisms, insults, threats, and sexist remarks in social media comments. Based on a survey experiment with more than 700 politicians from the local, regional, and national levels, we show that politicians’ assessment of the abusiveness of negative messages are not only affected by the content of the messages but also by political ideology and gender. Left-leaning politicians find negative social media comments to be more abusive than right-leaning politicians do. In addition, politicians are more averse to negative comments targeting women than negative comments targeting men, and male politicians are generally less averse to negative comments than female politicians are. Somewhat surprisingly, politicians are generally not affected by in-party biases, as they exhibit similar levels of aversion towards negative messages directed at co-partisans and out-partisans. While previous research has focused on the extent to which politicians receive online abuse, few studies have examined how politicians perceive potentially abusive messages, and our study thus provides novel insights into what exactly politicians consider to be abusive behavior on social media.

Testing the person-positivity bias in a political context: Voters’ affective responses to (non-)personalized individual political actors versus collective political actors

The person-positivity bias states that evaluations of real-life individuals are more positive compared to evaluations of collectivities or impersonal objects. This paper aims to test the prevalence of a person-positivity bias in a political context in two respects: (1) we examine whether voters develop stronger negative affective reactions towards other-minded collective actors (i.e. political parties) compared to individual actors (i.e. politicians), and (2) we investigate whether the provision of personalized individuating information tempers the development of negative feelings towards politicians. A survey experiment conducted among a representative sample of the Flemish population (N = 1200) reveals patterns of vertical affective polarization. However, our study did not find significant evidence that voters dislike other-minded collective actors more than other-minded individual politicians. Also the extent to which individual MPs are personalized has little effect on voters' affective evaluations. Taken together, this study highlights that ideological (dis)agreement is primarily steering voters’ evaluation of political actors.

Do Recovery Efforts Have the Same Impact on Private and Public Complainants After Experiencing a Service Failure?

This study investigated service failures in a public organization, focusing on whether types of complaints (i.e., complaining publicly or privately), the exposure generated by complaints, and reperformance (i.e., expeditiously satisfying citizens’ needs after the service failure) influenced perceptions of justice and complaint effectiveness. Several important findings emerged after conducting a preregistered online survey experiment with 665 respondents. First, when respondents were given vignettes detailing a well-executed reperformance after a private complaint, they did not post higher justice perceptions than those given vignettes detailing that a well-executed reperformance was conducted after a public complaint. Second, garnering high social media exposure from the complaint was only found to positively impact complaint effectiveness when individuals were given the public complaint vignette and primed with a reperformance that was not well-executed. Last, respondents who were provided with vignettes regarding public complaints were found to value the reperformance more than the exposure their complaint generated.

The Political Psychology of Cancel Culture: Value Framing or Group Identity?

“Cancel culture” has become a prominent phrase in US political commentary, with supporters and opponents relying on different value-based arguments to express their views. At the same time, these camps often fall along predictable partisan political lines. What, then, are the real motivations for promoting and opposing behaviors sometimes labeled as “canceling” in contemporary American politics? We explore this question through survey data from Pew in 2020 and two original survey experiments, conducted on the 2021 CES and a 2023 online sample. We examine how Americans define the term “cancel culture,” to what extent cancel culture is linked to both a range of core democratic values, and the role of partisanship in shaping support or opposition to specific behaviors. We observe a significant range of views about cancel culture and document connections to various political values. From our experiments, we find that partisanship is a potent driver of support for and understanding of cancel culture and that value-based framing has a weaker impact as compared to partisanship. Cancel culture provides an important case study of how Americans process conflicting norms and values, including free speech and political accountability, beyond the typical constraints of the formal political system.

This proposal is racist: the (muted) consequences of invoking racism in policy arguments

ABSTRACT Using an original vignette survey experiment, we explore how white Americans’ evaluations of policy arguments in the local context change depending on the rhetoric used to criticize the policy. We compare the effects of three rhetorical frames used by critics of a policy: inefficiency, unfairness, and racism. We find that respondents use political rhetoric as a strong ideological cue: criticisms that invoke both unfairness and racism make respondents perceive policy critics as more liberal and policy supporters as more conservative. The effects of rhetoric on criticism evaluations are moderated by partisanship: Democrats find criticism about unfairness and racism more persuasive, while for Republicans the effect is opposite. We are unable to establish reliable effects of rhetorical framing on the respondents’ support for the policy itself or their readiness to engage in local political action.

Prominent misinformation interventions reduce misperceptions but increase scepticism

Current interventions to combat misinformation, including fact-checking, media literacy tips and media coverage of misinformation, may have unintended consequences for democracy. We propose that these interventions may increase scepticism towards all information, including accurate information. Across three online survey experiments in three diverse countries (the United States, Poland and Hong Kong; total n = 6,127), we tested the negative spillover effects of existing strategies and compared them with three alternative interventions against misinformation. We examined how exposure to fact-checking, media literacy tips and media coverage of misinformation affects individuals’ perception of both factual and false information, as well as their trust in key democratic institutions. Our results show that while all interventions successfully reduce belief in false information, they also negatively impact the credibility of factual information. This highlights the need for further improved strategies that minimize the harms and maximize the benefits of interventions against misinformation.