List Experiment Research Articles

Estimating Prevalence of Abortion Using List Experiments: Findings from a Survey of Women in Delaware and Maryland.

Widespread underreporting of abortion persists in survey data. The list experiment, a measurement tool designed to elicit truthful responses to sensitive questions, may alleviate underreporting. Using The Statewide Survey of Women of Reproductive Age in Delaware and Maryland (n=2,747), we estimate the prevalence of abortion in Maryland and Delaware using a double list experiment. We find 21% (95% confidence interval [CI]: 16.8%-25.3%) of respondents aged 18 to 44 ever had an abortion and we identify disparities in abortion prevalence by age, race, education, income, marital status, and insurance status. Respondents who were Black (37.0%; 95% CI: 27.1%-46.8%), had less than a college degree (24.8%; 95% CI: 18.3%-31.3%), were in a cohabiting relationship (39.0%; 95% CI: 29.1%-48.9%), were living in households with incomes less than $50,000 (28.6%; 95% CI: 19.7%-37.5%), and were currently covered by Medicaid (42.8%; 95% CI: 27.6%-58.0%) were more likely than their counterparts to have ever had an abortion. List experiments yield estimates of abortion substantially higher than those obtained from direct questions. Findings demonstrate external validity through consistency with estimates from administrative data sources and gold standard abortion provider survey data.

Broken Promises - Evaluating an Incomplete Cash Transfer Program

Interventions in highly insecure and fragile contexts are always confronted with the latent risk of not being able to implement the program as intended. Despite its high policy relevance, little is known about the impacts of program disruption or cancellation on beneficiaries. This study uses the unplanned cancellation of the South Sudan Youth Business Start-Up Grant Program to assess the socioeconomic, behavioral, and psychological consequences of a program that fails to be implemented as intended. Originally planned as a randomized trial, the Youth Startup Business Grant Program consisted of an unconditional cash grant combined with a business and life skills training targeting the youth in South Sudan. Due to the intensification of violence in the country, the disbursement of the grant was terminated in late 2016 before most of the intended beneficiaries had accessed the grant. The study uses survey data from face-to-face interviews and experimental data from lotteries, trust games, and a list experiment to assess the consequences of the cancellation in a comprehensive form. The empirical analysis employs instrumental variable regressions to control for individual characteristics that might have made it more likely to access the grant before disbursement was frozen. The results show that participants who received the originally planned treatment displayed significant improvements in their consumption, savings, and psychological well-being. However, participants who vainly expected to receive the cash grant showed reduced levels of consumption and women among this subgroup also experienced strong reductions in their trust level. In addition, the study finds some evidence that these women were less likely to migrate.

Understanding Labor Market Discrimination Against Transgender People: Evidence from a Double List Experiment and a Survey

Understanding Labor Market Discrimination Against Transgender People: Evidence from a Double List Experiment and a Survey

The Two Faces of Opposition to Chemical Weapons: Sincere Versus Insincere Norm-Holders

Prominent research holds that the use of weapons of mass destruction is taboo. But how strong are these norms? Investigating this question among the mass public, we argue that some citizens actually support taboo policies in private but are unwilling to express counter-normative opinions openly due to fear of social sanction. These insincere norm-holders are difficult to identify empirically because they are observationally equivalent to sincere norm-holders in direct-question surveys. To overcome this challenge, we use a list design, which allows survey respondents to indirectly express sensitive opinions. The results from three list experiments show that between 10% and 17% of Americans falsify their preferences over chemical weapons use when asked directly. In an extension, we explore our framework in the realm of nuclear weapons and elite behavior. Our findings advance a specific debate on the strength of weapons taboos, while our conceptualization of insincere norm-holders and methodological application have broader implications for how scholars might think about and measure norms in international politics.

Coercion versus facilitation: Context and the implementation of anti-FGM/C law

AbstractA common policy response to female genital mutilation/cutting (“FGM/C”) is the enactment of criminal prohibitions. The implementation of such prohibitions requires significant state capacity, which is often associated with the coercion. We examine compliance with anti-FGM/C law in Burkina Faso and Kenya, two countries which are thought to have markedly different state capacity and where the practice of FGM/C has been widespread. Using a hard-case comparison and both qualitative and quantitative approaches, including a list experiment to elicit accurate responses regarding the current state of FGM/C practice, we find that Burkina Faso and Kenya have achieved comparatively similar compliance with their respective anti-FGM/C laws. We explain this surprising finding by pointing to the different implementation approaches taken by each state, one coercive and one facilitative, and the degree to which those approaches were well-matched to the attitudes of their respective populations. This paired comparison suggests that coercively weak states may still be able to secure compliance with their laws. It also suggests that academics and policymakers alike should consider the numerous other capacities states possess beyond simple coercion.

Behind the veil: Social desirability bias and animal welfare ballot initiatives

Farm animal welfare (FAW) issues are becoming increasingly political in many countries, as evidenced by the increased use of regulations, legislation, and ballot initiatives. Available empirical evidence however, indicates that consumer valuation of improved animal welfare is low, although positive. As a result of the sensitive nature of FAW issues, public preferences for improved FAW standards can be susceptible to social desirability bias leading to disparities between regulatory standards and the public’s “true” preferences. Given the potential negative impacts of high mandated FAW standards on food costs and the associated consumer and producer welfare losses, this study examined the issue of effective public preference elicitation in animal welfare ballot initiatives. Specifically, we examined social desirability, the tendency to conform to the social norms, and its role in generating overenthusiasm in the support for FAW issues and policy instruments. We used data from an opt-in survey of respondents and compared results of a List Experiments (LE) to a conventional (direct) survey format. Our results show that public support for the FAW issues examined was consistently overestimated when elicited with the conventional survey format. We discuss the implications of these outcomes for animal welfare policy and offer suggestions to researchers and practitioners eliciting preferences in other sensitive food policy contexts.

Norms to be prejudiced: List experiments on attitudes towards immigrants in Japan

Are anti-prejudice norms against immigrants shared worldwide? Although previous studies found that prejudice against immigrants is considered socially undesirable, these studies were conducted exclusively in Western societies and we have little knowledge of the awareness of anti-prejudice norms against immigrants outside of these societies. We use the case of Japan, where people tend to believe that the society is ethnically homogeneous, expecting that this context-specific notion mitigates the awareness of anti-prejudice norms. We conducted two list experiments using online surveys and compared the answers to those of list experiments and direct questions about attitudes towards immigrants to reveal Japanese citizens’ perceptions of norms. The results show that Japanese citizens attempt to show more negative attitudes upon direct questions than in list experiments, indicating that it is normative to express prejudice against immigrants rather than suppressing it. These results suggest anti-prejudice norms against immigrants are context-dependent and not universally shared.

Effect of a story-based, animated video to reduce added sugar consumption: A web-based randomized controlled trial.

BackgroundShort and animated story-based (SAS) videos, which can be rapidly distributed through social media channels, are a novel and promising strategy for promoting health behaviors. In this study, we evaluate the effectiveness of a SAS video intervention to reduce the consumption of added sugars.MethodsIn December 2020, we randomized 4159 English-speaking participants from the United Kingdom (1:1:1) to a sugar intervention video, a content placebo video about sunscreen use (no sugar message), or a placebo video about earthquakes (no health or sugar message). We nested six list experiments in each arm and randomized participants (1:1) to a control list or a control list plus an item about consuming added sugars. The primary end-points were mean differences (on a scale of 0-100) in behavioral intent and direct restoration of freedom to consume added sugars.ResultsParticipants (N = 4013) who watched the sugar video had significantly higher behavioral intent to cut their daily intake of added sugar (mean difference (md) = 16.7, 95% confidence interval (CI) = 1.5-31.8, P = 0.031), eat fresh fruit daily (md = 16.7, 95% CI = 0.5-32.9, P = 0.043), and check food labels for sugar content (md = 20.5, 95% CI = 2.6-38.5, P = 0.025) when compared with the sunscreen (content placebo) video. The sugar video did not arouse intent to restore freedom and consume added sugars when compared with the two placebo videos.ConclusionsOur SAS intervention video did not arouse reactance and increased short-term behavioral intent among participants to reduce their consumption of added sugars. SAS videos, which draw on best practices from the entertainment-education media, communication theory, and the animation industry, can be an effective strategy for delivering emotionally compelling narratives to promote health behavior change.Trial registrationGerman Clinical Trials Register: DRKS00022340

Linking individual and group motives for violent conflict

Research on the role of grievances in civil conflict is surprisingly inconclusive, with well-cited studies disagreeing on the relationship between perceived deprivation and violence. I argue that the role of grievances depends on an interaction between individual and group-level incentives. Individuals who perceive themselves as personally deprived are more likely to support or participate in anti-regime violence, but only if a successful rebellion would enhance their group’s power relative to the status quo. I test this argument in the context of Iraq’s sectarian civil war using data from a 2016 survey of 800 Baghdad residents. Using a list experiment to measure individuals’ willingness to consider violence against a government they feel is ignoring their needs, I find that minority Sunnis who are economically dissatisfied are significantly more willing to consider violence than similarly aggrieved Shias. However, as economic satisfaction increases, Sunnis’ propensity for violence decreases until it becomes indistinguishable from Shias’ propensity. These results clarify the joint impacts of vertical and horizontal grievances. Group inequality and individual deprivation are each necessary but not sufficient to fully explain individuals’ propensities for anti-state violence.

Authoritarian Rallying as Reputational Cascade? Evidence from Putin’s Popularity Surge after Crimea

When international conflict causes an authoritarian leader’s popularity to soar, extant theories lead us to treat such “rallying” as sincere preference change, the product of surging patriotism or cowed media. This study advances a theory of less-than-fully sincere rallying more appropriate for nondemocratic settings, characterizing it as at least partly reflecting cascading dissembling driven by social desirability concerns. The identification strategy combines a rare nationally representative rally-spanning panel survey with a list experiment and econometric analysis. This establishes that three quarters of those who rallied to Putin after Russia annexed Crimea were engaging in at least some form of dissembling and that this rallying developed as a rapid cascade, with social media joining television in fueling perceptions this was socially desirable.

Altered Processing of Hand-Related Words in Children With Upper Limb Motor Disorder (Arthrogryposis Multiplex Congenita): EEG Evidence

A high level of corruption usually constrains economic development in emerging countries. However, anti-corruption campaigns often fail because the relevant policies need to be implemented by existing corrupt governments. This article studies the extent of bureaucrats’ heterogeneity in attitude to the problem of corruption. Due to the sensitivity of direct questions on corruption, we conduct the list experiment among public procurement officials in Russia. We show that female bureaucrats consider corruption an obstacle to public procurement development, and find no such evidence for male bureaucrats. This heterogeneity holds even at the high-level occupied positions. Although the negative attitude to corruption does not necessarily imply the anti-corruption activity by women, recognition of the problem seems to be a prerequisite for supporting an anti-corruption policy.

Ideology and compliance with health guidelines during the COVID-19 pandemic: A comparative perspective.

ObjectiveWe measure the prevalence of noncompliance with public health guidelines in the COVID‐19 pandemic and examine how it is shaped by political ideology across countries.MethodsA list experiment of noncompliance and a multi‐item scale of health‐related behaviors were embedded in a comparative survey of 11,000 respondents in nine OCED countries. We conduct a statistical analysis of the list experiment capturing degrees of noncompliance with social distancing rules and estimate ideological effect heterogeneity. A semiparametric analysis examines the functional form of the relationship between ideology and the propensity to violate public health guidelines.ResultsOur analyses reveal substantial heterogeneity between countries. Ideology plays an outsized role in the United States. No association of comparable magnitude is found in the majority of the other countries in our study. In many settings, the impact of ideology on health‐related behaviors is nonlinear.ConclusionOur results highlight the importance of taking a comparative perspective. Extrapolating the role of ideology from the United States to other advanced industrialized societies might paint an erroneous picture of the scope of possible nonpharmaceutical interventions. Heterogeneity limits the extent to which policymakers can learn from experiences across borders.

Profiling racial prejudice during COVID-19: Who exhibits anti-Asian sentiment in Australia and the United States?

Following the COVID‐19 outbreak, anti‐Asian racism increased around the world, as exhibited through greater instances of abuse and hate crimes. To better understand the scale of anti‐Asian racism and the characteristics of people who may be expressing racial prejudice, we sampled respondents in Australia and the United States over 31 August–9 September 2020 (1375 Australians and 1060 Americans aged 18 or above; source YouGov). To address potential social desirability bias, we use both direct and indirect (list experiment) questions to measure anti‐Asian sentiment and link these variables to key socioeconomic factors. We find that, instead of being universal among general populations, anti‐Asian sentiment is patterned differently across both country contexts and socioeconomic groups. In the United States, the most significant predictor of anti‐Asian bias is political affiliation. By contrast, in Australia, anti‐Asian bias is closely linked to a wide range of socioeconomic factors including political affiliation, age, gender, employment status and income.

Do Americans perceive diverse judges as inherently biased?

ABSTRACT Although women and minorities hold an increasing share of judgships in the United States, they remain underrepresented. We explore Americans’ perceptions of the bias of women and minority judges – one of the possible challenges to creating a diverse bench. We argue that prejudice against these groups manifests in a subtle way, in the belief that diverse judges cannot fairly adjudicate controversies that involve their ingroup. To test our theory, we use a list experiment specifically developed to minimize social desirability effects. We find that many respondents rate female and Hispanic judges to be biased decision makers. Our results highlight the nature of prejudice against female and Hispanic judges and suggest multiple important implications. They shed light on the reasons why female and Hispanic candidates for judgships may win at a lower rate and also suggest negative implications for the legitimacy of their decisions.

Prejudice, Political Ideology, and Interest: Understanding Attitudes Toward Affirmative Action in Brazil

Few public policies have been as consequential or divisive as affirmative action. Proponents have argued for the need for equity and the redress of past and present discrimination, whereas opponents enlist claims over individual liberty and merit. Scholars have examined support to affirmative action, asking to what extent citizens' support is shaped by their political ideology, interest, prejudice, or some combination thereof. Much work to date has focused on the United States, where disentangling theoretical explanations has proved challenging. We turn our attention to an understudied but important case: Brazil. Brazil has implemented a broad form of affirmative action for admission to federal universities that include consideration of the applicant's education, income, and race. Adopting both a conventional question and a list experiment embedded in a face‐to‐face survey among a nationally representative sample of adult Brazilians, we find that public support for affirmative action suffers from social desirability bias, and in our subsequent regression analysis, that attitudes about affirmative action are structured especially by individuals' interests.

Can youth empowerment programs reduce violence against girls during the COVID-19 pandemic?

This paper shows that a youth empowerment program in Bolivia reduced the reported prevalence of violence against girls during the COVID-19 lockdown. The program offered training in soft skills and technical skills, sexual education, mentoring and job-finding assistance. To measure the effects of the program, the study conducted a randomized control trial with 600 vulnerable adolescents. Results indicate that 7 months after its completion, the program increased girls’ earnings and decreased violence against girls. Violence was measured with both direct self-report questions and list experiments. These findings suggest that multi-faceted empowerment programs can reduce the level of violence experienced by young women during high-risk periods.

The Effect of a Wordless, Animated, Social Media Video Intervention on COVID-19 Prevention: Online Randomized Controlled Trial.

BackgroundInnovative approaches to the dissemination of evidence-based COVID-19 health messages are urgently needed to counter social media misinformation about the pandemic. To this end, we designed a short, wordless, animated global health communication video (the CoVideo), which was rapidly distributed through social media channels to an international audience.ObjectiveThe objectives of this study were to (1) establish the CoVideo’s effectiveness in improving COVID-19 prevention knowledge, and (2) establish the CoVideo’s effectiveness in increasing behavioral intent toward COVID-19 prevention.MethodsIn May and June 2020, we enrolled 15,163 online participants from the United States, Mexico, the United Kingdom, Germany, and Spain. We randomized participants to (1) the CoVideo arm, (2) an attention placebo control (APC) arm, and (3) a do-nothing arm, and presented 18 knowledge questions about preventive COVID-19 behaviors, which was our first primary endpoint. To measure behavioral intent, our second primary endpoint, we randomized participants in each arm to five list experiments.ResultsGlobally, the video intervention was viewed 1.2 million times within the first 10 days of its release and more than 15 million times within the first 4 months. Knowledge in the CoVideo arm was significantly higher (mean 16.95, 95% CI 16.91-16.99) than in the do-nothing (mean 16.86, 95% CI 16.83-16.90; P<.001) arm. We observed high baseline levels of behavioral intent to perform many of the preventive behaviors featured in the video intervention. We were only able to detect a statistically significant impact of the CoVideo on one of the five preventive behaviors.ConclusionsDespite high baseline levels, the intervention was effective at boosting knowledge of COVID-19 prevention. We were only able to capture a measurable change in behavioral intent toward one of the five COVID-19 preventive behaviors examined in this study. The global reach of this health communication intervention and the high voluntary engagement of trial participants highlight several innovative features that could inform the design and dissemination of public health messages. Short, wordless, animated videos, distributed by health authorities via social media, may be an effective pathway for rapid global health communication during health crises.Trial RegistrationGerman Clinical Trials Register DRKS00021582; https://tinyurl.com/6r4zkbbnInternational Registered Report Identifier (IRRID)RR2-10.1186/s13063-020-04942-7

Gender heterogeneity of bureaucrats in attitude to corruption: Evidence from list experiment

A high level of corruption usually constrains economic development in emerging countries. However, anti-corruption campaigns often fail because the relevant policies need to be implemented by existing corrupt governments. This article studies the extent of bureaucrats’ heterogeneity in attitude to the problem of corruption. Due to the sensitivity of direct questions on corruption, we conduct the list experiment among public procurement officials in Russia. We show that female bureaucrats consider corruption an obstacle to public procurement development, and find no such evidence for male bureaucrats. This heterogeneity holds even at the high-level occupied positions. Although the negative attitude to corruption does not necessarily imply the anti-corruption activity by women, recognition of the problem seems to be a prerequisite for supporting an anti-corruption policy.

Current surveys may underestimate climate change skepticism evidence from list experiments in Germany and the USA.

Strong public support is a prerequisite for ambitious and thus costly climate change mitigation policy, and strong public concern over climate change is a prerequisite for policy support. Why, then, do most public opinion surveys indicate rather high levels of concern and rather strong policy support, while de facto mitigation efforts in most countries remain far from ambitious? One possibility is that survey measures for public concern fail to fully reveal the true attitudes of citizens due to social desirability bias. In this paper, we implemented list-experiments in representative surveys in Germany and the United States (N = 3620 and 3640 respectively) to assess such potential bias. We find evidence that people systematically misreport, that is, understate their disbelief in human caused climate change. This misreporting is particularly strong amongst politically relevant subgroups. Individuals in the top 20% of the income distribution in the United States and supporters of conservative parties in Germany exhibit significantly higher climate change skepticism according to the list experiment, relative to conventional measures. While this does not definitively mean that climate skepticism is a widespread phenomenon in these countries, it does suggest that future research should reconsider how climate change concern is measured, and what subgroups of the population are more susceptible to misreporting and why. Our findings imply that public support for ambitious climate policy may be weaker than existing survey research suggests.

Can you repeat the question? Paradata as a lens to understand respondent experience answering cognitively demanding, sensitive questions.

Survey researchers hope that respondents will provide high-quality data, but evidence suggests that social desirability bias may be commonplace. Social desirability can lead to significant underreporting or overreporting of sensitive behaviors. With better understanding of the cognitive processes that respondents use to prepare and deliver their responses, survey designers could hope to minimize social desirability bias or at least detect settings that lessen its impact. The primary objective of this study was to use survey paradata to understand the psychology of responding to certain types of survey questions. More specifically, we sought to determine how emotional triggering can alter response latencies to cognitively demanding and sensitive survey questions on induced abortion, which is underreported. We hypothesize that having had a prior abortion might lengthen response times to an indirect question about abortion among respondents who have experienced this sensitive reproductive outcome as they hesitate in deciding whether and how to respond to the question. Data come from a representative survey of 6,035 reproductive age women in Rajasthan, India. We used list experiment question active screen time paradata in conjunction with responses from direct questions on abortion to assess our hypothesis. Our final model was a multivariate linear regression with random effects at the level of the interviewer, including adjustments for respondent, community, and interviewer characteristics to estimate within-respondent effects. Results suggest that women who reported an abortion on the direct abortion questions took 5.11 (95% CI 0.21, 10.00) seconds longer to respond to the list experiment treatment list compared to the control list in comparison to women who did not report an abortion on the direct abortion questions. This study demonstrates the additional insights gained when focusing on response latencies to cognitively demanding questions involved in the measurement of sensitive behaviors.