List Experiment Research Articles

A Comprehensive Guide to Assessing and Adjusting for Social Desirability Bias Through Advanced Multivariate Analysis using R: A Case Study on Sports Fan Surveys

ABSTRACT The purpose of the current paper is to provide a step-by-step tutorial for employing the list experiment to control for Social Desirability Bias (SDB). Sport management scholars routinely rely on self-report data in cross-sectional settings to test research hypotheses. However, concerns exist regarding the use of self-reports without adjusting for sources of systematic misreporting such as SDB. This study offers a walkthrough guideline using the R programming language to utilize the novel method called “multivariate list experiment” to control for SDB, using Black athletes’ kneeling protests during the national anthem on the playing field as an example. All materials, including the dataset, R codes, and markdown, are provided for replicating the results.

Measuring the Prevalence of Earnings Manipulations: A Novel Approach

ABSTRACTWe provide prevalence estimates for five forms of earnings manipulation based on executives’ reports about their firms’ actual reporting practices. After preregistering our methods and analyses via the Journal of Accounting Research’s registration‐based editorial process, we recruit nearly a thousand executives from firms listed in the Russell 3000 Index to participate in either a survey or a list experiment; the hallmark of the latter being additional privacy protections designed to promote honest disclosure about self‐incriminating information. In our survey, 26.8% of executives disclose at least one form of earnings manipulation at their firm in the 2018–2023 period: 18.0% report changing an operational activity to meet a near‐term earnings target at the expense of long‐term value (i.e., real earnings management), 8.8% report intentionally obfuscating unfavorable information, 6.6% report manipulating accruals, 3.9% report withholding material information, and 0.0% report accounting fraud. Our list experiment produces an economically higher result in two cases, estimating that 29.9% of firms engaged in real earnings management and 12.4% committed accounting fraud over the same time period. We conclude that while a traditional survey can provide credible lower‐bound estimates for the prevalence of many forms of earnings manipulation, list experiments encourage more honest disclosure in some cases.

Do middle-aged and older people underreport loneliness? experimental evidence from the Netherlands

Despite the growing acknowledgment of the importance of loneliness among older individuals, questionnaire length constraints may hinder the inclusion of common multi-item loneliness scales in surveys. Direct, single-item loneliness measures are a practical alternative, but scholars have expressed concerns that such measures may lead to underreporting. Our aim was to test whether such reservations are justified. We conducted a preregistered list experiment among 2,553 people aged 50 + who participated in the Dutch Longitudinal Internet studies for the Social Sciences (LISS) panel. The list experiment method has been developed to unobtrusively gather sensitive information. We compared the list experiment estimate of the prevalence of frequent loneliness with the corresponding direct question estimate to assess downward bias in the latter. Next to pooled models, we estimated models stratified by gender to assess whether loneliness underreporting differed between women and men. Relying on the direct question, we estimated that 5.9% of respondents frequently felt lonely. Our list experiment indicated that the prevalence of frequent loneliness was 13.1%. Although substantial in magnitude, the difference between both estimates was only marginally significant (Δb: 0.072, 95% CI: − 0.003;0.148, p = .06). No evidence of gender differences was found. Although we cannot be conclusive that loneliness estimates are biased downward when a direct question is used, our results call for caution with direct, single-item measures of loneliness if researchers want to avoid underreporting. Replications are needed to gain more precise insights into the extent to which direct, single-item loneliness measures are prone to downward reporting bias.

Merit recruitment, professional advancement opportunities and prosocial rule-breaking among public servants in Greece

Abstract Public servants have a significant impact on peoples’ lives, but we don’t have many reliable estimates of how many employees engage in prosocial rule-breaking (PSRB), a form of constructive deviance. We collected original survey data (N = 497) among a representative sample of Greek public servants and implemented a list experiment to gauge how pervasive PSRB is in Greece’s public sector. Greece is a particularly useful setting in which to study PSRB as the euro crisis created strong reform pressure. We find that public servants who were hired via merit competitions are not less likely to break rules, but this is conditional on their beliefs about career prospects and the fairness of the promotion system. They perceive the professional reward system as biased towards those with political connections and compensate for lack of efficiency by doing favours. This finding raises concerns about the quality of EU reform assessments.

Lifetime abortion incidence when abortion care is covered by Medicaid: Maryland versus five comparison states.

To estimate the association of Medicaid coverage of abortion care with cumulative lifetime abortion incidence among women insured by Medicaid. We use 2016-2019 (Pre-Dobbs) data from the Survey of Women studies that represent women aged 18-44 living in six U.S. states. One state, Maryland, has a Medicaid program that has long covered the cost of abortion care. The other five states, Alabama, Delaware, Iowa, Ohio, and South Carolina, have Medicaid programs that do not cover the cost of abortion care. Our sample includes 8972 women residing in the study states. Our outcome, cumulative lifetime abortion incidence, is identified using an indirect survey method, the double list experiment. We use a multivariate regression of cumulative lifetime abortion on variables including whether women were Medicaid-insured and whether they were residing in Maryland versus in one of the other five states. This study used secondary survey data. We estimate that Medicaid coverage of abortion care in Maryland is associated with a 37.0 percentage-point (95% CI: 12.3-61.4) higher cumulative lifetime abortion incidence among Medicaid-insured women relative to women not insured by Medicaid compared with those differences by insurance status in states whose Medicaid programs do not cover the cost of abortion care. We found that Medicaid coverage of abortion care is associated with a much higher lifetime incidence of abortion among individuals insured by Medicaid. We infer that Medicaid coverage of abortion care costs may have a very large impact on the accessibility of abortion care for low-income women.

The Swiss common garden network: testing assisted migration of tree species in Europe

A warmer climate with drier summers will affect the suitability of tree species in their current range in most of Europe. To preserve ecosystem services in the future, many European countries are looking for tree species adapted to the expected future climate and are setting up trials to test them in different environments. In collaboration with forest practitioners and federal and regional authorities, we have established a network of 57 common gardens across large environmental gradients in Switzerland. Over a period of 30 to 50 years, the vitality, growth and survival of 18 tree species from 117 seed sources will be monitored to develop tree species recommendations for forest managers. In this article, we outline the considerations, challenges and trade-offs involved in designing this experiment, as well as the participatory process with a variety of stakeholders, from local foresters to the Federal Office for the Environment. Further, we list experiments testing multiple species on multiple sites in Europe and compare experimental designs, tree species and environmental gradients. The Swiss common garden network complements other European experiments and broadens the potential network by extending the covered environmental gradients to colder and moister conditions. Such targeted assisted migration trials are key to promoting tree species that can cope with the future climate and can help us to foster forest ecosystems that can adapt to rapidly changing climates.

Physician behavior: Experimental evidence from physician and patient perspectives

Physician motives remain a hotly debated and sensitive topic. Given the sensitive nature of this issue, direct questions may elicit either dishonest responses or no response at all. To mitigate this risk, we carried out two list experiments between November 2022 and February 2023 to examine two important physician behaviors from both the physician and patient perspectives. In these experiments, participants in the control and treatment groups were shown a list of non-sensitive statements. Additionally, each treatment group received a sensitive statement related to either demand inducement or demand enablement behavior. Participants were then asked to report only the number of statements with which they agreed. The difference in the mean number of statements agreed upon between the control and treatment groups revealed the prevalence rates of sensitive behaviors. In the physician experiment, we observed a 25 % prevalence rate (standard error (SE) = 0.127, p = 0.046) for demand inducement behavior and a 65 % rate (SE = 0.125, p < 0.001) for demand enablement behavior. In the patient experiment, compared to the control group, 52 % of participants (SE = 0.074, p < 0.001) perceived that physicians exhibit demand inducement behavior, while 39 % (SE = 0.073, p < 0.001) demonstrated behavior consistent with demand enablement. These findings indicate that physicians are more prone to offer unnecessary medical services when patients actively participate, underscoring the significant influence of patients on physician behavior.

Survey Experiment on Political Use of Force: The Case of Operation Guardian of the Walls in May 2021

ABSTRACT This study uses an online survey to conduct a list experiment to determine whether the military conflict between Hamas and Israel (mid-May 2021) – the so-called Operation Guardian of the Walls – should be considered political use of force by Prime Minister Netanyahu to preserve his threatened political career and life. Our study analyzed the experimental data and contributed to the studies on the political use of force from the perspective of political psychology. The result revealed that citizens with perceptions of political use of force rated Netanyahu’s war leadership more highly than citizens without such perceptions.

When Push Comes to Shove: An Experimental Analysis of Voter Support of a Woman President and the 2024 Nomination

A woman has not yet shattered the “hardest, highest glass ceiling” of the American presidency. Our research answers two questions: Which groups are more likely to believe electing a woman president to be historically important? (R1), and When a presidential election is at stake, who is likely to support a woman candidate? (R2). Using observational data ( n = 1075), our findings indicate that women, people who recognize sexism within politics, Democrats, and liberals are more likely to view a woman president as historic. Utilizing a list experiment of hypothetical 2024 presidential matchups, few who claimed to view a woman president as historic were willing to cast a vote in their favor. When push came to shove, Democratic women were the group most likely to vote for a woman presidential candidate. As parties look toward the future, this study offers insight into how voters respond to potential nominees and who parties will nominate.

Lip service to liberal democracy in Western Europe?

Abstract Political scientists heavily rely on standard survey questions referring to “democracy” when they study citizens’ attitudes toward (liberal) democracy. However, we only know little about the way in which citizens respond to these questions. This article focuses on two frequently highlighted issues: social desirability and the consistency between citizens’ understanding and researchers’ understanding of the term “democracy.” To address these issues, I collected novel survey data via YouGov from 14,000 British, French, German, and Italian respondents. I use a list experiment to show that respondents do not feel socially pressured to misreport their support for democracy. However, what citizens have in mind when they claim to support democracy only reflects norms and institutions of minimal conceptions of democracy. Overall, this encourages the usage of questions regarding citizens’ support for democracy widely, although this should not be interpreted as the support for anything going beyond minimal conceptions of democracy (providing freedom and allowing for citizens’ influence on political decisions).

The gendered costs of stigma: How experiences of conflict‐related sexual violence affect civic engagement for women and men

AbstractA common understanding emphasizes the destructive effects of conflict‐related sexual violence (CRSV) on social cohesion and community life. Stressing the agency of survivors, we present an alternative argument. Our theory predicts that survivors seek to counteract the stigma attached to CRSV by contributing to the community in the form of civic engagement. Drawing on three original surveys from Democratic Republic of Congo (DRC), Liberia, and Sri Lanka and relying on list experiments to reduce underreporting bias, we find that survivors of CRSV indeed show increased levels of civic engagement. This civic effect is consistent across the three contexts and very likely causal. We also rule out an alternative mechanism based on posttraumatic growth and dispel concerns that increased civic engagement comes at the expense of decreased intergroup relations. However, looking at sex differences, our results are more sobering. While in line with our prediction, they do not support the optimistic notion that survivors' mobilization results in female empowerment and the closing of existing gender gaps in civic behavior. Our findings have important implications for our understanding of CRSV, the legacy of violent conflict, and the gendered nature of politics.

Understanding Labor Market Discrimination Against Transgender People: Evidence from a Double List Experiment and a Survey

Using a double list experiment designed to elicit views free from social desirability bias, we find that support in the United States for transgender people in the labor market is significantly overreported by 8%–10%. After correcting for this overreporting, we still find that over two-thirds of respondents would be comfortable with a transgender manager and support employment nondiscrimination protection for transgender people. However, respondents severely underestimate this level of support. We also show that stated labor market support for transgender people is lower than support for gay, lesbian, and bisexual people. Our results advance our understanding of employment discrimination against transgender people. This paper was accepted by Marie Claire Villeval, behavioral economics and decision analysis. Funding: Financial support from The Vanderbilt LGBTQ+ Policy Lab. is gratefully acknowledged. Supplemental Material: The online appendix and data files are available at https://doi.org/10.1287/mnsc.2023.02567 .

Conviction of Multiparty Competition and Elections among Chinese College Students: A List Experiment Study

Conviction of Multiparty Competition and Elections among Chinese College Students: A List Experiment Study

Child online safety education: A program evaluation combining a randomized controlled trial and list experiments in Vietnam

With the rapid expansion of internet accessibility, children urgently need to be educated in online safety skills. We conducted a randomized controlled trial (RCT) with 1399 students from 22 junior high schools in Vietnam to assess the impact of an online safety program on their online behaviors. We designed our experiment to capture both the direct and spillover effects of the training program. We combined the RCT with list experiments, an indirect questioning method, to mitigate bias in the measurement of sensitive outcomes. Our impact estimates show the positive effects of the program. Specifically, participation in the program reduced students' identity disclosure on social networking sites (SNSs) by 9.3–16.8 percentage points, increased their online account security practices by 8.9–17.9 percentage points, and decreased their interactions with strangers online by 13.6–23.4 percentage points. Notably, using outcomes measured by the list experiments, we show that the program reduced sensitive and risky online behaviors by 34.7–47.0 percentage points, a result that would not have been revealed with conventional direct questioning. We also observed evidence of spillover effects on nonparticipants in the same schools. Our findings support the implementation of educational programs to enhance children's online safety.

Self-restraint behavior and partisanship during the COVID-19 pandemic: evidence from a list experiment in Japan

Abstract During the COVID-19 pandemic, lockdowns were considered a critical measure to control the spread of the virus. While Europe and the United States implemented strict lockdowns that restricted movement, Japan adopted a soft lockdown approach which relied on voluntary compliance without any legal enforcement. As a result, some citizens (especially young people) did not comply with the government’s request for voluntary restraint, and were frequently criticized in the media. To address the question of who did not comply with the government’s soft lockdown and why, this study analyzes a list experiment which measured participation in non-essential activities during Japan’s third COVID-19 wave, when economic stimulus policies and COVID-19 suppression measures were simultaneously implemented. The results showed that in particular young citizens who strongly disliked the ruling party did not comply with the government’s requests to stay at home. It is suggested that during a national crisis, the success of policies based on voluntary cooperation depends on citizens’ pre-crisis levels of trust in the government and the ruling parties.

Teacher's social desirability bias and Migrant students: A study on explicit and implicit prejudices with a list experiment

Scholarly research has consistently shown that teachers present negative assessments of and attitudes toward migrant students. However, previous studies have not clearly addressed the distinction between implicit and explicit prejudices, or identified their underlying sources. This study identifies the explicit and implicit prejudices held by elementary and middle school teachers regarding the learning abilities of an ethnic minority group: Haitian students within the Chilean educational system. We use a list experiment to assess how social desirability and intergroup attitudes toward minority students influence teachers' prejudices. The findings reveal that teachers harbor implicit prejudices towards Haitian students and are truthful in reporting their attitudes, thereby contradicting the desirability bias hypothesis. We suggest that teachers rely on stereotypes associated with the students' nationality when assessing Haitian students’ learning abilities. The implications of these results are discussed in relation to theories grounded in stereotypes and intergroup attitudes.

Unmasking academic cheating behavior in the artificial intelligence era: Evidence from Vietnamese undergraduates

The proliferation of artificial intelligence (AI) technology has brought both innovative opportunities and unprecedented challenges to the education sector. Although AI makes education more accessible and efficient, the intentional misuse of AI chatbots in facilitating academic cheating has become a growing concern. By using the indirect questioning technique via a list experiment to minimize social desirability bias, this research contributes to the ongoing dialog on academic integrity in the era of AI. Our findings reveal that students conceal AI-powered academic cheating behaviors when directly questioned, as the prevalence of cheaters observed via list experiments is almost threefold the prevalence of cheaters observed via the basic direct questioning approach. Interestingly, our subsample analysis shows that AI-powered academic cheating behaviors differ significantly across genders and grades, as higher-grade female students are more likely to cheat than newly enrolled female students. Conversely, male students consistently engage in academic cheating throughout all grades. Furthermore, we discuss potential reasons for the heterogeneous effects in academic cheating behavior among students such as gender disparity, academic-related pressure, and peer effects. Implications are also suggested for educational institutions to promote innovative approaches that harness the benefits of AI technologies while safeguarding academic integrity.

Using a list experiment to measure intimate partner violence: cautionary evidence from Ethiopia

ABSTRACT While indirect response methods are increasingly used in surveys to measure sensitive behaviours such as intimate partner violence, important questions persist around how respondents understand and react to these methods. This article presents evidence from a list experiment measuring multiple forms of intimate partner violence in rural Ethiopia. We find that the list experiment does not generate estimates of intimate partner violence that are higher than direct response questions; rather, prevalence estimates using the list experiment are lower vis-à-vis prevalence estimates using the direct reports, and are sometimes even negative. We provide suggestive evidence that this pattern may reflect ‘fleeing’ behaviour by respondents who do not wish to be associated with statements indicating their exposure to intimate partner violence.

Do Chinese Citizens Conceal Opposition to the CCP in Surveys? Evidence from Two Experiments

Abstract Most public opinion research in China uses direct questions to measure support for the Chinese Communist Party (CCP) and government policies. These direct question surveys routinely find that over 90 per cent of Chinese citizens support the government. From this, scholars conclude that the CCP enjoys genuine legitimacy. In this paper, we present results from two survey experiments in contemporary China that make clear that citizens conceal their opposition to the CCP for fear of repression. When respondents are asked directly, we find, like other scholars, approval ratings for the CCP that exceed 90 per cent. When respondents are asked in the form of list experiments, which confer a greater sense of anonymity, CCP support hovers between 50 per cent and 70 per cent. This represents an upper bound, however, since list experiments may not fully mitigate incentives for preference falsification. The list experiments also suggest that fear of government repression discourages some 40 per cent of Chinese citizens from participating in anti-regime protests. Most broadly, this paper suggests that scholars should stop using direct question surveys to measure political opinions in China.

A New Era of Political Research: The 'List Experiment' Method and Its Impact on the Study of Central Asia

A New Era of Political Research: The 'List Experiment' Method and Its Impact on the Study of Central Asia