Randomized Field Experiment Research Articles

Can Facebook Ads and Email Messages Increase Fiscal Capacity? Experimental Evidence from Venezuela

Tax compliance is a big challenge in weakly institutionalized countries, in which citizens do not trust the State and prefer to evade. Can compliance be increased, and hence fiscal state capacity strengthened, using online strategies? We perform a randomized field experiment in Venezuela's capital, Caracas, to determine if email messages, Facebook ads, or a combination of both, can increase tax compliance. By varying the mechanism used to approach taxpayers, we are able to test if more direct and personalized methods, such as email messages, are more effective than general advertisement tools, such as Facebook ads. Moreover, our design allows us to test if both strategies complement each other and boost the capacity of a local government to increase compliance. Preliminary results suggest that these strategies are cost-effective methods for increasing tax revenues, even in a context of low confidence in the State, as is the case of Venezuela nowadays.

Incentive Misalignments in Programmatic Advertising: Evidence from a Randomized Field Experiment

In programmatic advertising, firms outsource the bidding for ad impressions to ad platforms. Although firms are interested in targeting consumers that respond positively to advertising, ad platforms are usually rewarded for targeting consumers with high overall purchase probability. We develop a theoretical model that shows if consumers with high baseline purchase probability respond more positively to advertising, then firms and ad platforms agree on which consumers to target. If, conversely, consumers with low baseline purchase probability are the ones for which ads work best, then ad platforms target consumers that firms do not want to target—the incentives are misaligned. We conduct a large-scale randomized field experiment, targeting 208,538 individual consumers, in a display retargeting campaign. Our unique data set allows us to both causally identify advertising effectiveness and estimate the degree of incentive misalignments between the firm and ad platform. In accordance with the contracted incentives, the ad platform targets consumers that are more likely to purchase. Importantly, we find no evidence that ads are more effective for consumers with higher baseline purchase probability, rendering the ad platform’s bidding suboptimal for the firm. A welfare analysis suggests that the ad platform’s bidding optimization leads to a loss in profit for the firm and an overall decline in welfare. To remedy the incentive misalignment, we propose a solution in which the firm restricts the ad platform to target only consumers that are profitable based on individual consumer-level estimates for baseline purchase probability and ad effectiveness. This paper was accepted by Anandhi Bharadwaj, information systems. Funding: This work was funded by Vereniging Trustfonds Erasmus Universiteit Rotterdam [Grant 97090/17.16/741/oe], Fundacão para a Ciência e a Tecnologia [UID/ECO/00124/2019, UIDB/00124/2020, UIDP/00124/2020, and Social Sciences DataLab - PINFRA/22209/2016], POR Lisboa, and POR Norte [Social Sciences DataLab, PINFRA/22209/2016]. Supplemental Material: The online appendices are available at https://doi.org/10.1287/mnsc.2022.4438 .

Social Capital Accumulation through Social Media Networks: Evidence from a Randomized Field Experiment and Individual-Level Panel Data

Work-related social media networks (SMNs) like LinkedIn introduce novel networking opportunities and features that promise to help individuals establish, extend, and maintain social capital (SC). Typically, work-related SMNs offer access to advanced networking features exclusively to premium users in order to encourage basic users to become paying members. Yet little is known about whether access to these advanced networking features has a causal impact on the accumulation of SC. To close this research gap, we conducted a randomized field experiment and recruited 215 freelancers in a freemium, work-related SMN. Of these recruited participants, more than 70 received a randomly assigned voucher for a free 12- month premium membership. We observe that individuals do not necessarily accumulate more SC from their ability to access advanced networking features, as the treated freelancers did not automatically change their online networking engagement. Those features only reveal their full utility if individuals are motivated to proactively engage in networking. We found that freelancers who had access to advanced networking features increased their SC by 4.609% for each unit increase on the strategic networking behavior scale. We confirmed this finding in another study utilizing a second, individual-level panel dataset covering 52,392 freelancers. We also investigated the dynamics that active vs. passive features play in SC accumulation. Based on these findings, we introduce the “theory of purposeful feature utilization”: essentially, individuals must not only possess an efficacious “networking weapon”—they also need the intent to “shoot” it.

A Randomized Field Experiment to Explore the Impact of Herding Cues as Catalysts for Adoption

A herding cue is a lean information signal that an individual receives about the aggregate number of others who have engaged in a behavior that may result in herd behavior. Given the ease with which they can be leveraged as implementation interventions or design features on online sites, herding cues hold the promise to provide a means to influence adoption behaviors. Yet, little attention has been devoted in the IS adoption literature to understanding the effects of herding cues. Given that herding cues are just one of several forms of social influence on adoption behaviors and are relatively lean in nature, understanding their viability as an implementation intervention necessitates understanding their effects in the presence of (1) other forms of social influence, which also serve to reduce uncertainty and signal the appropriateness of technology adoption, and (2) an individual’s own beliefs about adopting. In this vein, we conducted a randomized field experiment to examine the use of a herding cue as an implementation intervention to hasten adoption behaviors. The research model was evaluated using survival analysis by combining the data from the field experiment with two waves of surveys, and archival logs of adoption. Our results show that a herding cue (1) directly impacts the time it takes an individual to adopt a technology, (2) amplifies the effects of peer behaviors (another type of informative social influence), but has no impact on the effect of subjective norm (a form of normative social influence), and (3) dampens the effects of an individual’s private beliefs about the usefulness of a technology. Our paper disentangles herding information signals to define a herding cue as distinct from other herd behavior triggers, explores how it may interact with other forms of social influences and private beliefs to influence adoption behaviors, and, on a practical level, provides evidence of how a herding cue can be a tangible intervention to accelerate technology adoption.

Economic corollaries of personalized recommendations

The impact of recommendation systems (RSs) on the diversity of consumption is not transparent or well understood. Available studies, whether experimental or theoretical, show inconsistent and even opposite results, which manifests as debate in the literature. In this paper, we investigate the impact of two main recommender systems, neural collaborative filtering and deep content filtering, on sales diversity via a randomized field experiment. Our results confirm the capability of recommender engines in increasing or decreasing aggregate sales diversity. Nonetheless, they amplify homogenization and reduce individual-level consumption diversity. In conclusion, our research reconciles seemingly contradict previous findings and illustrates that the design of the RS is the decisive factor in homogenizing or diversifying product sales.

Sex Workers, Stigma, and Self-Image: Evidence from Kolkata Brothels

AbstractThis paper studies the link between self-image and behavior among those who face stigma due to poverty and social exclusion. Using a randomized field experiment with sex workers in Kolkata (India), we examine whether a psychological intervention to mitigate adverse effects of internalized stigma can induce behavior change. We find significant improvements in participants' self-image, their savings choices, and health clinic visits. Administrative data confirm that these changes in savings and preventive health behavior persist 15 and 21 months later, respectively. Our findings highlight the potential of purely psychological interventions to improve the life choices and outcomes of marginalized groups.

News credibility labels have limited average effects on news diet quality and fail to reduce misperceptions.

As the primary arena for viral misinformation shifts toward transnational threats, the search continues for scalable countermeasures compatible with principles of transparency and free expression. We conducted a randomized field experiment evaluating the impact of source credibility labels embedded in users’ social feeds and search results pages. By combining representative surveys (n = 3337) and digital trace data (n = 968) from a subset of respondents, we provide a rare ecologically valid test of such an intervention on both attitudes and behavior. On average across the sample, we are unable to detect changes in real-world consumption of news from low-quality sources after 3 weeks. We can also rule out small effects on perceived accuracy of popular misinformation spread about the Black Lives Matter movement and coronavirus disease 2019. However, we present suggestive evidence of a substantively meaningful increase in news diet quality among the heaviest consumers of misinformation. We discuss the implications of our findings for scholars and practitioners.

Active learning improves financial education: Experimental evidence from Uganda

We conduct a randomized field experiment to study the effects of two financial education interventions offered to small-scale retailers in rural western Uganda. The treatments contrast “active learning” with traditional “lecturing” within standardized lesson-plans. After six months, active learning has a positive effect on savings and investment outcomes, in contrast to small or zero effects for lecturing. After four years, estimates come with substantial uncertainty but are generally larger for the active learning group, such as a 60 percent increase in investments. As an adverse outcome, reported late payment on loans increases by about 30 percent for both treatments. The findings suggest that teaching methods can play an important role in affecting how financial education programs impact financial behavior and outcomes.

Effects of dialogue meetings on sickness absence—Evidence from a large field experiment

Sickness absence entails large individual and societal costs. Dialogue Meetings (DMs) where the absentee, the employer, and the physician discuss arrangements for full or partial work resumption have been in place in Norway since 2007. In collaboration with the Labour and Welfare Administration, we conducted a large-scale, pre-registered, randomized field experiment to evaluate aspects of the Norwegian DMs policy. We do not find statistically significant effects of summoning to a meeting and we can reject even small threat (notification) effects of sending out letters. We also conduct an extensive search for heterogeneous treatment effects but find no evidence of these.

Mentoring as a grassroots effort for integrating refugees – evidence from a randomised field experiment

ABSTRACT About 80 million people were displaced worldwide at the end of 2020. To support this highly vulnerable group, in recent years, local bottom-up initiatives proliferated to support refugee integration in hosting communities. This study examines a network intervention for refugees in collaboration with a social start-up whose mission is to match refugees and local volunteers to form friendships. We apply an innovative randomised controlled trial approach with 446 participants integrated into a survey of almost 8000 randomly sampled refugees who moved to Germany between 2013 and 2016. Despite the field experimental study design, statistical imbalances between treatment and control groups arise in the process of enrolment and matching up to the re-interview approximately one year after recruitment, which we address using propensity score weighting. Out of 85 successfully matched individuals, for the 30 refugees with the highest intensity of the intervention we find positive treatment effects on social connectedness, housing satisfaction, and, although less robust, German language proficiency. Thus, a general-purpose mentoring program can promote subjective integration. Effects on objective indicators, such as employment, may only indirectly come about in the longer run.

Social Referral Programs for Freemium Platforms

We examine how freemium platforms can design social referral programs to encourage growth and engagement without sacrificing revenue. On the one hand, social referral programs generate new referrals from users who would not have paid for the premium features. On the other hand, they also attract new referrals from users who would have paid but prefer to invite others, resulting in more referrals but fewer paying users. We use data from a large-scale randomized field experiment in an online dating platform to assess the effects of adding referrals programs to freemium platforms and changing the referral requirements on users’ behavior, namely, on their decisions to invite, pay, and engage with the platform. We find that introducing referral programs in freemium platforms can significantly contribute to increasing the number of referrals at the expense of revenue. Platforms can avoid the loss in revenue by reserving some premium features exclusively for paying users. We also find that increasing referral requirements in social referral programs can work as a double-edged sword. Increasing the referral threshold results in more referrals and higher total revenue. Yet these benefits appear to come at a cost. Users become less engaged, decreasing the value of the platform for all users. We explore two mechanisms that help to explain the differences in users’ social engagement. Finally, and contrary to prior findings, we find that the quality of the referrals is not affected by the referral requirements. We discuss the theoretical and practical implications of our research. This paper was accepted by Chris Forman, information systems. Funding: This work was funded by Fundac¸ão para a Ciência e a Tecnologia [UID/ ECO/00124/2019, UIDB/00124/2020, UIDP/00124/2020, and Social Sciences DataLab - PINFRA/22209/2016], POR Lisboa, and POR Norte [Social Sciences DataLab, PINFRA/22209/2016]. Supplemental Material: The online appendix and data are available at https://doi.org/10.1287/mnsc.2022.4301 .

Do institutional pressures increase reactive transparency of government? Evidence from a field experiment

ABSTRACT This study aims to contribute to the literature on government reactive transparency based on the new institutional theory by testing the influences of regulative pressure and mimetic pressure on government compliance with information requests from the public. Through a randomized field experiment, requests were sent to 198 subdistrict governments in Shanghai, China, with the textual records of the requesting process and follow-up interviews. The findings confirm that regulative pressure can increase government compliance with requests. Further, governments prefer to use three strategies, namely selectivity, bargaining, and avoidance, as alternatives to conformity to institutional pressures for transparency.

Combining randomized field experiments with observational satellite data to assess the benefits of crop rotations on yields

With climate change threatening agricultural productivity and global food demand increasing, it is important to better understand which farm management practices will maximize crop yields in various climatic conditions. To assess the effectiveness of agricultural practices, researchers often turn to randomized field experiments, which are reliable for identifying causal effects but are often limited in scope and therefore lack external validity. Recently, researchers have also leveraged large observational datasets from satellites and other sources, which can lead to conclusions biased by confounding variables or systematic measurement errors. Because experimental and observational datasets have complementary strengths, in this paper we propose a method that uses a combination of experimental and observational data in the same analysis. As a case study, we focus on the causal effect of crop rotation on corn (maize) and soybean yields in the Midwestern United States. We find that, in terms of root mean squared error, our hybrid method performs 13% better than using experimental data alone and 26% better than using the observational data alone in the task of predicting the effect of rotation on corn yield at held-out experimental sites. Further, the causal estimates based on our method suggest that benefits of crop rotations on corn yield are lower in years and locations with high temperatures whereas the benefits of crop rotations on soybean yield are higher in years and locations with high temperatures. In particular, we estimated that the benefit of rotation on corn yields (and soybean yields) was 0.85 t ha−1 (0.24 t ha−1) on average for the top quintile of temperatures, 1.03 t ha−1 (0.21 t ha−1) on average for the whole dataset, and 1.19 t ha−1 (0.16 t ha−1) on average for the bottom quintile of temperatures. This association between temperatures and rotation benefits is consistent with the hypothesis that the benefit of the corn-soybean rotation on soybean yield is largely driven by pest pressure reductions while the benefit of the corn-soybean rotation on corn yields is largely driven by nitrogen availability.

Encouraging COVID-19 vaccination through behaviorally informed reminders: Results from a national randomized field experiment in Israel

Inducing people to get vaccinated is critical for controlling the spread of COVID-19. We explored the effectiveness of two text messaging strategies for encouraging unvaccinated individuals to get their COVID-19 vaccination. One message emphasized social norms to harness people's tendency to act in ways that line up with society's expectations. The other message underscored the personal medical benefits of vaccination. Both messages indicated that the vaccine was reserved for the recipient at a nearby location. Over the course of eight days, the percentage of people who got vaccinated after receiving the medical benefit message was 2.1% higher than the percentage of people who got vaccinated after receiving the social norm message ( p < .001). Our findings indicate that designing vaccination reminders that highlight the medical benefits of vaccination in addition to the availability of the vaccines can increase vaccination rates.

Framing effects on risk-taking behavior: evidence from a field experiment in multiple-choice tests

We exploit testing data to gain better understanding on framing effects on decision-making and performance under risk. In a randomized field experiment, we modified the framing of scoring rules for penalized multiple-choice tests. In penalized multiple-choice tests, right answers are typically framed as gains while wrong answers are framed as losses (Mixed-framing). In the Loss-framing proposed, both non-responses and wrong answers are presented in a loss domain. According to our theoretical model, we expect the change in the framing to decrease students’ non-response and to increase students’ performance. Under the Loss-framing, students’ non-response reduces by a 18%-20%. However, it fails to increase students’ scores. Indeed, our results support the possibility of impaired performance in the Loss-framing.

Augmenting Password Strength Meter Design Using the Elaboration Likelihood Model: Evidence from Randomized Experiments

In this research, we study an effective method to encourage users to generate stronger passwords. Specifically, we propose a novel design of password strength meters that incorporates contextual information to help users digest the message generated by the password strength meter. We evaluate our design by leveraging three independent and complementary methods: a survey-based experiment using students to evaluate the saliency of our conceptual design (proof of concept), a controlled laboratory experiment conducted on Amazon Mechanical Turk to test the effectiveness of the proposed design (proof of value), and a randomized field experiment conducted in collaboration with an online forum in Asia to establish proof of use. In each study, we observe that users exposed to the proposed password strength meter are more likely to change their passwords, leading to a new password that is significantly stronger. Our findings suggest that the proposed design of augmented password strength meters is an effective method for promoting secure password behavior among end users. Our design also requires minimal computational resources and technical capabilities.

Does Familiarity Breed Esteem? A Field Experiment on Emergent Attitudes Toward Members of Congress

Canonical theories of democratic representation envision legislators cultivating familiarity to enhance esteem among their constituents. Some scholars, however, argue that familiarity breeds contempt, which if true would undermine incentives for effective representation. Survey respondents who are unfamiliar with their legislator tend not to provide substantive answers to attitude questions, and so we are missing key evidence necessary to adjudicate this important debate. We solve this problem with a randomized field experiment that gave some constituents an opportunity to gain familiarity with their Member of Congress through an online Deliberative Town Hall. Relative to controls, respondents who interacted with their member reported higher esteem as a result of enhanced familiarity, a mediation effect supporting canonical theories of representation. This effect is statistically significant among constituents who are the same political party as the member but not among those of the opposite party, although in neither case did familiarity breed contempt.

Communication frames and beneficiary engagement in corporate social initiatives: Evidence from a randomized controlled trial in France

AbstractResearch SummaryThis article asks how distinct communication frames used by corporate social initiatives (CSIs) affect beneficiary engagement, specifically their reach vis‐à‐vis disadvantaged groups. Through a unique randomized field experiment in France, we assess the effectiveness of a series of communication frames on beneficiaries' decisions to enroll and use the social goods provided. Our results show that empathy‐ and simplicity‐oriented frames significantly increase program enrollment and utilization rates relative to other strategies, notably the widely used charity frame. We find that none of the tested communication frames raised take‐up by nonnative French beneficiaries, and only the empathy frame augmented the response of the very poor. Our work opens new research directions for strategic management studies on CSIs, communication frames, inclusion and social outcomes.Managerial SummaryMany corporate social initiatives (CSIs) launch large information campaigns to raise awareness about the social goods they provide. Findings from a field experiment reveal that the communication frames that CSIs deploy matters. Communication that expresses empathy (how the CSI understands and cares about its target beneficiaries) or presents simple information (what the CSI does) is far more effective at raising take‐up than communication frames that emphasize what for (why) and the charitable identity (who) about the CSI. However, results also reveal that foreigners and the economically most disadvantaged beneficiaries responded the least to these communication frames. This suggests that more is needed to effectively raise take‐up by these subgroups. Our work speaks to strategy researchers and practitioners interested in CSIs, communication frames, and social inclusion.

Doing Well by Doing Good: Improving Retail Store Performance with Responsible Scheduling Practices at the Gap, Inc.

We estimate the causal effects of responsible scheduling practices on store financial performance at the U.S. retailer Gap, Inc. The randomized field experiment evaluated a multicomponent intervention designed to improve dimensions of work schedules—consistency, predictability, adequacy, and employee control—shown to foster employee well-being. The experiment was conducted in 28 stores in the San Francisco and Chicago metropolitan areas for nine months between November 2015 and August 2016. Intent-to-treat (ITT) analyses indicate that implementing responsible scheduling practices increased store productivity by 5.1%, a result of increasing sales (by 3.3%) and decreasing labor (by 1.8%). Drawing on qualitative interviews with managers and quantitative analyses of employee shift-level data, we offer evidence that the intervention improved financial performance through improved store execution. Our experiment provides evidence that responsible scheduling practices that take worker well-being into account can enhance store productivity by motivating additional employee effort and reducing barriers to employees adhering to the scheduled labor plan. This paper was accepted by David Simchi-Levi, operations management. Funding: This research was supported by generous grants from the W.K. Kellogg Foundation, the Washington Center for Equitable Growth, the Robert Wood Johnson Foundation, the Institute of International Education in collaboration with the Ford Foundation, the Center for Popular Democracy, the Suzanne M. Nora Johnson and David G. Johnson Foundation, and Gap, Inc. Supplemental Material: The data files and online appendix are available at https://doi.org/10.1287/mnsc.2021.4291 .

Does Taekwondo improve children's self-regulation? If so, how? A randomized field experiment.

Emerging evidence suggests interventions can improve childhood self-regulation. One intervention approach that has shown promise is Taekwondo martial arts instruction, though little is known about its acceptability among stakeholders or its mechanisms of effect. We extend evidence on Taekwondo interventions in three ways: (a) testing the efficacy of a standard introductory course of Taekwondo, (b) assessing the acceptability of Taekwondo instruction among school children, and (c) investigating two self-regulatory mechanisms by which Taekwondo may operate (executive functions and motivation). This article reports findings from a randomized control trial implementing a standard 11-week beginners' course of Taekwondo. Participants were from a mixed-sex, nonselective U.K. primary school (N = 240, age range 7 to 11 years). Measures of self-regulation included teacher-rated effortful control, impulsivity, prosocial behavior, and conduct problems; computer-based assessments of executive functions; and child self-reported expectancies and values to use self-regulation. Postintervention, children in the Taekwondo condition were rated by teachers as having fewer symptoms of conduct problems and better effortful control (specifically attentional control), and they also had better executive attention assessed by a flanker task. Effects were not found for teacher-rated inhibitory control, activation control, impulsivity, and prosocial behavior or for assessments of response inhibition, verbal working memory, and switching. Taekwondo was rated very positively by children. Finally, there was evidence that children who completed Taekwondo classes reported higher expectancies and values to use self-regulation and that expectancies and values mediated intervention effects on self-regulation. We conclude that short standard Taekwondo courses are well received by pupils, improve attentional self-regulation, and reduce symptoms of conduct problems. (PsycInfo Database Record (c) 2022 APA, all rights reserved).