Randomized Field Experiment Research Articles

A multi-sensory tutoring program for students at risk of reading difficulties: Evidence from a randomized field experiment

This study examined a literacy program that targeted students most at risk of reading difficulties in kindergarten and first grade of 12 Swedish schools. The program used multi-sensory learning methods that focused on phonological awareness and phonics, and was delivered during 10 weeks over 30–35 sessions by teachers in a one-to-one or one-to-two setting. In total, 161 students aged 6–7 years were randomly assigned to a treatment group or a waiting list control group. The treatment group showed large and statistically significant improvements compared to the control group on the two pre-registered primary outcome measures: a standardized test of decoding (Hedges’ g = 1.07) and a standardized test of letter knowledge (g = 1.03). The improvements were also significantly larger on measures of phonological awareness (g = 0.56) and self-efficacy (g = 0.57), but not on measures of enjoyment and motivation. The program appears cost-effective relative to other tutoring programs.

Improving food safety on the farm: Experimental evidence from Kenya on incentives and subsidies for technology adoption

• Technologies to combat fungal toxins in stored maize are not widely used. • We test how subsidies and a food safety incentive affect farmers’ practices. • Pure subsistence farmers use better post-harvest practices than those who sell maize. • Both a price incentive and technology subsidies partially close this gap. • Providing training and drying sheets is the most cost-effective option considered. Evidence is emerging that foodborne illness accounts for a staggering health burden in developing countries. However, standard approaches used by developed country governments to ensure food safety are not appropriate in settings where regulatory enforcement capacity is weak and most firms are small and informal. Using a randomized field experiment, we test the impacts of subsidies and a price premium for safer produce on farmer adoption of technologies that improve food safety. We find that the food safety practices of farmers who produce maize for sale are inferior to those of farmers who produce maize only for household consumption, but that both a price incentive and technology subsidies can partially close this gap. We combine our experimental adoption results with prior evidence on the efficacy of the technologies studied to simulate the public health impacts of alternative policies. Our simulations show that interventions to reduce aflatoxin exposure are likely to be cost-effective based on averted poisoning deaths and cancer cases alone. Potential impacts on stunting, which are not as well established and more difficult to value, would imply additional health benefits. Of the policy options considered, providing training and plastic drying sheets to farmers free of charge is the most cost-effective.

Effectiveness of Integrated Offline-and-Online Promotions in Omnichannel Targeting: A Randomized Field Experiment

ABSTRACT Leveraging omnichannel has become a new norm of strategic marketing in the retail industry, with many vendors foregrounding the value of customers who wish to maximize their shopping experiences across all channels. Notwithstanding such heightened attention, little is known about the effectiveness of omnichannel targeting and promotional strategies. Whereas previous studies assessed the economic worth of channel promotions independently of each other, our study delved into the effects of integrated omnichannel campaigns. Using a randomized field experiment design, we provide empirical evidence of an offline direct experience effect and revealed short-term channel substitution (spillover) effects among customers who only use the online-channel (offline-channel). We further examined omnichannel conversion behaviors after exposure to online promotion and developed different coupon schemes based on responses to the previously offered offline initiative. Finally, we detected significant patterns of post-treatment omnichannel migration and confirmed the effectiveness of integrated omnichannel promotions in fostering a shift to omnichannel shopping.

A story induces greater environmental contributions than scientific information among liberals but not conservatives

<h2>Summary</h2> Scientific evidence that links human activities to environmental damage frequently fails to motivate people to act. Meanwhile, research on emotion, imagery, and identifiable victims has found these factors to influence behavior, and scientists and environmentalists are increasingly advocating for the use of narratives depicting personal stories of loss. We tested the behavioral effects of a narrative compared with scientific facts in a randomized field experiment with over 1,200 adults in a polluted urban watershed. Prior to making real purchase decisions about landscaping products that reduce nutrient runoff, consumers saw either scientific information about runoff's impacts or a narrative with tenuous scientific foundations. When exposed to the narrative rather than scientific information, consumers were willing to pay 11% more (95% confidence interval [4%, 18%]). This average effect, however, masks heterogeneity by political affiliation. While Democrats paid more after reading the narrative, Republicans paid less by roughly the same amount.

Biochar combined with nitrogen fertilizer affects soil properties and wheat yield in medium‐low‐yield farmland

AbstractBiochar combined with fertilizer as a soil amendment benefits to improving soil fertility, especially soil organic carbon and crop yield. However, the effect of biochar on the improvement of soil properties and crop yield was varied from soil properties and limited for medium–low‐yield farmland in the North China. During the completely randomized field experiment, SIX treatments (biochar applied as 0, 15 and 30 t·ha‐1, under 240 and 300 kg N ha‐1 nitrogen fertilizer) were applied in wheat season and examined to reveal changes in the SOC and other properties of 0‐ to 10‐cm and 10‐ to 20‐cm soil layers. The results showed that two years after the application of biochar, a significant increase in the SOC was observed, ranging from 19.52% to 97.50% (p &lt; 0.05) in the 0‐ to 20‐cm soil layer. Wheat yield and SOC content increased with increasing amount of biochar applied under the same amount of nitrogen fertilizer. The content of soil available potassium increased significantly under 30 t·ha‐1 biochar application (p &lt; 0.05). Both biochar and nitrogen fertilizer application could increase wheat yield, and the effect of biochar application for increasing wheat yield was better than that of nitrogen fertilizer. Wheat yield and SOC content increased with increasing nitrogen fertilizer at the same amount of biochar application. The principal component analysis results showed that biochar input, SOC, available potassium and total nitrogen were the key factors affecting wheat yield. Biochar application is a fast and effective measure to improve SOC and wheat yield in medium‐ and low‐yield farmlands.

Early Detection of Stripe Rust in Winter Wheat Using Deep Residual Neural Networks.

Stripe rust (Pst) is a major disease of wheat crops leading untreated to severe yield losses. The use of fungicides is often essential to control Pst when sudden outbreaks are imminent. Sensors capable of detecting Pst in wheat crops could optimize the use of fungicides and improve disease monitoring in high-throughput field phenotyping. Now, deep learning provides new tools for image recognition and may pave the way for new camera based sensors that can identify symptoms in early stages of a disease outbreak within the field. The aim of this study was to teach an image classifier to detect Pst symptoms in winter wheat canopies based on a deep residual neural network (ResNet). For this purpose, a large annotation database was created from images taken by a standard RGB camera that was mounted on a platform at a height of 2 m. Images were acquired while the platform was moved over a randomized field experiment with Pst-inoculated and Pst-free plots of winter wheat. The image classifier was trained with 224 × 224 px patches tiled from the original, unprocessed camera images. The image classifier was tested on different stages of the disease outbreak. At patch level the image classifier reached a total accuracy of 90%. To test the image classifier on image level, the image classifier was evaluated with a sliding window using a large striding length of 224 px allowing for fast test performance. At image level, the image classifier reached a total accuracy of 77%. Even in a stage with very low disease spreading (0.5%) at the very beginning of the Pst outbreak, a detection accuracy of 57% was obtained. Still in the initial phase of the Pst outbreak with 2 to 4% of Pst disease spreading, detection accuracy with 76% could be attained. With further optimizations, the image classifier could be implemented in embedded systems and deployed on drones, vehicles or scanning systems for fast mapping of Pst outbreaks.

Disability Disadvantage: Experimental Evidence of Hiring Discrimination against Wheelchair Users

Abstract Disability is associated with persistent labour market disadvantages. What is not clear is the extent to which these disadvantages result from employers’ discriminatory hiring decisions. Although observational research and laboratory experiments provide indicative evidence of its existence, few studies have used randomized field experiments such as correspondence studies to investigate the occurrence of disability discrimination. This article extends current knowledge by presenting the results of a correspondence study used to measure discrimination against wheelchair users in a new context: the Norwegian labour market. In the experiment, 1,200 fictitious applications with randomly assigned information about the applicants’ disability status were sent in pairs to 600 private sector employers with job openings. The experiment documents negative effects of disability on callbacks from employers across various occupations. The findings suggest that discrimination in hiring processes is a mechanism through which disability-related inequality in employment outcomes is perpetuated.

Does design thinking training increase creativity? Results from a field experiment with middle-school students

ABSTRACT Design thinking remains mired in controversy. Its proponents claim that it enhances not just confidence but also creativity, while the opponents question whether it does anything beyond building unfounded confidence. To bring rigorous evidence to this debate, we designed a randomised field experiment amongst school children served by a major non-governmental organisation in rural India. The findings reveal that the design thinking training did not just increase confidence: it also significantly increased ideational fluency and elaboration in a divergent thinking task, although the originality and flexibility of the generated ideas was on average lower in the treatment group than the control group. We also find that the increase in confidence occurred primarily among female students, whereas the increase in ideational fluency and elaboration occurred for both genders in our study.

Information, switching costs, and consumer choice: Evidence from two randomised field experiments in Swedish primary health care

Consumer choice policies may improve the matching of consumers and providers, and may spur competition over quality dimensions relevant to consumers. However, the gains from choice may fail to materialise in markets characterised by information frictions and switching costs. We use two large-scale randomised field experiments in primary health care to examine if individuals reconsider their provider choice when receiving leaflets with comparative information and pre-paid choice forms by postal mail. The first experiment targeted a representative subset of the 1.3 million residents in a Swedish region. The second targeted new residents in the same region, a group expected to have less prior information and lower switching costs than the general population. The propensity to switch providers increased after the interventions in both the population-representative sample (by 0.6–0.8 percentage points, 10–14%) and among new residents (2.3 percentage points, 26%). The results demonstrate that there are demand side frictions in the primary care market. Exploratory analyses indicate that the effects on switching were larger in urban markets and that the interventions had heterogeneous effects on the type of providers chosen, and on health care and drug consumption.

Improving the outcomes of youth with medical limitations: Evidence from the National Job Corps Study

AbstractImproving work outcomes for youth with disabilities and reducing their reliance on disability benefits are important policy priorities, but existing interventions have shown limited promise. We provide new evidence to inform this discussion by re‐analyzing data from the 1990s National Job Corps Study, a randomized field experiment conducted nationwide in the United States. Job Corps, which provides comprehensive training to economically disadvantaged youth, is the nation's largest youth program outside of the school system. We examine youth who had medical limitations when they enrolled in the experiment, a group that has not previously been studied. During the 4 years after random assignment, participation in Job Corps increased the earnings of youth with medical limitations—substantially more so than for youth without medical limitations—and additionally reduced their receipt of disability cash benefits. Interventions designed specifically for such youth have not typically demonstrated reductions in benefit receipt. Hence, our re‐analysis of the field experiment suggests that Job Corps could be a promising model for helping some youth with disabilities gain a foothold in the labor market and achieve greater self‐sufficiency.

Wikipedia matters

AbstractWe document a causal impact of online user‐generated information on real‐world economic outcomes. In particular, we conduct a randomized field experiment to test whether additional content on Wikipedia pages about cities affects tourists' choices of overnight visits. Our treatment of adding information to Wikipedia increases overnight stays in treated cities compared to nontreated cities. The impact is largely driven by improvements to shorter and relatively incomplete pages on Wikipedia. Our findings highlight the value of digital public goods for informing individual choices.

Social Norms Based Eco-Feedback for Household Water Consumption

Physical water scarcity is a growing threat to people’s lives around the world. Non-pecuniary interventions that encourage water conservation amongst households are an effective tool to promote sustainable consumption. In a randomised field experiment on 3461 UK households, a social norms based eco-feedback intervention was found to reduce water consumption by around 5.43 L a day or by 1.8% over 29 months. This effect did not persist for the 10 months after the intervention was stopped suggesting a lack of habit formation. Unlike previous studies, households with low consumption at baseline reduced their consumption the most, while high consumers did not. Heterogeneity was also found across quantile treatment effects, where households in the top and bottom quantiles increased their consumption. These results further contribute to the growing evidence on the effectiveness of combining social norms and eco-feedback as an intervention for conservation.

Evaluation of a Health Education Intervention for Rural Preschool and Kindergarten Children in the Southeastern United States: A Cluster Randomized Trial

This research employed a matched-pairs randomized field experiment design to evaluate a classroom-based health education intervention for pre-Kindergarten and Kindergarten children in a rural region of the southeastern United States. Schools were matched on demographic characteristics, then one school from each pair was randomly assigned to the treatment group and one to the delayed treatment group. The intervention included a field trip experience and an integrated curriculum designed to increase knowledge about nutrition, physical activity, and sleep. Staff conducted individual assessments of changes in knowledge with a random sample of children from each classroom (252 children from treatment classrooms; 251 children from delayed treatment classrooms). We used a multilevel linear regression with maximum likelihood estimation to incorporate the effects of clustering at the classroom and school level while examining the effects of the intervention on individual assessment change scores. During the intervention period, an estimated 3,196 children (treatment: 1,348 students in 68 classrooms in 10 schools; delayed treatment: 1,848 students in 86 classrooms in 10 schools) participated in the intervention. Children in the treatment group had significantly larger assessment change scores than children in the delayed treatment group. Findings suggest significant beneficial effects of the intervention on health knowledge.

Can ATMs get out the vote? Evidence from a nationwide field experiment

We report on a large-scale (randomized) field experiment we designed and conducted to assess ATMs’ (automatic teller machines) capacity to “get out the vote”. This is a heretofore unexploited method of voter mobilization. Our experimental design used the full universe of functioning ATMs in Portugal, which benefits from a sophisticated world class system, with wide national coverage. We randomly selected a set of treatment civil parishes, where a civic message took over the totality of ad time in ATMs, which we compare with a set of control civil parishes where advertisements ran as usual. The ATM campaign we follow was active for three days immediately before and during the 2017 local elections in Portugal. When we consider the intensity of treatment, for both the entire campaign period and the weekend, results show a statistically significant increase in the likelihood of voting. All three proxies for treatment intensity we employ deliver robust positive coefficients. Placebo tests using turnout rates in previous elections do not show significant coefficients, strengthening our interpretation of the experimental results.

Experimental evidence of limited attention at the gym

In a randomized field experiment, we show that a low-cost intervention, weekly email reminders, increases the weekly exercising frequency of gym members by 13%, with the effect being larger for class training (19%) than for free training (11%). We find that nearly all types of individuals benefit from the reminders (e.g., students, nonstudents, women, men, new members, recurring members), and this effect persists in a three-month posttreatment period. Furthermore, the increase in class visits is partly driven by an increase in the number of bookings of gym classes and a slight decrease in the share of bookings that are canceled. Limited attention and habit formation can best explain these results. In contrast to the increase in gym attendance, we do not find any effect on the duration and renewal of membership contracts.

Effects of experimental flea removal and plague vaccine treatments on survival of northern Idaho ground squirrels and two coexisting sciurids

Plague is a non-native disease in North America that reduces survival of many mammals. Previous studies have focused on epizootic plague which causes acute mortality events and dramatic declines in local abundance. We know much less about enzootic plague which causes less punctuated reductions in survival and abundance of infected populations. As a result, enzootic plague is much more difficult to detect because changes in population attributes are more subtle and Yersinia pestis prevalence is likely lower relative to epizootic plague outbreaks. The northern Idaho ground squirrel (Urocitellus brunneus) is a threatened species which coexists with Columbian ground squirrels (Urocitellus columbianus) and yellow-pine chipmunks (Neotamias amoenus) throughout their restricted distribution in central Idaho. Columbian ground squirrels and yellow-pine chipmunks are more abundant and widespread than northern Idaho ground squirrels and both are known hosts for plague. Hence, enzootic plague may be one cause of rarity for northern Idaho ground squirrels but its effect on this threatened species has not been evaluated. We conducted three controlled and randomized field experiments to examine the effects of plague in northern Idaho ground squirrels and the two coexisting species: 1) a plague vaccine experiment, 2) a paired flea-reduction experiment, and 3) a non-paired flea-reduction experiment. For Experiment 1, we hypothesized that if enzootic plague is present, vaccinated animals would have higher survival. Furthermore, Experiments 2 and 3 tested the prediction that untreated, control animals should have lower survival than those in areas where fleas are experimentally removed or reduced because fleas are the main vector for plague. In the plague vaccine experiment, vaccinated chipmunks had 4.65% higher apparent survival compared to chipmunks that received a placebo for intervals when the vaccine is believed to be effective. Apparent annual survival increased for all three species on experimental flea-reduction plots compared to non-treated plots for the paired experiment but results were mixed for the non-paired experiment. Taken together, our results suggest that enzootic plague is present and negatively impacting survival of northern Idaho ground squirrels and two coexisting species.

Combating Procrastination on Massive Online Open Courses via Optimal Calls to Action

Massive online open courses (MOOCs) are a booming phenomenon in the digital era. However, the online nature of educational delivery via MOOCs creates every opportunity for digital distraction and procrastination, resulting in difficulties for students and instructors. According to a new study in Information Systems Research, the authors Ni Huang (University of Houston), Jiayin Zhang (Tsinghua University), Gordon Burtch (University of Minnesota), Xitong Li (HEC Paris), and Peiyu Chen (Arizona State University) report a randomized field experiment on a large MOOC platform to examine several calls to action (CTAs) pertaining to the completion and submission of course assignments with an eye toward combating student procrastination on MOOCs. Their results show that descriptive norms (i.e., informing the completion rates of the assignments) lead to higher probabilities of assignment completion and a shorter time to completion. In contrast, a deadline reminder in the form of a planning prompt (i.e., informing the target deadline for assignment submission and the importance of planning ahead) has a surprisingly counterproductive effect, in particular, if students’ active course load is low. One possible explanation is that the students with low course loads may perceive the deadline to be distant, which reduces their sense of urgency and leads to complacency.

Early warning systems for more effective student counselling in higher education: Evidence from a Dutch field experiment

AbstractEarly Warning Systems (EWS) in higher education accommodate student counsellors by identifying at‐risk students and allow them to intervene in a timely manner to prevent student dropout. This study evaluates an EWS that shares student‐specific risk information with student counsellors, which was implemented at a large Dutch university. A randomised field experiment was conducted to estimate the effect of EWS‐assisted counselling on first‐year student dropout and academic performance. The results show that the EWS accurately predicts at‐risk students. Yet, EWS‐assisted counselling did not reduce dropout, nor improved academic performance. Solving the underlying problem of poor academic performance might require additional actionable feedback and recommended counselling practices.

Burnout of intrinsically motivated GPs when exposed to external regulation: A combined panel data survey and cluster randomized field experiment

Burnout among general practitioners (GPs) is a problem in many countries. Research indicates that burnout is less likely to occur among intrinsically motivated employees. Based on self-determination theory, we investigate 1) whether intrinsically motivated GPs are less burned out than their colleagues, and 2) whether the most intrinsically motivated GPs are more likely to burn out when exposed to an external regulatory accreditation programme. General practices in Denmark were cluster randomized to mandatory accreditation in 2016, 2017 or 2018. We measure GPs’ intrinsic motivation and burnout levels one and two years into the accreditation process. We use a balanced panel of GPs (n = 846) to estimate mixed effects ordered logit models. We find that GPs with high intrinsic motivation are less burned out than their colleagues. However, the most intrinsically motivated GPs are significantly more burned out when exposed to accreditation compared to their colleagues. We conclude that being intrinsically motivated may not shield from burnout when external regulation is imposed.

Preserving User Privacy Through Ephemeral Sharing Design: A Large-Scale Randomized Field Experiment in the Online Dating Context

Due to privacy concerns, users of online dating platforms often refrain from voluntary sharing of sensitive personal information in the initial interaction stage. However, the lack of information sharing impedes trust-building, further hurting their probability of securing matches and receiving follow-up conversations from prospective dating partners due to information asymmetry. We propose ephemeral sharing as a privacy-preserving mechanism to navigate the balance between users’ privacy concerns and voluntary information sharing in the initial interaction stage in online dating. Ephemeral sharing refers to the digital design that once the information being shared by a sender is disclosed to a receiver for a relatively short period, it will be invisible and non-retrievable to the receiver in the future. In partnership with an online dating platform, we report a large-scale randomized field experiment with over 70k users to understand how ephemeral sharing influences users’ information sharing, match outcomes, and follow-up conversational engagements. We find that the subjects in the ephemeral treatment group (versus those in the control group) send a significantly more significant number of personal photos along with their matching request and a more significant number of photos disclosing the human face. Meanwhile, the ephemeral sharing treatment also leads to a significant number of more matches. Further, through a set of mediation tests, we find that the observed effect on the number of matches is explained by the increases in the senders’ disclosure of personal photos. The sequential mediation tests further show that the increased sharing of personal photos also induces receivers’ follow-up conversational engagements, mitigating the communication cold-start issue. Lastly, we apply a recursive partitioning algorithm to explore the heterogeneous treatment effects, and the results illustrate that the platform can perform treatment optimization based on user’s popularity, education, and gender. Our study contributes to the literature and practice on the design of matching platforms and the privacy-preserving mechanisms.