Randomized Field Experiment Research Articles

Improving Cancer Outreach Effectiveness Through Targeting and Economic Assessments: Insights from a Randomized Field Experiment

Patients at risk for hepatocellular carcinoma or liver cancer should undergo semiannual screening tests to facilitate early detection, effective treatment options at lower cost, better recovery prognosis, and higher life expectancy. Health care institutions invest in direct-to-patient outreach marketing to encourage regular screening. They ask the following questions: (1) Does the effectiveness of outreach vary among patients and over time?; (2) What is the return on outreach?; and (3) Can patient-level targeted outreach increase the return? The authors use a multiperiod, randomized field experiment involving 1,800 patients. Overall, relative to the usual-care condition, outreach alone (outreach with patient navigation) increases screening completion rates by 10–20 (13–24) percentage points. Causal forests demonstrate that patient-level treatment effects vary substantially across periods and by patients’ demographics, health status, visit history, health system accessibility, and neighborhood socioeconomic status, thereby facilitating the implementation of the targeted outreach program. A simulation shows that the targeted outreach program improves the return on the randomized outreach program by 74%–96% or $1.6 million to $2 million. Thus, outreach marketing provides a substantial positive payoff to the health care system.

What is the impact of bilingual communication to mobilize Latinos? Exploratory evidence from experiments in New Jersey, North Carolina, and Virginia

Should materials aimed at increasing Latino voter turnout be in English or bilingual? Credible, theoretical arguments can be made both ways: bilingual materials may be more effective if signaling cultural awareness or less effective if seen as pandering. We tested these competing hypotheses with two rounds of randomized field experiments in New Jersey and Virginia in 2015, and North Carolina in 2016. While some GOTV experiments have used bilingual mailers, previous scholarship has not tested whether bilingual mailers are more effective than English-language materials. In the 2015 elections, both treatments increased turnout, and the monolingual English version was more effective at increasing turnout than the bilingual version. These results are replicated in the high salience 2016 election in North Carolina. These results indicate that further research is needed about bilingual communication across political and demographic contexts and about how household composition may condition the effects of bilingual communication.

Discretionary Remote Working Helps Mothers Without Harming Non-mothers: Evidence from a Field Experiment

Because mothers remain disproportionately responsible for childcare, the daily requirement for physical presence at work disadvantages them compared with otherwise equivalent men and childless women. Relaxing this requirement may therefore enhance the well-being and productivity of working mothers. I tested this idea with a randomized field experiment, using a within-subjects analysis from a repeated crossover design. The 187 participants in the experiment, which ran for four weeks and yielded 748 person-week observations, revealed a preference for about two remote working days per week. I observed no significant differences in the uptake of remote working days between men, women, parents, nonparents, fathers, and mothers. Mothers reported meaningfully reduced family–work conflict during remote working weeks, but fathers did not. Remote working generally increased job performance, but the effect was greatest for mothers. The coordination costs of remote working, with respect to coworker helping and job interdependence, did not appear prohibitive. Interviews with study participants corroborate and contextualize these findings. This paper was accepted by Olav Sorenson, organizations.

Social influence tactics in e-commerce onboarding: The role of social proof and reciprocity in affecting user registrations

Given the increasing competition to attract new customers, e-commerce providers (i.e., e-tailers) are being urged to optimize their website onboarding journeys. Design mechanisms employing social influence tactics seem to be promising avenues to effectively improve user onboarding experiences. Specifically, two social influence tactics—namely, reciprocity and social proof—are widely used in practice to reliably enhance user registrations. Yet, researchers have only tentatively examined how these combined tactics affect user onboarding behavior. While combining both tactics may hold synergistic potential, it may also be counterproductive, as too much social influence may backfire. We address this research gap by investigating the distinct and joint effects of reciprocity and social proof on user registration behavior. Drawing on two forms of reciprocity (i.e., utility- and monetary-based reciprocity), we conduct an online experiment (N = 249) and a related randomized field experiment (N = 475,495) to compare both reciprocity variants in combination with social proof and investigate their efficacy for actual user onboarding decisions (i.e., user registrations). Our results show that both social influence tactics and both reciprocity variants have positive effects on users' registration behavior if applied individually. However, when both tactics are employed together, the effect of reciprocity is moderated by social proof such that social proof nullifies the effect of monetary-based reciprocity whereas it amplifies the effect of utility-based reciprocity. Our study thereby contributes to a more advanced understanding of the interactive relationship between reciprocity and social proof and their joint effect on user onboarding decisions.

Influence Activities and Bureaucratic Performance: Evidence from a Large-Scale Field Experiment in China

Subjective performance evaluation is widely used by firms and governments to provide work incentives. However, delegating evaluation power to senior leadership could induce influence activities: agents might devote much efforts to please their supervisors, rather than focusing on productive tasks that benefit their organizations. We conduct a large-scale randomized field experiment among Chinese local government employees and provide the first rigorous empirical evidence on the existence and implications of influence activities. We find that state employees are able to impose evaluator-specific influence to affect evaluation outcomes and that this process could be partly observed by their co-workers. Furthermore, introducing uncertainty in the identity of the evaluator, which discourages evaluator-specific influence activities, can significantly improve the work performance of state employees.

Learning Mathematics Problem Solving through Test Practice: a Randomized Field Experiment on a Global Scale

We measure the effect of a single test practice on 15-year-old students’ ability to solve mathematics problems using large, representative samples of the schooled population in 32 countries. We exploit three unique features of the 2012 administration of the Programme for International Student Assessment (PISA), a large-scale, low-stakes international assessment. During the 2012 PISA administration, participating students were asked to sit two separate tests consisting of problem-solving tasks. Both tests included questions that covered the same internationally recognized and validated framework for mathematics assessment. Students were randomly assigned in the first, 2-h-long test to one of three test versions containing varying amounts of mathematics, reading, and science problems. We found that the amount of mathematics problems in the first test had a small positive effect on mean mathematics performance on the second test, but no effect on general reasoning and problem-solving ability. Subject-specific effects of test practice on subsequent test performance were found over both short lags (same day) and medium lags (1–7 days). The learning gains ascribed to mathematics problem-solving practice were larger for boys than for girls.

Beneficial, adverse, and spiraling health-promotion effects: Evidence from a longitudinal randomized controlled trial of working at sit-stand desks.

In a 6-month longitudinal randomized field experiment, we examined how using height-adjustable sit-stand desks could have beneficial, adverse, and spiraling effects on people's musculoskeletal and psychovegetative complaints, and on positive (vitality and vigilance) and negative psychological symptoms, namely, stressor uncontrollability (i.e., perceived uncontrollability of workload), psychological tension, and mental tiredness. A total of 127 employees in various, mostly sedentary, occupations were randomly assigned to either the intervention or the control group. Variables were assessed monthly for 6 months on a self-reported basis. Bayesian structural equation modeling showed that the intervention produced large inhibiting between-subjects effects for musculoskeletal problems in the neck, back, and shoulders (β ranged between -.26 and -.21). Within-subject analyses revealed that the intervention produced large inhibiting effect sizes for intensity (g = 3.06) and prevalence of musculoskeletal (g = 1.19) and psychovegetative complaints (g between 0.76 and 1.57). For negative psychological symptoms (i.e., psychological tension and mental tiredness), participants in the intervention group showed a steeper decrease than participants in the control group (g between 2.34 and 3.74). For positive indicators (i.e., vitality and vigilance), the intervention produced large promoting effects for participants in the intervention group compared with participants in the control group (g between 0.70 and 1.65). There was no change in stressor uncontrollability between the two groups. Finally, findings suggest that sit-stand desks can be effective in improving occupational health by weakening a downward-spiraling effect. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Designing Demand Information Disclosure in the Presence of Capacity Constraints: A Large-Scale Randomized Field Experiment on a Matching Platform

A common issue in online dating platforms is that many users focus their attention on a subset of popular peers, which leads to congestion and thus lower matching rates. Our study examines the results of a randomized natural field experiment, wherein our corporate partner, a large online dating platform, explored information sharing as a means to resolving the congestion problem, providing a random set of users with information on peers’ recent demand. We examine whether and for whom this demand information disclosure facilitated the redistribution of attention away from popular individuals, reducing demand concentration. In doing so, we first consider that two competing effects may emerge with demand disclosure: on the one hand, demand information may signal (a lack of) available capacity, which may attenuate demand concentration (capacity effect); on the other hand, demand information may signal quality, which in turn may exacerbate demand concentration (popularity effect). Although prior work on online platforms for the sale of goods and services has reported evidence in support of both effects, our work is one of the first to explore either relationship in online dating. Online dating platforms differ from other marketplaces in important ways, differences that call into question the likelihood that either effect will arise. Moreover, online dating markets incorporate unique platform dynamics that are likely to drive heterogeneity in the relative strength of the competing effects. Most notably, males and females (in heterosexual dating markets) face starkly different levels of competition; males typically outnumber females by a ratio of more than 2:1, which raises the prospect of very different responses to demand information across gender lines, and in turn, presents a targeting opportunity. Our results show that demand information disclosure increases the average user’s attention toward low-demand partners, decreases their pursuit of high-demand partners, and thereby drives a higher number of successful matches. Heterogeneity analyses further demonstrate that these average results are driven by the larger representation of males in the population, as females are largely unresponsive to the treatments. Our study provides theoretical implications for the literature on online dating and platform design.

Holding back from theory: limits and methodological alternatives of randomized field experiments in development economics

ABSTRACT In this paper, we critically and constructively examine the methodology of evidence-based development economics, which deploys randomized field experiments (RFEs) as its main tool. We describe the context in which this movement started, and illustrate in detail how RFEs are designed and implemented in practice, drawing on a series of experiments by Pascaline Dupas and her colleagues on the use of bednets, saving and governance in Kenya. We show that this line of experiments have evolved to address the limitation of obtaining policy-relevant insights from RFEs alone, characterized as their lack of external validity in the literature. After examining the two prominent responses by leading figures of evidence-based development economics, namely machine learning and structured speculation, we propose an alternative methodological strategy that incorporates two sub-fields, namely experimental economics and behavioral economics, to complement RFEs in investigating the data-generating process underlying the treatment effects of RFEs. This strategy highlights promising methodological developments in RFEs neither captured by the two proposals nor recognized by methodologists, and also guides how to combine different sub-fields of economics.

Building collective action at crime hot spots: Findings from a randomized field experiment

The study examined whether Assets Coming Together (ACT), a policing intervention directed at increasing collective action and collective efficacy at crime hot spots in Brooklyn Park, Minnesota, would have impacts on these outcomes, as well as police legitimacy, crime and fear of crime. We used a block-randomized experimental design in which hot spots of crime were randomly allocated to treatment and control conditions. The treatment condition received the ACT program, and the control condition received normal police response. We analyzed crime data using an ANOVA approach, taking into account treatment and block. We analyzed survey data collected at each hot spot using mixed-effects linear regression models with robust standard errors to account for the nesting of responses within hot spots. We find that the intervention increased citizen reporting of collective actions (including collaboration in problem solving and contacts with the police) at hot spots, but it had little impact on general measures of collective efficacy or police legitimacy. Fear of crime increased at the treatment sites. We found that crime reporting was significantly inflated in the treatment sites. Crime outcomes were non-significant without accounting for this reporting inflation, but the treatment areas had a significant crime decrease when adjusting estimates based on reporting inflation. Our experimental findings show that collective actions at hot spots can be encouraged through programs like ACT and that ordinary policing resources—patrol officers in this case—can be successfully used to carry out such programs. We find preliminary evidence that the program also impacted crime. At the same time, our study points to a bias in using official crime data to assess outcomes in programs that encourage community collaboration.

Bidding for Business: Tax Discrimination as Local Industrial Policy

Local and state governments routinely compete for economic activity by offering firms tax breaks and subsidies. While we have empirical evidence to show that large industrial plants produce significant local benefits, we know little about the type of firm that local governments most seek to attract. To further our understanding of local tax discrimination, we conduct a randomized field experiment across 312 communities in the United States. We observe how each of the governments in our sample adjusts its property tax in response to variation in firm characteristics and local economic conditions. We find that our towns pursue a systematic local industrial policy whose goal is to attract manufacturing jobs and work for unskilled labor. Surprisingly, we find no evidence that towns seek to generate rents targeted at the current local population. Local tax policy also appears to disregard agglomeration effects and concerns over industrial diversification.

Does training location matter? Evidence from a randomized field experiment in Rural Indonesia

Informal network helps disseminate agricultural knowledge in the rural area where formal extension is lacking. However, how the interplay between formal training and informal network promotes technology adoption is still under-studied. This paper aims to examine the effects of training locations upon knowledge and adoption of agricultural techniques via strengthened informal and formal social networks. We administer agricultural training of identical contents in farmers’ hometown and in more remote but advanced locations and arrange farmers to travel to the respective locations. Then, we examine locational heterogeneity effects upon conservation and plant-rehabilitation techniques, and social network variables with formal extension services, informal network, and between participants and non-participants. Only farmers trained in the most remote location increase the size and depth of social network with their peers, extension expert, and non-participants upon returning from training. These changes in social networks may encourage them to adopt the technology. While formal training is important for knowledge diffusion, informal network is the key for successful adoption, and these networks are strengthened via training held in the most remote location.

(Poly)phenols, Carotenoids, and Tocochromanols in Corn (Zea mays L.) Kernels As Affected by Phosphate Fertilization and Sowing Time.

Corn (Zea mays L.) growth and development is often limited by the availability of phosphate. We thus hypothesized that phosphate fertilization may increase the contents of (poly)phenols, carotenoids, and tocochromanols (vitamin E) in corn grains. Corn plants cultivated on a soil fertilized with 44 kg phosphorus/ha were compared to plants grown on soil with low plant-available phosphate (1.6 mg CAL-P/100 g of soil), each sown early (April) and late (May) in a randomized field experiment. HPLC-DAD-(HR)-ESI-MSn revealed 19 soluble and 10 insoluble (poly)phenols, comprising phenolic acids, phenolic amines, diferulic, and triferulic acids in corn grains. Contents of individual (poly)phenols, carotenoids, and tocochromanols in whole grains were significantly (p < 0.05) increased by sowing time, but not by phosphate fertilization. In conclusion, low phosphate availability did not impair the biosynthesis of (poly)phenols, carotenoids, and tocochromanols in corn grains.

Эксперименты над нищетой (Нобелевская премия по экономике 2019 года)

&lt;span&gt;В обзорной статье представлены научные достижения трех лауреатов Нобелевской премии по экономическим наукам 2019 г. Выделены основные заслуги лауреатов, отмеченные Нобелевским комитетом: рандомизированные полевые эксперименты, установление новых связей между микро- и макроэкономикой, оригинальная методика исследований причин бедности. Автор выделяет еще одно достижение лауреатов&lt;/span&gt;&lt;span class="myHorizontalScaleForEmDashSpaceBefore"&gt; &lt;/span&gt;&lt;span&gt;–&lt;/span&gt;&lt;span class="myHorizontalScaleForEmDashSpaceAfter"&gt; &lt;/span&gt;&lt;span&gt;освоение способов получения денежных средств на проведение масштабных экспериментов по всему миру. Общая идея лауреатов сегодня признана всеми исследователями и многими практиками: для преодоления бедности гораздо эффективнее не прямая финансовая помощь, а создание условий тому, чтобы сами бедные семьи улучшали свое положение. Из всех таких условий лауреаты считают определяющими решение проблем здравоохранения и образования. Их выводы основаны на результатах многих экспериментальных исследований в странах Африки, Азии и Латинской Америки. Достижения нобелевских лауреатов сопоставляются автором с результатами масштабных исследований новосибирских социологов, которые проводились в 1960-е годы.&lt;/span&gt;

Cream Skimming by Health Care Providers and Inequality in Health Care Access: Evidence from a Randomized Field Experiment

Cream Skimming by Health Care Providers and Inequality in Health Care Access: Evidence from a Randomized Field Experiment

Optional Verification and Signaling in Online Matching Markets: Evidence from a Randomized Field Experiment

Online matching platforms that lack common informational mechanisms such as ratings and reviews often resort to mandatory verification of their users to enhance credibility. However, such mandatory verification may suppress the transmission of useful information for individual users. This study focuses on an alternate role of verification - its ability to serve as a credible signal for a user, when such verification is made optional and visible to other users. In collaboration with a leading online dating platform with no reputation mechanisms and where most of the information is self-disclosed, we design and conduct a randomized field experiment to examine not only who chooses to verify, but also the effectiveness of such optional verification for different types of users. Interestingly, we find users on the two sides use the same signal very differently. Males act consistent with the conventional prediction of signaling with High type males being more likely to opt-in to verification. As for females, we find Medium type females are the most likely to opt-in to verification as compared to High type females. We also find that such differential opt-in decisions are related to the differences in the credibility of the existing key attribute of each side, viz. income for males and beauty for females. In examining the outcomes of verification, we find that verified users receive more contacts from higher-type users, with the High type males and Medium type females benefiting the most. More interestingly, we find that verified users become more proactive and reach out to more and better potential partners. Further, the introduction of this voluntary verification facilitates desirable matching outcomes and benefits the platform as a whole. These findings have useful implications for research as well as practice.

Autonomy and Specificity in Agricultural Technology Adoption: Evidence from Mexico

This paper explores heterogeneity in soil quality, lack of knowledge, and autonomy as explanations for the low adoption of improved agricultural practices. The paper uses data from a randomized field experiment that combined localized soil analyses, tailored input recommendations, extension services, and an in-kind grant. The analysis finds that while neither the degree of recommendation specificity (plot versus cluster level) nor the extent of autonomy (defined as the freedom of choice in spending the in-kind grant) had any effect on adoption during the intervention, farmers with autonomy had substantially higher adoption of improved practices two years after the intervention ended.

Framing Effects on Risk-Taking Behavior: Evidence from a Field Experiment

We exploit testing data to gain a better understanding of the influence of framing effects on decision-making and performance when facing risk. In a randomized field experiment we modified the framing of grading instructions of multiple-choice tests. In the business-as-usual framing (BAU framing), correct answers were framed as gains while wrong answers were framed as losses. Under our proposed loss framing, both non-responses and wrong answers were presented in the loss domain. According to our theoretical model, we expect the change in framing to decrease student non-response and increase student test scores. Our experiment confirms that framing matters for risky decisions in the context of testing. It reveals that under the loss framing, student non-response is reduced by 17%-19%, with risk-aversion being the main candidate for explaining this reduction. However, it fails to increase students’ scores. Indeed, our results support the possibility of impaired performance under the loss framing.

Fun Shopping: A Randomized Field Experiment on Gamification

Gamification utilizes game-like features to engage participants, widely implemented in a variety of contexts. Such an IT-enabled engagement strategy serves as a marketing device to boost sales and customer loyalty. This study focuses on two significant game elements (i.e., badges and leaderboards) that promote consumer motivations and social comparisons. To qualify the impacts, we conduct a randomized field experiment at one of the largest shopping malls in Asia. In the experiment, we contrast the two elements against coupons regarding various shopping outcomes. A two-period design (consisting of the treatment and post-treatment periods) identifies the long-term behavior changes after the treatment removals. The main results suggest that badging and leaderboarding promote sales by 21.5% and 22.5% in the treatment period, respectively, while couponing delivers a more potent effect of 31.7%. In the post-treatment period, the gamification impacts remain significant compared with the baseline, whereas the influence of couponing fades out. Besides, the additional analyses document the salient heterogeneous treatment effects across demographics. We further discover the substantial differences in the within-group heterogeneity across the treatments. Specifically, badging is a balanced tool for attracting the general public, whereas leaderboarding is a double-edged sword that could encourage self-reinforcing or self-banishing. Last, gamification brings more explorations that lead to additional sales and engagements. Overall, the robust results can be translated into actionable strategies to utilize gamification proactively.

Seeing Eye to Eye: Can Leadership Training Align Perceptions of Leadership?

There is a huge gap between how employees see leaders’ behavior and how leaders see themselves regardless of sector and functional area. Because this gap can be a serious problem in managing organizations, scholars have investigated how the gap can be reduced. This article focuses on leadership training and tests whether and under what conditions it narrows the gap. Using quantitative and qualitative data from a randomized field experiment with several hundred Danish leaders from public and private organizations, we find that a yearlong leadership training course decreases the differences between leader and employee perceptions of transformational and transactional leadership behaviors in public sector organizations but not in private organizations. The findings imply that leadership training can be one way for public organizations to align perceptions of leadership.