Randomized Field Experiment Research Articles

Doing Well By Doing Good: Improving Store Performance with Employee-friendly Scheduling Practices at the Gap, Inc

We estimate the causal effects of responsible scheduling practices on store financial performance at the US retailer Gap, Inc. The randomized field experiment evaluated a multi-component intervention designed to improve dimensions of work schedules – inconsistency, unpredictability, inadequacy, and lack of employee control – shown to undermine employee well-being. The experiment was conducted in 28 stores in the San Francisco and Chicago metropolitan areas for nine months between November 2015 and August 2016. Intent-to-treat (ITT) analyses indicate that implementing responsible scheduling practices increased store productivity by 5.1%, a result of increasing sales (by 3.3\%) while also decreasing labor (by 1.8%). Drawing on qualitative interviews with managers and quantitative analyses of employee shift-level data, we offer evidence that the intervention improved financial performance through improved store execution. Our experiment provides evidence that responsible scheduling practices that take worker well-being into account can enhance store productivity by motivating additional employee effort and reducing barriers to employees adhering to the scheduled labor plan.

Creating Social Contagion Through Firm-Mediated Message Design: Evidence from a Randomized Field Experiment

We study whether and how a firm can enhance social contagion simply by varying the message shared by customers with their friends. We focus on two key components of information contained in the message—information about the sender’s purchase status prior to referral and information about the existence of referral rewards—and their impacts on the recipient’s purchase decision and further referral behavior. In collaboration with an online daily-deal platform, we design and conduct a large-scale randomized field experiment involving more than 75,000 customers to identify the causal effect of different message designs on creating social contagion. We find that small variations in message content can have a significant impact on both recipients’ purchase and referral behaviors. Specifically, we find that (1) adding only information about the sender’s purchase status increases the likelihood of the recipient’s purchase but has no impact on follow-up referrals, (2) adding only information about referral reward increases the recipient’s follow-up referrals but has no impact on purchase likelihood, and (3) adding information about both the sender’s purchase and the referral rewards increases neither the likelihood of purchase nor follow-up referrals. We build a model to analyze the tradeoff between more adoption and more diffusion and implement the best-performing message design in a production system with millions of shared messages per year (with a projected increase in net profits of more than US$1 million per year). We further exploit the rich heterogeneity in deal, recipient, sender, and social-tie characteristics and examine the mechanisms underlying the effect of message design. The results suggest that both social learning and social utility are at work, and the attenuation in the recipient’s purchase is mainly driven by a decrease in social learning resulting from credibility concerns. The findings of the study provide actionable guidelines to firms for optimal design of messages at the aggregate and more granular levels. This paper was accepted by Anandhi Bharadwaj, information systems.

The Effects of Chess Instruction on Academic and Non-cognitive Outcomes: Field Experimental Evidence from a Developing Country

We conduct a randomized field experiment to investigate the benefits of an intensive chess training program undertaken by primary school students in a developing country context. We examine the effects on academic outcomes, and a number of non-cognitive outcomes: risk preferences, patience, creativity and attention/focus. Our main finding is that chess training reduces the level of risk aversion almost a year after the intervention ended. We also find that chess training improves math scores, reduces the incidence of time inconsistency and the incidence of non-monotonic time preferences. However, these (non-risk preference) results are less conclusive once we account for multiple hypothesis testing. We do not find any evidence of significant effects of chess training on other academic outcomes, creativity, and attention/focus.

Inducing flexibility of household electricity demand: The overlooked costs of reacting to dynamic incentives

Flexible electricity demand through dynamic pricing11We use the term ‘dynamic pricing’ to describe electricity tariffs that are continually adjusted to (partially) reflect the current price at the spot market or at the day ahead market such as real-time pricing. Time of use pricing is not covered by our use of the term because time of use tariffs are pre-declared and not continually adjusted to reflect current market conditions. Note that the term dynamic pricing is sometimes defined more broadly e.g. as any tariff structure different from a flat tariff. is becoming more critical for net-balancing as the share of intermittent wind and solar electricity production increases. However, people are prone to habits and regularity, so this kind of dynamic pricing schemes may also impose a greater cost of reacting to price incentives. In a randomized field experiment we compare the household consumer welfare effects of reacting to dynamic electricity price incentives to those of reacting to time of use incentives. We utilized smart-metered hourly electricity consumption data to unobtrusively measure treatment effects. We found that a dynamic incentive of 1 DKK/kWh induced an average change in electricity consumption per household of 0.182 kWh during the 3-hour target period. The corresponding time of use incentive induced an average change in electricity consumption of 0.351 kWh. This implies that dynamic incentives reduce consumer surplus from reacting by half, compared to reacting to corresponding time of use incentives and therefore that households derive a substantial welfare benefit from the regularity and predictability provided by time of use pricing. This suggests that the tariff structure that optimally balances demand flexibility and the associated consumer welfare costs is likely a combination of time of use and dynamic pricing incentives.

Impact of Incentive Mechanism in Online Referral Programs: Evidence from Randomized Field Experiments

ABSTRACT Despite the growing popularity of online referral programs, a minimal amount is known regarding the theoretical foundations that drive the key actions associated with successful referrals. In this paper, we study which type of referral reward structure is most effective in maximizing word-of-mouth by conducting two randomized experiments in mobile gaming context. Specifically, we examine the effect of three incentive schemes: selfish reward (inviter gets all the reward), equal-split reward (50-50 split), and generous reward (invitee gets all the reward). Consistent across the two experiments, we find that pro-social referral incentive schemes, namely the equal-split and generous schemes, tend to dominate purely selfish schemes in creating WOM. Our mechanism-level analysis shows that both equal-split and generous schemes result in higher number of conversions by significantly increasing the invitee’s likelihood to accept referrals, which we further show that is partially due to selective and better targeted referrals. Our results contribute to the understanding of the optimal design of online referral programs and provide important implications for designing effective referral reward schemes in the digital world.

Cream Skimming by Health Care Providers and Inequality in Health Care Access: Evidence from a Randomized Field Experiment

Using a randomized field experiment, we show that health care specialists cream-skim patients by their expected profitability. In the German two-tier system, outpatient reimbursement rates for both public and private insurance are centrally determined but are more than twice as high for the privately insured. In our field experiment, following a standardized protocol, the same hypothetical patient called 991 private practices in 36 German counties to schedule appointments for allergy tests, hearing tests and gastroscopies. Practices were 7% more likely to offer an appointment to the privately insured. Conditional on being offered an appointment, wait times for the publicly insured were twice as long than for the privately insured. Our findings show that structural differences in reimbursement rates lead to structural differences in health care access.

Effect of Drought on Various Agro-physical Parameters of Chickpea (<i>Cicer arietinum</i> L.) Genotypes in a Field Experiment

Drought stress is one of the major factors limiting the growth and development of legumes. In order to improve water deficit tolerance of this crop, several techniques have been put into practice such as seed priming or the selection of tolerant genotypes to water stress. In 2020, field experiment was conducted to assess the drought tolerance of thirty-six chickpea genotypes (<i>C. arietinum</i>) by analyzing the behaviour of certain physiological and biochemical parameters of plants harvested in a randomized field experiment. The genotypes analysed presented a diversity of behavior concerning the accumulation of mineral elements under drought. The results showed an accumulation of inorganic ions, especially calcium and potassium (1.8 and 2 mg.g^-1, respectively) and increased proline and protein content (3.4 and 1.7 mg.g^-1, respectively) has been observed in drought tolerant chickpea genotypes. Also, the results obtained showed that the P contents in the aerial parts are generally higher for plants with a high biomass, such as the case of genotypes V36 and V32. This tends to prove the positive effect of P on plant growth. After analysis of the various parameters, the results obtained allowed us to classify the tolerant genotypes:, V36, V38 andV41, intermediates: V40 and V4 and sensitive: V17 and V28.

Social Learning in Prosumption: Evidence from a Randomized Field Experiment

Digital technologies enable consumers to actively participate in the product design and production process for a wide range of products, leading rise to the concept of a 'prosumer'. A significant portion of the value for such products is generated through the prosumption process, and a variety of firms are investing in building such capabilities. However, a major, largely unexplored, friction in prosumption is the customers’ effort involved to undertake a creative exercise of designing products and extracting value from it. In this study, we ask whether and how social learning, the act of showing creations made by other customers to the focal customer, can ameliorate such friction. Arguably, by showing others’ product designs to the focal customer, the firm may help the customers gain design ideas and garner knowledge about product features. Such an action is also likely to influence their belief about their own ability, namely, their self-efficacy, to design a valuable product that they would like to purchase. This implies that, if not carefully done, displaying others’ design could be detrimental to prosumption. Certain designs may be perceived to be out of the creative reach of the focal user, and therefore reduce their likelihood of designing a product and purchasing it. If social learning is effective, what can we say about the nature of images to show to different sub-groups of users? In close collaboration with an e-commerce platform specialized in customized photo products, we examine the effectiveness of social learning by means of a large scale in-vivo randomized field experiment. We exogenously vary both the availability of others’ design and the characteristics of images shown to the treated users. Our analysis shows that showing other users’ design can be highly effective in influencing the purchase and design behavior of the focal customer, but firms must choose the right customers and carefully select the type of user image design for display. We develop a novel ‘honest-bagging’ approach guided by principles of causal forests to personalize the high-dimensional treatment around which images to show to what types of users.

Stigma and Misconceptions in the Time of the Covid-19 Pandemic: A Field Experiment in India

A hidden cost of the COVID-19 pandemic is the stigma associated with the disease for those infected and groups that are considered as more likely to be infected. This paper examines whether the provision of accurate and focused information about COVID-19 from a reliable source can reduce stigmatization. We carry out a randomized field experiment in the state of Uttar Pradesh, India, in which we provide an information brief about COVID-19 by phone to a random subsample of participants to address stigma and misconceptions. We find that the information brief decreases stigmatization of COVID-19 patients and certain groups such as religious minorities, lower-caste groups, and frontline workers (healthcare, police), and reduces the belief that infection cases are more prevalent among certain marginalized social and economic groups (Muslims, low caste, rural-poor population). We provide suggestive evidence that improved knowledge about the prevention and transmission of COVID-19 and reduced stress about the disease are important channels for these effects.

Small Group Instruction to Improve Student Performance in Mathematics in Early Grades: Results from a Randomized Field Experiment

Small Group Instruction to Improve Student Performance in Mathematics in Early Grades: Results from a Randomized Field Experiment

Ethnic Mixing in Early Childhood: Evidence from a Randomized Field Experiment and a Structural Model

We study the social integration of ethnic minority children in the context of an early childhood program conducted in Turkey aimed at preparing 5-year-old native and Syrian refugee children for primary school. We randomly assign children to groups with varying ethnic composition and find that exposure to children of the other ethnicity leads to an increase in the formation of interethnic friendships, especially for Turkish children. We also find that the Turkish language skills of Syrian children are better developed in classes with a larger presence of Turkish children. We then develop a model of friendship formation with two key mechanisms: preference bias and congestion in the friendship formation process. Structural estimation of the model suggests that interethnic exposure reduces the share of own-ethnicity friends (homophily) and has a non-monotonic effect on the propensity to form own-ethnicity friendships beyond what would be expected given the size of the group (inbreeding homophily). Counterfactual analysis indicates that improvement in the language skills of Syrian children can offset more than half of the effect that ethnic bias has on friendship formation patterns. Finally, we find that for Syrian children exposure to Turkish children in the pre-school program has a long-term effect on primary school absenteeism.

The Effect of Information Nudges on Energy Saving: Observations from a Randomized Field Experiment in Finland

The Effect of Information Nudges on Energy Saving: Observations from a Randomized Field Experiment in Finland

Mind the App: Information Design and Consumer Behavior

In a randomized field experiment, I test if the way in which information is presented influences consumer behavior. Users of an online account aggregation app received a personalized index representing their net worth as a lifetime monthly cash flow. The presentation of the index varied in the framing used to describe the index and in the salience of the comparison between the index and the user's historical spending. Consumers that received a consumption frame (promoting a mental reflection about the affordability of future consumption) and a salient comparison with historical spending decreased their discretionary spending by 15% relative to other treatments throughout the eight months of the experiment. The effect persisted for an additional eight months after the removal of the experiment content from the app. The sensitivity of consumers' spending to information design presents opportunities for policies aimed at influencing saving rates.

From Free to Paid: Monetizing a Non-advertising-based App

Non-advertising-based mobile apps face several critical challenges when trying to monetize their free services: the choice of pricing strategies (whether to continue offering limited free services to current users) and aspects of product design (whether to provide exclusive secondary offerings to paying users). We implemented a large-scale randomized field experiment to test the causal effects of such pricing strategies and product design. Results show that providing either feature decreases existing app users’ willingness to subscribe to the service after monetization. Further, we find a positive interaction between providing limited free services and exclusive secondary offerings on subscriptions. We provide a discussion of potential mechanisms that might be at play, and conduct robustness checks to rule out several alternative explanations. To assess generalizability, we conducted a second field experiment with a different operationalization of limited free content and obtained consistent results. We also report the results from the actual implementation of our proposed monetization strategy by the firm. Our research provides guidance on possible theoretical underpinnings of users’ responses and important managerial implications for app monetization.

A Data-Driven Research of Sales and Purchases on JD.com Platform

Unlike consumers in the mall or supermarkets, online consumers are “intangible” and their purchasing behaviors are affected by multiple factors, including product pricing, promotion and discounts, quality of products and brands, and the platforms where they search for the product. In this research, I study the relationship between product sales and consumer characteristics, the relationship between product sales and product qualities, demand curve analysis, and the search friction effect for different platforms. I utilized data from a randomized field experiment involving more than 400 thousand customers and 30 thousand products on JD.com, one of the world’s largest online retailing platforms. There are two focuses of the research: 1) how different consumer characteristics affect sales; 2) how to set price and possible search friction for different channels. I find that JD plus membership, education level and age have no significant relationship with product sales, and higher user level leads to higher sales. Sales are highly skewed, with very high numbers of products sold making up only a small percentage of the total. Consumers living in more industrialized cities have more purchasing power. Women and singles lead to higher spending. Also, the better the product performs, the more it sells. Moderate pricing can increase product sales. Based on the research results of search volume in different channels, it is suggested that it is better to focus on app sales. By knowing the results, producers can adjust target consumers for different products and do target advertisements in order to maximize the sales. Also, an appropriate price for a product is also crucial to a seller. By the way, knowing the search friction of different channels can help producers to rearrange platform layout so that search friction can be reduced and more potential deals may be made.

Selection on Welfare Gains: Experimental Evidence from Electricity Plan Choice

We study a problem in which policymakers need to screen self-selected individuals by unobserved heterogeneity in social welfare gains from a policy intervention. In our framework, the marginal treatment effects and marginal treatment responses arise as key statistics to characterize social welfare. We apply this framework to a randomized field experiment on electricity plan choice. Consumers were offered socially efficient dynamic pricing with randomly assigned take-up incentives. We find that price-elastic consumers—who generate larger welfare gains—are more likely to self-select. Our counterfactual simulations quantify the optimal take-up incentives that exploit observed and unobserved heterogeneity in selection and welfare gains.

How Local Government Vaccination Campaigns Can Increase Willingness to Get Vaccinated Against Covid-19: A Field Experiment on Psychological Ownership

With the increasing availability of life-saving vaccines against the SARS-CoV-2 virus, government agencies face the challenge of promoting vaccine uptake. Thus, encouraging vaccine uptake marks an urgent policy challenge in fighting the COVID-19 pandemic. This study builds on the theory of psychological ownership to design a behaviorally-inspired local government vaccination campaign. We conducted a large-scale, cluster-randomized field experiment (N=27,298 residents nested in 6,442 addresses) delivered to all registered residents of a German municipality via an official mailing campaign. The campaign included a psychological ownership intervention designed to boost residents’ intentions to get vaccinated – measured through unique link clicks on a municipal website where people can schedule a COVID-19 vaccination appointment. Findings suggest that adding possessive pronouns (i.e., “YOUR vaccination”) increases vaccination intentions by 39 per cent, or 2.5 percentage points (p < .001 [95 per cent CI=1.8 per cent, 3.3 per cent], control letter: 6.4 per cent, treatment letter: 8.9 per cent). The discussion outlines the value of using psychological ownership-based nudges to increase vaccine uptake and other desirable behaviors more generally.

Selecting Cover Images for Restaurant Reviews: AI vs. Wisdom of the Crowd

Restaurant review platforms, such as Yelp and Tripadvisor, routinely receive large numbers of photos in their review submissions. These photos provide significant value for users who seek to compare restaurants. In this context, the choice of cover images (i.e., representative photos of the restaurants) can greatly influence the level of user engagement on the platform. Unfortunately, selecting these images can be time consuming and often requires human intervention. At the same time, it is challenging to develop a systematic approach to assess the effectiveness of the selected images. In this paper, we collaborate with a large review platform in Asia to investigate this problem. We discuss two image selection approaches, namely crowd-based and AI-based systems. The AI-based system we propose learn complex latent image features, which is further enhanced by transfer learning to overcome the scarcity of labeled data. We collaborate with the platform to deploy our AI-based system through a randomized field experiment to carefully compare both systems. We find that the AI-based system outperforms the crowd-based counterpart and boosts user engagement by 12.43%-16.05% on average. We then conduct empirical analyses on observational data to identify the underlying mechanisms that drive the superior performance of the AI-based system. Finally, we infer from our findings that the AI-based system outperforms the crowd-based system for restaurants with a (i) longer tenure on the platform, (ii) more expensive offering, (iii) lower star rating, and (iv) limited number of user-generated photos.

Igniting Deliberation in High Stake Decisions: A Field Study

We conduct a large scale randomized field experiment to study whether providing recipients – 42,454 Chinese households in a rural area – with information on the costs of a real decision they make can help to improve the quality of their choices. The decisions are of high financial impact, as the objects of deliberation – air conditioners – have upfront prices exceeding the average monthly salary of a household. Besides providing nominal cost information, we conduct two additional treatments, where we either present the same information by making the real opportunity costs salient, or by administering the information via a quiz. The former aims at facilitating the comparison of effective costs, while the latter aims at enhancing attention and cognitive involvement. We find that providing cost information substantially affects the choices made, and reduces the decision mistakes, in particular in the two additional treatments.

The Impact of Short-Term Employment for Low-Income Youth: Experimental Evidence from the Philippines

We use a randomized field experiment to test the causal impact of short-term work experience on employment and school enrollment among disadvantaged, in-school youth in the Philippines. This experience leads to a 4.4 percentage point (79-percent) increase in employment 8 to 12 months later. While we find no aggregate increase in enrollment, we also do not find that the employment gains push youth out of school. Our results are most consistent with work experience serving as a signal of unobservable applicant quality, and these findings highlight the role of temporary work as a stepping- stone to employment for low-income youth.