Randomized Field Experiment Research Articles

Rationally trust, but emotionally? The roles of cognitive and affective trust in laypeople's acceptance of AI for preventive care operations

AbstractArtificial intelligence (AI) is transforming healthcare operations. Nevertheless, particularly in the context of preventive care, little is known about how laypeople perceive and accept AI and change their behavior accordingly. Grounded in a solid theoretical framework of trust, this study bridges this gap by exploring individuals’ acceptance of AI‐based preventive health interventions and following health behavior change, which is critical for preventive care providers’ operational and business performance. Through a randomized field experiment with 15,000 users of a mobile health app complemented by a survey, we first show that the use and disclosure of AI in preventive health interventions improve their effectiveness. However, individuals are less likely to accept and achieve the health behavior change suggested by AI than when they receive similar interventions from health experts. We also observe that the effectiveness of AI‐based interventions can be improved by combining them with human expert opinions, increasing their algorithmic transparency, or emphasizing their genuine care and warmth. These results collectively suggest that, different from conventional technologies, AI's deficient affective trust, rather than comparable cognitive trust, play a decisive role in the acceptance of AI‐based preventive health interventions. This study sheds light on the literature on the role of new‐age information technologies in behavioral operations management, consumer marketing, and healthcare as well as the role of trust in technology acceptance. Valuable practical implications for more effective management of AI for preventive care operations and promotion of consumers’ health behavior are also provided.

Raising student engagement using digital nudges tailored to students' motivation and perceived ability levels

AbstractParticipation in educational activities is an important prerequisite for academic success, yet often proves to be particularly challenging in digital settings. Therefore, this study set out to increase participation in an online proctored formative statistics exam by digital nudging. We exploited targeted nudges based on the Fogg Behaviour Model, highlighting the relevance of acknowledging differences in motivation and ability in allocating nudges to elicit target behaviour. First, we assessed whether pre‐existing levels of motivation and perceived ability to participate are effective in identifying different propensities of responsiveness to plain untailored nudges. Next, we evaluated whether tailoring nudges to students' motivation and perceived ability levels increases target behaviour by means of a randomized field experiment in which 579 first‐year university students received 6 consecutive emails over the course of three weeks to nudge behaviour regarding successful participation in the online exam. First, the results point out that motivation explains differences in engagement as indicated by student responsiveness and participation, whereas the perceived ability to participate does not. Second, the results from the randomized field experiment indicate that tailored nudging did not improve observed engagement. Implications for the potential of providing motivational information to improve participation in online educational activities are discussed, as are alternatives for capturing perceived ability more effectively.Practitioner notesWhat is already known about this topic Participation in educational activities is an important prerequisite for academic success, yet often proves to be particularly challenging in digital settings. Students' internal barriers to online participation and persistence in higher education are lack of motivation and perceived ability. Nudging interventions tackle students' behavioural barriers, and are particularly effective when guided by a theory of behaviour change, and when targeting students who suffer most from those barriers. What this paper adds This study examines whether the Fogg Behaviour Model is suited to guide a nudging intervention with the aim to increase student engagement in online higher education. This study examines whether students with different levels of motivation and perceived ability vary in their online behaviour in response to nudges. This study experimentally evaluates whether targeted nudges—targeted at students' motivation and perceived ability—are more effective than plain (not‐targeted) nudges. Implications for practice and/or policy The results indicate the importance of motivation for performing nudged behaviours regarding successful participation in an online educational activity. The results do not provide evidence for the role of perceived digital ability, yet do show prior performance on a similar educational activity can effectively distinguish between students' responsiveness. Targeted nudges were not more effective than plain nudges, but the potential of other motivational nudges and how to increase perceived performance are discussed.

Front-of-Package Labels on Unhealthy Packaged Foods in India: Evidence from a Randomized Field Experiment.

Policies to require front-of-package labels (FOPLs) on packaged foods may help Indian consumers to better identify foods high in nutrients of concern, including sugar, saturated fat, and sodium, and discourage their consumption, which are outcomes that are critical for preventing rises in diet-related non-communicable disease. The objective was to test whether FOPLs helped Indian consumers identify “high-in” packaged foods and reduce intentions to purchase them. We conducted an in-person randomized experiment (n = 2869 adults between ages 18 and 60 years old) in six states of India in 2022. Participants were randomized to one of five FOPLs: a control label (barcode), warning label (octagon with “High in [nutrient]”), Health Star Rating (HSR), Guideline Daily Amount (GDA), or traffic light label. Participants then viewed a series of packaged foods high in sugar, saturated fat, or sodium with the assigned FOPL, and rated product perceptions and label reactions. Fewer than half of participants in the control group (39.1%) correctly identified all products high in nutrient(s) of concern. All FOPLs led to an increase in this outcome, with the biggest differences observed for the warning label (60.8%, p < 0.001), followed by the traffic light label (54.8%, p < 0.001), GDA (55.0%, p < 0.001), and HSR (45.0%, p < 0.01). While no FOPLs led to a reduction in intentions to purchase the packaged foods, the overall pattern of results suggested that warning labels are the most effective FOPL to help Indian consumers identify unhealthy foods.

The effect of intermediary coverage on disclosure: Evidence from a randomized field experiment

Intermediaries play a significant role in capital markets by reducing disclosure processing costs to market participants. Yet, due to selection and other empirical challenges, the extent and mechanisms through which intermediaries influence firm disclosure are largely unknown. To address these challenges, I conduct a field experiment that randomizes coverage of a mandatory disclosure by two key intermediaries—non-governmental organizations and the media. I show that this coverage causes noncompliant firms to publish the disclosure and firms with low-quality disclosures to improve their existing disclosures. Surprisingly, new disclosures from previously noncompliant firms are on average low quality. Survey evidence and cross-sectional tests suggest the main effect may be driven by firms perceiving pressure from the regulator, whose processing costs may have been reduced. My results yield causal estimates of the effect of intermediary coverage on disclosure and shed light on the nature of the stakeholders that subsequently exert costs on firms.

Effects of social information signals on user engagement: evidence from randomized field experiments

ABSTRACT In this study, we examine the social information signal as a driver of user engagement. Specifically, we model two distinct outcomes related to user engagement: content consumption (e.g. watching a video) and content organisation (e.g. adding a video to a playlist). Given the increasingly social nature of digital platforms, we predict user engagement actions as a function of Peer and Expert social information signals. We employ a series of field experiments on a mobile e-learning application to tease out the distinction between content consumption (CC) and content organisation (CO) based on users’ responses to information signals distinguished based on the source. Our results indicate that the Peer and Expert information signals significantly affect the focal user's CC action but have no effect on their CO actions. These findings, coupled with interesting nuances from two allied field experiments on the same app, reveal the uneven effects of social information signals on both user engagement actions. To sum up, the study presents important implications for predicting user engagement in digital platforms and the social information signaling these platforms can adopt.

Do Handwritten Notes Benefit Online Retailers? A Field Experiment

Despite the technological advances in online retailing, the human touch continues to be essential to relationships between retailers and customers. Although a handwritten note is proposed as a simple means to help establish a personal relationship, its economic significance and alignment with current practices have been insufficiently studied. In this article, the authors evaluate the benefits of a handwritten note for an online retailer and reveal the boundary conditions of its benefits. A randomized field experiment demonstrates that presenting a handwritten note has a positive and significant effect on customer spending. More importantly, the authors observe that an additional marketing incentive (i.e., giveaway, price discount) attenuates this beneficial effect. Further, the study shows that these two effects arise with loyal customers, but not with nonloyal customers. A follow-up experiment reveals that warmth underlies the key findings, lending support for the theoretical prediction. The article concludes with a general discussion that the authors hope will inspire future research on relationship management in online retailing.

Informing employees in small and medium‐sized firms about training: Results of a randomized field experiment

AbstractWe mailed brochures to 10,000 randomly chosen employed German workers who were eligible for a subsidized occupational training program called WeGebAU, informing them about the importance of skills‐upgrading training in general and about WeGebAU in particular. Using survey and register data, we estimate effects of the informational brochure on awareness of the program, on take‐up of WeGebAU and other training, and on subsequent employment. The brochure more than doubles awareness of the program. There are no effects on WeGebAU take‐up, but participation in other (unsubsidized) training increases among employees aged under 45. Short‐term labor market outcomes are not affected.

The impact of employment programs on common mental disorders: A systematic review.

Background:While employment programs were not created with the intent to improve common mental disorders (CMDs), they may have a positive impact on the prevalence, incidence, and severity of CMD by reducing poverty and increasing access to economic mobility.Aim:To examine and synthesize the available quantitative evidence of the impact of employment programs on outcomes of CMD.Methods:Embase, Econlit, Global Health, MEDLINE, APA PsychINFO, and Social Policy and Practice were searched for experimental and quasi-experimental studies which investigated the impact of employment programs on primary and secondary outcomes of a CMD. A narrative synthesis according to Popay was conducted. The methodological quality of studies was assessed with the Cochrane Risk of Bias tool and the Newcastle-Ottawa Assessment Scale.Results:Of the 1,327 studies retrieved, two randomized controlled trials, one retrospective cohort, one pilot study with a non-randomized experimental design, and one randomized field experiment were included in the final review. Employment programs generally included multiple components such as skills-based training, and hands-on placements. Depression and anxiety were the CMDs measured as primary or secondary outcomes within included studies. Findings regarding the impact of employment programs on CMD were mixed with two studies reporting significantly positive effects, two reporting no effects, and one reporting mixed effects. The quality among included studies was good overall with some concerns regarding internal validity.Conclusion:Employment programs may support a decrease in the prevalence, incidence, and severity of CMDs. However, there is high heterogeneity among study effects, designs, and contexts. More research is needed to gain further insight into the nature of this association and the mechanisms of impact. This review highlights the potential for employment programs and other poverty-reduction interventions to be utilized and integrated into the wider care, prevention, and treatment of common-mental disorders.

The influence of food recommendations: Evidence from a randomized field experiment

AbstractWe report results from a randomized field experiment conducted at two food festivals. Our primary aim is to assess the impact of two types of recommendations commonly observed in food settings: most popular and chef's choice. We find that the most popular is the only recommendation that significantly increased consumers' demand relative to a baseline without recommendations. Furthermore, we find that this effect only holds for subjects from outside the local region. Our results are consistent with previous studies indicating information on peers' choices is a powerful influence on consumers' decisions in the absence of prior knowledge.

More than words can say: A randomized field experiment on the effects of consumer self-disclosure in the sharing economy

Mechanism design has emerged as relevant field for information systems management. Specifically, for two-sided markets, design choices have significant effects on user behavior and thus on how supply and demand are matched. On sharing economy platforms, not only actors from the demand side, but also from the supply side must trust their respective counterparts for any transaction to be realized. With this paper, we shed light on the effects of consumer self-disclosure by means of a field experiment on Airbnb (n = 400). Specifically, we investigate how the availability of profile photos and/or self-descriptions affects the chances of their booking requests being accepted. In view of recent anti-discrimination initiatives pushing back profile photos, we consider the compensatory and complementary roles of textual self-descriptions for creating trust. Findings suggest that, while not as powerful as photos, self-descriptions can partially compensate the trust loss induced by the renunciation of photos. Our findings have immediate implications for the design and operation of sharing economy platforms.

Cash-like vouchers improve psychological well-being of vulnerable and displaced persons fleeing armed conflict.

The psychological burden of conflict-induced displacement is severe. Currently, there are 80 million displaced persons around the world, and their number is expected to increase in upcoming decades. Yet, few studies have systematically assessed the effectiveness of programs that assist displaced persons, especially in settings of extreme vulnerability. We focus on eastern Democratic Republic of Congo, where myriad local armed conflicts have driven cycles of displacement for over 20 years. We conducted a within-village randomized field experiment with 976 households, across 25 villages, as part of the United Nations' Rapid Response to Population Movements program. The program provided humanitarian relief to over a million people each year, including vouchers for essential nonfood items, such as pots, pans, cloth, and mattresses. The vouchers led to large improvements in psychological well-being: a 0.32 standard deviation unit (SDU) improvement at 6 weeks, and a 0.18 SDU improvement at 1 year. There is no evidence that the program undermined social cohesion within the village, which alleviates worries related to programs that target some community members but not others. Finally, there was no improvement in child health.

Managing Congestion in a Matching Market via Demand Information Disclosure

In online dating platforms, users tend to focus their attention on a subset of popular peers, leading to congestion. We consider the potential efficacy of an informational intervention, namely, the disclosure of peers’ recent demand. We evaluate our treatment’s efficacy in mitigating congestion and improving matching efficiency, conducting a randomized field experiment at a large mobile dating platform. Our results show that the intervention is particularly effective at improving matching efficiency when presented in tandem with a textual message-framing cue that highlights the capacity implications of the peer demand information. Heterogeneity analyses further indicate that these effects are driven primarily by those users who most contend with congestion in the form of competition, namely, male users and those who rely more heavily upon outbound messages for matches.

Short- and Medium-term Impacts of Employability Training: Evidence from a Randomised Field Experiment in Rwanda

Abstract We use a randomised field experiment to study short-term and medium-term impacts of a training intervention that aims to increase employability of Rwandan (underemployed) youths. The training includes networking and mentorship as well as modules on developing entrepreneurship, technical skills and soft social skills. We evaluate intended outcomes of the training on attitudes towards work, employability and labour market outcomes. We also consider unintended social impacts of the training. The outcomes of the intervention are modest. While on the short term the training positively impacted ‘work readiness’ and networking, we do not document significant effects on employment status or income. In the medium term, we do not find any significant effect. Non-compliance and attrition reduced statistical power of our analysis.

Order Matters: Rating Service Professionals First Reduces Tipping Amount

As customer ratings have become ubiquitous and digital platforms can directly request ratings and tips from customers, it is important to understand how a customer rating influences tipping. The authors investigate whether, how, why, and when the order of rating and tipping affects both consumer behaviors in seven studies, including one quasi-field experiment, one archival data analysis, one randomized field experiment, and four randomized lab experiments. They show that asking customers to rate a service professional before tipping negatively impacts the tip amount but that tipping first does not affect subsequent rating scores. The authors propose that the negative effect of rating on tipping occurs because, when rating a service professional first, customers categorize their feedback as a reward for the service professional, which partially alleviates the felt obligation to tip, resulting in a smaller tip. This negative effect is more evident when customers (1) tip from their own pocket, (2) have higher categorization flexibility, or (3) perceive that the service professional benefits from the rating. Moreover, highlighting the consistency motivation after rating but before tipping can attenuate this effect. These boundary conditions not only support the proposed mechanism and evaluate alternative processes but also have significant practical implications.

Delay Information in Virtual Queues: A Large-Scale Field Experiment on a Major Ride-Sharing Platform

The growing adoption of virtual queues in the service and retail industries has been greatly accelerated in recent times. In collaboration with a major ride-sharing platform, we study how the wait time information (WTI), both its initial magnitude and its subsequent progress over time, impacts customers’ abandonment behavior in virtual queues. The study was conducted through a randomized field experiment that included 1,425,745 rides: one-third of the rides received a neutral WTI, one-third received an optimistic WTI shorter than the neutral WTI (hence less frequent updates), and one-third received a pessimistic WTI (hence more frequent updates). The underlying wait time did not vary across the three groups. We find that both the magnitude of the initial WTI and the update frequency of the WTI have a significant impact on customer abandonment. Specifically, when adjusting the initial WTI by one minute, it did not impact customer abandonment. This is because the magnitude effect of the initial WTI is cancelled out by the opposite update-frequency effect. However, when adjusting the WTI by more than one minute, the magnitude effect dominates: when comparing the pessimistic WTI of four minutes with the neutral initial WTI of two minutes, five minutes with three minutes, and eight minutes with five minutes, customers’ likelihood to abandon increases by 6.2%, 14.1%, and 19.6%, respectively. Similar but opposite effects are found when comparing the optimistic WTI with the neutral WTI. We discuss how firms can use our findings and insights to design and operate better virtual queues. This paper was accepted by Vishal Gaur, operations management.

Relative performance feedback and the effects of being above average — field experiment and replication

In a randomized field experiment, we give first-year students in higher education feedback on their relative performance and show that the type of feedback matters, as feedback increases performance only if it informs the student that they placed above average in the past. We reproduce the results in a replication experiment and investigate mechanisms: The effects are not driven by above-average students reacting particularly well to feedback due to individual characteristics; rather, the information about being above average makes feedback effective. We present evidence that individuals focus on good news to adjust their beliefs, and that feedback can offset disadvantages faced by individuals who are held back by their own underestimation of relative abilities. Once beliefs between controls and the treated converge, repeated treatment does not add to the effects.

Pushy Patients Or Pushy Providers? Effect Of Patient Knowledge On Antibiotic Prescribing In Tanzania.

Antimicrobial resistance is one of the most serious threats to global health, but little progress has been made in reversing its spread. Inappropriate use of antibiotics in humans is a major driver of antimicrobial resistance, and rates are high and growing in lower- and middle-income countries. Antibiotics are thought to be subject to supplier-induced demand, whereby providers prescribe them to patients who do not know they are unnecessary. We conducted a randomized field experiment in 227 private health facilities in Tanzania, with standardized patients presenting uncomplicated upper respiratory tract infection symptoms. Standardized patients were randomly assigned to express knowledge (informed) or not (uninformed) that antibiotics were not required to treat them. There was a very high rate of inappropriate antibiotic prescription, with 86.0percent of informed standardized patients and 94.8percent of uninformed standardized patients prescribed an antibiotic, for an adjusted difference of 7.8percentage points between the groups. This small effect suggests that broader health systems factors are at play and that interventions should be aimed at systems, health facilities, and providers.

Customer Complaint Avoidance: A Randomized Field Experiment of Platform Governance Based on Value Co-Creation and Appropriation

From the theoretical perspectives of value co-creation and value appropriation in platform governance, this study explores how a platform can design motivation mechanisms to induce the proactive efforts of providers to reduce customer complaints. We conducted a field experiment on a major Chinese peer-to-peer real estate platform offering long-term rental properties. Specifically, when a renter (customer) complaint about a host (provider) occurs, we sent reminder messages to other unaffected hosts in the same neighborhoods (as the affected host) and urged them to proactively prevent similar complaints. The reminder messages varied in terms of their emphasis of the roles of different stakeholders in platform-based value co-creation (provider-emphasized vs. customer-emphasized) and how they explicated different value appropriation mechanisms (competition-based vs. cooperation-based). Results show that compared to the control message, customer-emphasized messages effectively motivated providers’ proactive efforts to reduce customer complaints. Contrastingly, provider-emphasized messages led to the undesirable outcome of increasing customer complaints, possibly due to providers’ shirking behavior. We also found that a competition-based value appropriation mechanism strengthened the motivating effect, whereas a cooperation-based mechanism undermined the motivating effect. The study provides important theoretical and practical implications for platforms on the design of effective governance mechanisms.

Standardize or Let a Thousand Flowers Bloom? Interface Design Coordination between Software Platforms and Hosted Apps

Software platform ecosystems are emerging as a dominant model for IT-based services. To accelerate third-party development, platform owners often waive restrictions on app user interfaces to provide app developers with more autonomy in interface design. The literature has indicated, however, that standardized interface design across products is a strategic necessity for firms to introduce and manage their product lines, suggesting that platform owners should not waive restrictions on app user interfaces. This tension between the two approaches raises a crucial research question: Does standardized interface design used by apps and their host platform benefit the platform and apps that it hosts? Specifically, does the coordinated design approach enhance or hinder usage between the platform and apps such that apps attract usage from their host platform and vice versa? Drawing on the theory of basic Gestalts, we theorize the three interface design properties of interface similarity, embeddedness, and synchrony, and extend the theory to investigate how these properties help both apps and their host platform attract each other’s users, thus reflecting derivative usage between them. We performed a randomized field experiment to test the proposed hypotheses. The results reveal that interface similarity, embeddedness, and synchrony of the app-platform interface design strengthen users’ grouping perception of the platform and its apps, thus enhancing platform-to-app forward derivative usage. Contrary to our expectations, we found that this strengthened grouping perception does not improve the app-to-platform backward derivative usage. These findings have crucial implications for the value of interface design in software platform ecosystems.

Perspective-Taking to Reduce Affective Polarization on Social Media

The intensification of affective polarization worldwide has raised new questions about how social media platforms might be further fracturing an already-divided public sphere. As opposed to ideological polarization, affective polarization is defined less by divergent policy preferences and more by strong negative emotions towards opposing political groups, and thus arguably poses a formidable threat to rational democratic discourse. We explore if prompting perspective-taking on social media platforms can help enhance empathy between opposing groups as a first step towards reducing affective polarization. Specifically, we deploy a randomized field experiment through a browser extension to 1,611 participants on Twitter, which enables participants to randomly replace their feeds with those belonging to accounts whose political views either agree with or diverge from their own. We find that simply exposing participants to "outgroup" feeds enhances engagement, but not an understanding of why others hold their political views. On the other hand, framing the experience in familiar, empathic terms by prompting participants to recall a disagreement with a friend does not affect engagement, but does increase their ability to understand opposing views. Our findings illustrate how social media platforms might take simple steps that align with business objectives to reduce affective polarization.