Incentives For Accuracy Research Articles

Belief elicitation under competing motivations: Does it matter how you ask?

Beliefs, alongside preferences, are an important driver of behaviour. While preferences are often inferred by the choices made, measuring beliefs is not straightforward. We design a giving experiment to compare different methods of measuring beliefs, with and without monetary incentives. Consistent with a simple theoretical framework, we find that elicited beliefs about the giving decisions of others are biased and self-serving when no incentive is offered, with non-donors reporting that giving is rare. Offering a simple incentive does not reduce the bias in beliefs; however, this bias is not observed when using an incentivised method which makes the monetary outcome associated with accurately predicting beliefs more prominent. Our findings suggest that when self-interested motivations compete with accuracy incentives, beliefs are sensitive to how they are measured.

Anticipatory Anxiety and Wishful Thinking

Across five experiments (N = 1,714), we test whether people engage in wishful thinking to alleviate anxiety about adverse future outcomes. Participants perform pattern recognition tasks in which some patterns may result in an electric shock or a monetary loss. Diagnostic of wishful thinking, participants are less likely to correctly identify patterns that are associated with a shock or loss. Wishful thinking is more pronounced under more ambiguous signals and only reduced by higher accuracy incentives when participants’ cognitive effort reduces ambiguity. Wishful thinking disappears in the domain of monetary gains, indicating that negative emotions are important drivers of the phenomenon. (JEL C91, D12, D83, D91)

Evaluation of Probabilistic Project Cost Estimates

Evaluation of cost estimates should be fair and give incentives for accuracy. These goals, we argue, are challenged by a lack of precision in what is meant by a cost estimate and the use of evaluation measures that do not reward the most accurate cost estimates. To improve the situation, we suggest the use of probabilistic cost estimates and propose guidelines on how to evaluate such estimates. The guidelines emphasize the importance of a match between the type of cost estimate provided by the estimators and the chosen cost evaluation measure, and the need for an evaluation of both the calibration and the informativeness of the estimates. The feasibility of the guidelines is exemplified in an analysis of a set of 69 large Norwegian governmental projects. The evaluation indicated that the projects had quite accurate and unbiased P50 estimates and that the prediction intervals were reasonably well-calibrated. It also showed that the cost prediction intervals were noninformative with respect to differences in cost uncertainty and, consequently, not useful to identify projects with higher cost uncertainty. The results demonstrate the usefulness of applying the proposed cost estimation evaluation guidelines.

Tokenization of social media engagements increases the sharing of false (and other) news but penalization moderates it

Some major social media companies are announcing plans to tokenize user engagements, derived from blockchain-based decentralized social media. This would bring financial and reputational incentives for engagement, which might lead users to post more objectionable content. Previous research showed that financial or reputational incentives for accuracy decrease the willingness to share misinformation. However, it is unclear to what extent such outcome would change if engagements instead of accuracy were incentivized, which is a more realistic scenario. To address this question, we conducted a survey experiment to examine the effects of hypothetical token incentives. We find that a simple nudge about the possibility of earning token-based points for the achieved user engagements increases the willingness to share different kinds of news, including misinformation. The presence of penalties for objectionable posts diminishes the positive effect of tokenization rewards on misinformation sharing, but it does not eliminate it. These results have policy implications for content moderation practices if platforms embrace decentralization and engagement tokenization.

Hidden inefficiency: Strategic inflation of project schedules

Establishing realistic project plans and completing the resulting business projects on schedule is crucial for organizations striving to effectively utilize their resources. However, incentivizing on-time project delivery may result in moral hazard, as people could respond to estimation accuracy incentives by strategically inflating duration estimates and subsequently prolonging project execution. While the project is delivered on time, the resources are underutilized. We conjecture that the possibility of moral hazard can be mitigated by introducing speed incentives in addition to the schedule accuracy incentives. We conduct a diagnostic test of the effect of accuracy and speed incentives on the process of project estimation and delivery. Our study presents direct empirical evidence that the incentive structure rewarding solely the estimation accuracy can result in hidden inefficiency due to inflated estimates and deliberately slower project execution. When speed incentives are implemented alongside estimation accuracy incentives, the estimates are significantly lower and the project is completed more quickly, without compromising the schedule accuracy or output quality. Aligning the objectives of a project owner with those of planners, by incentivizing the planners for both estimation accuracy and quick project completion, fosters more compressed but still accurate and reliable project schedules, and accelerated project delivery.

Is Attention Produced Optimally? Theory and Evidence From Experiments With Bandwidth Enhancements

This paper develops and deploys a methodology for testing whether people correctly value tools that reduce attention costs. We call these tools bandwidth enhancements (BEs) and characterize how demand for BEs varies with the pecuniary incentives to be attentive, under the null hypothesis of correct perceptions and optimal choice. We examine if the theoretical optimality conditions are satisfied in three experiments. The first is a field experiment ( n = 1373) with an online education platform, in which we randomize incentives to complete course modules and incentives to utilize a plan‐making tool to complete the modules. In the second experiment ( n = 2306), participants must complete a survey in the future. We randomize survey‐completion incentives and how long participants must wait to complete the survey, and we elicit willingness to pay for reminders. The third experiment ( n = 1465) involves a psychometric task in which participants must identify whether there are more correct or incorrect mathematical equations in an image. We vary incentives for accuracy, elicit willingness to pay to reduce task difficulty, and examine the impact of learning and feedback. In all experiments, demand for reducing attention costs increases as incentives for accurate task completion increase. However, in all experiments—and across all conditions—our tests imply that this increase in demand is too small relative to the null of correct perceptions. These results suggest that people may be uncertain or systematically biased about their attention cost functions, and that experience and feedback do not necessarily eliminate bias.

Incentivising accuracy reduces bias in the imposter phenomenon

The imposter phenomenon (IP) involves a bias towards negative performance evaluation. While dispositional theories predict that such biases are inflexible, situational theories suggest that they may respond to accuracy incentives. In this study, we tested whether performance evaluation bias in IP is responsive to financially incentivising accuracy. Participants (n = 225) high and low in IP solved a set of reasoning problems that they were informed were designed to test their intelligence. After the task, participants estimated their performance, in either an incentivised or unincentivized condition. The results replicated previous findings, as participants high in IP evaluated their performance more negatively. However, when incentivised to do so, high IP participants became more positive in their self-assessments. Amongst low IP participants, no effect of incentive was revealed. These findings emphasise the importance of situational factors in understanding and intervening on the biases associated with IP.

Self-Persuasion: Evidence from Field Experiments at International Debating Competitions

Laboratory evidence shows that when people have to argue for a given position, they persuade themselves about the position’s factual and moral superiority. Such self-persuasion limits the potential of communication to resolve conflict and reduce polarization. We test for this phenomenon in a field setting, at international debating competitions that randomly assign experienced and motivated debaters to argue one side of a topical motion. We find self-persuasion in factual beliefs and confidence in one’s position. Effect sizes are smaller than in the laboratory, but robust to a one-hour exchange of arguments and a tenfold increase in incentives for accuracy. (JEL C93, D12, D72, D83, D91, I23)

Rationalizing forecast inefficiency

We show analysts’ own earnings forecasts predict error in their own forecasts of earnings at other horizons, which we argue provides a measure of the extent to which analysts inefficiently use information. We construct our measure by exploiting two sources of variation in analysts’ incentives: (i) more recent forecasts have greater salience at the time of the earnings release so accuracy incentives are higher (lower) at shorter (longer) forecast horizons and (ii) analysts have greater incentives for optimism (pessimism) at longer (shorter) horizons. Consistent with these incentives affecting the incorporation of information into forecasts, we document (i) current year forecasts underweight (overweight) information in shorter (longer) horizon forecasts and (ii) the mis-weighting is more pronounced when recent news is negative—when analysts have greater (weaker) incentives to incorporate the news into shorter (longer) horizon forecasts. Finally, returns tests suggest that forecasts adjusted for the inefficiency we document better represent market expectations of earnings.

Motivated Reasoning in an Explore‐Exploit Task

The current research investigates how prior preferences affect causal learning. Participants were tasked with repeatedly choosing policies (e.g., increase vs. decrease border security funding) in order to maximize the economic output of an imaginary country and inferred the influence of the policies on the economy. The task was challenging and ambiguous, allowing participants to interpret the relations between the policies and the economy in multiple ways. In three studies, we found evidence of motivated reasoning despite financial incentives for accuracy. For example, participants who believed that border security funding should be increased were more likely to conclude that increasing border security funding actually caused a better economy in the task. In Study 2, we hypothesized that having neutral preferences (e.g., preferring neither increased nor decreased spending on border security) would lead to more accurate assessments overall, compared to having a strong initial preference; however, we did not find evidence for such an effect. In Study 3, we tested whether providing participants with possible functional forms of the policies (e.g., the policy takes some time to work or initially has a negative influence but eventually a positive influence) would lead to a smaller influence of motivated reasoning but found little evidence for this effect. This research advances the field of causal learning by studying the role of prior preferences, and in doing so, integrates the fields of causal learning and motivated reasoning using a novel explore-exploit task.

Accuracy incentives and framing effects to minimize the influence of cognitive bias among advanced cancer patients.

Many patients with advanced illness have unrealistic survival expectations, largely due to cognitive biases. Studies suggests that when people are motivated to be accurate, they are less prone to succumb to these biases. Using a randomized survey design, we test whether offering advanced cancer patients (n = 200) incentives to estimate their prognosis improves accuracy. We also test whether presenting treatment benefits in terms of a loss (mortality) rather than a gain (survival) reduces willingness to take up a hypothetical treatment. Results are not consistent with the proposed hypotheses for either accuracy incentives or framing effects.

Misjudgment of interrupted time-series graphs due to serial dependence: Replication of

AbstractInterrupted time-series graphs are often judged by eye. Such a graph might show, for example, patient symptom severity (y) on each of several days (x) before and after a treatment was implemented (interruption). Such graphs might be prone to systematic misjudgment because of serial dependence, where random error at each timepoint persists into later timepoints. An earlier study (Matyas & Greenwood, 1990) showed evidence of systematic misjudgment, but that study has often been discounted due to methodological concerns. We address these concerns and others in two experiments. In both experiments, serial dependence increased mistaken judgments that the interrupting event led to a change in the outcome, though the pattern of results was less extreme than in previous work. Receiver operating characteristics suggested that serial dependence both decreased discriminability and increased the bias to decide that the interrupting event led to a change. This serial dependence effect appeared despite financial incentives for accuracy, despite feedback training, and even in participants who had graduate training relevant to the task. Serial dependence could cause random error to be misattributed to real change, thereby leading to judgments that interventions are effective even when they are not.

How Robust Is Evidence of Partisan Perceptual Bias in Survey Responses?

Abstract Partisans often offer divergent responses to survey items ostensibly unrelated to politics. These gaps could reveal that partisanship colors perception or, alternatively, that in answering survey questions, individuals communicate partisan proclivities by providing insincere, or “expressive” responses, to send a partisan message. This study tests two techniques for reducing expressive responding that (1) avoid criticisms about using monetary incentives for accuracy, which have reduced measured partisan differences for objective facts; and (2) can be used in contexts where incentives are infeasible, such as when objective benchmarks for correct responses are unavailable. This study experimentally tests these techniques in replicating a study that found that partisanship affected attractiveness evaluations. These interventions, which allow partisans to express their partisan sentiments through other survey items, substantially reduce apparent partisan differences in beauty evaluations and show that standard survey items likely confound sincere partisan differences with elements of expressive responding.

Rationalizing Forecast Inefficiency

We show analysts’ own earnings forecasts predict error in their own forecasts of earnings at other horizons, which we argue provides a measure of the extent to which analysts inefficiently use information. We construct our measure by exploiting two sources of variation in analysts’ incentives: (i) more recent forecasts have greater salience at the time of the earnings release so accuracy incentives are higher (lower) at shorter (longer) forecast horizons and (ii) analysts have greater incentives for optimism (pessimism) at longer (shorter) horizons. Consistent with these incentives affecting the incorporation of information into forecasts, we document (i) current year forecasts underweight (overweight) information in shorter (longer) horizon forecasts and (ii) the mis-weighting is more pronounced when recent news is negative—when analysts have greater (weaker) incentives to incorporate the news into shorter (longer) horizon forecasts. Finally, returns tests suggest that forecasts adjusted for the inefficiency we document better represent market expectations of earnings.

Comparing fast thinking and slow thinking: The relative benefits of interventions, individual differences, and inferential rules

AbstractResearch on judgment and decision making has suggested that the System 2 process of slow thinking can help people to improve their decision making by reducing well-established statistical decision biases (including base rate neglect, probability matching, and the conjunction fallacy). In a large pre-registered study with 1,706 participants and 23,292 unique observations, we compare the effects of individual differences and behavioral interventions to test the relative benefits of slow thinking on performance in canonical judgment and decision-making problems, compared to a control condition, a fast thinking condition, an incentive condition, and a condition that combines fast and slow thinking. We also draw on the rule-based reasoning literature to examine the benefits of having access to a simple form of the rule needed to solve a specific focal problem. Overall, we find equivocal evidence of a small benefit from slow thinking, evidence for a small benefit to accuracy incentives, and clear evidence of a larger cost from fast thinking. The difference in performance between fast-thinking and slow-thinking interventions is comparable to a one-scale point difference on the 4-point Cognitive Reflection Test (CRT). Inferential rules contribute unique explanatory power and interact with individual differences to support the idea that System 2 benefits from a combination of slower processes and knowledge appropriate to the problem.

Comparing Thinking Fast and Thinking Slow: Interventions, Individual Differences, and Rules

Research on judgment and decision making has suggested that the System 2 process of ‘thinking slow’ can help people to improve their decision making by reducing well-established statistical decision biases (including base rate neglect, probability matching, and the conjunction fallacy). In a large (N = 1709) pre-registered study, we compare the effects of individual differences and behavioral interventions to test the relative benefits of thinking slow compared to a control condition, a thinking fast condition, an incentive condition, and a condition that combines fast and slow thinking. We also draw on the rule-based reasoning literature to examine the benefits of possessing a simple form of a rule needed to solve a specific focal problem. Overall, we find a large cost from thinking fast and a small benefit from thinking slow or from pursuing accuracy incentives. The difference in performance between interventions to think fast and think slow is comparable to a one-scale point difference on the 4-point Cognitive Reflection Test (CRT). Inferential rules contribute unique explanatory power and interact with specific conditions and individual differences to support the idea that System 2 benefits from a combination of slower processes and appropriate task knowledge.

Inferring Sexually Transmitted Infection Risk From Attractiveness in Online Dating Among Adolescents and Young Adults: Exploratory Study.

BackgroundSexually transmitted infection (STI) rates are on the rise among adolescents and young adults in the United States. With the popularity of online dating, adolescents and young adults must increasingly rely on limited cues to make initial judgments about potential sexual partners, including judgments about STI risk.ObjectiveThis study aimed to assess whether in the context of online dating, an attractiveness heuristic would be used for STI risk assessment. We hypothesized that consistent with research on halo effects, decision makers would judge more attractive people to be less likely to have STIs.MethodsIn a survey experiment, we asked participants to determine which individual in each of 20 sets of paired photographs was enrolled in a personals website for people with publicly disclosed STIs.ResultsDespite financial incentives for accuracy and high levels of self-confidence in their judgments, participants performed no better than chance at identifying individuals with self-reported STIs. Contrary to our hypothesis, however, more attractive people were judged as being more likely to have an STI. This relationship appears to be mediated by inferences regarding the target individual’s sexual behavior, with more attractive individuals considered to have more partners.ConclusionsOn showing adolescents and young adults photographs offering no diagnostic information about STIs, they appeared to use attractiveness as a cue for sexual risk, which was mediated by the belief that attractive individuals have more sexual opportunities. Health care providers may wish to address this heuristic process among their adolescent patients in discussions about sexual health.

Hidden Inefficiency: Strategic Inflation of Project Schedules

Establishing realistic project plans and completing the resulting business projects on schedule is crucial for organizations striving to effectively utilize their resources. However, incentivizing on-time project delivery may introduce moral hazard, as people may respond to estimation accuracy incentives by strategically inflating project duration estimates and subsequently prolonging the project execution. While the project is delivered on time, the resources are underutilized. We conjecture that these perverse effects can be mitigated by introducing incentives to complete the project as soon as possible (speed incentives) in addition to the schedule accuracy incentives. We conduct a diagnostic test of the effect of accuracy and speed incentives on the process of project estimation and delivery. Our study presents direct empirical evidence that the incentive structure rewarding solely the estimation accuracy can result in hidden inefficiency due to inflated estimates and deliberately prolonged project execution. However, when speed incentives are implemented alongside estimation accuracy incentives, the estimates are significantly lower and the project is completed more quickly, without compromising the schedule accuracy. Aligning the objectives of a project owner with those of planners, by incentivizing the planners for both estimation accuracy and quick project completion, can therefore foster more compressed but still accurate and reliable project schedules and accelerated project delivery. In summary, our study contributes to the economic analysis of incentive structures in project management by identifying a hidden inefficiency that could be present in projects delivered “on time” and by pointing out a mechanism that mitigates the risk of moral hazard.

The Effects of Certain and Uncertain Incentives on Effort and Knowledge Accuracy

AbstractIn many situations, incentives exist to acquire knowledge and make correct political decisions. We conduct an experiment that contributes to a small but growing literature on incentives and political knowledge, testing the effect of certain and uncertain incentives on knowledge. Our experiment builds on the basic theoretical point that acquiring and using information is costly, and incentives for accurate answers will lead respondents to expend greater effort on the task and be more likely to answer knowledge questions correctly. We test the effect of certain and uncertain incentives and find that both increase effort and accuracy relative to the control condition of no incentives for accuracy. Holding constant the expected benefit of knowledge, we do not observe behavioral differences associated with the probability of earning an incentive for knowledge accuracy. These results suggest that measures of subject performance in knowledge tasks are contingent on the incentives they face. Therefore, to ensure the validity of experimental tasks and the related behavioral measures, we need to ensure a correspondence between the context we are trying to learn about and our experimental design.

Anticipatory Anxiety and Wishful Thinking

It is widely hypothesized that anxiety and worry about an uncertain future lead to the adoption of comforting beliefs or However, there is little direct causal evidence for this effect. In our experiment, participants perform a visual pattern recognition task where some patterns may result in the delivery of an electric shock, a proven way of inducing anxiety. Participants engage in significant wishful thinking, as they are less likely to correctly identify patterns that they know may lead to a shock. Greater ambiguity of the pattern facilitates wishful thinking. Raising incentives for accuracy does not significantly decrease it.