Base-rate Problems Research Articles

Conflict detection and base-rate extremity

People tend to ignore the probabilistic rules cued by the base-rate information and rely on the heuristic intuition cued by the descriptive information to make “stereotypical” responses in base-rate problems. Conflict detection studies have shown that reasoners can detect conflicts between heuristic intuition and probabilistic considerations despite ultimately stereotypical responses. However, these studies primarily used extreme base-rate tasks. A critical open question is the extent to which successful conflict detection relies on an extreme base rate. The present study explores this issue by manipulating the base-rate extremity of problems in which the descriptive information and the base-rate information conflict or not. As a result, when reasoners made stereotypical responses in the conflict version of the moderate base-rate task, they took longer to respond, had lower confidence in their responses, and were slower to evaluate their confidence than in the no-conflict version of the task. All three measures indicate that stereotypical reasoners can stably detect conflict in moderate base-rate tasks, which expands the scope of successful conflict detection. Moreover, our response confidence data found a larger detection effect size in the extreme base-rate condition than in the moderate base-rate condition. This suggests that conflict detection is more efficient as the base-rate extremity increases. Implications for the boundary conditions of conflict detection are discussed.

The paradoxical effects of time pressure on base rate neglect

Base rate neglect refers to the well-documented tendency for people to primarily rely on diagnostic information to identify event probabilities while discounting information about relative probabilities (base rates). It is often postulated that using base rate information requires some form of working memory intensive processes. However, recent studies have put this interpretation into doubt, showing that rapid judgments can also lead to base rate use. Here we examine the idea that base rate neglect can be explained by the degree of attention paid to diagnostic information, which predicts that having more time should lead to greater rates of base rate neglect. Participants were presented with base rate problems either with a limited time to respond or with no time restrictions. Results show that having more time results in a decrease in base rate use.

When Type 2 Processing Misfires: The Indiscriminate Use of Statistical Thinking about Reasoning Problems.

Research on dual-process theories of judgment makes abundant use of reasoning problems that present a conflict between Type 1 intuitive responses and Type 2 rule-based responses. However, in many of these reasoning tasks, there is no way to discriminate between the adequate and inadequate use of rules based on logical or probabilistic principles. To experimentally discriminate between the two, we developed a new set of problems: rule-inadequate versions of standard base-rate problems (where base rates are made irrelevant). Across four experiments, we observed conflict sensitivity (measured in terms of response latencies and response confidence) in responses to standard base-rate problems but also in responses to rule-inadequate versions of these problems. This failure to discriminate between real and merely apparent (or spurious) conflict suggests that participants often misuse statistical information and draw conclusions based on irrelevant base rates. We conclude that inferring the sound use of statistical rules from normatively correct responses to standard conflict problems may be unwarranted when this kind of reasoning bias is not controlled for.

Can a fast thinker be a good thinker? The neural correlates of base-rate neglect measured using a two-response paradigm

ABSTRACT Traditionally, it has been assumed that logical thinking requires deliberation. However, people can also make logical responses quickly, exhibiting logical intuitions. We examined the neural correlates of logical intuitions by administering base rate problems during fMRI scanning using a two-response paradigm where participants first responded quickly and then reflectively to problems that did or did not pit a normative response against an intuitively-cued stereotypical response (i.e., conflict vs. non-conflict problems). As predicted, participants were less likely to make judgments in accordance with base rates on conflict problems. Critically, in only 4% of cases did longer deliberation change an initially biased response to a normatively correct response. The fMRI data revealed that intuitively-made initial biased judgments nevertheless activate regions typically involved in cognitive control, executive functions and attention, including anterior, inferior, middle and superior frontal cortex, suggesting that even when errors are made, there might be very early awareness of conflict.

From slow to fast logic: the development of logical intuitions

Recent reasoning accounts suggest that people can process elementary logical principles intuitively. These controversial “logical intuitions” are believed to result from a learning process in which developing reasoners automatize their application. To verify this automatization hypothesis, we contrasted the reasoning performance of younger (7th grade) and older (12th grade) reasoners with a two-response paradigm. Participants initially responded with the first intuitive response that came to mind and subsequently were allowed to deliberate on classic “bias” problems (base-rate problems and syllogisms). Results showed that in addition to showing less deliberate correction of an initial erroneous response, younger reasoners were specifically less likely to generate the correct response from the outset. The findings lend credence to the role of a developmental automatization process and indicate that developmental improvements in reasoning accuracy are at least partially driven by an improvement in the accuracy of our intuitions.

Metacognitive Feelings, Conflict Detection and Illusion of Linearity

Studies on conflict detection have suggested that people are sensitive to conflict between their heuristic judgment and logical or probabilistic principles, but due to the inhibition failure, they do not disregard appealing heuristic answer. However, these studies were mostly conducted on syllogistic reasoning and base-rate problems. The question is whether findings about conflict detection can be applied to other materials and areas of reasoning. Current study focused on the illusion of linearity, in which people over-rely on linearity heuristic. Therefore, the aim of this study was to examine whether students detect the conflict between heuristic answer and logical/mathematical principles. Participants were 113 secondary school students from Zagreb, Croatia. Data were collected using a computer program, which consisted of instructions, 20 problems (10 linear and 10 non-linear) and sociodemographic questions. Problems were presented randomly in multiple-choice format and had three offered answers (correct answer, distractor, "none of the answers"). Response time for each problem was also measured. Results demonstrated that students mostly solved non-linear problems incorrectly and in accordance with linearity heuristic. Furthermore, the analysis of metacognitive feelings of confidence and difficulty revealed that students detected conflict between heuristic answer and mathematical principles. Moreover, overriding heuristic answer and generating correct answer to non-linear problems resulted in increased response time. Comparison between metacognitive feelings and response time in linear and non-linear problems indicates the importance of processing fluency and inhibition failure in the occurrence of the illusion of linearity.

Fast and slow thinking: Electrophysiological evidence for early conflict sensitivity

Popular dual process models have characterized reasoning as an interplay between fast, intuitive (System 1) and slow, deliberate (System 2) processes, but the precise nature of the interaction between the two systems is much debated. Here we relied on the temporal resolution of electroencephalogram (EEG) recordings to decide between different models. We adopted base-rate problems in which an intuitively cued stereotypical response was either congruent or incongruent with the correct response that was cued by the base-rates. Results showed that solving problems in which the base-rates and stereotypical description cued conflicting responses resulted in an increased centro-parietal N2 and frontal P3. This early conflict sensitivity suggests that the critical base-rates can be processed fast without slow and deliberate System 2 reflection. Findings validate prior EEG work and support recent hybrid dual process models in which the fast System 1 is processing both heuristic belief-based responses (e.g., stereotypes) and elementary logico-mathematical principles (e.g., base-rates).

Rule-based reasoning is fast and belief-based reasoning can be slow: Challenging current explanations of belief-bias and base-rate neglect.

It is commonly assumed that belief-based reasoning is fast and automatic, whereas rule-based reasoning is slower and more effortful. Dual-Process theories of reasoning rely on this speed-asymmetry explanation to account for a number of reasoning phenomena, such as base-rate neglect and belief-bias. The goal of the current study was to test this hypothesis about the relative speed of belief-based and rule-based processes. Participants solved base-rate problems (Experiment 1) and conditional inferences (Experiment 2) under a challenging deadline; they then gave a second response in free time. We found that fast responses were informed by rules of probability and logical validity, and that slow responses incorporated belief-based information. Implications for Dual-Process theories and future research options for dissociating Type I and Type II processes are discussed. (PsycINFO Database Record

A meta-analysis of confidence and judgment accuracy in clinical decision making.

The overconfidence bias occurs when clinicians overestimate the accuracy of their clinical judgments. This bias is thought to be robust leading to an almost universal recommendation by clinical judgment scholars for clinicians to temper their confidence in clinical decision making. An extension of the Meta-Analysis of Clinical Judgment (Spengler et al., 2009) project, the authors synthesized over 40 years of research from 36 studies, from 1970 to 2011, in which the confidence ratings of 1,485 clinicians were assessed in relation to the accuracy of their judgments about mental health (e.g., diagnostic decision making, violence risk assessment, prediction of treatment failure) or psychological issues (e.g., personality assessment). Using a random effects model a small but statistically significant effect (r = .15; CI = .06, .24) was found showing that confidence is better calibrated with accuracy than previously assumed. Approximately 50% of the total variance between studies was due to heterogeneity and not to chance. Mixed effects and meta-regression moderator analyses revealed that confidence is calibrated with accuracy least when there are repeated judgments, and more when there are higher base rate problems, when decisions are made with written materials, and for earlier published studies. Sensitivity analyses indicate a bias toward publishing smaller sample studies with smaller or negative confidence-accuracy effects. Implications for clinical judgment research and for counseling psychology training and practice are discussed.

Base rates: Both neglected and intuitive.

Base-rate neglect refers to the tendency for people to underweight base-rate probabilities in favor of diagnostic information. It is commonly held that base-rate neglect occurs because effortful (Type 2) reasoning is required to process base-rate information, whereas diagnostic information is accessible to fast, intuitive (Type 1) processing (e.g., Kahneman & Frederick, 2002). To test this account, we instructed participants to respond to base-rate problems on the basis of "beliefs" or "statistics," both in free time (Experiments 1 and 3) and under a time limit (Experiment 2). Participants were given problems with salient stereotypes (e.g., "Jake lives in a beautiful home in a posh suburb") that either conflicted or coincided with base-rate probabilities (e.g., "Jake was randomly selected from a sample of 5 doctors and 995 nurses for conflict; 995 doctors and 5 nurses for nonconflict"). If utilizing base-rates requires Type 2 processing, they should not interfere with the processing of the presumably faster belief-based judgments, whereas belief-based judgments should always interfere with statistics judgments. However, base-rates interfered with belief judgments to the same extent as the stereotypes interfered with statistical judgments, as indexed by increased response time and decreased confidence for conflict problems relative to nonconflict. These data suggest that base-rates, while typically underweighted or neglected, do not require Type 2 processing and may, in fact, be accessible to Type 1 processing.

Development of heuristic bias detection in elementary school.

Although human reasoning is often biased by intuitive heuristics, recent studies have shown that adults and adolescents detect the biased nature of their judgments. The present study focused on the development of this critical bias sensitivity by examining the detection skills of young children in elementary school. Third and 6th graders were presented with child-friendly versions of classic base-rate problems in which a cued heuristic response could be inconsistent or consistent with the base rates. After each problem children were asked to indicate their subjective response confidence to assess their bias detection skills. Results indicated that 6th graders showed a clear confidence decrease when they gave a heuristic response that conflicted with the base rates. However, this confidence decrease was not observed for 3rd graders, suggesting that they did not yet acknowledge that their judgment was not fully warranted. Implications for the development of efficient training programs and the debate on human rationality are discussed.

The role of answer fluency and perceptual fluency as metacognitive cues for initiating analytic thinking

Although widely studied in other domains, relatively little is known about the metacognitive processes that monitor and control behaviour during reasoning and decision-making. In this paper, we examined the conditions under which two fluency cues are used to monitor initial reasoning: answer fluency, or the speed with which the initial, intuitive answer is produced (Thompson, Prowse Turner, & Pennycook, 2011), and perceptual fluency, or the ease with which problems can be read (Alter, Oppenheimer, Epley, & Eyre, 2007). The first two experiments demonstrated that answer fluency reliably predicted Feeling of Rightness (FOR) judgments to conditional inferences and base rate problems, which subsequently predicted the amount of deliberate processing as measured by thinking time and answer changes; answer fluency also predicted retrospective confidence judgments (Experiment 3b). Moreover, the effect of answer fluency on reasoning was independent from the effect of perceptual fluency, establishing that these are empirically independent constructs. In five experiments with a variety of reasoning problems similar to those of Alter et al. (2007), we found no effect of perceptual fluency on FOR, retrospective confidence or accuracy; however, we did observe that participants spent more time thinking about hard to read stimuli, although this additional time did not result in answer changes. In our final two experiments, we found that perceptual disfluency increased accuracy on the CRT (Frederick, 2005), but only amongst participants of high cognitive ability. As Alter et al.’s samples were gathered from prestigious universities, collectively, the data to this point suggest that perceptual fluency prompts additional processing in general, but this processing may results in higher accuracy only for the most cognitively able.

Adaptive redundancy, denominator neglect, and the base-rate fallacy

AbstractHomo sapiens have evolved a dual-process cognitive architecture that is adaptive but prone to systematic errors. Fuzzy-trace theory predicts that nested or overlapping class-inclusion relations create processing interference, resulting in denominator neglect: behaving as if one ignores marginal denominators in a 2 × 2 table. Ignoring marginal denominators leads to fallacies in base-rate problems and conjunctive and disjunctive probability estimates.

Nested sets and base-rate neglect: Two types of reasoning?

AbstractBarbey & Sloman (B&S) claim that frequency formats and other task manipulations induce people to substitute associative thinking for rule-based thinking about nested sets. My critique focuses on the substitution assumption. B&S demonstrate that nested sets are important to solve base-rate problems but they do not show that thinking about these nested sets relies on a different type of reasoning.

The operating characteristics of the major HRNES-R measures

The operating characteristics and base rate effects of tests and indexes in the Halstead Russell Neuropsychological Evaluation System—Revised (HRNES-R) were obtained to determine its accuracy in assessing brain damage. Since operating characteristics and base rate problems are not well understood they were discussed in some detail. The operating characteristics of Sensitivity, Specificity, Positive Predictive Power, Negative Predictive Power and Overall Predictive Power along with base rate effects were obtained for 2 HRNES-R indexes, 10 index tests and 3 other Halstead Reitan Battery (HRB) tests. The indexes were found to be as accurate as the most accurate indexes from other HRB studies. The accuracy of the various tests were high but varied according to their function and design.

Discrepancies Between Normative and Descriptive Models of Decision Making and the Understanding/Acceptance Principle

Several tasks from the heuristics and biases literature were examined in light of Slovic and Tversky's (1974) understanding/acceptance principle—that the deeper the understanding of a normative principle, the greater the tendency to respond in accord with it. The principle was instantiated both correlationally and experimentally. An individual differences version was used to examine whether individuals higher in tendencies toward reflective thought and in cognitive ability would be more likely to behave normatively. In a second application of the understanding/acceptance principle, subjects were presented with arguments both for and against normative choices and it was observed whether, on a readministration of the task, performance was more likely to move in a normative direction. Several discrepancies between performance and normative models could be explained by the understanding/acceptance principle. However, several gaps between descriptive and normative models (particularly those deriving from some noncausal base rate problems) were not clarified by the understanding/acceptance principle—they could not be explained in terms of varying task understanding or tendencies toward reflective thought. The results demonstrate how the variation and instability in responses can be analyzed to yield inferences about why descriptive and normative models of human reasoning and decision making sometimes do not coincide.

A farewell to normative null hypothesis testing in base rate research

I agree with Gibbs that the message of the base rate literature reads differently depending on which null hypothesis is used to frame the issue. But I argue that the normative null hypothesis, H0: “People use base rates in a Bayesian manner,” is no longer appropriate. I also challenge Adler's distinction between unused and ignored base rates, and criticize Goodie's reluctance to shift research attention to the field. Macchi's arguments about textual ambiguities in traditional base rate problems suggest that empirical testing is needed to tease apart the effects of problem clarification and problem framing. Macdonald's, Fletcher's and Snow's skepticism about the value of Bayesian methods in real world judgment tasks is treated as a challenge for the next generation of empirical base rate studies.

Base rates and randomness

In base rate problems the estimated probability must equal the base rate only where random sampling is assumed. Otherwise there is uncertainty over and above that which can be included in any probability model and inductive inference is involved. Subjects should use base rates to the extent that the problem suggests a simple random sampling model.

Overcoming Ineffective Mental Representations in Base-Rate Problems

Many biases have been observed in probabilistic reasoning, hindering the ability to follow normative rules in decision-making contexts involving uncertainty. One systematic error people make is to neglect base rates in situations where prior beliefs in a hypothesis should be taken into account when new evidence is obtained. Incomplete explanations for the phenomenon have impeded the development of effective debiasing procedures or tools to support decision making in this area. In this research, we show that the main reason behind these judgment errors is the causal representation induced by the problem context. In two experiments we demonstrate that people often possess the appropriate decision rules but are unable to apply them correctly because they have an ineffective causal mental representation. We also show how this mental representation may be modified when a graph is used instead of a problem narrative. This new understanding should contribute to the design of better decision aids to overcome this bias.

A Reconsideration of the High-Risk Strategy: The Problem of Base Rates

Services which require practitioners to identify individuals at risk or having an early problem so that intervention can be delivered before these individuals are aware of their need for professional assistance run the risk of misidentification. In services which target at low base-rate problems, many “false positives” will occur unless better diagnostic procedures are available. Individuals who are misidentified and are being actively approached for service will have to bear whatever negative consequences these services induce. The implication for planning for prevention is discussed.