Use Of Simple Heuristics Research Articles

Inconsistencies in rater-based assessments mainly affect borderline candidates: but using simple heuristics might improve pass-fail decisions.

Research in various areas indicates that expert judgment can be highly inconsistent. However, expert judgment is indispensable in many contexts. In medical education, experts often function as examiners in rater-based assessments. Here, disagreement between examiners can have far-reaching consequences. The literature suggests that inconsistencies in ratings depend on the level of performance a to-be-evaluated candidate shows. This possibility has not been addressed deliberately and with appropriate statistical methods. By adopting the theoretical lens of ecological rationality, we evaluate if easily implementable strategies can enhance decision making in real-world assessment contexts. We address two objectives. First, we investigate the dependence of rater-consistency on performance levels. We recorded videos of mock-exams and had examiners (N=10) evaluate four students' performances and compare inconsistencies in performance ratings between examiner-pairs using a bootstrapping procedure. Our second objective is to provide an approach that aids decision making by implementing simple heuristics. We found that discrepancies were largely a function of the level of performance the candidates showed. Lower performances were rated more inconsistently than excellent performances. Furthermore, our analyses indicated that the use of simple heuristics might improve decisions in examiner pairs. Inconsistencies in performance judgments continue to be a matter of concern, and we provide empirical evidence for them to be related to candidate performance. We discuss implications for research and the advantages of adopting the perspective of ecological rationality. We point to directions both for further research and for development of assessment practices.

The common origin of both oversimplified and overly complex decision rules

AbstractMany deviations from rational choice imply the neglect of important evidence and suggest the use of simple heuristics. In contrast, other deviations imply sensitivity to irrelevant evidence and suggest the use of overly complex rules. The current analysis takes two steps toward identifying the conditions that trigger these contradictory deviations from efficient reasoning. The first step involves a theoretical analysis. It shows that the contradictory deviations can be captured without assuming the use of rules of different complexity in different settings. Both deviations can be the product of a reliance on small samples of similar past experiences. This reliance on small samples triggers apparent overcomplexity when the optimal rule is simple, but more complex rules yield better outcomes in most cases; the opposite tendency, oversimplification, emerges when the optimal rule is complex, and simple rules yield better outcomes in most cases. The second step involves a preregistered experiment with 325 participants (Mechanical Turk workers). The experiment shows that human decision makers exhibit the pattern predicted by the reliance‐on‐small‐samples assumption. In the experiment, participants chose between the status quo and a risky alternative in a multi‐attribute decision with three binary cues. They used uninformative cues when this strategy was best in most cases yet ignored two informative cues when this strategy was best in most cases. In addition, describing the cues as recommendations given by three experts increased the tendency to follow the modal recommendation (even when reliance on only one of the experts was optimal), but people still behaved as though they relied on a small sample of past experiences.

Learning in Networks: An Experiment on Large Networks with Real-World Features

Subjects observe a private signal and make an initial guess; they then observe their neighbors’ guesses, update their own guess, and so forth. We study learning dynamics in three large-scale networks capturing features of real-world social networks: Erdös–Rényi, Stochastic Block (reflecting network homophily), and Royal Family (that accommodates both highly connected celebrities and local interactions). We find that the Royal Family network is more likely to sustain incorrect consensus and that the Stochastic Block network is more likely to persist with diverse beliefs. These patterns are consistent with the predictions of DeGroot updating. It lends support to the notion that the use of simple heuristics in information aggregation is prevalent in large and complex networks. This paper was accepted by Yan Chen, behavioral economics and decision analysis. Funding: The authors thank the Keynes Fund (University of Cambridge), the Creative-Pioneering Researchers Program (Seoul National University), and C-BID (NYUAD) for financial support. Supplemental Material: The data files and e-companion are available at https://doi.org/10.1287/mnsc.2023.4680 .

Knowing when to fold 'em: Problem attributes and strategy differences in the Paper Folding test

Spatial visualization—the ability to mentally imagine and manipulate objects—has frequently been measured using the Paper Folding Test (PFT). In this task, participants view diagrams of a paper being folded and a hole being punched. They then identify the resulting punch pattern. Although task instructions promote mentally unfolding the paper, the extent to which people follow this spatial visualization strategy is unknown. The present work assesses hypothesized PFT solution strategies and their relation to problem attributes. Accuracy was impacted by the interaction between fold types, linear mixed models revealed greater use of simple heuristics compared to the suggested unfolding. Furthermore, most participants used a single strategy but simple heuristics were more often used than unfolding. Given this, we recommend redesigning the PFT to utilize the prevalence of strategy use to assess individual differences.

From anomalies to forecasts: Toward a descriptive model of decisions under risk, under ambiguity, and from experience.

Experimental studies of choice behavior document distinct, and sometimes contradictory, deviations from maximization. For example, people tend to overweight rare events in 1-shot decisions under risk, and to exhibit the opposite bias when they rely on past experience. The common explanations of these results assume that the contradicting anomalies reflect situation-specific processes that involve the weighting of subjective values and the use of simple heuristics. The current article analyzes 14 choice anomalies that have been described by different models, including the Allais, St. Petersburg, and Ellsberg paradoxes, and the reflection effect. Next, it uses a choice prediction competition methodology to clarify the interaction between the different anomalies. It focuses on decisions under risk (known payoff distributions) and under ambiguity (unknown probabilities), with and without feedback concerning the outcomes of past choices. The results demonstrate that it is not necessary to assume situation-specific processes. The distinct anomalies can be captured by assuming high sensitivity to the expected return and 4 additional tendencies: pessimism, bias toward equal weighting, sensitivity to payoff sign, and an effort to minimize the probability of immediate regret. Importantly, feedback increases sensitivity to probability of regret. Simple abstractions of these assumptions, variants of the model Best Estimate and Sampling Tools (BEAST), allow surprisingly accurate ex ante predictions of behavior. Unlike the popular models, BEAST does not assume subjective weighting functions or cognitive shortcuts. Rather, it assumes the use of sampling tools and reliance on small samples, in addition to the estimation of the expected values. (PsycINFO Database Record

The development of adaptive decision making: Recognition-based inference in children and adolescents.

Judgments about objects in the world are often based on probabilistic information (or cues). A frugal judgment strategy that utilizes memory (i.e., the ability to discriminate between known and unknown objects) as a cue for inference is the recognition heuristic (RH). The usefulness of the RH depends on the structure of the environment, particularly the predictive power (validity) of recognition. Little is known about developmental differences in use of the RH. In this study, the authors examined (a) to what extent children and adolescents recruit the RH when making judgments, and (b) around what age adaptive use of the RH emerges. Primary schoolchildren (M = 9 years), younger adolescents (M = 12 years), and older adolescents (M = 17 years) made comparative judgments in task environments with either high or low recognition validity. Reliance on the RH was measured with a hierarchical multinomial model. Results indicated that primary schoolchildren already made systematic use of the RH. However, only older adolescents adaptively adjusted their strategy use between environments and were better able to discriminate between situations in which the RH led to correct versus incorrect inferences. These findings suggest that the use of simple heuristics does not progress unidirectionally across development but strongly depends on the task environment, in line with the perspective of ecological rationality. Moreover, adaptive heuristic inference seems to require experience and a developed base of domain knowledge. (PsycINFO Database Record

Time to broaden the scope of research on anticipatory behavior: a case for the role of probabilistic information.

Time to broaden the scope of research on anticipatory behavior: a case for the role of probabilistic information.

Slime moulds use heuristics based on within-patch experience to decide when to leave.

Animals foraging in patchy, non-renewing or slowly renewing environments must make decisions about how long to remain within a patch. Organisms can use heuristics ('rules of thumb') based on available information to decide when to leave the patch. Here, we investigated proximate patch-departure heuristics in two species of giant, brainless amoeba: the slime moulds Didymium bahiense and Physarum polycephalum. We explicitly tested the importance of information obtained through experience by eliminating chemosensory cues of patch quality. In P. polycephalum, patch departure was influenced by the consumption of high, and to a much lesser extent low, quality food items such that engulfing a food item increased patch-residency time. Physarum polycephalum also tended to forage for longer in darkened, 'safe' patches. In D. bahiense, engulfment of any food item increased patch residency irrespective of that food item's quality. Exposure to light had no effect on the patch-residency time of D. bahiense. Given that these organisms lack a brain, our results illustrate how the use of simple heuristics can give the impression that individuals make sophisticated foraging decisions.

Effort reduction after self-control depletion: The role of cognitive resources in use of simple heuristics

Heuristics have been described as decision strategies that save time and effort. Given this advantageous property, heuristics should be more often used when cognitive resources are scarce. We tested this general view with respect to the fast and frugal recognition heuristic which assumes one-reason decision making based on recognition alone whenever one object in a paired comparison is recognised and the other is not. In an experiment, we manipulated cognitive resources through an executive control procedure previously used in ego-depletion research. From the choice frequencies, the probability of using the recognition heuristic was estimated by means of a multinomial processing tree model. Results confirmed that use of the recognition heuristic was indeed more likely under depleted resources and thus limited cognitive capacity. These findings corroborate the general notion that a need for effort reduction fosters use of simple decision heuristics.

Simple heuristics in sports

How do people make decisions under conditions of limited knowledge, time, and cognitive capacity in real-life situations such as sports? In this review I will introduce the concept of simple heuristics – rules of thumb that are based on the building blocks of decision making: how to search for information, stop information search, and decide quickly and accurately – and how they can help us understand the decisions made by athletes, coaches, referees, managers, and fans in tasks involving high uncertainty, such as predicting tournament outcomes, allocating balls to teammates, or determining when to buy or sell a talented player. I will present an ‘adaptive toolbox’ of such heuristics, that is, a collection of strategies that work effectively in specific environments. Additional building blocks will be added to explain motor behavior itself, which is central to many sport applications. Finally, principles for studying the use of simple heuristics by people involved in sports will be presented to guide future applications.

Strategic adaptation of humans playing computer algorithms in a repeated constant-sum game

This paper examines strategic adaptation in participants’ behavior conditional on the type of their opponent. Participants played a constant-sum game for 100 rounds against each of three pattern-detecting computer algorithms designed to exploit regularities in human behavior such as imperfections in randomizing and the use of simple heuristics. Significant evidence is presented that human participants not only change their marginal probabilities of choosing actions, but also their conditional probabilities dependent on the recent history of play. A cognitive model incorporating pattern recognition is proposed that capture the shifts in strategic behavior of the participants better than the standard non-pattern detecting model employed in the literature, the Experience Weighted Attraction model (and by extension its nested models, reinforcement learning and fictitious play belief learning).

Humans versus Computer Algorithms in a Repeated Constant-Sum Game

This paper examines strategic adaptation in participants’ behavior conditional on the type of their opponent. Participants played a constant-sum game for 100 rounds against each of three pattern-detecting computer algorithms designed to exploit regularities in human behavior such as imperfections in randomizing and the use of simple heuristics. Significant evidence is presented that human participants not only change their marginal probabilities of choosing actions, but also their conditional probabilities dependent on the recent history of play. A cognitive model incorporating pattern recognition is proposed that capture the shifts in strategic behavior of the participants better than the standard non-pattern detecting model employed in the literature, the Experience Weighted Attraction model (and by extension its nested models, reinforcement learning and fictitious play belief learning).

Parallel Constraint Satisfaction in Memory-Based Decisions

Three studies sought to investigate decision strategies in memory-based decisions and to test the predictions of the parallel constraint satisfaction (PCS) model for decision making (Glöckner & Betsch, 2008). Time pressure was manipulated and the model was compared against simple heuristics (take the best and equal weight) and a weighted additive strategy. From PCS we predicted that fast intuitive decision making is based on compensatory information integration and that decision time increases and confidence decreases with increasing inconsistency in the decision task. In line with these predictions we observed a predominant usage of compensatory strategies under all time-pressure conditions and even with decision times as short as 1.7 s. For a substantial number of participants, choices and decision times were best explained by PCS, but there was also evidence for use of simple heuristics. The time-pressure manipulation did not significantly affect decision strategies. Overall, the results highlight intuitive, automatic processes in decision making and support the idea that human information-processing capabilities are less severely bounded than often assumed.

Parallel Constraint Satisfaction in Memory-Based Decisions

Three studies sought to investigate decision strategies in memory-based decisions and to test the predictions of the parallel constraint satisfaction (PCS) model for decision making (Glockner & Betsch, 2008). Time pressure was manipulated and the model was compared against simple heuristics (take the best and equal weight) and a weighted additive strategy. From PCS we predicted that fast intuitive decision making is based on compensatory information integration and that decision time increases with increasing inconsistency in the decision task. In line with these predictions we observed a predominant usage of compensatory strategies under all time pressure conditions and even with decision times as short as 1.7 seconds. For a substantial number of participants, choices and decision times were best explained by PCS, but there was also evidence for use of simple heuristics. The time pressure manipulation did not significantly affect decision strategies. Overall, the results highlight intuitive, automatic processes in decision making and support the idea that human information-processing capabilities are less severely bounded than often assumed.

Variable Decision Strategies, Rational Choice, and Situation-Related Travel Demand

Classical economic theory following Bentham and Mill assumes decision makers are completely informed, highly sensitive to differences between alternatives, and above all rational in being able to rank order the possible alternatives and choose among them on some measure of welfare (utility maximization). In recent years, this view has been successfully challenged by empirical studies of decision making, and many alternative decision rules have emerged (such as lexicographic, elimination by aspects, counting, conjunctive, heuristic additive difference, affective, take the best, and so on). Despite thirty years’ work on boundedly rational and cognitive approaches, the classical view is still widely accepted in urban travel demand modeling. This paper develops a newer conceptual framework for shopping travel situtations—namely, choice of shopping place on different kinds of tour, for different orders of goods, and for regional malls and neighborhood centers. Hypotheses concerning the use of different kinds of decision rule are generated for the different situations or contexts. A statistical design is provided for tests of these hypotheses using fieldwork, verbal protocol analysis, and low-level t-tests. Fifty-five MIT students are interviewed, and their protocols for the shopping situations recorded and analyzed. The findings are that decision rules are multiple and vary with context. Although the specific hypotheses of the paper are not strongly supported, there is overwhelming evidence of the widespread use of simple heuristics. Implications of the findings are drawn for future work in the travel demand field and the preference and choice field. In addition, the contribution of this paper to behavioral decision making is noted, through the extension of work on choice heuristics to a different field (travel demand) and the use of a fieldwork qualitative methodology.

The ‘Good Enough’ Approach to Language Comprehension

AbstractFerreira and colleagues argued that the language comprehension system creates syntactic and semantic representations that are merely ‘good enough’ (GE) given the task that the comprehender needs to perform. GE representations contrast with ones that are detailed, complete, and accurate with respect to the input. In this article, we review the original argument for GE processing, and we present new evidence that supports the concept: first, local interpretations are computed, which can interfere with global ones; second, new findings based on the recording of event‐related potentials show the use of simple heuristics rather than compositional algorithms for constructing sentence meaning; and recent studies show that the comprehension system has mechanisms for handling disfluencies, but they work imperfectly. We argue that the GE approach to language comprehension is similar to the use of fast and frugal heuristics for decision‐making, and that future research should explore this connection more thoroughly.

Using argumentation to extract key sentences from biomedical abstracts

PROBLEM: key word assignment has been largely used in MEDLINE to provide an indicative “gist” of the content of articles and to help retrieving biomedical articles. Abstracts are also used for this purpose. However with usually more than 300 words, MEDLINE abstracts can still be regarded as long documents; therefore we design a system to select a unique key sentence. This key sentence must be indicative of the article's content and we assume that abstract's conclusions are good candidates. We design and assess the performance of an automatic key sentence selector, which classifies sentences into four argumentative moves: PURPOSE, METHODS, RESULTS and CONCLUSION. METHODS: we rely on Bayesian classifiers trained on automatically acquired data. Features representation, selection and weighting are reported and classification effectiveness is evaluated on the four classes using confusion matrices. We also explore the use of simple heuristics to take the position of sentences into account. Recall, precision and F-scores are computed for the CONCLUSION class. For the CONCLUSION class, the F-score reaches 84%. Automatic argumentative classification using Bayesian learners is feasible on MEDLINE abstracts and should help user navigation in such repositories.

Empirical tests of “Take-the-Best” with non-human subjects

The fast and frugal heuristics of ABC and rules of thumb of behavioural biologists represent strategies that humans and other animals might use to make decisions under time constraints and with a minimum of information. If experimental psychologists could demonstrate use of simple heuristics by non-humans in experimental settings, quantitative and empirical evaluation of those heuristics would benefit from additional formal, controlled avenues of study.

Waking Up from the Dream of Reason

For centuries, philosophy and psychology in one way or another identified thought with logic, or logic with thought. Only in recent decades has it become clear that there is more to thought than logic. Evolutionary psychology has stressed the adaptive function of thought, as of other processes often identified as psychological. Gigerenzer et al. have worked for some years now on a program of research in which thinking is approached as an adaptive process. According to them, it is misleading to use the complexities of logic, mathematics or statistics as the standards for correct thinking or as a model for thought. On the contrary, thinking is the effective use of simple heuristics. The four volumes discussed in this review essay demonstrate that the approach can be very fruitful in many domains of human reasoning, and can be effective in practical problem solving as well, without having to fall back on the intricacies of logic, statistics, mathematics and other tools of the trade of scientific decision making.

Use of simple heuristics to target macrolide prescription in children with community-acquired pneumonia.

Macrolides are the first-line antibiotic treatment of community-acquired pneumonia (CAP). Owing to alarming resistance rates among invasive Streptococcus pneumoniae isolates, particularly in young children, macrolide use should be restricted to patients infected with susceptible pathogens, eg, Mycoplasma pneumoniae. To develop a simple clinical prediction rule for identifying M pneumoniae as the cause of CAP in children. Prospective cohort study in 253 children with radiologically confirmed CAP in a walk-in clinic of a tertiary care hospital. Mycoplasma infection, proven by results of antibody testing of paired serum samples (gold standard). We compared the area under the receiver operating characteristic curve (c statistic) of the following 2 prediction models: a scoring system derived from logistic regression analysis and a fast-and-frugal decision tree. Mycoplasma pneumoniae infection was confirmed in 32 (13%) of 253 children. A scoring system based on duration of fever and patient age yielded a c statistic of 0.84 (95% confidence interval [CI], 0.77-0.91), compared with that of the decision tree (c = 0.76 [95% CI, 0.70-0.83]). The scoring system identified 75% of all cases as being at high or very high risk for M pneumoniae infection; the decision tree, 72% at high risk. The scoring system would curtail macrolide prescriptions by 75%; the decision tree, by 68%. In children with CAP, simple clinical decision rules identify patients at risk for M pneumoniae infection. At present US macrolide resistance rates among invasive S pneumoniae isolates, both rules increase the chance of prescribing effective first-line antibiotics compared with general macrolide administration.