Natural Frequency Formats Research Articles

The Effects of Working Memory and Probability Format on Bayesian Reasoning.

Bayesian reasoning is common and critical in everyday life while the performance on Bayesian reasoning is rather poor. Previous studies showed that people could enhance their performance by applying cognitive resources under the natural frequency format condition. Working memory is one of the crucial cognitive resources in the reasoning process. However, the role of working memory on Bayesian reasoning remains unclear. In our study, we verified the effect of working memory on Bayesian reasoning by evaluating the performance of participants with high and low working memory span (WMS); we also investigated if working memory as a kind of cognitive resource can affect Bayesian reasoning performance by manipulating the cognitive load in a dual-task paradigm among participants with no-, low-, and high-loads. We found the following: (1) The Bayesian reasoning performance of high WMS participants was significantly higher than that of low WMS participants. (2) Performance under natural frequency condition was noticeably higher than that in standard probability condition. (3) Interaction between working memory and probability format was significant, and the performance of participants with high-load in natural frequency condition was higher when compared to those of participants with no- and low-load. Therefore, we can conclude that: (1) Working memory resource is a major factor in Bayesian reasoning. The performance of Bayesian reasoning is influenced by working memory span and working memory load. (2) A Bayesian facilitation effect exists, and replacing the standard probability format with a natural frequency format can significantly improve Bayesian performance. (3) Bayesian facilitation occurs only in participants with sufficient working memory resources.

Why Can Only 24% Solve Bayesian Reasoning Problems in Natural Frequencies: Frequency Phobia in Spite of Probability Blindness.

For more than 20 years, research has proven the beneficial effect of natural frequencies when it comes to solving Bayesian reasoning tasks (Gigerenzer and Hoffrage, 1995). In a recent meta-analysis, McDowell and Jacobs (2017) showed that presenting a task in natural frequency format increases performance rates to 24% compared to only 4% when the same task is presented in probability format. Nevertheless, on average three quarters of participants in their meta-analysis failed to obtain the correct solution for such a task in frequency format. In this paper, we present an empirical study on what participants typically do wrong when confronted with natural frequencies. We found that many of them did not actually use natural frequencies for their calculations, but translated them back into complicated probabilities instead. This switch from the intuitive presentation format to a less intuitive calculation format will be discussed within the framework of psychological theories (e.g., the Einstellung effect).

From reading numbers to seeing ratios: a benefit of icons for risk comprehension.

Promoting a better understanding of statistical data is becoming increasingly important for improving risk comprehension and decision-making. In this regard, previous studies on Bayesian problem solving have shown that iconic representations help infer frequencies in sets and subsets. Nevertheless, the mechanisms by which icons enhance performance remain unclear. Here, we tested the hypothesis that the benefit offered by icon arrays lies in a better alignment between presented and requested relationships, which should facilitate the comprehension of the requested ratio beyond the represented quantities. To this end, we analyzed individual risk estimates based on data presented either in standard verbal presentations (percentages and natural frequency formats) or as icon arrays. Compared to the other formats, icons led to estimates that were more accurate, and importantly, promoted the use of equivalent expressions for the requested probability. Furthermore, whereas the accuracy of the estimates based on verbal formats depended on their alignment with the text, all the estimates based on icons were equally accurate. Therefore, these results support the proposal that icons enhance the comprehension of the ratio and its mapping onto the requested probability and point to relational misalignment as potential interference for text-based Bayesian reasoning. The present findings also argue against an intrinsic difficulty with understanding single-event probabilities.

Visualizing the Bayesian 2-test case: The effect of tree diagrams on medical decision making.

In medicine, diagnoses based on medical test results are probabilistic by nature. Unfortunately, cognitive illusions regarding the statistical meaning of test results are well documented among patients, medical students, and even physicians. There are two effective strategies that can foster insight into what is known as Bayesian reasoning situations: (1) translating the statistical information on the prevalence of a disease and the sensitivity and the false-alarm rate of a specific test for that disease from probabilities into natural frequencies, and (2) illustrating the statistical information with tree diagrams, for instance, or with other pictorial representation. So far, such strategies have only been empirically tested in combination for “1-test cases”, where one binary hypothesis (“disease” vs. “no disease”) has to be diagnosed based on one binary test result (“positive” vs. “negative”). However, in reality, often more than one medical test is conducted to derive a diagnosis. In two studies, we examined a total of 388 medical students from the University of Regensburg (Germany) with medical “2-test scenarios”. Each student had to work on two problems: diagnosing breast cancer with mammography and sonography test results, and diagnosing HIV infection with the ELISA and Western Blot tests. In Study 1 (N = 190 participants), we systematically varied the presentation of statistical information (“only textual information” vs. “only tree diagram” vs. “text and tree diagram in combination”), whereas in Study 2 (N = 198 participants), we varied the kinds of tree diagrams (“complete tree” vs. “highlighted tree” vs. “pruned tree”). All versions were implemented in probability format (including probability trees) and in natural frequency format (including frequency trees). We found that natural frequency trees, especially when the question-related branches were highlighted, improved performance, but that none of the corresponding probabilistic visualizations did.

Natural Frequencies Do Foster Public Understanding of Medical Tests: Comment on Pighin, Gonzalez, Savadori, and Girotto (2016).

Patients and doctors often need to make decisions based on the results of medical tests. When these results are presented in the form of conditional probabilities, even doctors find it difficult to interpret them correctly. There is over 20 y of research supporting the finding that people are better able to calculate the correct positive predictive value of a test when given information in natural frequencies, as opposed to conditional probabilities. Natural frequencies are one of a few psychological tools that have made it into evidence-based medicine. Recently, Pighin and others (Med Decis Making 2016;36:686-91) argued that natural frequencies could hinder informed decision making, a critique based on a single task and a crude scoring criterion we refer to as the 50%-Split. Our commentary addresses these criticisms based on three analyses. First, we show how the 50%-Split scoring used by Pighin and others misclassifies known errors, such as solely attending to the hit rate (true-positive rate) of the test, as strategies that support understanding. Second, we reanalyze data from 21 additional problems completed by various participant groups to show that their scoring criterion does not support their results in 19 out of 21 cases. Third, we apply the mean deviation scoring method and show that, when given information in natural frequency formats, participants provide estimates that are closer to the correct Bayesian solution than for conditional probability formats. In each analysis, natural frequencies lead to more correct judgements and therefore promote informed decision making relative to conditional probabilities. We welcome further discussions of performance metrics that can provide insight into how the public and therefore patients understand the implications of medical test results.

Meta-analysis of the effect of natural frequencies on Bayesian reasoning.

The natural frequency facilitation effect describes the finding that people are better able to solve descriptive Bayesian inference tasks when represented as joint frequencies obtained through natural sampling, known as natural frequencies, than as conditional probabilities. The present meta-analysis reviews 20 years of research seeking to address when, why, and for whom natural frequency formats are most effective. We review contributions from research associated with the 2 dominant theoretical perspectives, the ecological rationality framework and nested-sets theory, and test potential moderators of the effect. A systematic review of relevant literature yielded 35 articles representing 226 performance estimates. These estimates were statistically integrated using a bivariate mixed-effects model that yields summary estimates of average performances across the 2 formats and estimates of the effects of different study characteristics on performance. These study characteristics range from moderators representing individual characteristics (e.g., numeracy, expertise), to methodological differences (e.g., use of incentives, scoring criteria) and features of problem representation (e.g., short menu format, visual aid). Short menu formats (less computationally complex representations showing joint-events) and visual aids demonstrated some of the strongest moderation effects, improving performance for both conditional probability and natural frequency formats. A number of methodological factors (e.g., exposure to both problem formats) were also found to affect performance rates, emphasizing the importance of a systematic approach. We suggest how research on Bayesian reasoning can be strengthened by broadening the definition of successful Bayesian reasoning to incorporate choice and process and by applying different research methodologies. (PsycINFO Database Record

Measures of Bayesian Reasoning Performance on ‘Normal’ and ‘Natural’ Frequency Tasks

ABSTRACTWhile the majority of similar studies examining Bayesian reasoning investigate how participants avoid common errors such as base-rate neglect, the current research also examines whether different formats (frequency and probability) lead to a difference in levels of absolute accuracy. In Study One, older (≥60 years) and younger (18 to 29 years) participants completed tasks in probability and normalized frequency formats. In Study 2, participants completed tasks in probability and natural frequency formats. Findings are that frequencies lead to less over-estimation, particularly in natural frequency tasks, which also reveal an interaction between age and task format whereby older adults seem unaffected by format. There was no association found between format and the avoidance of errors such as base-rate neglect. Findings are discussed in the light of dual and multi-process theories of reasoning, having failed to support the theory that frequency formats elicit System 2 reasoning processes.

Communicating HIV Results to Low-Risk Individuals: Still Hazy After All These Years.

Revised Centers for Disease Control and Prevention recommendations on HIV testing now promote testing of most risk groups. However, positive results for low-risk individuals are more likely to be false positives than for high-risk individuals, making clear communication of test results even more imperative. In a study, we evaluated current counseling of low-risk test recipients via a sample of 29 HIV hotline counselors from U.S. state and national hotlines. 100% of counselors interviewed failed to provide an accurate conditional HIV risk for low-risk women, but were more likely than a 1998 German sample to report that false positives could occur. In a second study, undergraduates read idealized transcripts of interviews with HIV counselors and computed conditional risk for a low-risk individual. The natural frequency format offered a small but significant improvement in conditional reasoning, comparable to the effect of numerical literacy. Applications for ecologically valid numerical presentations of risk and implications for numeracy are discussed.

Retrospective Frequency Formats Promote Consistent Experience‐Based Bayesian Judgments

SummaryOn the basis of their experiences with pregnant patients in their practice, obstetrician/gynecologists estimated the posterior probability of Down syndrome given a positive screening result. They also estimated the base rate of Down syndrome in their practice, along with the hit and false alarm rates for the screening test; for each subject, these numbers were combined to calculate a posterior probability to which the initial estimated posterior probability could be compared. Physicians gave highly consistent estimates when asked to think about their past experiences in terms of event frequencies. However, those told to respond using single event probabilities or to use past experiences to predict prospective frequencies gave inconsistent Bayesian estimates. Thus, when making Bayesian judgments based on real life experience, natural frequency formats only lead to better judgments, compared with single event probability formats, if people think retrospectively, not when using past experiences to make prospective predictions. Copyright © 2011 John Wiley & Sons, Ltd.

Natural frequencies and Bayesian reasoning: the impact of formal education and problem context

Research suggests that children and adults are able to solve Bayesian problems when natural frequencies are used, but most studies have involved only students or academics. The present research focuses on random samples from the general population. Experiments 1 and 2 showed that the natural-frequency format resulted in more Bayesian responses than the probability format. Results of Experiment 3 showed that when the task is not framed as a medical problem and the numbers are not too difficult, about 40% of the general population could correctly solve the problems. More importantly, the natural-frequency format resulted in a substantial increase in correct answers in the lower level of formal education group. Experiment 4 suggests that the context of a problem influences people’s ability to solve a Bayesian problem. Participants were twice as likely to solve a social problem compared with a medical problem. The results of the four experiments show that natural frequency resulted in much better estimates than probabilities. It should be emphasized, however, that even when natural frequencies were used, a majority of participants were not able to solve the problems correctly.

The Effects of Absolute Risks, Relative Risks, Frequencies, and Probabilities on Decision Quality

It is important to understand how the quality of people's decision making may be affected by the format used to present treatment benefits. Two experiments compared the accuracy of presenting the benefits of cancer screening tests or vaccines using either absolute or relative risk formats that included baseline risk information. Moreover, the absolute and/or baseline risks were presented using either natural frequencies or probabilities. In both experiments, accuracy was measured by the sensitivity of choices to differences in absolute rather than relative risks. Experiment 1 showed no significant differences in sensitivity between the relative and absolute risk formats when the risks were presented as natural frequencies. Sensitivity was, however, poor in both probability versions. Experiment 2 tested the natural frequency versions more stringently by presenting choices with different levels of difficulty. The author found that decision quality was significantly less affected by increases in difficulty in the absolute risk format. Presenting baseline risks using natural frequencies may help to reduce the biasing effects of relative risks but decision quality may not be on a par with the accuracy of decisions made when absolute risks are presented in natural frequency formats.

Facilitating Effect of Natural Frequencies: Size Does Not Matter

The question of whether humans are able to work in a Bayesian way is currently a topic of substantial investigation. An important finding, reported by Gigerenzer and Hoffrage in 1995 is that Bayesian reasoning is facilitated when the information format corresponds to natural frequencies. The present concern was whether the facilitating effect of frequencies persists when natural frequencies relate to samples which are not convenient multiples of 10. 150 undergraduates participated as volunteers (42 men, 108 women; M age = 23 yr.). Analysis showed the effect of natural frequency formats was not dependent on size of reference class. Theoretical and practical implications are discussed.

Enhancing sensitivity to base-rates: Natural frequencies are not enough

AbstractWe present evidence supporting the target article's assertion that while the presentation of base-rate information in a natural frequency format can be helpful in enhancing sensitivity to base rates, method of presentation is not a panacea. Indeed, we review studies demonstrating that when subjects directly experience base rates as natural frequencies in a trial-by-trial setting, they evince large base-rate neglect.

New tools for decision analysts

This paper presents psychological research that can help people make better decisions. Decision analysts typically: 1) elicit outcome probabilities; 2) assess attribute weights; and 3) suggest the option with the highest overall value. Decision analysis can be challenging because of environmental and psychological issues. Fast and frugal methods such as natural frequency formats, frugal multiattribute models, and fast and frugal decision trees can address these issues. Not only are the methods fast and frugal, but they can also produce results that are surprisingly close to or even better than those obtained by more extensive analysis. Apart from raising awareness of these findings among engineers, the authors also call for further research on the application of fast and frugal methods to decision analysis

Nurses as information providers: facilitating understanding and communication of statistical information

Nurses are increasingly being called upon to be the conveyers of important statistical information to patients. This trend is particularly evident in the domains of genetics and cancer screening. These new roles, however, demand new competencies, such as the ability to solve statistical problems, and the skill to communicate the answers effectively, as effective communication is an important ingredient in shared decision making. Genetic testing, perhaps more than other medical domains, relies heavily on the use of statistics. Being able to convey statistical information effectively is vital. In this paper, we illustrate the problems health care professionals have had in tackling and communicating statistical information. We introduce the natural frequencies method of solving Bayesian inference problems and review empirical evidence that shows the superiority of this format. Being able to transform probabilities into natural frequencies facilitates correct Bayesian inferences. It is argued that the conventional approach to educating nurses in Bayesian problem solving should be reconsidered and their statistical curriculum should be supplemented with instruction in using the natural frequency format.

A sampling approach to biases in conditional probability judgments: Beyond base rate neglect and statistical format.

Conditional probability judgments of rare events are often inflated when some meaningful relation exists between the condition and the low-baserate event. While traditional explanations assume that human judgments are generally insensitive to statistical baserates, more recent evidence shows much better performance when the problems are presented in natural frequency (as opposed to probability) formats and when the conditions refer to natural categories. The theory advanced here suggests a different explanation. Rather than postulating an a priori advantage of natural formats or categories, we emphasize sampling decisions as a key to understanding biased probability judgments. Experiment 1 shows that the seeming advantage of frequencies over probabilities is confined to conditions in which probabilities are scaled with reference to unequal subsamples. In Experiment 2, an active information search paradigm is employed that always provides a natural frequency format. When sampling by the predictor condition, the conditional probability to be estimated, p(criterion/ predictor), is conserved in the samples and the resulting judgments are quite accurate. However, when sampling by the criterion, the low-baserate event is strongly overrepresented in the samples. This sampling bias is even stronger than the resulting judgment bias. In general, judgments reflect the statistics of the actually acquired samples rather accurately, but judges do not understand the logical constraints imposed by their own sampling. This interpretation is corroborated in Experiment 3, where judges can freely choose between predictor sampling and criterion sampling, and in Experiment 4 using direct evaluations of the appropriateness of different sampling procedures.