Statistical Illiteracy Research Articles

Illusion of knowledge in statistics among clinicians: evaluating the alignment between objective accuracy and subjective confidence, an online survey

Healthcare professionals’ statistical illiteracy can impair medical decision quality and compromise patient safety. Previous studies have documented clinicians’ insufficient proficiency in statistics and a tendency in overconfidence. However, an underexplored aspect is clinicians’ awareness of their lack of statistical knowledge that precludes any corrective intervention attempt. Here, we investigated physicians’, residents’ and medical students’ alignment between subjective confidence judgments and objective accuracy in basic medical statistics. We also examined how gender, profile of experience and practice of research activity affect this alignment, and the influence of problem framing (conditional probabilities, CP vs. natural frequencies, NF). Eight hundred ninety-eight clinicians completed an online survey assessing skill and confidence on three topics: vaccine efficacy, p value and diagnostic test results interpretation. Results evidenced an overall consistent poor proficiency in statistics often combined with high confidence, even in incorrect answers. We also demonstrate that despite overconfidence bias, clinicians show a degree of metacognitive sensitivity, as their confidence judgments discriminate between their correct and incorrect answers. Finally, we confirm the positive impact of the more intuitive NF framing on accuracy. Together, our results pave the way for the development of teaching recommendations and pedagogical interventions such as promoting metacognition on basic knowledge and statistical reasoning as well as the use of NF to tackle statistical illiteracy in the medical context.

Evaluation of statistical illiteracy in Latin American clinicians and the piloting evaluation of a short course across multiple timepoints

BackgroundAll clinicians require statistical interpretation skills to keep up to date with evidence-based recommendations in their field. However, statistical illiteracy among clinicians is a highly prevalent problem with far-reaching consequences. The few available that report statistical literacy improvements after educational interventions do not measure for how long these benefits last. To estimate statistical illiteracy among Latin-American clinicians across multiple levels of training and to evaluate a 10-h course at multiple timepoints.MethodsUsing an online questionnaire, we evaluated; self-perceived statistical proficiency, scientific literature reading habits and statistical literacy (using an adaptation of the Quick Risk Test). Separately, we evaluated statistical proficiency after a 10-h statistics course in a group of Internal Medicine residents at a tertiary center in Mexico City across multiple time points between November 2020 and February 2021.ResultsData from 392 clinicians from 9 Latin American countries were analyzed. Most clinicians (85%) failed our adaptation of the Quick Risk Test (mean score = 2.6/10, IQR:1.4). The 10-h course significantly improved the scores of the Internal Medicine Residents (n = 16) from 3.8/10, IQR:1.8 to 8.3/10, IQR:1.4 (p < 0.01). However, scores dropped after one and 2 months to 7.7/10, IQR:1.6 and 6.1 / 10, IQR:2.2, respectively.ConclusionStatistical Illiteracy is highly prevalent among Latin American clinicians. Short-term educational interventions are effective but, their benefits quickly fade away. Medical boards and medical schools need to periodically teach and evaluate statistical proficiency to ameliorate these issues.

The jigsaw puzzle of fraudulent health claims: Missing psychological pieces.

Fraudulent health claims have become an inescapable fixture of the contemporary information (or misinformation) landscape. MacFarlane, Hurlstone, and Ecker (2020) provided a five-fold framework for conceptualizing susceptibility to fraudulent health claims, and proposed potential remedies for each driver of these claims. We build on their analysis by arguing that a complete account of fraudulent health claim susceptibility additionally requires a thoroughgoing consideration of (a) individual differences in cognitive styles and personality traits, (b) innumeracy and statistical illiteracy, and (c) persuasive appeals involving logical fallacies and commonplaces. We further contend that dual processing models of cognition may help to synthesize a variety of psychological variables relevant to fraudulent health claim vulnerability. In conjunction with our commentary, MacFarlane's framework underscores the broader point that complex, multifactorial psychological phenomena demand complex, multifactorial psychological explanations and solutions.

The Barrier to Informed Choice in Cancer Screening: Statistical Illiteracy in Physicians and Patients.

An efficient health care requires both informed doctors and patients. Our current healthcare system falls short on both counts. Most doctors and patients do not understand the available medical evidence. To illustrate the extent of the problem in the setting of cancer screening: In a representative sample of some 5000 women in nine European countries, 92% overestimated the reduction of breast cancer mortality by mammography by a factor of 10-200, or did not know. For a similar sample of about 5000 men with respect to PSA screening, this number was 89%. Of more than 300 US citizens who regularly attended one or more cancer screening test, more than 90% had never been informed about the biggest harms of screening-overdiagnosis and overtreatment-by their physicians. Among 160 German gynecologists, some 80% did not understand the positive predictive value of a positive mammogram, with estimates varying between 1 and 90%. In a national sample of 412 US primary care physicians, 47% mistakenly believed that if more cancers are detected by a screening test, this proves that the test saves lives, and 76% wrongly thought that if screen-detected cancers have better 5-year survival rates than cancers detected by symptoms, this would prove that the screening test saves lives. And of 20 German gynecologists, not a single one provided a woman with all information on the benefits and harms of cancer screening required in order to make an informed choice. Why is risk literacy so scarce in health care? One frequently discussed explanation assumes that people suffer from cognitive deficits that make them predictably irrational and basically hopeless at dealing with risks, so that they need to be "nudged" into healthy behavior. Yet research has demonstrated that the problem lies less in stable cognitive deficits than in how information is presented to physicians and patients. This includes biased reporting in medical journals, brochures, and the media that uses relative risks and other misleading statistics, motivated by conflicts of interest and defensive medicine that do not promote informed physicians and patients. What can be done? Every medical school should teach its students how to understand evidence in general and health statistics in particular. To cultivate informed patients, elementary and high schools should start teaching the mathematics of uncertainty-statistical thinking. Guidelines about complete and transparent reporting in journals, brochures, and the media need to be better enforced, and laws need to be changed in order to protect patients and doctors alike against the practice of defensive medicine instead of encouraging it. A critical mass of informed citizens will not resolve all healthcare problems, but it can constitute a major triggering factor for better care.

Judgment of Association between Potential Factors and Associated Risk in 2x2 Tables: A Study with Psychology Students

(ProQuest: ... denotes formulae omitted.)IntroductionIn the past decades we have observed an increasing interest towards risk perception and management in many organizations, such as private companies, banks, hospitals, or schools. Slovic (2000) described a risk as an uncertain event or condition that when happening has a positive or negative effect on a person or a group of people. Gigerenzer (2003) suggested that such an event should be associated w7ith a probability or a frequency with is based on empirical data about its past or potential occurrence. In this definition the author includes different interpretations of probability; such as subjective probability (degree of belief of the person assigning the probability), propensity (physical property of an object; for example, regularity in a dice) or frequentist estimation (infonnation from a large number of observations of the event). Consequently, an event could be perceived or not as a risk, depending on the person's conception of probability.Today an increasing number of events are described in terms of risk using mathematical formats, such as probabilities, proportions or percentages. The underlying concepts have to be learned in school and for this reason, mathematics educators are becoming interested in students' perception and understanding of risk (Martignon, 2014). In the same way, the assessment of students' potential biases, wrong strategies and misconceptions when interpreting risks is a relevant area of research in mathematics education (Nunes, & Bryant, 2011).In health or clinical contexts, including psychological evaluation or diagnosis, risk is synonymous of hazards and dangers; for example an illness or the undesirable effect of a treatment (Power, 2007). Risks in these contexts are often associated to decision making in such a way that it is impossible to make a 'risk-free' decision, unless we leave some potential risk factors unmanaged. A sound understanding of risk and of the associated numerical information is essential for these professionals, in order to make adequate decisions. However, numerous examples that this understanding is not complete are described in the literature related with risk perception and management. For example, Gigerenzer et al. (2007) termed collective statistical illiteracy the fact that many professionals have difficulties in interpreting health statistics and draw wrong conclusions in the clinical practice without noticing. Gigerenzer and Edwards (2003) suggest that misperception of risk factors occur by confusing single event probabilities, conditional probabilities, and relative risks. They also suggest that the situation can be improved by education and by representing the information regarding risks in ways that are transparent for the human mind, such as natural frequencies, tree diagrams or two-way tables.In fact, two-way tables are a main representational tool for bivariate data, and are often used in professional journals to report the influence of risk factors on different pathologies. Its understanding is related to risk perception (Nunes, & Bryant, 2011). Adi, Karp lus, Lawson, and Pulos (1978) suggested that providing subjects with information already organised in tables improves their performance in tasks in which they are asked to assess whether there is an association between two events.These tables; in particular 2x2 tables (two-way tables with only two columns and two rows) are also an important tool in diagnosis and psychological evaluation, where psychologists are confronted with different potential risk factors that may be associated or not with a disorder (risk) (Diaz, & Gallego, 2006). The estimation of association in these tables is the first step to determine which factors are associated to specific risks. However, even when association judgment is a priority learning issue in statistics courses (Zieffler, 2006), little attention is paid to its teaching, in assuming that the interpretation of 2x2 tables is easy. …

Altitude Does Not Reduce Concussion Incidence: Letter to the Editor.

Altitude Does Not Reduce Concussion Incidence: Letter to the Editor.

Application of Analysis of the Statistical Data to Formation of the State Order for the Teacher Training

As a source of socially significant information, statistics plays an important role in the modern society. Inability to value and understand this role can be referred to as statistical illiteracy, which can cause some negative tendencies in the development of social processes. In such activities, a disregard of statistical data analysis can lead to an inefficient use of public funds. The paper deals with the illustration of the application of statistical regularities in such an important area of education as the formation of the state order in the process of teacher training. However, the main purpose of this paper is not only to show that the reduced number of study vouchers is not proportional to the reduced number of teachers working at secondary schools but also to emphasise that the structure of the state order for teacher training is totally unbalanced, and, in fact, is being formed without training of a young generation of mathematics, informatics, biology, geography, chemistry, physics and astronomy teachers.

Statistical Illiteracy in Residents: What They Do Not Learn Today Will Hurt Their Patients Tomorrow

Statistical Illiteracy in Residents: What They Do Not Learn Today Will Hurt Their Patients Tomorrow

Krebsfrüherkennung und Risikokommunikation

In most psychological and medical research, patients are assumed to have difficulties with health statistics but clinicians not. However, studies indicate that most doctors have problems in understanding health statistics, including those of their own speciality. For example, only two out of 20 urologists knew the information relevant for a patient to make an informed decision about whether to take PSA screening for prostate cancer, just 14 out of 65 physicians in internal medicine understood that 5-year survival rates do not tell anything about screening's benefit, and merely 34 out of 160 gynecologists were able to interpret the meaning of a positive test result. This statistical illiteracy has a direct effect on patients understanding and interpretation of medical issues. Not rarely their own limited health literacy and their doctors' misinformation make them suffer through a time of emotional distress and unnecessary anxiety. The main reasons for doctors' statistical illiteracy are medical schools that ignore the importance of teaching risk communication. With little effort doctors could taught the simple techniques of risk communication, which would make most of their statistical confusion disappear.

Transparency in numbers: the dangers of statistical illiteracy

Transparency in numbers: the dangers of statistical illiteracy

Making sense of health statistics

Many physicians, patients, health journalists and politicians do not un-derstand health statistics. Yet we make little effort to educate our children in statistical thinking or our medical students in understanding health statis-tics. This collective statistical illiteracy has resulted in serious consequences for health.

Statistical illiteracy undermines informed shared decision making

Shared decision making relies on the exchange of information between the physician and the patient and the involvement of both patient and physician in making the decision. Informed shared decision making thus requires that patients and doctors understand the benefits and harms of different treatment options. This, however, is severely undermined by what we call collective statistical illiteracy. Both patients and physicians have difficulties to understand the meaning of numbers so that an effective risk communication cannot take place. Risk communication based on misunderstandings, however, renders the "informed" in informed shared decision making obsolete. We show that the problem of statistical illiteracy can largely be solved by changing the representation of statistical information. Insight can be achieved by communicating risks in absolute, not relative terms; by using a frequentist formulation, which makes the reference class clear instead of communicating single event probabilities; and by communicating natural frequencies instead of conditional probabilities.

Helping Doctors and Patients Make Sense of Health Statistics

Many doctors, patients, journalists, and politicians alike do not understand what health statistics mean or draw wrong conclusions without noticing. Collective statistical illiteracy refers to the widespread inability to understand the meaning of numbers. For instance, many citizens are unaware that higher survival rates with cancer screening do not imply longer life, or that the statement that mammography screening reduces the risk of dying from breast cancer by 25% in fact means that 1 less woman out of 1,000 will die of the disease. We provide evidence that statistical illiteracy (a) is common to patients, journalists, and physicians; (b) is created by nontransparent framing of information that is sometimes an unintentional result of lack of understanding but can also be a result of intentional efforts to manipulate or persuade people; and (c) can have serious consequences for health. The causes of statistical illiteracy should not be attributed to cognitive biases alone, but to the emotional nature of the doctor-patient relationship and conflicts of interest in the healthcare system. The classic doctor-patient relation is based on (the physician's) paternalism and (the patient's) trust in authority, which make statistical literacy seem unnecessary; so does the traditional combination of determinism (physicians who seek causes, not chances) and the illusion of certainty (patients who seek certainty when there is none). We show that information pamphlets, Web sites, leaflets distributed to doctors by the pharmaceutical industry, and even medical journals often report evidence in nontransparent forms that suggest big benefits of featured interventions and small harms. Without understanding the numbers involved, the public is susceptible to political and commercial manipulation of their anxieties and hopes, which undermines the goals of informed consent and shared decision making. What can be done? We discuss the importance of teaching statistical thinking and transparent representations in primary and secondary education as well as in medical school. Yet this requires familiarizing children early on with the concept of probability and teaching statistical literacy as the art of solving real-world problems rather than applying formulas to toy problems about coins and dice. A major precondition for statistical literacy is transparent risk communication. We recommend using frequency statements instead of single-event probabilities, absolute risks instead of relative risks, mortality rates instead of survival rates, and natural frequencies instead of conditional probabilities. Psychological research on transparent visual and numerical forms of risk communication, as well as training of physicians in their use, is called for. Statistical literacy is a necessary precondition for an educated citizenship in a technological democracy. Understanding risks and asking critical questions can also shape the emotional climate in a society so that hopes and anxieties are no longer as easily manipulated from outside and citizens can develop a better-informed and more relaxed attitude toward their health.