Understanding the Shades of Gray in Diagnosis-An Online Course in Bayesian Reasoning.

Abstract

Foundational skills in Bayesian reasoning may aid clinical decision-making, decrease overreliance on single diagnostic tests, and improve patient care. However, clinicians are often unskilled in applying Bayes' rule at the bedside, such as using likelihood ratios to calculate posttest probabilities.1While published literature measures clinicians' abilities to update conditional probabilities based on dichotomous test results, many diagnostic tests used in clinical practice are not dichotomous.1 One of the gaps in clinical decision-making revolves around the interpretation of continuous tests such as D-dimer, troponin, and procalcitonin. Although often dichotomized as “normal” and “abnormal” for simplicity, the test's degree of abnormality may have a profound effect on how the result changes the disease probability. Explicit instruction in understanding the “shades of gray” in diagnostic test interpretation is lacking.Clinicians who are deficient in knowledge of Bayesian reasoning are often involved in trainee education. Faculty development initiatives to improve Bayesian reasoning would presumably have positive downstream effects on learners. Educational efforts may be limited by faculty time, availability, and lack of local experts in Bayesian reasoning. Furthermore, the recent COVID-19 pandemic has presented additional challenges to designing and delivering in-person educational curricula.We developed an online, self-paced, asynchronous, case-based curriculum in Bayesian reasoning. The course uses a flipped classroom approach and revolves around 10 core clinical cases. Each case represents a realistic patient scenario. The learner reads each scenario and answers a series of questions assessing a range of skills involved in Bayesian reasoning. The learner is asked to incorporate various findings in the estimation of disease probability, updating disease probability based on new information and integrating this information to guide subsequent management decisions. Each case is followed by video explanations that guide learners through the fundamentals of Bayesian reasoning. The course gradually builds from simple test characteristics, such as sensitivity and specificity, to understanding multilevel likelihood ratios, receiver operating characteristic curves, and action thresholds when making decisions under uncertainty. Course completion is estimated to take 6 to 8 hours. All course materials were created by the study authors.The course was piloted to faculty volunteers beginning June 2021 for the Division of Hospital Medicine faculty at the University of Colorado. Informed consent was obtained. Each participant took 15-question pre- and posttests to evaluate the baseline and gained knowledge. Participants also self-assessed their prior knowledge, ranked the importance of Bayesian reasoning in their clinical care, and noted their satisfaction with the course on a 4-point Likert scale that excluded neutral responses. The Colorado Multiple Institutional Review Board granted the study an exempt status.As of October 2021, 18 out of 20 invited faculty members started the course, and 12 (67%) completed it. Overall, 11 of 12 (92%) faculty members rated their prior knowledge in Bayesian reasoning as “somewhat” or “very weak.” Most (10 of 12) participants rated the importance of Bayesian reasoning in clinical care as “very important,” and the rest rated it as “somewhat important.” All participants (100%) reported feeling “very satisfied” with the course and indicated that it is “very likely” they would recommend the course to a colleague. The pretest assessment demonstrated serious deficiencies in all tested domains, and the posttest assessment showed marked improvement in all of them (Figure). This online course is now integrated into the internal medicine residency curriculum, and all trainees completed the course by February 2022. Further dissemination is planned to medical schools, residency training programs, and faculty outside of our institution.