Mean Diagnostic Accuracy Scores Research Articles

The Effect of Information Presentation Order on Residents' Diagnostic Accuracy of Online Simulated Patients With Chest Pain.

Physicians may receive diagnostic information in different orders, and there is a lack of empirical evidence that the order of presentation may influence clinical reasoning. We investigated whether diagnostic accuracy of chest pain cases is influenced by the order of presentation of the history and electrocardiogram (EKG) to cardiology residents. We conducted an experimental study during a resident training in 2019. Twelve clinical cases were presented in 2 diagnostic rounds. Residents were randomly allocated to seeing the EKG first (EKGF) or the history first (HF). The mean diagnostic accuracy scores (range 0-1) and confidence level (0-100) in each diagnostic round and time needed to make the diagnosis were evaluated. The final diagnostic accuracy was higher than the initial in both groups. After the first round, diagnostic accuracy was higher in HF (n=24) than in EKGF (n=28). Time taken to judge the history was comparable in both groups. Time taken to judge the EKG was shorter in HF (40±11 vs 64±13 seconds; P<.01). Time invested in the second round was significantly correlated with changing the initial diagnosis. A significant difference was observed in confidence ratings after the initial diagnosis, with EKGF reporting less confidence relative to HF. The order in which history and EKG are presented influences the clinical reasoning process.

Notes from the Editor Emeritus, 1996–1998

<h3>Background</h3> Literature suggests that patients who display disruptive behaviours in the consulting room fuel negative emotions in doctors. These emotions, in turn, are said to cause diagnostic errors. Evidence substantiating this claim is however lacking. The purpose of the present experiment was to study the effect of such difficult patients’ behaviours on doctors’ diagnostic performance. <h3>Methods</h3> We created six vignettes in which patients were depicted as difficult (displaying distressing behaviours) or neutral. Three clinical cases were deemed to be diagnostically simple and three deemed diagnostically complex. Sixty-three family practice residents were asked to evaluate the vignettes and make the patient9s diagnosis quickly and then through deliberate reflection. In addition, amount of time needed to arrive at a diagnosis was measured. Finally, the participants rated the patient9s likability. <h3>Results</h3> Mean diagnostic accuracy scores (range 0–1) were significantly lower for difficult than for neutral patients (0.54 vs 0.64; p=0.017). Overall diagnostic accuracy was higher for simple than for complex cases. Deliberate reflection upon the case improved initial diagnostic, regardless of case complexity and of patient behaviours (0.60 vs 0.68, p=0.002). Amount of time needed to diagnose the case was similar regardless of the patient9s behaviour. Finally, average likability ratings were lower for difficult than for neutral-patient cases. <h3>Conclusions</h3> Disruptive behaviours displayed by patients seem to induce doctors to make diagnostic errors. Interestingly, the confrontation with difficult patients does however not cause the doctor to spend less time on such case. Time can therefore not be considered an intermediary between the way the patient is perceived, his or her likability and diagnostic performance.

Flexible e-learning video approach to improve fundus examination skills for medical students: a mixed-methods study

BackgroundTraining for the fundus examination using traditional teaching is challenging, resulting in low generalist physicians’ confidence in performing the funduscopic examination. There is growing evidence suggesting a flexible e-learning video approach’s value in teaching physical examination procedures. However, whether the flexible e-learning video approach is superior to the traditional, face-to-face (F2F) lecture-based teaching for the funduscopic exam and the cognitive processes supporting its effectiveness has not yet been determined.MethodsWe conducted a sequential explanatory mixed-method study to compare the flexible e-learning video approach’s effectiveness versus the F2F lecture-based approach for teaching the funduscopic exam to medical students at Chiba University in Japan. Medical students were randomly assigned to either a flexible e-learning video approach group or a F2F lecture approach group. We then quantitatively measured the diagnostic accuracy of funduscopic findings before and after attending the specific classrooms. Next, we conducted student focus groups to explore the students’ thinking processes in the flexible e-learning video approach vs. the F2F lecture-based teaching of fundus examination. The qualitative data were analyzed using the qualitative content analysis method.ResultsThe mean diagnostic accuracy scores in the post-test significantly increased from pre-test in the intervention group (36.6 to 63.4%, p < 0.001). Post-post comparisons across the two groups revealed a significant difference (intervention group 63.4% vs. control group 34.6%, p < 0.001). Six semi-structured focused group interviews were conducted (n = 36). In the flexible e-learning video approach group, we identified ten categories corresponding to four levels of the revised Bloom’s taxonomy: remember, understand, apply, analyze. Five categories were identified in the traditional F2F lecture approach group corresponding to three revised Bloom’s taxonomy levels: understand, apply, analyze. Interrater reliability was substantial (Cohen’s kappa = 0.81).ConclusionsTeaching medical students funduscopic examination using the flexible e-learning video approach leads to improved diagnostic accuracy of funduscopic examinations. The flexible e-learning video teaching method enabled higher cognitive activity levels than the traditional, lecture-based classroom, as assessed using the revised Bloom’s taxonomy.Trial registrationThis study was registered with the University Hospital Medical Information Network Clinical Trials Registry on 08/02/2020 (Unique trial number: UMIN 000039434).

Think Twice: Effects on Diagnostic Accuracy of Returning to the Case to Reflect Upon the Initial Diagnosis.

Diagnostic errors have been attributed to failure to sufficiently reflect on initial diagnoses. However, evidence of the benefits of reflection is conflicting. This study examined whether reflection upon initial diagnoses on difficult cases improved diagnostic accuracy and whether reflection triggered by confrontation with case evidence was more beneficial than simply revising initial diagnoses. Participants were physicians in Bern, Switzerland, registered for the 2018 Swiss internal medicine certification exam. They diagnosed written clinical cases, providing an initial diagnosis by following the same instructions and returning to the case to provide a final diagnosis. The latter required different types of reflection depending on the physician's experimental condition: return without instructions, identify confirmatory evidence, identify contradictory evidence, or identify both confirmatory and contradictory evidence. The authors examined diagnostic accuracy scores (range 0-1) as a function of diagnostic phase and reflection type. One hundred and sixty-seven physicians participated. Diagnostic accuracy scores did not significantly differ between the 4 groups of physicians in the initial (I) or the final (F) diagnostic phase (mean [95% CI]: return without instructions, I: 0.21 [0.17, 0.26], F: 0.23 [0.18, 0.28]; confirmatory evidence, I: 0.24 [0.19, 0.29], F: 0.31 [0.25, 0.37]; contradictory evidence, I: 0.22 [0.17, 0.26], F: 0.26 [0.22, 0.30]; confirmatory and contradictory evidence, I: 0.19 [0.15, 0.23], F: 0.25 [0.20, 0.31]). Regardless of type of reflection employed while revising the case, accuracy increased significantly between initial and final diagnosis, I: 0.22 (0.19, 0.24) vs F: 0.26 (0.24, 0.29); P < .001. Physicians' diagnostic accuracy improved after reflecting upon initial diagnoses provided for difficult cases, independently of the evidence searched for while reflecting. The findings support the importance attributed to reflection in clinical teaching. Future research should investigate whether revising the case can become more beneficial by triggering additional reflection.

Teaching clinical reasoning through hypothetico-deduction is (slightly) better than self-explanation in tutorial groups: An experimental study.

BackgroundSelf-explanation while individually diagnosing clinical cases has proved to be an effective instructional approach for teaching clinical reasoning. The present study compared the effects on diagnostic performance of self-explanation in small groups with the more commonly used hypothetico-deductive approach.MethodsSecond-year students from a six-year medical school in Saudi Arabia (39 males; 49 females) worked in small groups on seven clinical vignettes (four criterion cases representing cardiovascular diseases and three ‘fillers’, i.e. cases of other unrelated diagnoses). The students followed different approaches to work on each case depending on the experimental condition to which they had been randomly assigned. Under the self-explanation condition, students provided a diagnosis and a suitable pathophysiological explanation for the clinical findings whereas in the hypothetico-deduction condition students hypothesized about plausible diagnoses for signs and symptoms that were presented sequentially. One week later, all students diagnosed eight vignettes, four of which represented cardiovascular diseases. A mean diagnostic accuracy score (range: 0–1) was computed for the criterion cases. One-way ANOVA with experimental condition as between-subjects factor was performed on the mean diagnostic accuracy scores.ResultsStudents in the hypothetico-deduction condition outperformed those in the self-explanation condition (mean = 0.22, standard deviation = 0.14, mean = 0.17; standard deviation = 0.12; F(1, 88) = 4.90, p = 0.03, partial η2 = 0.06, respectively).ConclusionsStudents in the hypothetico-deduction condition performed slightly better on a follow-up test involving similar cases, possibly because they were allowed to formulate more than one hypothesis per case during the learning phase.

Erratum to: The effect of self-explanation of pathophysiological mechanisms of diseases on medical students' diagnostic performance.

Self-explanation while diagnosing clinical cases fosters medical students’ diagnostic performance. In previous studies on self-explanation, students were free to self-explain any aspect of the case, and mostly clinical knowledge was used. Elaboration on knowledge of pathophysiological mechanisms of diseases has been largely unexplored in studies of strategies for teaching clinical reasoning. The purpose of this two-phase experiment was to investigate the effect of self-explanation of pathophysiology during practice with clinical cases on students’ diagnostic performance. In the training phase, 39 4th-year medical students were randomly assigned to solve 6 criterion cases (3 of jaundice; 3 of chest pain), either self-explaining the pathophysiological mechanisms of the findings (n = 20) or without self-explaining (n = 19). One-week later, in the assessment phase, all students solved 6 new cases of the same syndromes. A repeated-measures analysis of variance on the mean diagnostic accuracy scores showed no significant main effects of study phase (p = 0.34) and experimental condition (p = 0.10) and no interaction effect (p = 0.42). A post hoc analysis found a significant interaction (p = 0.022) between study phase and syndrome type. Despite equal familiarity with jaundice and chest pain, the performance of the self-explanation group (but not of the non-self-explanation group) on jaundice cases significantly improved between training and assessment phases (p = 0.035) whereas no differences between phases emerged on chest pain cases. Self-explanation of pathophysiology did not improve students’ diagnostic performance for all diseases. Apparently, the positive effect of this form of self-explanation on performance depends on the studied diseases sharing similar pathophysiological mechanisms, such as in the jaundice cases.

The effect of self-explanation of pathophysiological mechanisms of diseases on medical students' diagnostic performance.

Self-explanation while diagnosing clinical cases fosters medical students' diagnostic performance. In previous studies on self-explanation, students were free to self-explain any aspect of the case, and mostly clinical knowledge was used. Elaboration on knowledge of pathophysiological mechanisms of diseases has been largely unexplored in studies of strategies for teaching clinical reasoning. The purpose of this two-phase experiment was to investigate the effect of self-explanation of pathophysiology during practice with clinical cases on students' diagnostic performance. In the training phase, 39 4th-year medical students were randomly assigned to solve 6 criterion cases (3 of jaundice; 3 of chest pain), either self-explaining the pathophysiological mechanisms of the findings (n=20) or without self-explaining (n=19). One-week later, in the assessment phase, all students solved 6 new cases of the same syndromes. A repeated-measures analysis of variance on the mean diagnostic accuracy scores showed no significant main effects of study phase (p=0.34) and experimental condition (p=0.10) and no interaction effect (p=0.42). A post hoc analysis found a significant interaction (p=0.022) between study phase and syndrome type. Despite equal familiarity with jaundice and chest pain, the performance of the self-explanation group (but not of the non-self-explanation group) on jaundice cases significantly improved between training and assessment phases (p=0.035) whereas no differences between phases emerged on chest pain cases. Self-explanation of pathophysiology did not improve students' diagnostic performance for all diseases. Apparently, the positive effect of this form of self-explanation on performance depends on the studied diseases sharing similar pathophysiological mechanisms, such as in the jaundice cases.

Exposure to Media Information About a Disease Can Cause Doctors to Misdiagnose Similar-Looking Clinical Cases

Anecdotal evidence indicates that exposure to media-distributed disease information, such as news about an outbreak, can lead physicians to errors; influenced by an availability bias, they misdiagnose patients with similar-looking but different diseases. The authors investigated whether exposure to media-provided disease information causes diagnostic errors and whether reflection (systematic review of findings) counteracts bias. In 2010, 38 internal medicine residents first read the Wikipedia entry about one or another of two diseases (Phase 1). Six hours later, in a seemingly unrelated study, they diagnosed eight clinical cases (Phase 2). Two cases superficially resembled the disease in the Wikipedia entry they had read (bias expected), two cases resembled the other disease they had not read about (bias not expected), and four were filler cases. In Phase 3, they diagnosed the bias-expected cases again, using reflective reasoning. Mean diagnostic accuracy scores (Phase 2; range: 0-1) were significantly lower on bias-expected cases than on bias-not-expected cases (0.56 versus 0.70, P = .016) because participants misdiagnosed cases that looked similar to a Wikipedia description of a disease more often when they had read the Wikipedia description (mean = 0.61) than when they had not (mean = 0.29). Deliberate reflection (Phase 3) restored performance on bias-expected cases to pre-bias levels (mean = 0.71). Availability bias may arise simply from exposure to media-provided information about a disease, causing diagnostic errors. The bias's effect can be substantial. It is apparently associated with nonanalytical reasoning and can be counteracted by reflection.

Why Do Doctors Make Mistakes? A Study of the Role of Salient Distracting Clinical Features

Diagnostic errors have been attributed to faulty reasoning and cognitive biases, but minimizing errors requires understanding the mechanisms underlying biases. The authors investigated whether salient distracting features (SDFs)-case findings that tend to grab physicians' attention because they are strongly associated with a particular disease, but are indeed unrelated to the problem-misdirect diagnostic reasoning, causing errors. In a 2012 study conducted at Erasmus Medical Centre, Rotterdam, 72 internal medicine residents diagnosed 12 clinical cases (6 simple, 6 complex) in three different formats: without a SDF, with a SDF in the beginning, and with a SDF at the end. In a within-subjects design, each participant solved 2 simple cases and 2 complex cases in each format. Proportions of correct diagnoses in each case type were compared by performing repeated-measures analysis of variance (ANOVA). There was a significant main effect of SDFs and a significant interaction effect between SDFs and case complexity. The presence of SDFs in the beginning of complex cases caused errors decreasing the proportion of correct diagnoses in comparison both with cases without SDFs (0.18, 95% CI, 0.13-0.23 versus 0.43, 95% CI, 0.35-0.51; P < .001) or with SDFs at the end (0.18, 95% CI, 0.13-0.23 versus 0.36, 95% CI, 0.29-0.43; P < .001). SDFs did not affect performance when presented near the end of cases. SDFs early in a case are apparently an important source of diagnostic errors. Physicians should be aware of the need to overcome their influence.

How Can Students’ Diagnostic Competence Benefit Most From Practice With Clinical Cases? The Effects of Structured Reflection on Future Diagnosis of the Same and Novel Diseases

To develop diagnostic competence, students should practice with many examples of clinical problems to build rich mental representations of diseases. How to enhance learning from practice remains unknown. This study investigated the effects of reflection on cases compared with generating a single or differential diagnosis. In 2012, during the learning phase, 110 fourth-year medical students diagnosed four cases of two criterion diseases under three different experimental conditions: structured reflection, single-diagnosis, or differential-diagnosis. One week later, they diagnosed two novel exemplars of each criterion disease and four cases of new diseases that were not among the cases of the learning phase but were plausible alternative diagnoses. Diagnostic performance did not differ among the groups in the learning phase. One week later, the reflection group obtained higher mean diagnostic accuracy scores (range: 0-1) than the other groups when diagnosing new exemplars of criterion diseases (reflection: 0.67; single-diagnosis: 0.36, P < .001; differential-diagnosis: 0.51, P = .014) and cases of new diseases (reflection: 0.44; single-diagnosis: 0.32, P = .010; differential-diagnosis: 0.33, P = .015). No difference was found between the single-diagnosis and the differential-diagnosis conditions. Structured reflection while practicing with cases enhanced learning of diagnosis both of the diseases practiced and of their alternative diagnoses, suggesting that reflection not only enriched mental representations of diseases practiced relative to more conventional approaches to clinical learning but also influenced the representations of adjacent but different diseases. Structured reflection seems a useful addition to the existing clinical teaching methods.

Effect of enhanced analytic reasoning on diagnostic accuracy: A randomized controlled study

Background: Diagnostic error can be caused by several types of cognitive bias, which may be reversed by enhancing analytic reasoning.Aims: To evaluate whether enhancing analytic reasoning can increase diagnostic accuracy in objective structured clinical examination (OSCE) in medical students.Methods: All fourth-year medical students, randomly assigned to the analytic reasoning or control groups, undertook the OSCE with four cases using standardized patients. The analytic reasoning group was requested to list differential diagnoses and findings compatible or not compatible with each diagnosis prior to providing a diagnosis, while the control group provided a diagnosis without these processes. Mean diagnostic accuracy scores (perfect score, 4.0) from four cases of OSCE were compared between the two groups.Results: One hundred forty-five students were randomly assigned to the analytic reasoning group (n = 65) or the control group (n = 80). The baseline characteristics, including grade point average and the scores from each patient encounter, were comparable between groups. Mean diagnostic accuracy scores were significantly higher in the analytic reasoning group than in the control group (3.40 ± 0.66 versus 3.05 ± 0.98; p = 0.011).Conclusion: Enhancement of analytic reasoning may improve diagnostic accuracy in novice doctors.

Reflection as a strategy to foster medical students’ acquisition of diagnostic competence

Developing diagnostic competence in students is a major goal of medical education, but there is little empirical evidence on instructional strategies that foster the acquisition of this competence. The aim of this study was to investigate the effects of structured reflection compared with the generation of immediate or differential diagnosis while practising with clinical cases on learning clinical diagnosis. This was a three-phase experimental study. During a learning phase, 46 Year 4 students diagnosed six clinical cases under different experimental conditions: structured reflection, immediate diagnosis, or differential diagnosis. This was followed by an immediate test and a delayed test administered 1 week later. Each test consisted of diagnosing four different cases of diseases presented in the learning phase. Performance in diagnosing these new cases was used as a measure of learning. Repeated-measures analysis of variance on the mean diagnostic accuracy scores (range: 0-1) showed a significant interaction between performance moment (i.e. performance in the learning phase and on each test) and instructions followed during the learning phase (p=0.003). Follow-up analyses of this interaction showed that diagnostic performance did not differ between conditions in the learning phase. On the immediate test, scores in the reflection condition (mean=0.48, 95% confidence interval [CI] 0.38-0.58) were significantly lower than scores in the differential diagnosis condition (mean=0.62, 95% CI 0.54-0.70; p=0.012) and marginally lower than those in the immediate diagnosis condition (mean=0.61, 95% CI 0.52-0.70; p=0.04). One week later, however, scores in the reflection condition (mean=0.66, 95% CI 0.56-0.76) significantly outperformed those in the other conditions (differential diagnosis: mean=0.48, 95% CI 0.37-0.58 [p<0.01]; immediate diagnosis: mean=0.52, 95% CI 0.43-0.60 [p=0.01]). Comparisons within experimental conditions showed that performance from the immediate to the delayed test decreased in the immediate and differential diagnosis conditions (immediate diagnosis: p=0.042; differential diagnosis: p=0.012), but increased in the reflection condition (p=0.003). Structured reflection while practising with cases appears to foster the learning of clinical knowledge more effectively than the generation of immediate or differential diagnoses and therefore seems to be an effective instructional approach to developing diagnostic competence in students.

Effect of Availability Bias and Reflective Reasoning on Diagnostic Accuracy Among Internal Medicine Residents

Diagnostic errors have been associated with bias in clinical reasoning. Empirical evidence on the cognitive mechanisms underlying biases and effectiveness of educational strategies to counteract them is lacking. To investigate whether recent experience with clinical problems provokes availability bias (overestimation of the likelihood of a diagnosis based on the ease with which it comes to mind) resulting in diagnostic errors and whether reflection (structured reanalysis of the case findings) counteracts this bias. Experimental study conducted in 2009 at the Erasmus Medical Centre, Rotterdam, with 18 first-year and 18 second-year internal medicine residents. Participants first evaluated diagnoses of 6 clinical cases (phase 1). Subsequently, they diagnosed 8 different cases through nonanalytical reasoning, 4 of which had findings similar to previously evaluated cases but different diagnoses (phase 2). These 4 cases were subsequently diagnosed again through reflective reasoning (phase 3). Mean diagnostic accuracy scores (perfect score, 4.0) on cases solved with or without previous exposure to similar problems through nonanalytical (phase 2) or reflective (phase 3) reasoning and frequency that a potentially biased (ie, phase 1) diagnosis was given. There were no main effects, but there was a significant interaction effect between "years of training" and "recent experiences with similar problems." Results consistent with an availability bias occurred for the second-year residents, who scored lower on the cases similar to those previously encountered (1.55; 95% confidence interval [CI], 1.15-1.96) than on the other cases (2.19; 95% CI, 1.73-2.66; P =.03). This pattern was not seen among the first-year residents (2.03; 95% CI, 1.55-2.51 vs 1.42; 95% CI, 0.92-1.92; P =.046). Second-year residents provided the phase 1 diagnosis more frequently for phase 2 cases they had previously encountered than for those they had not (mean frequency per resident, 1.44; 95% CI, 0.93-1.96 vs 0.72; 95% CI, 0.28-1.17; P =.04). A significant main effect of reasoning mode was found: reflection improved the diagnoses of the similar cases compared with nonanalytical reasoning for the second-year residents (2.03; 95% CI, 1.49-2.57) and the first-year residents (2.31; 95% CI, 1.89-2.73; P =.006). When faced with cases similar to previous ones and using nonanalytic reasoning, second-year residents made errors consistent with the availability bias. Subsequent application of diagnostic reflection tended to counter this bias; it improved diagnostic accuracy in both first- and second-year residents.