Teaching trainees how to critically evaluate the literature - a crossover study at two pediatric residency programs.

Benjamin Nelson,David Nelson,Catherine Ingard

doi:10.5116/ijme.58ce.5f04

Abstract

ObjectivesThe purpose of this study was to assess the efficacy of a concise, evidence based medicine curriculum in improving the knowledge of pediatric residents at two institutions. MethodsSixty first and second year pediatric residents at MassGeneral Hospital for Children and MedStar Georgetown University Hospital participated in a crossover study. The evidence based medicine curriculum, consisting of 4 ninety minute sessions grounded in adult learning theory principles, was developed using the methodology described in the book ‘Studying a Study’. A validated 20 question evidence based medicine multiple choice test was administered on three separate occasions to measure baseline knowledge, efficacy of the curriculum in improving knowledge, and long term retention of that knowledge. Results Post curriculum, the fall group’s scores improved 23% from baseline (M=10.3, SD=2.4) to (M=12.7, SD=3.0) students (t(26)=-3.29, p=0.0018) while the spring group improved by 41% (M=10.0, SD=2.8) to (M=14.1, SD=2.2) students (t(32)=-6.46, p<0.0001). When re-tested 4-6 months later, the fall group’s scores did not significantly decline from their immediate post curriculum scores (M=12.7, SD=3.0) to (M=11.7, SD=3.0) students (t(32) =1.33, p=0.190). There was an association between number of sessions attended and increase in post curriculum score (χ2(3, N=60) =11.75, p=0.0083). ConclusionsFindings demonstrate our curriculum was effective in teaching evidence based medicine to pediatric residents, and fostered long term retention of knowledge. Based on these results, we believe this curriculum could be implemented at any institution.

Highlights

Understanding the concepts of evidence based medicine (EBM) with the ability to assess the medical literature is a necessary skill for all clinicians
Many pediatric programs offer an EBM curriculum in some form, a survey of chief residents at North American residency programs found that 7% (n=153) felt confident in their ability to teach EBM, and only 20% (n=153) were able to evaluate EBM effectiveness.[2]
Basing curriculum on adult learning theory is critical[3] in engaging adult learners and fostering knowledge retention, but these principles are rarely applied in the context of these journal clubs

Summary

Introduction

Understanding the concepts of evidence based medicine (EBM) with the ability to assess the medical literature is a necessary skill for all clinicians. Using a 5 point Likert scale anchored with no confidence (1) and complete confidence (5), residents were asked to rate their confidence in interpreting commonly used statistical tests before and after participating in the EBM curriculum. These included their ability to interpret p values and other statistical measures, determining if the correct statistical procedure was used, and identifying factors that relate to a study’s power. At the completion of the course, residents were asked to rate their experience using a 5 point Likert scale (1=poor, 5=great) They were asked their level of overall enjoyment, to assess the relevance of the course, if they learned skills which they could apply to daily practice, and whether they would recommend the course to their peers.

Objectives

Methods

Results

Discussion

Conclusion