A Pediatrics-Based Instrument for Assessing Resident Education in Evidence-Based Practice

Abstract

The principles of evidence-based practice (EBP) are a mandated component of the pediatric residency curriculum; however, a pediatrics-based assessment tool validated with pediatric residents does not exist. We designed an assessment instrument composed of items in 4 categories: 1) demographics; 2) comfort level; 3) self-reported practice of EBP; and 4) EBP knowledge. This last section required participants to identify best evidence and most appropriate study design by using pediatric-based scenarios, develop searchable questions, and use existing published research to address diagnostic and treatment issues. Four groups completed the instrument: preclinical medical students (MS-2), incoming pediatric interns (PGY-1), incoming second- and third-year pediatric residents (PGY2-3), and expert tutors (expert). We determined internal consistency, interrater reliability, content validity, item difficulty, and construct validity. Fifty-six subjects completed tests (MS-2, n = 13; PGY-1, n = 13; PGY2-3, n = 22; expert, n = 8). Internal reliability was good, with Cronbach's alpha = .80. Interrater reliability was high (kappa = 0.94). Items were free of floor or ceiling effects. Comfort level and self-reported practice of EBP increased with expertise level and prior EBP experience (P < .01). Scores on the knowledge section (out of 50 +/- SD) rose with training level (MS-2: 14.8 +/- 5.7; PGY-1: 22.2 +/- 3.4; PGY2-3: 31.7 +/- 6.1; experts: 43 +/- 4.0; P < .01). Scores also correlated with prior EBP education. We have developed a reliable and valid instrument to assess knowledge and skill in EBP taught to pediatric residents. This instrument can aid pediatric educators in monitoring the impact of the EBP curriculum.