MON-003 Enhancing Formative Feedback of Pediatric Endocrinology Fellows through Structured Oral Exams

Abstract

INTRODUCTION: The inadequacy of formative feedback (FF) in graduate medical education (GME) has been a concern of physician-educators and trainees for more than three decades. FF in GME is most often obtained through two assessments, the global ratings scale and In-Training Exam (ITE), both with well documented limitations. The global ratings scale (i.e. “milestones”) is subject to numerous threats to validity including limited direct observations of trainees, rater bias, inaccurate recall, and lack of specific clinical skills to rate. The ITE suffers from construct underrepresentation as atypical clinical scenarios and basic science knowledge is frequently assessed. The multiple-choice format can result in cueing, seem artificial, and removed from real situations. The structured oral exam (SOE), used for FF, does not have limitations of the assessments noted above. To the best of the authors’ knowledge, there have been no published reports of the use of a SOE to provide FF in GME. OBJECTIVE: To determine the educational impact, as well as the feasibility, cost, and acceptability of a SOE for FF of Pediatric Endocrinology (PE) fellows. METHODS: Four cases (precocious and delayed puberty, PCOS, congenital hypothyroidism) were developed for this pilot SOE. Each question and scoring rubric was evidence-based and peer-reviewed by four PE attending physicians. A 5-point Likert scale survey with short answers was developed utilizing best practice guidelines to address our research question. Three faculty/fellow dyads completed the pilot. RESULTS: With regards to educational impact, all fellows and faculty reported that the exam was helpful for both identifying previously unrecognized knowledge deficits, and for providing guidance for the fellow’s future learning, with the majority stating it was quite helpful or extremely helpful. There were no significant concerns regarding need for specialized training to administer the exam, clarity of questions, subjectivity in scoring, or stress on the fellow. Several participants proposed shorter, more frequent sessions to enable longitudinal assessment. An estimated 4 hours of faculty time would be required per fellow per year to administer our comprehensive SOE, consisting of 35 cases, over the course of a 3 year fellowship. Administration by senior fellows was suggested if faculty time is limited. CONCLUSIONS: To the authors’ knowledge, this is the first description of a SOE as a FF tool in GME. Fellows and faculty found this SOE to be educationally impactful, however, modifications could further improve it’s feasibility, cost, and acceptability. The full-scale, multi-center study will provide further investigation of this SOE as a FF tool for PE fellows.