Educational Needs Assessment of US Allergy/Immunology Fellowship Programs: Assessment Methods for Determining Competency of Fellows in-Training

Abstract

ACGME introduced a milestone framework to assist in competency development in GME trainees. Little is known about current use of assessment methods in U.S. Allergy/Immunology (A/I) Fellowship programs. A recommendation by the A/I Milestone Working Group was that programs “should make use of all available data” from a wide variety of assessment sources. IRB approval was obtained for a prospective survey study designed as a needs assessment. A link to a REDcap questionnaire, previously piloted, was e-mailed to 74 U.S. Program Directors (PDs) to identify currently used assessment methods. We were also interested in the provision of feedback to trainees. Program characteristics were examined for continuous and categorical variables. Twenty-one of 74 PDs (28%) responded, with 19 completing questions regarding assessment methods. Thirteen PDs completed all questionnaire items. The most commonly used assessment methods were the ITE (100%), direct observation of trainees (95%), faculty assessments (89%), and 360-degree assessments (89%). Commonly used methods (>50% of programs) included self-assessments, portfolios, and patient surveys. More information about multiple choice examinations (23%), teamwork assessments (15%), chart-stimulated recall (15%), and OSCEs (15%) was wanted. Feedback, verbal and written, was typically provided to trainees every six months and classified as both formative and summative in nature. Our results indicate consistency in the type of assessment methods used in respondents’ training programs. Peer assessments and OSCEs, while recommended, were less commonly used in this sample. Faculty development regarding recommended methods may increase their use and provide additional valuable feedback to trainees.