The Continuing Quest for Meaningful Faculty Evaluations of Residents.

Abstract

Listen

Translate article

In this issue of JAMA Surgery, Ray et al1 report their findings at their institution, the University of Miami, Miami, Florida, on the (non)relationship betweenperformance on theAmerican Board of Surgery In-Training Examination (ABSITE) and themean scores of faculty evaluations. Several thoughts and conjectures stem from their study. First, the authors cite their use of the measures from the General Surgery Milestone Project as their base for evaluations in the New Innovations Residency Management Suite online system for the studyperiod 2011 to 2014; however, the General Surgery Milestone Project was under development in 2011 and only became effective for general surgery in July 2014. The 6-monthly residentperformance reviewand conversation with the programdirector or associate programdirector have been part of the Accreditation Council for Graduate Medical Education Surgical Residency Review Committee program requirements for decades. Second, their faculty evaluations were nonanonymous. Manyof our own faculty, citing that theywill beworkingwith residents over years, are leery of providing less than laudatory comments (constructive criticism)evenanonymouslybecause theybelieve residents can identify the sourceof the lessthan-A+ evaluation. They are not keen on nonanonymous evaluations.Ofnote,New Innovationsnowprovides away for faculty to make a “not for the resident to see” comment in a supplemental box at the bottom of the evaluation form. The residents in the studybyRay and colleaguesweremostly perceived as “A” performers (mean evaluation score, 92.24; median evaluation score, 92.65), as shown in the histogram (Figure 1A) with all annual evaluation scores in the top quartile, while ABSITE scores varied more. Finally, one can argue that the ABSITE and the faculty evaluationsmeasure different domains, not unlike the qualifying examination and the certifying examination for the American Board of Surgery, and that they complement one another rather than necessarily predict or correlate with one another. Some irascible residents ace theABSITE,while other residents with excellent professional demeanor and people skillsmaystrugglewithmultiple-choicequestions.Booksmarts and street smarts do not have an r value of 1, nor domultiplechoice question skills and the soft competencies of interpersonal and communication skills and professionalism. Should we expect an examination that measures 1 portion of 1 domain,medical knowledge, to correlatewitha subjectiveevaluation measuring 6 or 7 competency domains? Thisarticlechallengesus to improveourevaluationsof residentperformance,aworthyquestofanelusive target.Thenext major hurdle is evaluation of technical and operative skills.