Psychiatry Clinical Skills Evaluation: a Multisite Study of Validity.

Abstract

Since 2007, the American Board of Psychiatry and Neurology (ABPN) has required that residency programs conduct a specific clinical skills evaluation (CSE) of physician-patient interaction, psychiatric interview and mental status examination, and case presentation on a directly observed patient interview as a prerequisite for certification. The authors examined a multisite database of CSE assessments to investigate the validity of the evaluation. The authors collected 1156 CSE assessments from 4 residency programs conducted over a 6-year period, compared scoring patterns among the programs, score improvement over 4 years of residency, time and number of CSEs required to meet ABPN requirements, and patterns of scoring for individual faculty evaluators. The distribution of scores within each of the 4 programs showed similar, but nonidentical patterns. The number of CSEs required to meet the ABPN standards (3.5) and the point in training at which this was completed (late PGY-2) were the same in all programs. CSE scores were highly correlated with year of training but were not correlated with performance on an unrelated cognitive examination. Individual faculty members tended to stay within a moderate range of scores over multiple residents, partially attributable to year of training. Taken together, these findings support the validity of the CSE as a measure of residents' clinical skills in the specified areas and demonstrate a moderate-high degree of consistency in the scoring of the CSE across these 4 programs.