The Patient's Voice in Assessing Resident Communication Skills

Abstract

To generate validity evidence for using patient-reported satisfaction ratings of residents' communication skills to determine progress along the Interpersonal and Communication Skills (ICS) Milestones. A single-institution, retrospective study analyzed a CAHPS Clinician and Group Survey (CG-CAHPS) database which collects inpatients' ratings of residents' communication skills using 6 questions on a scale of 1 (very poor) to 5 (very good). CG-CAHPS results for each resident were averaged for each question as well as across the 6 questions. The averaged ratings were compared between low and high performer groups. A large, academic, mid-western General Surgery residency program. General Surgery residents with 3 or more survey responses from July 2020 to June 2021 were included. Residents were dichotomized into low or high performer groups based on their end-of-year ICS1 sub-competency milestone within their post-graduate year (PGY) cohort. 543 CG-CAHPS responses across 44 residents were analyzed with a median of 9 (Interquartile range 6, 17) responses per resident. When residents were compared based on PGY, ratings for the question "resident's knowledge about your medical care of condition" demonstrated statistically significant differences with PGY5s receiving the lowest score (p = 0.05). PGY5s received the lowest averaged ratings across all questions (p = 0.08, η2 = 0.10). When residents were dichotomized into low (n = 21) and high performer groups (n = 23) based on ICS1 milestones, statistically significant differences were noted in ratings for the questions "concern the resident showed for your questions or worries" (4.81 vs. 5, p = 0.047) and "courtesy and respect of the resident" (4.75 vs. 5, p = 0.046). Analysis of patient ratings of surgery residents' communication skills demonstrated concordant findings between ICS1 milestone and 2 domains of CG-CAHPS responses. When low and high performer groups were compared, CG-CAHPS responses demonstrated a similar pattern. These findings provide validity evidence for CG-CAHPS data as a source of information for ICS1 sub-competency assessment.