The feasibility and value of using patient satisfaction ratings to evaluate internal medicine residents

Abstract

To determine the feasibility and value of using patient satisfaction ratings to evaluate the physician-patient relationship skills of medical residents. A cross-sectional survey was used to collect patient satisfaction information for medical residents from all patients attending the out-patient teaching clinics over a three-month period. The feasibility of patient rating was assessed by evaluating the reliability of resident satisfaction scores and potential sources of bias in rating. The value of using this method was assessed by evaluating the proportion of variance in ratings attributable to residents, the prevalence of residents with ratings substantially below average, and the effect of training on patient satisfaction. The medical clinics of two teaching hospitals at McGill University. 91 medical residents and clinical clerks, 1,003 patients, and 1,219 visit ratings. An average of 12.2 satisfaction ratings were collected per resident, providing a reliability of 0.56 (intraclass correlation) for resident satisfaction score. Adjustment for differences in patient populations would be necessary to obtain comparable scores among residents. Fifteen percent of the residents had a substantially greater proportion of poor and fair satisfaction ratings than the study average. Residents were responsible for a substantial proportion of the variance in clinic waiting time and patient satisfaction rating. Training appeared to have no effect on improving satisfaction scores, except among women residents. Patient satisfaction ratings provide valuable information about a resident's ability to establish an effective physician-patient relationship. However, the number of ratings required to obtain a reliable estimate of resident skill may limit the feasibility of using patient ratings as part of residency evaluation.