Do faculty show the ‘halo effect’ in rating students compared with standardized patients during a clinical examination?

Abstract

BackgroundWe compared standardized patient (SP) and faculty ratings of student performance to test the hypothesis that faculty systematically overestimate student clinical skills compared with SPs.MethodsWe conducted a study of three successive classes of second year medical students (n=303). Students were assessed by SPs duing an Objective Structured Clinical Examination and faculty simultaneously rated student performance using the modified mini-clinical evaluation exercise (miniCEX). The Patient Physician Interaction scale (PPI), a communication scale designed for use by SPs and the mini-CEX measure designed for use by faculty were each analyzed for internal consistency reliability. Within the SP and faculty measures scores for similar skills of history-taking, physical examination, humanism/communication and global score were analyzed in aggregate for concordance. Range and patterns of SP and faculty scores were examined and compared.ResultsThe PPI and mini-CEX each demonstrated high internal consistency (Cronbach’s α .86, .89). Correlation between clinical skill (history, examination, humanism/communication and global) score as rated by SPs and faculty was modest (Pearson’s r 0.213 to 0.288). SPs and faculty both used the entire range of scores within each measure with a normal distribution.ConclusionsThe PPI and mini-CEX have high internal consistency reliability. There is only modest correlation between SP and faculty ratings of student clinical skills. Differences should be recognized when giving student immediate feedback on their performance in a standardized examination. Faculty rate students without a ‘halo’ effect.