Relationship between candidate communication ability and oral certification examination scores

Abstract

Structured case-based oral examinations are widely used in medical certifying examinations in the USA. These orals assess the candidate's decision-making skills using real or realistic patient cases. Frequently mentioned but not empirically evaluated is the potential bias introduced by the candidate's communication ability. This study aimed to assess the relationship between candidate communication ability and medical certification oral examination scores. Non-doctor communication observers rated a random sample of 90 candidates on communication ability during a medical oral certification examination. The multi-facet Rasch model was used to analyse the communication survey and the oral examination data. The multi-facet model accounts for observer and examiner severity bias. anova was used to measure differences in communication ability between passing and failing candidates and candidates grouped by level of communication ability. Pearson's correlations were used to compare candidate communication ability and oral certification examination performance. Candidate separation reliability values for the communication survey and the oral examination were 0.85 and 0.97, respectively, suggesting accurate candidate measurement. The correlation between communication scores and oral examination scores was 0.10. No significant difference was found between passing and failing candidates for measured communication ability. When candidates were grouped by high, moderate and low communication ability, there was no significant difference in their oral certification examination performance. Candidates' communication ability has little relationship to candidate performance on high-stakes, case-based oral examinations. Examiners for this certifying examination focused on assessing candidate decision-making ability and were not influenced by candidate communication ability.