Oral Examination versus Simulation-Based Assessment in Assessing Patient Care Competencies for Emergency Medicine Residents

Abstract

Background: Assessment is essential for any accreditation process in the medical field. If a candidate passes a high-stakes assessment, they can work independently. While oral examinations are common, given the complexity of clinical competencies, such an approach may not be the most effective assessment method. A form of performance-based assessment, such as a simulation, may be beneficial in this context. Objectives: This study aims to determine whether the results of oral examinations match those of simulation-based assessments when both modalities are used to evaluate residents’ performance in scenarios featuring similar content. It also seeks to determine whether oral examinations under- or overestimate residents’ competencies concerning patient care when compared to their simulation performance. Methods: This is a cross-sectional, single-centre study. Emergency medicine residents underwent an oral examination and completed a simulation-based assessment. Standardized scenarios were used to assess the residents’ emergency medicine competencies. A global rating scale was used to rate participants’ performance in each assessment modality. Results: There was a moderate positive correlation between oral examination and simulation-based assessment results (r = 0.699, p < 0.05, n = 28). A paired t test indicates that the oral examination overestimates residents’ competency compared to the simulation-based assessment; the mean difference is 0.26 (confidence interval: 0.041–0.493). Conclusions: Emergency medicine residents whose knowledge was assessed at the “know-how” level of Miller’s pyramid in the oral examination were not necessarily able to move up to the level of “show-how” by demonstrating the ability to apply their knowledge in the simulation-based assessment. The findings of this study confirm that simulation-based assessments should be an essential aspect of high-stakes examinations intended to determine residents’ different clinical competencies.