Evaluating OSPE as a Formative Assessment Tool for Medical Students

Abstract

Listen

Translate article

Introduction. Objective structured practical examination (OSPE) is a globally recognized and widely implemented assessment method used for evaluating students in practical examinations. It requires examinees to complete various tasks within a set time frame, each assessed against predetermined criteria designed to measure practical skills and attitudes. This study aimed to assess the effectiveness of the OSPE as a tool for internal assessment in the subject of Anatomy for first-year MBBS students, comparing its performance against the traditional practical examination (TPE) method. Methods. In this prospective study, an OSPE tool specific to Anatomy was developed and validated for face and content to ensure that the OSPE stations were standardized and well-constructed. The part-leaving test covered the Thorax and Brain chapters using both the TPE and OSPE methods. A Bland-Altman plot was employed to assess the agreement between the quantitative assessments of both methods. Additionally, a 5-point Likert scale was utilized to gauge the perceptions of both students and faculty members regarding the assessment methods. Results. The analysis revealed a significant difference between the scores obtained from OSPE and TPE. The intraclass correlation coefficients for reliability between TPE and OSPE were 0.81 for the Thorax chapter and 0.84 for the Brain chapter. Additionally, the Bland-Altman plots illustrating the scores from both assessment methods indicated low bias and no definite pattern. While no significant outliers were identified for the Brain part, the Thorax part exhibited outliers at a rate of 6.43%, suggesting the presence of low-impact erratic variables. The Chi-square test for student perceptions yielded significant results for the entire questionnaire. In contrast, faculty perceptions showed significant results for all questions except for two. Conclusions. Due to the high intraclass correlation coefficients between TPE and OSPE and the absence of bias, systematic errors, and erratic variables, OSPE is a promising assessment tool. Students perceived that OSPE decreased the chances of failure and that the questions asked in OSPE were more relevant compared to the traditional method. Faculty members also noted that OSPE provided reasonably high transparency and decreased subjective bias compared to TPE. Considering these factors, it can be concluded that OSPE is a viable potential tool to replace TSE.