Implementing objective structured practical examination as a formative assessment tool for practical skills in 1st-year MBBS students: A comparative study in the physiology department at Surat Municipal Institute of Medical Education and Research, Surat

Abstract

Background: It is a well-known fact that assessment drives learning. One of the most important aspects of training a doctor is the acquisition of practical skills. The conventional practical examination (CPE) fails to guarantee the development of the skills expected from students by the end of the course. In addition, the scoring is affected based on the examiner’s variability for different students. Objective structured practical examination (OSPE) has been used to evaluate those areas of performance by students. CPE also does not provide any feedback except stating pass or fail. To overcome this problem, an earlier innovation in this regard is the OSPE described in 1975 and greater detail in 1979 by Harden and his group. The OSPE is now an accepted tool in the assessment of practical skills in both pre- and para-clinical subjects. In view of this, we have implemented the system of OSPE for the assessment of practical physiology in the Department of Physiology at SMIMER College in Surat. Aims and Objectives: The study aims to introduce OSPE as a formative assessment method for 1st-year MBBS students within the physiology department. It involves conducting a comparative analysis between CPE and OSPE to evaluate the efficacy of these assessment methods. In addition, the study seeks to assess students’ satisfaction with the implementation of OSPE for evaluating practical skills and to investigate faculty perceptions of OSPE as a formative assessment tool. Materials and Methods: The study was conducted in the physiology department of SMIMER on 138 students of 1st-year MBBS. The first CPE was conducted on 46 students each on 3 alternate days over a period of 1 week by six faculties. The students underwent a second evaluation of the same practical skills, next week through the OSPE method. Feedback from students and faculties was taken separately. CPE was compared with OSCE statistically to find the significance of the difference in means between the marks of these two methods. Results: The results indicate that while students achieved higher marks in the conventional method of practical assessment compared to OSPE, the difference between the two was not deemed statistically significant (P = 0.1). Interestingly, over 80% of students expressed a preference for integrating OSPE into future assessments. Notably, 60% of students perceived OSPE as potentially easier to navigate than traditional practical assessments, while over 80% acknowledged its broader coverage of topics. Furthermore, a significant majority, more than 70% of students, deemed OSPE as a fairer evaluation method compared to traditional approaches. Moreover, a majority of faculty members, exceeding 50%, endorsed the feasibility of conducting OSPE within the Department of Physiology at SMIMER. Their strong consensus highlighted the objectivity and efficacy of OSPE stations in evaluating cognitive and psychomotor skills, surpassing conventional assessment methods. Conclusion: Overall, the findings suggest that while there may not be a statistically significant difference in marks obtained between traditional practical assessment methods and OSPE, there is a strong inclination among students and faculty toward incorporating OSPE into future assessments. Students appreciated the wider coverage of topics in this method. Faculty members recognize the objectivity, validity, and reliability of OSPE but are unsure about its feasibility. The consensus leans toward utilizing OSPE for formative assessment in the future.