Enhancing Assessment towards Flexible Medical Education: A Deep Dive into Preclinical Short Answer Questions through Item Analysis

Abstract

Short answer questions (SAQs) and other forms of similarly structured examination methods are employed as evaluation tools to gauge the competency of medical students at various levels of study. However, item analyses for these questions have rarely been conducted. Evaluating the quality of examination questions through item analysis ensures that stakeholders, especially those undergoing learning in various flexible pathways in medical education, are provided with reliable and relevant assessments, promoting effective learning and competency development. In this study, we performed item analyses on SAQs by extracting the passing index (PI) and discrimination index (DI) for each sub-question. Data were analysed using Microsoft Excel (Microsoft Corporation, United States) and Jeffreys’s Amazing Statistics Program (JASP) (University of Amsterdam, The Netherlands). Twentyseven sub-questions from five SAQs were analysed. The DI of the sub-questions ranged from 0.043 to 0.935 with a mean of 0.449 ± 0.223, while the PI returned a range of 0.012 to 0.971 with a mean value of 0.597 ± 0.246. In conclusion, the SAQs administered during a professional examination of preclinical medical students exhibited an acceptable range and mix of PI and DI values. However, improvements must be made to the sub-questions that return poor PI and DI values.