Descriptive Analysis of the Psychometric Properties of Extended Matching Questions Conducted Among Anaesthesia Residents

Abstract

Background: Clinical reasoning is one of the core features of clinical competency. Training and assessing clinical reasoning is vital in post-graduate training. Extended matching questions(EMQs) are effective in assessing problem-solving and clinical reasoning abilities, but not commonly used in Postgraduate training. Covid-19 pandemic, which prevented both patient encounter and regular academic activities, warranted the introduction of innovative Teaching-Learning methods to sustain clinical reasoning skills. Hence, we aimed to introduce EMQs in formative assessment among anaesthesia residents and analyzed its psychometric properties. Methods: The study was conducted at the Department of Anaesthesiology, MGMCRI, Pondicherry. Four modules of EMQs as part of a formative assessment was conducted among residents (n=20). A total of 40 clinical vignettes and 60 options were administered. Post-validation of the EMQs was done by item analysis. Test reliability was estimated by the Kuder-Richardson 20 formula. Difficulty index(DIF-I), discrimination index(DIS-I), and distractor functionality were analysed. Results: The KR-20 reliability coefficient was 0.72. The mean DIF-I was 0.43±0.17, where 72.5%(29) were in the acceptable range, 20%(8) difficult, and 7.5%(3) easy. The mean DIS-I was 0.28±0.24, where 40%(16) had acceptable, 27.5%(11) excellent, and 20% had poor discrimination. Ninety percent of distractors were functional. The DIS-I exhibited a positive correlation with DIF-I (r= 0.2155, P=0.0185). Conclusions: The results of the present study indicates that EMQs have acceptable test reliability. The majority of the items (80%) followed the principles of MCQs. We conclude that EMQs can be effectively used as part of the postgraduate assessment to test higher-order knowledge and clinical competency.