Identification of technical item flaws leads to improvement of the quality of single best Multiple Choice Questions

Humaira Fayyaz Khan,Masood Anwar Anwar,Azra Saeed Awan Saeed Awan Awan,Khalid Farooq Danish

doi:10.12669/pjms.293.2993

Abstract

The purpose of the study was to identify technical item flaws in the multiple choice questions submitted for the final exams for the years 2009, 2010 and 2011. This descriptive analytical study was carried out in Islamic International Medical College (IIMC). The Data was collected from the MCQ's submitted by the faculty for the final exams for the year 2009, 2010 and 2011. The data was compiled and evaluated by a three member assessment committee. The data was analyzed for frequency and percentages the categorical data was analyzed by chi-square test. Overall percentage of flawed item was 67% for the year 2009 of which 21% were for testwiseness and 40% were for irrelevant difficulty. In year 2010 the total item flaws were 36% and 11% testwiseness and 22% were for irrelevant difficulty. The year 2011 data showed decreased overall flaws of 21%. The flaws of testwisness were 7%, irrelevant difficulty were 11%. Technical item flaws are frequently encountered during MCQ construction, and the identification of flaws leads to improved quality of the single best MCQ's.

Highlights

Altering the mode of examination to single best question type represents a major challenge for a faculty in any medical college
The results in this study show that the frequency of item flaws encountered in the year 2009 were 67% which is comparable to flaws encountered in a study conducted by Ellsworth et al in psychology test banks.[7]
The frequency of item writing flaws found in MCQs in a study by Tarrant et al (2005) is 46.2% which is a study conducted on nursing curriculum.[9]

Summary

Introduction

Altering the mode of examination to single best question type represents a major challenge for a faculty in any medical college. With change in the curriculum, the modality of assessment changes. The new system of examination focuses on application, problem solving and integration of the different concepts taught. 1. Dr Humaira Fayyaz Khan, MBBS, FCPS, Assistant Professor Physiology, 2. Dr Khalid Farooq Danish, MBBS, FCPS, Professor and Head of Surgery, 3. Dr Azra Saeed Awan, MBBS, FCPS, Professor of Gynaecology and Obstetrics, 4. Maj. Gen.(R) Masood Anwar, MBBS, FCPS, Principal, 1-4: Islamic International Medical College, Rawalpindi, Pakistan

Methods

Results

Discussion

Conclusion