Effectiveness of longitudinal faculty development programs on MCQs items writing skills: A follow-up study.

Hamza Mohammad Abdulghani,Kamran Sattar,Shafiul Haque,Mohammad Irshad,Mahmoud Salah Khalil,Tauseef Ahmad,Emmanuel Manalo

doi:10.1371/journal.pone.0185895

Hamza Mohammad Abdulghani, Kamran Sattar + Show 5 more

Open Access

https://doi.org/10.1371/journal.pone.0185895

Copy DOI

Journal: PloS one	Publication Date: Oct 10, 2017
Citations: 32	License type: CC BY 4.0

Affiliation: King Saud University, Jazan University

Abstract

This study examines the long-term impact of the faculty development programs on the multiple choice question (MCQ) items’ quality leading to study its effect on the students’ overall competency level during their yearly academic assessment. A series of longitudinal highly constructed faculty development workshops were conducted to improve the quality of the MCQs items writing skills. A total of 2207 MCQs were constructed by 58 participants for the assessment of 882 students’ cognitive competency level during the academic years 2012–2015. The MCQs were analyzed for the difficulty index (P-value), discriminating index (DI), presence/absence of item writing flaws (IWFs), and non-functioning distractors (NFDs), Bloom’s taxonomy cognitive levels, test reliability, and the rate of students’ scoring. Significant improvement in the difficulty index and DI were noticed during each successive academic year. Easy and poor discriminating questions, NFDs and IWFs were decreased significantly, whereas distractor efficiency (DE) mean score and high cognitive level (K2) questions were increased substantially during the each successive academic year. Improved MCQs’ quality leaded to increased competency level of the borderline students. Overall, the longitudinal faculty development workshops help in improving the quality of the MCQs items writing skills of the faculty that leads to students’ high competency levels.

Highlights

Effective delivery of medical science education demands precise and efficient assessment system(s) capable of examining students’ essential subjective knowledge, learning attitude and practical skills at high competency level
Out of 2207 multiple choice question (MCQ) items, 729 MCQs were constructed during the academic year 2012–2013, 690 MCQs were constructed in the academic year 2013–2014, and 788 MCQs were constructed in the academic year 2014–2015
The MCQ items were prepared according to the guidelines of AEC, college of medicine (COM), King Saud University (KSU) (Supporting Information: S1 Appendix)

Summary

Introduction

Effective delivery of medical science education demands precise and efficient assessment system(s) capable of examining students’ essential subjective knowledge, learning attitude and practical skills at high competency level. Earlier studies have reported that MCQs are superior and apposite competency test for evaluating the subject’s knowledge, comprehension, and can be designed to examine applications and analyses [9]. Construction of high quality MCQs can test advanced level critical thinking and application of the knowledge by evaluating the examinee’s ability to integrate, synthesize and evaluate medical information [10,11,12]. Use of high quality MCQs in the assessment process determines the deep learning approach(s) of the students towards higher Bloom’s taxonomical level of cognitive abilities such as interpretation, synthesis and application of the acquired knowledge instead of testing the recalling of the isolated facts [13]

Methods

Results

Discussion

Conclusion