Quality Of Multiple Choice Questions Research Articles

Educational Data Mining: Multiple Choice Question Classification in Vocational School

Data mining on student learning outcomes in the education sector can overcome this problem. This research aimed to provide a solution for selecting quality multiple choice questions (MCQ) using the results of students’ mid-semester exams in vocational high schools using a Data Mining approach. The research method used was the Cross-Industry Standard Process for Machine Learning (CRISP-ML) model. Steps to assess the accuracy of analyzing the difficulty level of questions based on student profile data and midterm test results. The data used in this research were the findings of basic computer tests on mid-term exams in mathematics disciplines at vocational high schools. This research used several classification algorithms, including SVM, Naive Bayes, Random Forest, Decision Three, Linear Regression, and KNN. The results of evaluating the classification

Analysis of Difficulty Index, Discrimination Index and Distractor Efficiency in Multiple Choice Questions

Background: The item analysis of multiple-choice questions (MCQs) is an essential tool that can provide input on validity and reliability of items. It helps to identify items, which can be revised or discarded, thus building a quality MCQ bank. Aim: The aim of the study to evaluate the quality of MCQs by analyzing DIF, DI and DE and to find out the association of MCQs having good difficulty and discrimination indices with DE. Method: The study was conducted in the Department of Maternity and Child Health Nursing at the College of Nursing during the academic year 2023, second semester. Students of level six who took the final exam of one-course second-semester academic year 2023, A total of 152 third-year Bachelor of Nursing. The item analysis explored the difficulty index (DIF I) and discrimination index (DI) with distractor effectiveness (DE). Data was kept confidential and the descriptive and inferential statistics will be computed using SPSS 22.0. Results: A total of 50 MCQs, the majority, that is, 74 (82%) MCQs had a good/acceptable level of difficulty with a mean DIF I of 94.94 ± 6.36 (mean ± SD); this indicates that, on average, the MCQs were relatively easy for the students, as the difficulty index is close to 100. The discrimination index, which assesses the ability of the MCQs to differentiate between high and low-performing students, had a mean of 0.11 and a standard deviation of 0.47. This relatively low mean value suggests that the MCQs had limited discriminatory power, meaning they did not effectively distinguish between students with different levels of performance. 150 (75%) were categorized as functional distractors (FDs). These FDs effectively served their purpose by distracting students from selecting the correct answer, challenging their knowledge and understanding. On the other hand, 50 distractors (25%) were classified as non-functional distractors (NFDs). Conclusion: The categorization of the MCQs based on distractor efficiency highlights the importance of having well-constructed and effective distractors. The presence of functional distractors challenges students' decision-making, while non-functional distractors need to be revised or replaced to improve the overall quality of the MCQs. This finding indicates a potential area for improvement in the construction and selection of MCQs to enhance their ability to differentiate student abilities

An assessment of preclinical removable prosthodontics based on multiple-choice questions: Stakeholders' perceptions.

Item analysis of multiple-choice questions (MCQs) is an essential tool for identifying items that can be stored, revised, or discarded to build a quality MCQ bank. This study analyzed MCQs based on item analysis to develop a pool of valid and reliable itemsand investigate stakeholders' perceptions regarding MCQs in a written summative assessment (WSA) based on this item analysis. In this descriptive study, 55 questions each from 2016 to 2019 of WSA in preclinical removable prosthodontics for fourth-year undergraduate dentistry students were analyzed for item analysis. Items were categorized according to their difficulty index (DIF I) and discrimination index (DI). Students (2021-2022) were assessed using this question bank. Students' perceptions of and feedback from faculty members concerning this assessment were collected using a questionnaire with a five-point Likert scale. Of 220 items when both indices (DIF I and DI) were combined, 144 (65.5%) were retained in the question bank, 66 (30%) required revision before incorporation into the question bank, and only 10 (4.5%) were discarded. The mean DIF I and DI values were 69% (standard deviation [Std.Dev] = 19) and 0.22 (Std.Dev = 0.16), respectively, for 220 MCQs. The mean scores from the questionnaire for students and feedback from faculty members ranged from 3.50 to 4.04 and from 4 to 5, respectively, indicating that stakeholders tended to agree and strongly agree, respectively, with the proposed statements. This study assisted the prosthodontics department in creating a set of prevalidated questions with known difficulty and discrimination capacity.

The relationship between question grid and quality of multiple choice questions in high school biology made by prospective teacher students

The problem grid is an important part of the problem formulation rules which serves to maintain quality so that the measurement function can be carried out properly. The purpose of this study was to analyze the relationship between the multiple-choice question grid and the quality of Biology questions. The approach used in this research is descriptive quantitative. The research was conducted at the University of Muhammadiyah Malang. The research sample is Biology Education students in 2020/2021 who are taking the learning evaluation course using a purposive sampling technique. The research instrument used was the form of a questionnaire containing the assessment of the item grid. Path analysis is used in data processing with the help of SmartPLS software. The results showed that there was a positive relationship between the grid of multiple choice questions on biology material and the quality of the questions which consisted of the accuracy of the material, the accuracy of the indicators, the accuracy of the item indicators, operational verbs, cognitive processes, answer choices, statement suitability, and language suitability. In addition, prospective educators must have the ability to compose a question grid following the components and quality of the grid so that they can compile quality instruments so that they can measure what should be measured.

Relations of the Number of Functioning Distractors With the Item Difficulty Index and the Item Discrimination Power in the Multiple Choice Questions.

Background Multiple choice questions (MCQ) are used nowadays in summative assessments for certification of courses, in a competitive examination, and for recruitment. A single-best-answer stem-type MCQ is one of the formats most commonly used for this purpose; it contains a question and options from which the examinee needs to select the correct answer. Item analysis is used for the determination of the quality, validity, and reliability of the MCQ. Item difficulty index, item discrimination power, and distractor effectiveness are critical for the quality of the MCQs. Aim This study was conducted to estimate the effect of distractor effectiveness on the item difficulty index and item discriminating power. Method A test paper consisting of 200 single-best-answer stem-type MCQs with four options responded to by 400 medical undergraduates was analyzed for item difficulty index, item discriminating power, and distractor effectiveness with established item analysis formulas. The effect of distractor effectiveness on item difficulty index and item discriminating power was statistically analyzed using Epi-Info 7TMsoftware. Result The mean item difficulty index, item discriminating power, and distractor effectiveness were 57.75% ± 28.65%, 0.4388 ± 0.36, and 84.17%, respectively (p<0.05). The item difficulty index was higher for items with single-functioning distractors in comparison to items with three-functioning distractors. Item discriminating power was higher in items with three functioning distractors in comparison to items with one and two functioning distractors. Conclusion Two or three functioning distractors show an appropriate item difficulty index and item discriminating power. In addition, item discriminating power is lower for the easy MCQs. Two to three distractors are appropriate to construct a quality MCQ.

Quality of Multiple Choice Questions (MCQs) as perceived by the postgraduate residents appearing in Mid Training Assessment (MTA) during January 2022 at Rawalpindi Medical University

Impartiality and reliability of Multiple-Choice Questions (MCQs) render them an ideal assessment tool for measuring higher order cognition. Objective: To determine the perception of postgraduate medical trainees pertinent to the quality of MCQs administered in Mid Training Assessment (MTA) January 2022. Methods: A cross-sectional descriptive study was carried out among 40 postgraduate trainees who appeared in MTA during January 2022. About 23, 12 and 05 of these trainees were doing MS and MD training at Benazir Bhutto Hospital, Holy Family Hospital and DHQ Hospital Rawalpindi respectively. Trainees were enrolled in the study through consecutive non-probability sampling. The feedback from the trainees about quality of MCQs was gathered by means of a structured questionnaire based on 5-point likert scale. Data analysis was done by using SPSS version 25.0. Results: About 67% of the trainees were pleased with the quality and construct of MCQ in their assessment in terms of illustration of the information in their stems needed to understand the clinical problem. More than 82% trainees satisfied with the time allocated for attempting MCQs papers. Only 55% trainees agreed with alignment of MCQs with given Table of Specification (TOS). Most (32.5%) wanted to have difficulty level of the MCQs in accordance with level of training. Around 27.5% and 22.5% of the residents proposed to incorporate more clinical reasoning and problem solving MCQs respectively. Conclusions: Trainees were substantially contented with the standard of MCQs. They suggested for inclusion of more clinical reasoning and problem-solving type questions in alignment with TOS.

TINGKAT KESUKARAN SOAL ULANGAN KELAS XI DI MA AS-SULAIMANIYAH MOJOAGUNG JOMBANG

This study aims to determine the level of difficulty of Indonesian test questions. This type of research is descriptive quantitative. Data collection techniques in the form of written tests and documentation. The written test consist of 10 multiple choice questions and 10 essay questions. The results of the research on the quality of multiple choice questions based on the level of difficulty of the questions are classified as easy, totaling 7 items, 2 items classified as moderate, and 1 item classified as difficult. In the description questions based on the level of difficulty of the questions there are 10 items that are classified as easy.

Exploring (Collaborative) Generation and Exploitation of Multiple Choice Questions: Likes as Quality Proxy Metric

Multiple Choice Questions (MCQs) are an established medium of formal educational contexts. The collaborative generation of MCQs by students follows the perspectives of constructionist and situated learning and is an activity that fosters learning processes. The MCQs generated are—besides the learning processes—further outcomes of collaborative generation processes. Quality MCQs are a valuable resource, so that collaboratively generated quality MCQs might also be exploited in further educational scenarios. However, the quality MCQs first need to be identified from the corpus of all generated MCQs. This article investigates whether Likes distributed by students when answering MCQs are viable as a metric for identifying quality MCQs. Additionally, this study explores whether the process of collaboratively generating MCQs and using the quality MCQs generated in commercial quiz apps is achievable without additional extrinsic motivators. Accordingly, this article describes the results of a two-stage field study. The first stage investigates whether quality MCQs may be identified through collaborative inputs. For this purpose, the Reading Game (RG), a gamified, web-based software aiming at collaborative MCQ generation, is employed as a semester-accompanying learning activity in a bachelor course in Urban Water Management. The reliability of a proxy metric for quality calculated from the ratio of Likes received and appearances in quizzes is compared to the quality estimations of domain experts for selected MCQs. The selection comprised the ten best and the ten worst rated MCQs. Each of the MCQs is rated regarding five dimensions. The results support the assumption that the RG-given quality metric allows identification of well-designed MCQs. In the second stage, MCQs created by RG are provided in a commercial quiz app (QuizUp) in a voluntary educational scenario. Despite the prevailing pressure to learn, neither the motivational effects of RG nor of the app are found in this study to be sufficient for encouraging students to voluntarily use them on a regular basis. Besides confirming that quality MCQs may be generated by collaborative software, it is to be stated that in the collaborative generation of MCQs, Likes may serve as a proxy metric for the quality of the MCQs generated.

Item Analysis of Multiple Choice Questions used as a tool for Summative Assessment in a Medical College in Haryana

Introduction: Multiple choice questions (MCQs) are one of the popular tools used for evaluating medical students since they can be used to test a large number of candidates with only minimal human intervention. These MCQs also need testing for quality and standard which will ultimately improve the teaching-learning process. This study is conducted to assess the quality of MCQs of our in-use question bank. Methods: A total of 80 MCQs from the biochemistry internal examinations conducted in 2021 held at Al-Falah Medical College, Haryana for 147 students of 1st year Bachelor of Medicine and Bachelor of Surgery (M.B.B.S) were analyzed for difficulty index, discrimination index, and distractor effectiveness. The MCQs were the single best response type with one key option and three distractors. Statistical analysis for calculation of Mean, Standard Deviation, and Pearson correlation was performed on the data obtained using Microsoft excel 2010. Results: The observed mean of difficulty index, discrimination index, and distractor effectiveness was 75.08±15.68, 0.32±0.17, and 53.30±29.37 respectively. No item had a negative discrimination index. The Pearson correlation between DIF I and DI was negative (r = -0.43) and insignificant. Conclusion: Item analysis helps to optimize the existing question bank by helping in identifying flawed items. It is recommended to use this tool to standardize the testing process so as to accurately assess the student performance and improve the teaching-learning outcome.

Assessment of Global Health Education: The Role of Multiple-Choice Questions.

Introduction: The standardization of global health education and assessment remains a significant issue among global health educators. This paper explores the role of multiple choice questions (MCQs) in global health education: whether MCQs are appropriate in written assessment of what may be perceived to be a broad curriculum packed with fewer facts than biomedical science curricula; what form the MCQs might take; what we want to test; how to select the most appropriate question format; the challenge of quality item-writing; and, which aspects of the curriculum MCQs may be used to assess.Materials and Methods: The Medical School for International Health (MSIH) global health curriculum was blue-printed by content experts and course teachers. A 30-question, 1-h examination was produced after exhaustive item writing and revision by teachers of the course. Reliability, difficulty index and discrimination were calculated and examination results were analyzed using SPSS software.Results: Twenty-nine students sat the 1-h examination. All students passed (scores above 67% - in accordance with University criteria). Twenty-three (77%) questions were found to be easy, 4 (14%) of moderate difficulty, and 3 (9%) difficult (using examinations department difficulty index calculations). Eight questions (27%) were considered discriminatory and 20 (67%) were non-discriminatory according to examinations department calculations and criteria. The reliability score was 0.27.Discussion: Our experience shows that there may be a role for single-best-option (SBO) MCQ assessment in global health education. MCQs may be written that cover the majority of the curriculum. Aspects of the curriculum may be better addressed by non-SBO format MCQs. MCQ assessment might usefully complement other forms of assessment that assess skills, attitude and behavior. Preparation of effective MCQs is an exhaustive process, but high quality MCQs in global health may serve as an important driver of learning.

Replacing Non-functional Distractors to Improve the Quality of MCQs: A Quasi-experimental Study

The quality of multiple-choice questions (MCQs) can be determined by the discrimination index (DsI), difficulty index (DI), and distractor efficiency (DE) in post exam analysis. Selection of distractors is one of the most neglected aspects of MCQ preparation. The present study evaluated the effect on DsI by replacing nonfunctional distractor (NFD) with more plausible alternatives using a quasi-experimental study model. The DsI and NFD values were obtained from the MCQ examinations that were conducted during the year 2017 from 37 courses with each containing 50 MCQ’s in the College of Dentistry, Majmaah University. The post exam analysis results were informed to the course directors and the faculty were given orientation program about replacing the NFD’s with more plausible options. In the following year (2018), sixty percent of the previous year MCQ’s were used with modified NFD’s in those questions (1100 MCQ’s). The average DsI and NFD were significantly improved from the year 2017 to 2018 (P&lt; 0.000) with significant negative correlation between them (P&lt;0.000). This study shows that the psychometric properties of MCQs can be improved by creating better distractors. The researchers recommend providing faculty with more training on how to use distractors effectively in MCQ writing.

Item analysis of multiple choice questions: A quality assurance test for an assessment tool

BackgroundThe item analysis of multiple choice questions (MCQs) is an essential tool that can provide input on validity and reliability of items. It helps to identify items which can be revised or discarded, thus building a quality MCQ bank. MethodsThe study focussed on item analysis of 90 MCQs of three tests conducted for 150 first year Bachelor of Medicine and Bachelor of Surgery (MBBS) physiology students. The item analysis explored the difficulty index (DIF I) and discrimination index (DI) with distractor effectiveness (DE). Statistical analysis was performed by using MS Excel 2010 and SPSS, version 20.0. ResultsOf total 90 MCQs, the majority, that is, 74 (82%) MCQs had a good/acceptable level of difficulty with a mean DIF I of 55.32 ± 7.4 (mean ± SD), whereas seven (8%) were too difficult and nine (10%) were too easy. A total of 72 (80%) items had an excellent to acceptable DI and 18 (20%) had a poor DI with an overall mean DI of 0.31 ± 0.12. There was significant weak correlation between DIF I and DI (r = 0.140, p < .0001). The mean DE was 32.35 ± 31.3 with 73% functional distractors in all. The reliability measure of test items by Cronbach alpha was 0.85 and Kuder-Richardson Formula 20 was 0.71, which is good. The standard error of measurement was 1.22. ConclusionOur study helped teachers identify good and ideal MCQs which can be part of the question bank for future and those MCQs which needed revision. We recommend that item analysis must be performed for all MCQ-based assessments to determine validity and reliability of the assessment.

Question-Based Collaborative Learning for Constructive Curricular Alignment.

IntroductionWe designed a curriculum mapping tool which enables medical students to access intended learning outcomes (ILOs) on their iPads in the workplace. Students were encouraged to use the online curriculum map in a specially planned teaching session: question-based collaborative learning (QBCL). The aim of the session was to empower medical students to constructively align their experiential learning with the learning outcomes of the undergraduate curriculum. In doing so, our session aimed to provide students with a greater understanding of the curriculum, improve their insights into assessment and their question-writing abilities.MethodsThe QBCL pre-session preparation involved reviewing a patient with a presentation that aligned to the year-specific ILOs. During a 150 minute QBCL session, students received training on how to write high quality multiple choice questions (MCQs) delivered by a faculty member of Imperial College School of Medicine. They then worked collaboratively in groups and created MCQs based on their clinical encounters. Their questions were tagged to the relevant learning objective and submitted online via the curriculum map. The student-generated MCQs were analyzed using an adjusted version of Bloom’s taxonomy. We also conducted a quantitative evaluation of the session.ResultsOne hundred and sixty-three questions were submitted, with 81% of questions being tagged to ILOs considered to show evidence of learning consistent with the “Apply” tier of Bloom’s taxonomy. The majority of students agreed that the session was interactive (80%), thought-provoking (77%) and improved their team-working skills (70%). It gave them a greater understanding of the undergraduate curriculum (65%), improved their question-writing and insight into assessments (76%), and provided an opportunity to learn from their peers (86%). Students agreed that this session covered a variety of cases (82%) and deepened their understanding of medical conditions and presentations (87%).ConclusionWe encouraged students to actively interact with the curriculum map. Students were able to achieve their own constructive alignment by writing assessment items based on real patients and linking them to the appropriate intended learning outcomes.

Item Analysis of Multiple Choice Questions in Pharmacology

Background: MCQs type assessment in medical education is replacing old theory style. There are concerns regarding the quality of the Multiple Choice Questions.Objectives: To determine the quality of Multiple Choice Questions by item analysis. Material and Methods: Study was a cross sectional descriptive .Fifty Multiple Choice Questions in the final internal evaluation exams in 2015 of Pharmacology at Bacha khan Medical College were analyzed. The quality of each Multiple Choice Questions item was assessed by the Difficulty index (Dif.I), Discriminative Index (D.I) and Distracter Efficiency (D.E).Results: Multiple Choice Questions that were of moderate difficulty were 66%. Easy were 4% and high difficulty were 30%.Reasons for high difficult Multiple Choice Questions were analyzed as Item Writing Flaws 41%, Irreverent Difficulty 36% and C2 level 23%. Discrimination Index shows that majority of MCQs were of Excellent Level (DI greater than 0.25) i.e 52 , Good 32% . (DI=2.15-0.25), Poor 16%. MCQs Distracter Effectiveness (DE)= 4, 3,2,1 were 52%, 34%, 14%, and 0% respectively. Conclusion: Item analysis gives us different parameters with reasons to recheck MCQ pool and teaching programme. High proportions of difficult and sizable amount of poor discriminative indices MCQs were the finding in this study and need to be resolved

QUALITY AND FEATURE OF MULTIPLE-CHOICE QUESTIONS IN EDUCATION

The quality of multiple-choice questions (MCQs) as well as the student's solve behavior in MCQs are educational concerns. MCQs cover wide educational content and can be immediately and accurately scored. However, many studies have found some flawed items in this exam type, thereby possibly resulting in misleading insights into students’ performance and affecting important decisions. This research sought to determine the characteristics of MCQs and factors that may affect the quality of MCQs by using item response theory (IRT) to evaluate data. For this, four samples of different sizes from US and China in secondary and higher education were chosen. Item difficulty and discrimination were determined using item response theory statistical item analysis models. Results were as follows. First, only a few guessing behaviors are included in MCQ exams because all data fit the two-parameter logistic model better than the three-parameter logistic model. Second, the quality of MCQs depended more on the degree of training of examiners and less on middle or higher education levels. Lastly, MCQs must be evaluated to ensure that high-quality items can be used as bases of inference in middle and higher education. Keywords: higher education, item evaluation, item response theory, multiple-choice test, secondary education

Impact Measurement on Medical Faculty for Adhering to Appropriate Guidelines in Framing Effective Multiple-Choice Questions for Item Analysis

Background: Multiple-choice questions (MCQs) are the most frequently accepted tool for the evaluation of comprehension, knowledge, and application among medical students. In single best response MCQs (items), a high order of cognition of students can be assessed. It is essential to develop valid and reliable MCQs, as flawed items will interfere with the unbiased assessment. The present paper gives an attempt to discuss the art of framing well-structured items taking kind help from the provided references. This article puts forth a practice for committed medical educators to uplift the skill of forming quality MCQs by enhanced Faculty Development programs (FDPs). Objectives: The objective of the study is also to test the quality of MCQs by item analysis. Methods: In this study, 100 MCQs of set I or set II were distributed to 200 MBBS students of Late Shri Lakhiram Agrawal Memorial Govt. Medical College Raigarh (CG) for item analysis for quality MCQs. Set I and Set II were MCQs which were formed by 60 medical faculty before and after FDP, respectively. All MCQs had a single stem with three wrong and one correct answers. The data were entered in Microsoft excel 2016 software to analyze. The difficulty index (Dif I), discrimination index (DI), and distractor efficiency (DE) were the item analysis parameters used to evaluate the impact on adhering to the guidelines for framing MCQs. Results: The mean calculated difficulty index, discrimination index, and distractor efficiency were 56.54%, 0.26, and 89.93%, respectively. Among 100 items, 14 items were of higher difficulty level (DIF I &lt; 30%), 70 were of moderate category, and 16 items were of easy level (DIF I &gt; 60%). A total of 10 items had very good DI (0.40), 32 had recommended values (0.30 - 0.39), and 25 were acceptable with changes (0.20 - 0.29). Of the 100 MCQs, there were 27 MCQs with DE of 66.66% and 11 MCQs with DE of 33.33%. Conclusions: In this study, higher cognitive-domain MCQs increased after training, recurrent-type MCQ decreased, and MCQ with item writing flaws reduced, therefore making our results much more statistically significant. We had nine MCQs that satisfied all the criteria of item analysis.

Can automated item generation be used to develop high quality MCQs that assess application of knowledge?

The purpose of this study was to compare the quality of multiple choice questions (MCQs) developed using automated item generation (AIG) versus traditional methods, as judged by a panel of experts. The quality of MCQs developed using two methods (i.e., AIG or traditional) was evaluated by a panel of content experts in a blinded study. Participants rated a total of 102 MCQs using six quality metrics and made a judgment regarding whether or not each item tested recall or application of knowledge. A Wilcoxon two-sample test evaluated differences in each of the six quality metrics rating scales as well as an overall cognitive domain judgment. No significant differences were found in terms of item quality or cognitive domain assessed when comparing the two item development methods. The vast majority of items (> 90%) developed using both methods were deemed to be assessing higher-order skills. When compared to traditionally developed items, MCQs developed using AIG demonstrated comparable quality. Both modalities can produce items that assess higher-order cognitive skills.

Subjective quality of multiple choice questions used in undergraduate courses in orthopedics and other specialties.

Background and Objective:Multiple Choice Questions (MCQs) can sample broad domains of knowledge efficiently and reliably. The MCQs of lower order C1(Cognitive Level1=Recall of knowledge) do not fulfill this purpose and those of higher cognitive order C2 (Cognitive Level2=Interpret) &C3(Cognitive Level3=Analyze) are better at assessing the problem solving capabilities of the student. Every good educational activity must be supported by quality examination to complete the objectives of a curriculum. The objective of the study was to evaluate MCQs presently being used in internal examinations of medical colleges in Lahore.Methods:Papers consisting of MCQs from Orthopedics other specialties were collected in June 2019 from different medical colleges of Lahore and reviewed by a senior medical teacher without blinding and without his knowing the scores the students had been awarded before. Question statement, clinical scenarios, options and other mistakes were assessed in each item on predetermined criteria. Cognitive level of the item was determined if it was asking for a recall/identify/ analyze response. The results were tabulated and compared in two groups i.e. Miscellaneous and Orthopedics.Results:Most of the items(total=589) in both groups were of C1 cognitive level though Orthopedics (229) were slightly better (χ2 = 49.882 P-Value = 0.000 (Statistically Significant). Miscellaneous group (360) was better in quality in making clinical scenarios (χ2 = 29.952 P-Value = 0.000 (Statistically Significant) and writing a question statement without confusion. Options were better written in both groups. A good percentage of items needed to be corrected for mistakes in spellings, grammar and segregation into under graduate level.Conclusions:The cognitive level of assessment tool s MCQs is quiet low in both groups especially clinical scenario construction can be improved. Mistakes in spellings, grammar and conceptual mediocrity is common in both groups.

Nonfunctional distractor analysis: An indicator for quality of Multiple choice questions.

Objectives:To analyze the low to medium distractor efficiency items in a multiple-choice question (MCQ) paper for item writing flaws.Methods:This qualitative study was conducted at Islamic International Medical College Rawalpindi, in October 2019. Archived item- analysis report from a midyear medium stakes MCQ paper of 2nd year MBBS class, was analyzed to determine the non-functional distractors (NFDs) and distractor efficiency (DE) of items, in a total of 181 MCQs. DE was categorized as low (3-4 NFDs), medium (1-2 NFDs) and high (0 NFD). Subsequently, qualitative document analysis of the MCQ paper whose item analysis report was assessed was conducted to investigate the item flaws in the low to medium DE items. The flaws identified were coded and grouped as, within option flaws, alignment flaws between options and stem/ lead-in and other flaws.Results:Distractor efficiency was high in 69 items (38%), moderate in 75 items (42%) and low in 37 items (20%). The item-writing flaws identified in low to moderate DE items within distractors included, non-homogenous length (1.8%), non-homogenous content (8%) and repeat in distractor (1.7%). Alignment flaws between distractors and stem/ lead-in identified were linguistic cues (10%), logic cues (12.5%) and irrelevant distractors (16%). Flaws unrelated to distractors were low cognitive level items (40%) and unnecessarily complicated stems (11.6%).Conclusions:Analyzing the low to medium DE items for item writing flaws, provides valuable information about item writing errors which negatively impact the distractor efficiency.

Knowledge, application and how about competence? Qualitative assessment of multiple-choice questions for dental students

ABSTRACT Background: Many medical schools train their faculty members to construct high cognitive level multiple choice questions (MCQs) that demand a great deal of analytical and critical thinking, application, and competence. The purpose of this study is to determine the cognitive levels of MCQs by using Moore’s Expanded Outcomes Framework and to understand whether the quality of MCQs has an effect on students’ assessment performance. Methods: Four trained faculty members analysed 100 randomly selected questions developed at the University Clinic of Dentistry (UCD) and 100 questions developed by the National Board of Dental Examinations (NBDE). Moore’s framework was applied to assist the review process. Results: The majority of questions was at the level of declarative knowledge followed by questions at the level of procedural knowledge. The cognitive level of UCD questions from 2002 to 2009 was significantly lower than that of NBDE questions but increased in questions written from 2010 to 2018. The improvement of quality of MCQs had no impact on assessment performance of students. Conclusion: The enhanced cognitive levels of UCD MCQs written 2010–2018 coincides with the implementation of a faculty training program for writing high-ordered MCQs. In addition, this study shows that the use of Moore’s expanded framework is on par with other known taxonomies in supporting educators in writing items and reviewing the process. Abbreviations: MCQs: Multiple Choice Questions; UCD: University Clinic of Dentistry; NBDE: National Board of Dental Examinations