Flawed Items Research Articles

Evaluating the Quality of Examination Items From the Pathophysiology, Drug Action, and Therapeutics Course Series

ObjectiveTo determine the impact of item-writing flaws and cognitive level on student performance metrics in 1 course series across 2 semesters at a single institution. MethodsFour investigators reviewed 928 multiple-choice items from an integrated therapeutics course series. Differences in performance metrics were examined between flawed and standard items, flawed stems and flawed answer choices, and cognitive levels. ResultsReviewers found that 80% of the items were flawed, with the most common types being implausible distractors and unfocused stems. Flawed items were generally easier than standard ones, but the type of flaw significantly impacted the difficulty. Items with flawed stems had the same difficulty as standard items; however, those with flawed answer choices were significantly easier. Most items tested lower-level skills and have more flaws than higher-level items. There was no significant difference in difficulty between lower- and higher-level cognitive items, and higher-level items were more likely to have answer flaws than item flaws. ConclusionItem-writing flaws differently impact student performance. Implausible distractors artificially lower the difficulty of questions, even those designed to assess higher-level skills. This effect contributes to a lack of significant difference in difficulty between higher- and lower-level items. Unfocused stems, on the other hand, likely increase confusion and hinder performance, regardless of the question’s cognitive complexity.

EFFECT OF ITEM WRITING FLAWS IN MULTIPLE CHOICE QUESTIONS ON STUDENTS’ ACHIEVEMENT GROUPS AND RELIABILITIES OF TESTS IN AZAD JAMMU KASHMIR MEDICAL COLLEGE

BACKGROUND: MCQs can test different levels of cognitive knowledge from recall to analysis. This study was to determine the item writing flaws in multiple choice questions and their effects on students’ academic achievements and test reliabilities. Methods: This study was conducted at the Azad Jammu Kashmir (AJK) Medical College Muzaffarabad from December 2017 to June 2019. Ten end-of-block exams were used in this study. The item review committee went over every MCQ. Two tests from each class of MBBS were taken. The initial tests without being reviewed were deemed to be flawed tests. The outcomes of each test were assessed, and the students were ranked into three achievement levels high, moderate, and low based on their scores of >79.9, between 50 and 79.9, and <50%, respectively. After removal of flawed items from the exams, the scores of each test (the standard test) were calculated, compared to the flawed tests, and its impacts were evaluated among three achievement groups. The post-exam analysis of test items was done on optical mark reading (OMR) classic-4 programme and data analysis on SPSS-25. Results: In one test i.e. test no 6 the null hypothesis was rejected with p<0.05. However, in all other tests null hypothesis could not be rejected on statistical basis. Moderate achievers and low achievers had statistically significant correlation (p<0.05). The flawed tests had better reliabilities with statistically significant (p<0.05). Conclusion: Flawed MCQs had negative impact on high and moderate achievers. Reliabilities of flawed test were greater than standard tests.

TYPES OF ITEM WRITING FLAWS IN MULTIPLE CHOICE QUESTIONS IN MEDICAL EDUCATION AND THEIR EFFECT ON PASSING RATE OF STUDENTS

Background: The most popular written exam in health professions education is the multiple-choice style question (MCQ). This study sought to understand the effects of multiple-choice writing errors on students' academic performance in medical education. Methods: This descriptive study was done from December 2017 to June 2019 in Azad Jammu Kashmir (AJK) Medical College, Muzaffarabad. Ten block examinations were included. The item review committee reviewed all MCQs for flaws. Two tests from each class of MBBS were taken. The original tests containing all items were labeled as flawed tests and the result of each flawed test was evaluated. The students were graded into high, moderate and low achievement groups with scores of more than 79.9, between 50–79.9 and less than 50 percent respectively. Flawed items were then removed from the tests by the review committee and the scores of each test (standard test) were determined and compared with flawed tests and its effects were assessed in three achieving groups of students. Optical mark reading (OMR) classic-4 software was used for post-exam analysis and data were analyzed by using SPSS-25. Results: The passing rate ranged from 68.18% to 90.82% in flawed and 75.54% to 93.69% in standard tests. Most standard tests (7) had higher passing rates than flawed tests. Conclusion: Tests containing in-house developed MCQs have frequent item writing flaws and their inclusion in assessment did affect the passing rates of students but were not statistically significant. Pak J Physiol 2023;19(4):49–52

Twelve tips to aid interpretation of post-assessment psychometric reports

Post-assessments psychometric reports are a vital component of the assessment cycle to ensure that assessments are reliable, valid and fair to make appropriate pass-fail decisions. Students’ scores can be summarised by examination of frequency distributions, central tendency measures and dispersion measures. Item discrimination indicies to assess the quality of items, and distractors that differentiate between students achieving or not achieving the learning outcomes are key. Estimating individual item reliability and item validity indices can maximise test-score reliability and validity. Test accuracy can be evaluated by assessing test reliability, consistency and validity and standard error of measurement can be used to measure the variation. Standard setting, even by experts, may be unreliable and reality checks such as the Hofstee method, P values and correlation analysis can improve validity. The Rasch model of student ability and item difficulty assists in modifying assessment questions, pinpointing areas for additional instruction. We propose 12 tips to support test developers in interpreting structured psychometric reports, including analysis and refinement of flawed items and ensuring fair assessments with accurate and defensible marks.

Identifying the presence of context and item-writing flaws in practice items: The case of Turkish mathematics textbooks

This study seeks to ascertain the degree to which context-based items are offered in Turkish mathematics textbooks as well as the quality of the items in terms of item writing guidelines, whether or not they are given as traditional or context-based. A qualitative research approach is used in this study. The eighth-grade mathematics textbook used in public schools and a textbook used in certain private school chains constitute its sample. The practice items (i.e, exercises without solutions given) included in the textbooks were analyzed by performing document analysis. The results revealed that both textbooks contain several flawed items in terms of item writing rules, as well as having mainly non-contextual items.

Item Analysis of Multiple Choice Questions used as a tool for Summative Assessment in a Medical College in Haryana

Introduction: Multiple choice questions (MCQs) are one of the popular tools used for evaluating medical students since they can be used to test a large number of candidates with only minimal human intervention. These MCQs also need testing for quality and standard which will ultimately improve the teaching-learning process. This study is conducted to assess the quality of MCQs of our in-use question bank. Methods: A total of 80 MCQs from the biochemistry internal examinations conducted in 2021 held at Al-Falah Medical College, Haryana for 147 students of 1st year Bachelor of Medicine and Bachelor of Surgery (M.B.B.S) were analyzed for difficulty index, discrimination index, and distractor effectiveness. The MCQs were the single best response type with one key option and three distractors. Statistical analysis for calculation of Mean, Standard Deviation, and Pearson correlation was performed on the data obtained using Microsoft excel 2010. Results: The observed mean of difficulty index, discrimination index, and distractor effectiveness was 75.08±15.68, 0.32±0.17, and 53.30±29.37 respectively. No item had a negative discrimination index. The Pearson correlation between DIF I and DI was negative (r = -0.43) and insignificant. Conclusion: Item analysis helps to optimize the existing question bank by helping in identifying flawed items. It is recommended to use this tool to standardize the testing process so as to accurately assess the student performance and improve the teaching-learning outcome.

Analysis of Item Writing Flaws in a Communications Skills Test in a Ghanaian University

Adhering to the rules governing the writing of multiple-choice test items will ensure quality and validity. However, realizing this ideal could be challenging for non-native English language teachers and students. This is especially so for non-native English language teachers because developing test items in a language that neither they nor their students use as their mother tongue raises a multitude of issues related to quality and validity. A descriptive study on this problem was conducted at a Technical University in Ghana which focused on item writing flaws in a communication skills test. The use of multiple-choice test in Ghanaian universities has increased over the last decade due to increasing student intake. A 20-item multiple-choice test in communication skills was administered to 110 students. The test items were analyzed using a framework informed by standard item writing principles based on the revised taxonomy of multiple-choice item-writing guides by Haladyna, Downing and Rodriguez (2002). The facility and discrimination index (DI) was calculated for all the items. In total, 60% of the items were flawed based on standard items writing principles. The most violated guideline was wording stems negatively. Pearson correlation analysis indicated a weak relationship between the difficulty and discrimination indices. Using the discrimination indices of the flawed items showed that 84.6 % of them had discrimination indices below the optimal level of 0.40 and above. The lowest DI was recorded by an item with which was worded negatively. The mean facility of the test was 45%. It was observed that the flawed items were more difficult than the non-flawed items. The study suggested that test items must be properly reviewed before they are used to assess students’ knowledge.

QUALITY AND FEATURE OF MULTIPLE-CHOICE QUESTIONS IN EDUCATION

The quality of multiple-choice questions (MCQs) as well as the student's solve behavior in MCQs are educational concerns. MCQs cover wide educational content and can be immediately and accurately scored. However, many studies have found some flawed items in this exam type, thereby possibly resulting in misleading insights into students’ performance and affecting important decisions. This research sought to determine the characteristics of MCQs and factors that may affect the quality of MCQs by using item response theory (IRT) to evaluate data. For this, four samples of different sizes from US and China in secondary and higher education were chosen. Item difficulty and discrimination were determined using item response theory statistical item analysis models. Results were as follows. First, only a few guessing behaviors are included in MCQ exams because all data fit the two-parameter logistic model better than the three-parameter logistic model. Second, the quality of MCQs depended more on the degree of training of examiners and less on middle or higher education levels. Lastly, MCQs must be evaluated to ensure that high-quality items can be used as bases of inference in middle and higher education. Keywords: higher education, item evaluation, item response theory, multiple-choice test, secondary education

Impact Measurement on Medical Faculty for Adhering to Appropriate Guidelines in Framing Effective Multiple-Choice Questions for Item Analysis

Background: Multiple-choice questions (MCQs) are the most frequently accepted tool for the evaluation of comprehension, knowledge, and application among medical students. In single best response MCQs (items), a high order of cognition of students can be assessed. It is essential to develop valid and reliable MCQs, as flawed items will interfere with the unbiased assessment. The present paper gives an attempt to discuss the art of framing well-structured items taking kind help from the provided references. This article puts forth a practice for committed medical educators to uplift the skill of forming quality MCQs by enhanced Faculty Development programs (FDPs). Objectives: The objective of the study is also to test the quality of MCQs by item analysis. Methods: In this study, 100 MCQs of set I or set II were distributed to 200 MBBS students of Late Shri Lakhiram Agrawal Memorial Govt. Medical College Raigarh (CG) for item analysis for quality MCQs. Set I and Set II were MCQs which were formed by 60 medical faculty before and after FDP, respectively. All MCQs had a single stem with three wrong and one correct answers. The data were entered in Microsoft excel 2016 software to analyze. The difficulty index (Dif I), discrimination index (DI), and distractor efficiency (DE) were the item analysis parameters used to evaluate the impact on adhering to the guidelines for framing MCQs. Results: The mean calculated difficulty index, discrimination index, and distractor efficiency were 56.54%, 0.26, and 89.93%, respectively. Among 100 items, 14 items were of higher difficulty level (DIF I < 30%), 70 were of moderate category, and 16 items were of easy level (DIF I > 60%). A total of 10 items had very good DI (0.40), 32 had recommended values (0.30 - 0.39), and 25 were acceptable with changes (0.20 - 0.29). Of the 100 MCQs, there were 27 MCQs with DE of 66.66% and 11 MCQs with DE of 33.33%. Conclusions: In this study, higher cognitive-domain MCQs increased after training, recurrent-type MCQ decreased, and MCQ with item writing flaws reduced, therefore making our results much more statistically significant. We had nine MCQs that satisfied all the criteria of item analysis.

Knowledge of dental faculty in gulf cooperation council states of multiple-choice questions’ item writing flaws

ABSTRACT Multiple-Choice Questions provide an objective cost/time effective assessment. Deviation from appropriate question writing structural guidelines will most probably result in commonly ignored multiple-choice questions writing flaws, influencing the ability of the assessment to measure students’ cognitive levels thereby seriously affecting students’ academic performance outcome measures. To gauge the knowledge of multiple-choice question items writing flaws in dental faculty working at colleges in Gulf Cooperation Council (GCC) countries. A cross-sectional short online Survey MonkeyTM multiple-choice questions-based questionnaire was disseminated to dental faculty working in GCC countries during the academic year 2018/2019. The questionnaire included five test incorrect (flawed) multiple-choice questions and one correct control question. The participants were asked to identify flawed multiple-choice question items from the known 14 items writing flaws. Out of a total of 460 faculty, 216 respondents completed the questionnaires, 132 (61.1%) were from Saudi Arabia, while numbers of participants from United Arab Emirates, Kuwait and Oman were 59 (27.3), 14 (6.5%) and 11 (5.1%) respectively. Majority of participants were male (n = 141, 65.9%) compared to 73 females (34.1%). Eighty percent of the participants possessed more than five years of teaching experience. Assistant professors constituted the majority (43.3%) of the academic positions participating in this study. The overall fail rate ranged from 76.3% to 98.1% and almost 2/3rds of the participants were unable to identify one or more of the flawed item(s). No significant association was observed between the demographics (age, region, academic position and specialty) and knowledge except that of participant’s gender (p < 0.009). GCC dental faculty demonstrated below average knowledge of multiple-choice question items writing flaws. Training and workshops are needed to ensure substantial exposure to proper multiple-choice question items construction standards.

Improvement of Faculty’s Skills on the Creation of Items for Progress Testing Through Feedback to Item Writers: a Successful Experience

Abstract: The evaluation of knowledge is an important step in the teachers’ work, because it is fundamental that the evaluation results be valid and reliable. Several faculty development programs have been undertaken to improve teachers’ skills on item writing. Few studies, however, have explored the benefits of providing feedback to multiple-choice item writers. Progress Testing is a longitudinal assessment of students’ knowledge gain. Their results are useful for generating feedback to students, teachers and medical school managers. The aim of this report is to present a successful experience of faculty development on high-quality item writing for Progress Testing through feedback about the fate of the items written by faculty members. Annually, faculties from medical schools participating in a consortium for the application of the Progress Test are asked to formulate new items to compose the test version. However, the fate of the items is not known by their authors, that is, they do not know if the items are incorporated into the test, if the items have good quality, what is the students’ performance in each of the items, and what is the psychometric function of the items. In 2017, one of the schools participating in the consortium offered its item authors a feedback regarding the flaws in item writing, modification of items by the review committee and students’ performance, as well as psychometric function in each item. The percentage of flawed item (items without enough quality to be selected for the test) was more than 30%. There was an inverse relation between the number of flawed item and the number of items included in the final exam, when analyzing the different content areas of the test (that is, Internal Medicine, Surgery, Pediatrics, Obstetrics and Gynecology, Public Health, and Basic Sciences). In the following year, we observed a decrease in the number of flawed item (less than 10%) and an increase in the number of eligible items to be selected for the Progress Testing. Therefore, giving feedback to item writers seems to be a good strategy for developing the faculty’s ability on writing good quality items.

PENGENDALIAN KUALITAS PADA HOME INDUSTRY MOBIL MAINAN TRUCK TANGKI di PT. SELAMAT SENTOSA

Quality control is conducted so the product or service which are being produced could have a result that matched the standard that are desired and planned, also to fix product quality which are not matched yet with the standard that already have been set and wherever possible maintain quality which are fit. Companies need some way to realize the good quality on products that are they produce and maintain the consistency so it fulfill the market demands with implementing quality control system for process activity which is being passed. Control map is usually being used to map flawed items or the number of flawed items from a sample which is taken beforehand. In contrast with the p control map which could map the process with the number of sample in each equal or inequal observation, np control map could only being used when the taken sample from each observations have the same amount. This research draws whether flaw on production process still in the control threshold or not, and how many size variety ins the controlled threshold or not. After the data processing on PT. Selamat Sentosa, the result shows that attribute control map is not yet on the threshold control that iterarion have to be done and variable control map still on the control threshold. The result shows that CP number is 1.8, which is more than 1.33, so it shows a very good result, dimension which is tolerated is 99.999986%, dimension that reworks is 0.000014% and dimension that are denied is 0.000007%. Tight control is required because CPK &lt;1.00.

Differential Length and Overlap with the Stem in Multiple-Choice Item Options: A Pilot Experiment

Multiple-choice items are extensively used across different assessment contexts. A crucial ment for ensuring their validity is their correct development, and a number of item-writing guidelines have been proposed that support item developers. This experimental pilot study aimed to investigate the effect of violating two item-writing guidelines: the differential length of the correct option compared to distractors and its lexical overlap with the stem. Standard and flawed items, respectively adhering to and deviating from guidelines, were randomly assigned to 55 college students and compared in their psychometric ing. Results indicated that, in general, flawed items tended to be easier and less subject to random answers than standard ones, but significant differences were few. Discrepancies between standardand flawed subtests approached statistical significance with medium effect sizes. Although of interest, findings must be cautiously interpreted due to the small sample size. Implications for future research are discussed.

Dealing with flawed items in examinations: Using the compensation of disadvantage as used in German state examinations in items with partial credit scoring.

In the written part of German state exams, multiple choice questions which inherit a flaw are not always excluded but taken into consideration when awarding a grade to any individual candidate, if this is advantageous to the candidate. This process of elimination and compensation of disadvantage in flawed questions, as applied by the Institut für medizinische und pharmazeutische Prüfungsfragen (IMPP), can lead to different items being taken into consideration in the grading in individual cases and to the application of different pass and grade boundaries in individual cases.This procedure is applied here to examinations containing items with partial credit scoring. A simple numerical rule can be applied which can be used to decide whether a flawed item has be taken into account in the grading procedure in order to achieve a pass or a particular grade for a candidate. This rule – as in the state examinations – must take account of the individual number of scores achieved in the flawed item and, if the relative pass threshold ("automatic adjustment clause") is applied, the average score achieved in the item by all candidates. In addition, however, it also includes the grade boundaries.

The future of online testing and assessment: question quality in MOOCs

If MOOCs are to play a role in the future of higher education it is imperative that we critically examine how they are currently functioning. In particular, questions persist about the role MOOCs will play in the future of formal accredited learning. As the focus turns from informal and free to formal, accredited and paid, greater scrutiny will be brought to bear on the quality of the courses themselves. Although there have been some empirical studies into the quality of MOOCs, a notable gap exists in that such research has not examined Multiple Choice Questions (MCQs) which are a key component of much MOOC assessment and testing. Previous research suggests that flawed MCQ items may compromise the reliability and validity of these assessments, potentially leading to inconsistent outcomes for students. This study was hence designed to examine MCQ quality in MOOCs. 204 MCQs were analysed, from a selection of 18 MOOCs, sampling the domains of computing, social science and health sciences. Over 50% of MCQs (112) contained at least one item flaw; 57 MCQs contained multiple flaws. A large proportion of MOOC MCQs violated item-writing guidelines, which is comparable with previous studies examining the prevalence of flaws in assessments in more traditional educational contexts. The problem of low quality MCQs can be ameliorated by appropriate faculty training and pre- and post-test quality checks. These activities are essential if MOOCs are to become a force that can enable enhanced and improved pedagogies in the future of higher education, instead of simply proceeding to replicate existing poor practices at scale.

Examining the impact of specific types of item-writing flaws on student performance and psychometric properties of the multiple choice question.

This article was migrated. The article was marked as recommended. Background: Item-writing flaws (IWFs) are common in multiple choice questions (MCQs) despite item-writing guidelines. Previous studies have shown that IWFs impact validity as observed through student performance, item difficulty, and discrimination. Most previous studies have examined IWFs collectively and have shown that they have a diverse impact. The aim of the study was to determine if the effects of individual types of IWFs are systematic and predictable. Method: A cross-over study design was used. 100 pairs of MCQ items (with and without an IWF) were constructed to test 10 types of IWFs. Medical students were invited to participate in a mock examination. Paper A consisted of 50 flawed followed by 50 unflawed items. Paper B consisted of 50 unflawed followed by 50 flawed items. The effect of each of the IWFs on mean item scores, item difficulty and discrimination were examined. Results: The hypothesised effect of IWFs on mean item scores was confirmed in only 4 out of 10 cases. ' Longest choice is correct', 'Clues to the right answer (Eponymous terms)' and ' Implausible distractors' positively impacted, while 'Central idea in choices rather than stem' negatively impacted mean item scores. Other flaws had either the opposite or no statistically significant effect. IWFs did not impact item difficulty or discrimination. Conclusion: The effect of IWFs is neither systematic nor predictable. Unpredictability in assessment produces error and thus loss of validity. Therefore, IWFs should be avoided. Faculties should be encouraged to invest in item-writing workshops in order to improve MCQs. However, the cost of doing so should be carefully weighed against the benefits of developing programmes of assessment.

Item Quality Improvement: What Determines a Good Question? Guidelines for Interpreting Item Analysis Reports

Medical schools strive to administer high-quality exams. These exams are often authored and graded by basic science and clinical faculty with content expertise; however, these faculty may lack expertise in question writing as well as familiarity with the psychometrics involved in item analysis. This short communication overviews the background related to multiple choice questions, proposes how to interpret an item analysis report, and makes recommendations for evidence-based grading decisions. The guidelines described here have helped ensure that faculty at one Midwestern medical school appropriately address psychometrically flawed items and make defensible grading decisions.

Evaluating the Quality of Higher Education Instructor-Constructed Multiple-Choice Tests: Impact on Student Grades

Multiple-choice questions (MCQs) are commonly used in higher education assessment tasks because they can be easily and accurately scored, while giving good coverage of instructional content in a short time. However, studies that have evaluated the quality of MCQs used in higher education assessments have found many flawed items, resulting in misleading insights about student performance and contaminating important decisions. Thus, MCQs need to be evaluated statistically to ensure high quality items are used as the basis of inferences. This study evaluated the quality of 100 instructor-written MCQs used in an undergraduate midterm test (50 items) and final exam (50 items), making up 50% of the course grade, using the responses of 380 students enrolled in one 1st-year undergraduate general education course. Item difficulty, discrimination, and chance properties were determined using Classical Test Theory and Item Response Theory statistical item analysis models. The two-parameter logistic model consistently had the best fit to the data. The impact on overall course grades between the original raw score model and the IRT 2PL model showed 70% of students would receive the same grade (i.e., D to A), but only one-third would get the same mark using the standard augmented grade scale (i.e., A+ to D-). The analyses show that higher education institutions need to ensure MCQs are evaluated before student grading decisions are made.

Welchen Einfluss haben Autorenrichtlinien der Verlage auf die formale Qualität von CME-Fragen in der ärztlichen Fortbildung?

IntroductionThe widespread use of flawed multiple choice questions (MCQs) in continuing medical education (CME) has been demonstrated in different studies. In recent years, quality control measures have been established by some German publishing houses to ensure the quality of MCQs. ObjectivesThe purpose of the present study is to evaluate whether the quality of author's guidelines and the quality of MCQs correlate and whether the quality of MCQs in CME has improved following the implementation of quality control measures. Materials and MethodsA set of 720 items from 6 journals was selected for analysis. A list of 10 item-writing flaws provided by Kühne-Eversmann et al. was modified. The existing guidelines for authors of the six selected journals were categorized by their comprehensiveness. ResultsThere were substantial differences in item quality with the percentage of flawed items ranging between 34 % and 92 % of all items published by a journal.One journal showed considerable improvement of item quality following the implementation of guidelines for authors (61 % flawed items in 2006 vs. 33% in 2012). The comprehensiveness of the author's guidelines correlated negatively with the number of item-writing flaws. ConclusionsThe correlation between the existence of elaborate guidelines for authors and the absence of item-writing flaws in MCQs suggests that the publication of guidelines for authors could result in a higher quality of MCQs, even though the review process might have a major influence.

Evaluation of Cognitive levels and Item writing flaws in Medical Pharmacology Internal Assessment Examinations.

Objectives:This study aimed to evaluate the cognitive levels of Multiple Choice Questions (MCQs) & Short Answer Questions (SAQs) and types of Item Writing Flaws (IWFs) in MCQs in Medical Pharmacology internal assessment exams.Methods:This descriptive, study was conducted over a period of six months, from December 2015 to May 2016 and evaluated six internal assessment examinations comprising SAQs and MCQs. A total of 150 MCQs and 43 SAQs were analyzed. These questions were administered to third-year medical students in the year of 2015. All SAQs were reviewed for their cognitive levels and MCQs were reviewed for cognitive levels as well as for IWFs. Items were classified as flawed if they contained one or more than one flaw. The cognitive level of the questions was determined by the modified Bloom’s taxonomy.Results:The proportion of flawed items out of 150 items in six exams ranged from 16% to 52%. While the percentage of total flawed items was 28%. Most common types of flaws were implausible distractors 19.69% (26), extra detail in correct option 18.18% (24), vague terms 9.85% (13), unfocused stem 9.09% (12) and absolute terms 9.09% (12). The two-third of MCQs 97(64.67%) were assessing the recall of information, while 29 (19.33%) and 24 (16%) were assessing the interpretation of data and problem-solving skills respectively. The majority of the SAQs (90.7%) were assessing recall of the information and only 9.3% were assessing interpretation of data while none of the questions was assessing the problem-solving skills.Conclusions:The cognitive level of assessment tools (SAQs & MCQs) is low, and IWFS are common in the MCQs. Therefore, faculty should be urged and groomed to design problem-solving questions which are devoid of any flaws.