MCQ Examination Research Articles

A Simple Statistical Estimation of One’s Performance in an MCQ Examination, Based Upon Mock Test Results, Using Binomial Distribution of Probability

A simple statistical model is proposed regarding the estimation of one’s overall performance in an MCQ examination along with the calculation of probability of obtaining a certain percentage of marks in the same. Using the data obtained from the results of a sufficiently large number of mock examinations, conducted prior to the main examination, certain parameters quantifying one’s knowledge or preparation for the examination has been calculated. Based on those parameters, the probability of obtaining a certain percentage of marks has been computed using the theory of binomial probability distribution. The dependence of this probability function on various parameters has been depicted graphically. A parameter, called the performance index, has been defined in terms of the expectation value and standard deviation of marks computed from probability distribution. Using this parameter, a new parameter called the relative performance index has been defined. This index estimates one’s performance with respect to the best possible performance. The variation of relative performance index with respect to the preparation index has been shown graphically for different parameter values quantifying various aspects regarding the examination and the examinee.

Optimal correction for guessing in multiple-choice tests

Subjects’ decisions in multiple-choice tests are an interesting domain for the analysis of decision making under uncertainty. When the test is graded using a rule that penalizes wrong answers, each item can be viewed as a lottery where a rational examinee would choose whether to omit (sure reward) or answer (take the lottery) depending on risk aversion and level of knowledge. We formalize students as heterogeneous decision makers with different risk attitudes and levels of knowledge. Building on IRT, we compute the optimal penalty given students’ optimal behavior and the trade-off between bias and measurement error. Although MCQ examinations are frequently used, there is no consensus as to whether a penalty for wrong answers should be used or not. For example, examinations for medical licensing in some countries include MCQ sections with penalty while in others there is no penalty for wrong answers. We contribute to this discussion with a formal analysis of the effects of penalties; our simulations indicate that the optimal penalty is relatively high for perfectly rational students but also when they are not fully rational: even though penalty discriminates against risk averse students, this effect is small compared with the measurement error that it prevents.

Non‐traditional learners in higher education: comparison of a traditional MCQ examination with concept mapping to assess learning in a dental radiological science course

The aim of this study is to argue for alternative assessment methods (i.e. concept map) considering the changes in demography in higher education. In the case of school of dentistry, for example, there is an urgent call for a catalyst for new assessment methods in dental education in view of the drive to comprehensively assess professional competence to cater to the current work requirement; this quantitative study compared two different approaches to assessment following a first‐year undergraduate course for dental and hygiene and therapy students. The assessment methods compared were multiple‐choice questions and concept mapping. The data suggest that concept mapping can be appropriate for non‐traditional students (i.e. adult learners), particularly those who already have professional experience in related practice. This is because concept mapping allows learners to locate their new knowledge in a broader social and experiential frame. The implications are discussed.

Multivariate analysis of factors Influencing reliability of teacher made tests

Background: According to the measurements literature reliability of the test refers to the consistency of the test results and shows whether the obtained score is stable indication of the student’s performance in particular test Reliability can be measured by different statistics formula. Purpose: To determine the factors influenced the reliability of 392 MCQ examinations. Methods: The correlation of reliabilities of MCQ based examination and other characteristics of tests such as length difficult items, discrimination index, mean, standard deviation and time for answering was calculated based on the data available on examination center of Tehran University of Medical Sciences. Multivariate regression has been used for data analysis. Results: overall reliability of teacher made test is at satisfactory level in most cases. The mean value of reliability was 0.71 ±0.15. In comparing previous semester with last series of examination some improvement have been found during these years (P=0.000, for first semester, P=0.002 for second, P= 0.005 for third and P=0.005 for forth semester). Keeping other variable fixed the interaction of length of exam according to item difficulty showedl significant difference on value of test reliability. Comparing difficult and easy items question with moderate difficulty index can increase reliability 8 times more than difficult and 13 times more than easy items P=0.000. Conclusion: Our study showed that with documentation of tests’ metric features an analysis and evaluation of tests are within reach of medical school .

Student feedback from MCQ examinations.

A simple method is described for providing feedback to students from multiple choice question examinations. A mask with holes punched at positions corresponding to the correct answers and labelled with the subject of the questions is superimposed on the student's answer sheet. Correct answers appear as black circles and a photocopy may be taken as a permanent record. The method is inexpensive, not dependent upon computers, and is economical in staff time. It is appropriate in a wide variety of situations where MCQs are used.

The ‘don't know’ option in MCQ examinations

The ‘don't know’ option in MCQ examinations

Experience with multiple choice question papers in Nigeria. A practical guide to their use in a new medical school.

The application of MCQ examination techniques, in the context of a new medical school in tropical Africa, is described. The pitfalls are noted, the composition of papers discussed, and a simple method of marking is described. Some examples of questions are given, and their individual values are simply assessed. From our experience, there is no doubt that objective testing has a real place in a young medical school, where it can actually save time, as well as reducing variation in standards of examination.

A randomized controlled trial comparing instructions regarding unsafe response options in a MCQ examination

Background: Marking of multiple choice type examinations often just takes account of the correct responses. This may encourage guessing of incorrect and potentially unsafe responses. Ideally responses should contain a high proportion of correct, unsafe and the use of ‘don’t know’ response rather than incorrect.Aim: This study explored the effect of instructions on responses.Methods: Fourth- and fifth-year students sitting for an optional multiple choice examination were randomized to receive one of four instruction options: number-correct marking (the control group); alert to unsafe; mark deduction for unsafe; or correction for guessing.Results: A total of 210 students sat the test. For the fourth-year cohort, compared with the control group, being alerted to unsafe and mark deduction for unsafe or incorrect responses were associated with graduated increases in the use of ‘don’t know’ and reductions in incorrect responses. For the fifth-year cohort, there were no differences in responses between options. The fifth-year cohort, gave more correct, and fewer incorrect, unsafe and ‘don’t know’ responses.Conclusions: Both the year group and instructions had an effect. Being alerted to potentially unsafe responses, even when there is no mark deduction penalty, had an effect, although mark deduction still had the greatest effect. Assessment instructions may give subliminal messages that have important consequences.

Relationship Between Dental Students’ Performance on Standardized multiple‐Choice Examinations and OSCEs

The purpose of this study was to examine the relationship among dental students' performance on the National Board Dental Examination (NBDE Parts I and II), comprehensive written multiple-choice question examinations (MCQ examinations), and objective structured clinical examinations (OSCEs) administered at the Harvard School of Dental Medicine (HSDM). Outcomes were measured during the third and fourth years at HSDM for the classes of 2006 and 2007. Three separate OSCE exams and two comprehensive MCQ examinations are administered during years 3 and 4 at HSDM per class. The study group was made up of sixty-two students (thirty-two females and thirty males). The average NBDE Parts I and II scores of the study population were 93.32 +/-4.02 and 84.63 +/-4.25, respectively, and were associated with outcomes on all three OSCE examinations (p<or=0.044). However, using multiple regression models, the only statistically significant association occurred between NBDE Part II and OSCE 2 (p=0.003). Analysis showed that didactic predictors (NBDE Parts I and II and comprehensive MCQ examinations) explained 20.4 to 22.1 percent of the variability in OSCE scores. These results suggest that performance on OSCE examinations is not highly correlated with performance on NBDE Parts I and II and HSDM-administered MCQ examinations. The findings suggest that OSCE examinations are more likely to measure other qualities such as problem-solving ability, critical thinking, and communication skills.

Assessment of an undergraduate psychiatry course in an African setting

BackgroundInternational reports recommend the improvement in the amount and quality of training for mental health workers in low and middle income countries. The Scotland-Malawi Mental Health Education Project (SMMHEP) has been established to support the teaching of psychiatry to medical students in the University of Malawi. While anecdotally supportive medical educational initiatives appear of value, little quantitative evidence exists to demonstrate whether such initiatives can deliver comparable educational standards. This study aimed to assess the effectiveness of an undergraduate psychiatry course given by UK psychiatrists in Malawi by studying University of Malawi and Edinburgh University medical students' performance on an MCQ examination paper.MethodsAn undergraduate psychiatry course followed by an MCQ exam was delivered by the SMMHEP to 57 Malawi medical students. This same MCQ exam was given to 71 Edinburgh University medical students who subsequently sat their own Edinburgh University examination.ResultsThere were no significant differences between Edinburgh students' performance on the Malawi exam and their own Edinburgh University exam. (p = 0.65). This would suggest that the Malawi exam is a comparable standard to the Edinburgh exam. Malawi students marks ranged from 52.4%–84.6%. Importantly 84.4% of Malawi students scored above 60% on their exam which would equate to a hypothetical pass by UK university standards.ConclusionThe support of an undergraduate course in an African setting by high income country specialists can attain a high percentage pass rate by UK standards. Although didactic teaching has been surpassed by more novel educational methods, in resource poor countries it remains an effective and cost effective method of gaining an important educational standard.

Teach a man to fish and you feed him for a lifetime: physics, pharmacology and physiology for anaesthetists

The sheer depth and breadth of knowledge required for the FRCA examinations in anaesthesia is compelling. This is particularly true with regards to the basic science components you have to be familiar with during a challenging viva or MCQ examination.

Assessment of learning with multiple-choice questions

Nurse educators aspire to learning outcomes that will produce competent practitioners. Assessment strategies should be consistent with desired learning outcomes [Manogue, M., Kelly, M., Bartakova Masaryk, S., Brown, G., Catalanotto, F., Choo-Soo, T., Delap, E., Godoroja, P., Murio, I., Rotgans, J., Saag, M., 2002. Evolving methods of assessment. European Journal of Dental Education 6 (3) 53-66] and there is a requirement to diversify approaches. Objective testing, specifically multiple-choice questions (MCQ's) are one of the approaches that may diversify the assessment approach in Irish undergraduate nursing education. In this paper, Quinn's [Quinn, F.M., 2000. The Principles and Practice of Nurse Education, fourth ed., Stanley Thorne (Publishers) Ltd, Cheltenham] cardinal criteria of assessment, practicality, reliability, validity and discrimination provide a useful framework to discuss the issues associated with the implementation of this type of assessment strategy. MCQ examinations that are consistent with educational outcomes can be used to effectively assess aspects of student performance and can facilitate timely feedback and contribute to the process of self-learning. Significant commitment is required to prepare MCQ test items and examination formats that are reliable and consistent with curriculum objectives. Appropriately constructed MCQ examinations are efficient, objective, capable of discrimination and can be combined with other assessment strategies to contribute to a comprehensive student assessment strategy for use in nursing education.

Comparison of Performance 2 Years after the Old and New (Interactive) ATLS Courses

Background. We previously (1997) demonstrated superior clinical but similar cognitive performance after the new interactive compared to the old ATLS course. The present study is aimed at determining whether this difference was short term or maintained over time (2 years).Materials and methods. Two groups of 13 physicians out of the original 32 physicians were available for the study which compared performance in a 40-item MCQ examination on trauma topics and clinical performance in 4 trauma OSCE stations consisting of simulated trauma patients. Paired and unpaired t tests were used for within- and between-group comparisons, respectively, with P < 0.05 being considered statistically significant. Overall OSCE scores (maximum standard 20), adherence to priority scores (Priority, scale 1 to 7), and overall approach (Approach, scale 1 to 5) scores were analyzed.Results. Values are means ± SD; +−P < 0.05 compared to 1999; *P < 0.05 compared to old group.Conclusions. Although knowledge base decreases similarly with time after both courses, the new interactive course participants maintained a consistently higher clinical skill performance level at 2 years.

The effectiveness of a computer-aided instruction courseware developed using interactive multimedia concepts for teaching Phase III MD students

A prospective study on 64 final-year MD students who failed in their pre-test MCQ examination on admission to the Paediatric Cardiology Block was conducted to evaluate the effectiveness of an innovative computer-assisted instruction (CAI) courseware developed using interactive multimedia concepts. Students were randomly divided into two groups and were taught by either conventional lecture (Group A) or using the CAI multimedia courseware (Group B). They were required to sit for a post-test MCQ assessment immediately after exposure to the different teaching methods. There was no significant difference in the mean pre-test scores between Group A (37.4 ± 9.1) and Group B (40.3 ± 9.7). The results showed that 81.3% of students in Group B passed the post-test compared with 40.6% in Group A. Both groups showed significant improvement in their mean post-test scores compared with their respective mean pre-test scores. However, students taught using the CAI multimedia courseware did significantly better than students taught using the conventional method. This study suggests that CAI multimedia may be a more effective method for teaching cardiovascular subjects which required dynamic illustrations, such as Tetralogy of Fallot.

Der Ulmer Notfallparcours - Teil 2: Realisation und Akzeptanz des Praktikums der Notfallmedizin

The first part of this publication described the concept of the student course in emergency medicine at the University Hospital Ulm as part of a "vertical curriculum" of education in emergency medicine. METHODS. A training circuit (Fig. 1) was conceptualised using training manikins (i.e. BLS, intubation, megacode training, ATLS training), computer programmes (ECG-simulation) and other means of instruction to simulate twelve realistic situations. These practical skill sessions were linked with case presentation and lectures. Its topics were chosen depending on the prevalence, type and urgency. Separate guidelines for teachers and students were edited prior to the beginning of the course in 1994. RESULTS. Evaluation showed good acceptance by the students for most parts of the course except the ATLS skill station and CPR computer simulations (Table 5). The score of the national board examination (multiple-choice test) of the students from Ulm was nevertheless not better than the mean of all german examinees (72.1 to 71.7%). CONCLUSION. The concept of the course in emergency medicine is well accepted by our students. Results of the national examination were not improved, since the course aims at performance, skills and competence in emergency medicine and not at teaching factual knowledge that is measured by the MCQ examinations.

Portfolio learning: a proposal for undergraduate cancer teaching.

This paper describes a randomized study to evaluate a longitudinal approach to undergraduate education using cancer patients and cancer medicine as a model. Medical students were randomized to the study or control group. Those in the study were allocated to tutors (general practitioners or consultants) in groups of three. Each student was allocated three consenting patients with cancer to follow longitudinally. They compiled a portfolio to record events, interactions, relevant articles, etc. based around the patients' case histories. The students received tutorials to provide direction, discussion and support bimonthly during the course. Evaluation was by central review of the portfolios, appraisal in the clinical competence examination and hidden questions in the standard MCQ examinations. The aim was to provide the learner with a personal experience of cancer patients at all stages of the disease, an understanding of the natural history of cancer and an insight into the impact of the disease and its treatment on the patient and family. The study evaluates whether these aims were achieved to a greater extent in those students receiving the portfolio teaching compared with a control group receiving the standard curriculum teaching only.

Book Review : MCQ Examinations in Psychiatry

Book Review : MCQ Examinations in Psychiatry

Strategic marking in MCQ papers

The usual advice given to candidates attempting MCQ examination papers is to tick those answers which they are certain or fairly certain are correct and leave blank the ones they are unsure about. An alternative strategy, the details of which are set out below, is to ensure that every question is answered whether the candidate knows the answer or not. Users of the first type of strategy are referred to below as “chickens” and users of the second type of strategy as “gamblers”. It is clearly in the interests of a “fair” examination that all candidates should use the same marking strategy when answering the questions, so it is not surprising that teachers tend to encourage the “chicken” approach rather than the “gambler” approach since the latter may introduce spurious changes in examination results. Some candidates may enter this contest so supremely confident of their ability to score a high proportion of “certain correct” answers that they can afford to guarantee themselves as a zero score on the remaini...

MCQ and higher examinations as exemplified by the primary Fellowship of the Faculty of Anaesthetics, Royal College of Surgeons of England.

A major problem of the use of multiple choice examinations is the inherent difficulty of the questions, a fact which cannot be assessed in relation to questions not previously used. Questions which have been used during previous examinations, however, will have an available measure of the difficulty in the form of candidates' previous scores. The performance of candidates in a current examination can be compared with that in a previous examination. On the strength of these differences it is possible to decide whether the candidates in any single examination are better prepared than their predecessors or less well prepared. Pass marks can be appropriately adjusted.

The prediction of medical student performance using in-course assessment.

The value of in-course assessment at King's College Hospital Medical School at predicting student performance at final qualifying examinations in medicine and surgery has been analysed. Only a small proportion of the variation in marks at finals could be explained by the factors studied. Both the MCQ and firm report methods of internal assessment used in the first clinical year were equally valuable at predicting final outcome. Performance at 2nd MB BS, the possession of another degree and the sequence of firm attachment in the first year have little predictive value. Female students are predicted to perform better in medicine than in surgery. The development of a more standardized quantifiable firm assessment looks potentially more valuable at predicting performance than refining the first year MCQ examinations.