Multiple-choice Question Performance Research Articles

The effect of the attitude towards risk/ambiguity on examination grades: cross-sectional study in a Portuguese medical school.

Given the high prevalence of multiple-choice examinations with formula scoring in medical training, several studies have tried to identify other factors in addition to the degree of knowledge of students which influence their response patterns. This study aims to measure the effect of students' attitude towards risk and ambiguity on their number of correct, wrong, and blank answers. In October 2018, 233 3rd year medical students from the Faculty of Medicine of the University of Porto, in Porto, Portugal, completed a questionnaire which assessed the student's attitudes towards risk and ambiguity, and aversion to ambiguity in medicine. Simple and multiple regression models and the respective regression coefficients were used to measure the association between the students' attitudes, and their answers in two examinations that they had taken in June 2018. Having an intermediate level of ambiguity aversion in medicine (as opposed to a very high or low level) was associated with a significant increase in the number of correct answers and decrease in the number of blank answers in the first examination. In the second examination, high levels of ambiguity aversion in medicine were associated with a decrease in the number of wrong answers. Attitude towards risk, tolerance for ambiguity, and gender did not show significant association with the number of correct, wrong, and blank answers for either examination. Students' ambiguity aversion in medicine is correlated with their performance in multiple-choice examinations with negative marking. Therefore, it is suggested the planning and implementation of counselling sessions with medical students regarding the possible impact of ambiguity aversion on their performance in multiple-choice questions with negative marking.

Gender differences in multiple-choice questions and the risk of losing points

I study the gender differences in performance in multiple-choice questions in a setting where wrong answers are penalized and the objective is to score as high points as possible. I exploit data from an undergraduate level microeconomics course at a Finnish university across a six-year period of 2010 and 2012-2016. The course consists of two equally weighted exams that include both multiple-choice and open-ended questions. The results show that, when controlling for the performance in the first exam, women omit more multiple-choice question items (MCQ) in the second exam than men which, in turn, translates to fewer points. Women do not do worse in open-ended questions that are similar to the MCQ, neither is the probability of them answering incorrectly to the MCQ higher. Hence, gender differences in test results might reflect differences in behavior in a very particular test setting rather than genuine differences in skills.

Design and Application of English Assisted Learning System Based on Mobile Learning Platform

For the purpose of developing a new kind of auxiliary learning method or tool, the author puts forward a kind of personalized mobile EFL learning system based on Android and vocabulary learning. The system consists of Remote Management Server, Client Mobile Learning System and Data Synchronization Intelligence Agent. This system has carried on the corresponding design in view of the different student, the different new word level, the different study, the different memory stage. The study found that there were 15 multiple choice questions before and after the exam, and 15 closed questions, with a full score of 30. In order to ensure accurate testing, the difficulty levels of both tests are controlled within a similar range. Although learners' academic performance in multiple choice questions did not significantly improve, their vocabulary ability in cloze tests significantly improved. It has been proven that the proposed system can effectively improve English learning performance, and the simulation also verifies the effectiveness and practicality of the system.

Multiple Choice Questions Versus Very Short Answered Questions in the Evaluation of Students of Veterinary Pathology

Background: Well-developed assessment methods have a positive impact on students’ performance, and higher education institutes are usually encouraged to establish and use effective assessment methods that effectively enhance the learning process. Objectives: This study was designed to compare students’ performance in multiple choice questions (MCQs) including best single answer (BSA) and multiple true and false (MTF) types, and very short answer (VSA) questions, and to evaluate gender differences and feedback from students and faculty. Materials and methods: A test containing three different types of questions, BSA, MTF and VSA questions (10 each), was delivered to eight groups of veterinary pathology students (3rd level, Faculty of Veterinary Medicine, Assiut University, Assiut, Egypt) in April 2022. Students’ performance, test reliability and gender differences as well as feedback from students and faculty were evaluated. Results: Students did a significant higher performance in BSA and MTF questions compared to VSA questions. The reliability of BSA, MTF and VSA questions between different groups of students were 0.53, 0.52 and 0.13, respectively. Girl students achieved a higher performance in the whole test compared to boy students. Both boy and girl students showed nearly similar performance in BSA and MTF questions. However, girl students got significant higher score in VSA questions compared to boy students. Students’ and faculty opinions were in favor of BSA and MTF questions. Conclusion: Students did much better in MCQs than VSA questions. BSA and MTF questions are preferentially favored by students and faculty.

Timing is key to providing modified assessments for students with specific learning difficulties.

IntroductionMedical students who are diagnosed with a specific learning difficulty (SpLD) will typically receive a reasonable adjustment within examinations in the form of modified assessment provision (MAP). This study investigated whether the timing of SpLD diagnosis and subsequent implementation of MAP has an impact on performance in applied medical knowledge multiple choice question (MCQ) assessments.MethodThe MCQ performance of 108 students diagnosed with SpLD who received a MAP was monitored and compared with 1960 students who received an unmodified assessment, over 5 years of a medical program. Students who received a SpLD diagnosis in the latter years of the program were identified as not receiving a MAP in assessments prior to diagnosis.ResultsDifferences were found between declaration and diagnosis, with 44.4% of students who declared and 48.1% who did not declare subsequently receiving a diagnosis. Students with SpLD who receive a MAP increase their applied medical knowledge assessment performance, although there is a delay of up to a year for this impact to reach significance.ConclusionEarly diagnosis of SpLD is necessary to ensure the intended benefit is received from MAP.

Comparison of medical students' performance in multiple choice questions and integrated short answer essay questions in end unit examination at PBL School

The Arabian Gulf University (AGU) is a regional university serving the Gulf Cooperation Council Countries. One of the most distinguished features of the MD program at AGU is problem based learning curriculum where teaching and learning of basic medical sciences are delivered through nine integrated organ system based units. At the end of each unit, medical students are required to pass a comprehensive end unit exam, where multiple formats of assessment are currently used This retrospective study compared the performance of year II medical students in multiple choice (MCQ) and short answer integrated questions (SAQ) of the end unit I examination The grades of three cohorts (471 students) were analyzed. The average score was 69.8(±SD 12.) in the MCQ component and 65.5 (± SD 13.7) in the SAQ component. Regression analysis showed a significant correlation (r=0.819, P< 0.000) between MCQ and SAQ.. when students performance was grouped by final end unit grade, a significant correlation existed between MCQ and SAQ score for 40 students who failed the end unit exam, 234 students who received a passing or a good score and for 132 students who received a very good score A positive but none significant correlation existed between the MCQ score and the SAQ score for students who received an excellent grade in the end unit exam. Finally two ANOVA revealed that MCQ scores were higher than (P < 0.01) SAQ scores for all groups. The results of this study showed that for most students, there is strong correlation between MCQ and SAQ scores, indicating that student performance was independent of testing format.Support or Funding InformationArabain Gulf UniversityThis abstract is from the Experimental Biology 2018 Meeting. There is no full text article associated with this abstract published in The FASEB Journal.

DIFFERENCES IN MULTIPLE CHOICE AND SHORT ANSWER QUESTIONS PERFORMANCE AMONG MEDICAL AND DENTAL TRAINEES AS REFLECTED IN THEIR ANATOMY COURSE

The aim of this study was to compare Medical and Dental students’ performance in multiple-choice questions (MCQs) and short answer questions (SAQs) in the pre-clinical Anatomy examinations. The study also seeks to determine the interaction between the students’ examination final grade in Anatomy course with the exam formats (MCQs versus SAQs) and level of study. The study is retrospective type in which the data were collected from examination records in the Department of Anatomy, Faculty of Basic Medical Sciences, College of Health Science, Bayero University Kano. A total of 551 students’ records were used, this population comprised ofsecond and third year Medical and Dental students for two academic sessions. The mood’s median test showed differences in the median scores across different level of the study, program and academic session. It was observed that more students had scores greater than the median in the SAQs scores. In MCQs scores despite higher number of students with scores greater than median, more students had scores greater than median in SAQs components of the examination Final examination scores correlate higher with essay scores compared to its relationship with MCQs score in year two. A statistically significant interaction among examination format and year of the study and grades was also observed. In conclusion, medical students’ performance in multiple-choice questions and short answer essay questions depends on the student’s year of the study. It was suggested that the examination format and students’ year of study may influence the final examination grade in Anatomy course.

Role-Play Preceded by Fieldwork in the Teaching of Pharmacology: from “Raw Sap” to “Elaborated Sap”

ABSTRACT Background: The task of teaching abilities, attitudes and knowledge, which can sometimes be complex, related to the safe and efficient use of medications, stands as a great challenge faced by medical schools nowadays. The role of the prescriber, therefore, who promotes the rational use of medication at undergraduate level gains importance in professional training, with a direct impact on public healthcare. In this context, the implementation of teaching methods that allow an active, critical and reflexive medical training for students is desirable in order to enable them to develop the skills required to manage the main pharmacological classes used by the general practitioner. We intend to describe and analyse role-play preceded by fieldwork as an educational strategy. Methods: Following the fieldwork regarding the utilization of the main pharmacological classes used in primary healthcare, 5-6 groups of students prepared scripts and staged role-plays involving practical aspects of pharmacokinetics, pharmacodynamics, side effects, and potential drug interactions regarding the main drug types. The intervention was assessed using students’ responses to questionnaires coupled with Likert scales, the Dundee Ready Education Environment Measure (DREMM) and semi-structured interviews. There was a correlation between participation in teaching practice and performance in multiple-choice questions in the final course evaluation. Results: All students felt involved and motivated in the activity. 78.5% strongly agreed and 19% partially agreed that the method allowed reflection on knowledge, abilities and attitude, all important to professional practice regarding rational therapeutic prescribing. The DREMM revealed a score of 129.23, consistent with a more positive learning environment in a reliable sample (Cronbach's alpha=0.86). Analysis of the open interviews allowed us to infer that the students considered the method efficient, dynamic, fun, and enjoyable, which enabled greater understanding and consolidation of the content. The strategy was considered stimulating regarding group activities, with active participation. Furthermore, it allowed students an opportunity to experience their future professional reality. The main weaknesses found were the unequal involvement of individuals in some groups and the great length of time spent in the preparation of activities. In the final exam, the students who participated in the intervention had, on average, a higher performance than those who did not take part in it. However, there was no statistically significant difference. Discussion: Role-play preceded by fieldwork proved to be a promising pedagogical strategy and may be used in other basic sciences.

Using the regulation of accuracy to study performance when the correct answer is not known.

We examined memory performance in multiple-choice questions when correct answers were not always present. How do participants answer when they are aware that the correct alternative may not be present? To answer this question we allowed participants to decide on the number of alternatives in their final answer (the plurality option), and whether they wanted to report or withhold their answer (report option). We also studied the memory benefits when both the plurality and the report options were available. In two experiments participants watched a crime and then answered questions with five alternatives. Half of the questions were presented with the correct alternative and half were not. Participants selected one alternative and rated confidence, then selected three alternatives and again rated confidence, and finally indicated whether they preferred the answer with one or with three alternatives (plurality option). Lastly, they decided whether to report or withhold the answer (report option). Results showed that participants' confidence in their selections was higher, that they chose more single answers, and that they preferred to report more often when the correct alternative was presented. We also attempted to classify a posteriori questions as either presented with or without the correct alternative from participants' selection. Classification was better than chance, and encouraging, but the forensic application of the classification technique is still limited since there was a large percentage of responses that were incorrectly classified. Our results also showed that the memory benefits of both plurality and report options overlap.

Introduction and evaluation of case-based learning in the first foundational course of an undergraduate medical curriculum

Case-based learning (CBL) has been proposed as an effective method to promote student knowledge and motivation. The timing and methods for implementation have varied among schools, and data regarding the effectiveness of this pedagogy compared to other learning modalities are inconclusive. We introduced five different cases in the first course of our medical school as part of small-group CBL, and monitored student satisfaction and performance in summative evaluations. We observed that (1) students were very satisfied with the CBL sessions; (2) performance in essay examinations was significantly higher in CBL topics, compared to non-CBL topics, as evaluated at the end of the course, and three months after course conclusion; and (3) there was a trend of higher performance in multiple-choice questions about CBL topics, but this was not statistically significant. Our results indicate that CBL is an acceptable pedagogy to be incorporated from the beginning of undergraduate medical education to promote acquisition and retention of knowledge, students’ interest in foundational sciences, and integration of basic science disciplines and clinical medicine. When evaluating knowledge acquired using the CBL method, integrative essays represent a valuable tool to assess the depth of comprehension and application of information.

Validating a standardized laparoscopy curriculum for gynecology residents: a randomized controlled trial

Residency programs struggle with integrating simulation training into curricula, despite evidence that simulation leads to improved operating room performance and patient outcomes. Currently, there is no standardized laparoscopic training program available for gynecology residents. The purpose of this study was to develop and validate a comprehensive exvivo training curriculum for gynecologic laparoscopy. In a prospective, single-blinded randomized controlled trial (Canadian Task Force Classification I) postgraduate year 1 and 2 gynecology residents were allocated randomly to receive either conventional residency training or an evidence-based laparoscopy curriculum. The 7-week curriculum consisted of cognitive didactic and interactive sessions, low-fidelity box trainer and high-fidelity virtual reality simulator technical skills, and high-fidelity team simulation. The primary outcome measure was the technical procedure score at laparoscopic salpingectomy with the use of the objective structured assessment of laparoscopic salpingectomy tool. Secondary outcome measures related to performance in multiple-choice questions and technical performance at box trainer and virtual reality simulator tasks. A sample size of 10 residents per group was planned (n= 20). Results are reported as medians (interquartile ranges), and data were compared between groups with the Mann-Whitney U, chi-square, and Fisher's exact tests (P ≤ .05). In July 2013, 27 residents were assigned randomly (14 curriculum, 13 conventional). Both groups were similar at baseline. Twenty-one residents (10 curriculum, 11 conventional) completed the surgical procedure-based assessment in the operating room (September to December 2013). Our primary outcome indicated that curriculum-trained residents displayed superior performance at laparoscopic salpingectomy (P= .043). Secondary outcomes demonstrated that curriculum-trained residents had higher performance scores on the cognitive multiple-choice questions (P < .001), the nontechnical skills multiple-choice questions (P= .016), box trainer task time (P < .001), and all virtual reality simulator parameters. Participation in a comprehensive simulation-based training curriculum for gynecologic laparoscopy leads to a superior improvement in knowledge and technical performance in the operating room compared with conventional residency training.

Simulation training in critical care: Does practice make perfect?

Few data exist regarding the effectiveness of simulation in resident education in critical care. The purpose of this study was to determine whether multiple-simulation exposure (MSE) or single-simulation exposure (SSE) improved residents' recognition of shock and initial management of the critically ill simulated surgical patient. Data were collected at a level 1 trauma center. Surgery, anesthesiology, and emergency medicine residents were given a multiple-choice question (MCQ) pretest before a tutorial on the recognition and management of shock followed by high-fidelity simulation/debriefing and MCQ post-test. MSE residents had 1.5 hours of simulation per resident over 3 days, and SSE residents had 1.5 hours of simulation as a group in 1 day. Pre- and posttest comparisons overall and subgroup analysis for MSE versus SSE were performed. Data was available for 45 MSE residents and 15 SSE residents. Overall posttest percent correct was greater than pretest percent correct (81% ± 9% vs 75% ± 13%, post- versus pre-, P = .01). Subgroup analysis demonstrated significantly improved post- versus pretest performance for MSE residents only. There were no differences in pre- or posttest performance for MSE residents during the first 4 months of the academic year versus the last 4 months. Pretest performance over 12 months of observation for MSE residents showed no significant differences. Repeated simulation exposure was more effective than single simulation exposure at improving MCQ performance designed to measure the recognition and management of shock in the critically ill simulated surgical patient. Duration of training did not impact MCQ performance.

Simulation Training in Critical Care: Does Practice Make Perfect?

Background Few data exist regarding the effectiveness of simulation in resident education in critical care. The purpose of this study was to determine whether multiple-simulation exposure (MSE) or single-simulation exposure (SSE) improved residents' recognition of shock and initial management of the critically ill simulated surgical patient. Methods Data were collected at a level 1 trauma center. Surgery, anesthesiology, and emergency medicine residents were given a multiple-choice question (MCQ) pretest before a tutorial on the recognition and management of shock followed by high-fidelity simulation/debriefing and MCQ post-test. MSE residents had 1.5 hours of simulation per resident over 3 days, and SSE residents had 1.5 hours of simulation as a group in 1 day. Pre- and posttest comparisons overall and subgroup analysis for MSE versus SSE were performed. Results Data was available for 45 MSE residents and 15 SSE residents. Overall posttest percent correct was greater than pretest percent correct (81% ± 9% vs 75% ± 13%, post- versus pre-, P = .01). Subgroup analysis demonstrated significantly improved post- versus pretest performance for MSE residents only. There were no differences in pre- or posttest performance for MSE residents during the first 4 months of the academic year versus the last 4 months. Pretest performance over 12 months of observation for MSE residents showed no significant differences. Conclusion Repeated simulation exposure was more effective than single simulation exposure at improving MCQ performance designed to measure the recognition and management of shock in the critically ill simulated surgical patient. Duration of training did not impact MCQ performance.

The relationship of deep and surface study approaches on factual and applied test‐bank multiple‐choice question performance

With the advent of online test banks and large introductory classes, instructors have often turned to textbook publisher‐generated multiple‐choice question (MCQ) exams in their courses. Multiple‐choice questions are often divided into categories of factual or applied, thereby implicating levels of cognitive processing. This investigation examined the influence student study approaches have on performance of shallow compared to deep, cognitive process outcomes measured by scores on factual versus application MCQ test bank questions respectively. Fifty‐six introduction to psychology students, aged 18–52 years, completed a modified version of the Revised Two‐Factor Study Process Questionnaire (R‐SPQ‐2F) to access deep or surface study approaches. Mid‐term and final exam MCQ were equally divided between factual and application questions from the textbook publisher test bank. Overall, students performed significantly better on factual MCQ, with older students achieving higher factual and applied scores. Results suggest younger students tend to use a surface approach to study, with reproduction of what was taught to meet minimum requirements. With age statistically controlled, analyses demonstrated that the surface learning approach negatively impacts MCQ performance on questions categorised as factual and applied more than a deep learning approach benefits MCQ performance.

Using email for teaching

An account is given of a three-year study of the use of email for teaching purposes within two courses forming part of a psychology degree, in order to compare course delivery via email with delivery via ‘traditional’ lectures. The courses were a first year ‘Psychology and IT’ course, which was taught entirely via email and a second year ‘Cognitive Psychology’ course, parts of which were taught by email and part by lectures, some of which were supported by emailed summaries. Although there were drawbacks to the use of email for teaching, the email method was felt to have been successful, as was confirmed by generally positive student feedback. The examination performance of two successive student cohorts on an multiple-choice question (MCQ) paper was studied. In one cohort the poorest part-time students performed better on material taught by email than by lectures, while in the other cohort there was some evidence for lectures supported by emailed 1000-word summaries producing better performance. However, in this second cohort MCQ performance deriving from emailed lectures was worse than that deriving from other delivery formats.

The modified essay question: an evaluation of its use in a family medicine clerkship.

This paper describes the extensive experience of the Department of Family Medicine at Jefferson Medical College in utilizing the modified essay question (MEQ) as a final examination for its required third-year clerkship in family medicine. The results of a preliminary study are presented, comparing the MEQ as a method of evaluation with the multiple choice question (MCQ) format on similar content material. MCQ performance was found to be a better predictor of National Board Part I and II scores, but neither format was found to be a good predictor of postgraduate performance. The data were statistically significant (P less than 0.05) that MEQ performance was related to postgraduate performance in the area of professional attitude while the MCQ was not; however, the number of students in the analysis was small. Additional studies are planned to test this hypothesis further.

Linear study of medical undergraduate performance

The examination performance of a cohort of clinical undergraduates has been scrutinized. The cohort was notable for a high failure rate at the university preclinical examination, but this failure rate did not reappear consistently in all four final clinical subjects. The assaying examinations were not only valid for their objectives, but also measured a range of other attributes. Multiple choice question papers scores detect both quantitative and qualitative differences in knowledge between the high and low fail risk candidates. The effect of A-level grade is variable but is carried up to the final qualifying examinations. High performers tend to have good A-levels. The contribution of the preclinical course to the clinical qualifying examination was small, varied between the preclinical subjects and also between the clinical subjects and two streams of students. It is shown that lack of an adequate standard in science subjects before entry to university is associated with subsequent disadvantage in clinical training. Such disadvantage could be reduced by an introductory premedical course. The contributions made by the differing educational experiences within the cohort could be detected in many of the examinations; in particular, poor performance at the end of first clinical year pathology multiple choice questions paper examination was associated with poor performance at finals. Failure in the first part of finals indicated a likelihood of furhter failure, factors such as sex and motivation are shown to affect multiple choice question performance whilst others such as work overload do not. The high risk fail candidates can be detected 2 years before finals and might be given special educational treatment designed to improve their chance of first time success as there is no efficient way of identifying them before acceptance into the medical school.