Use Of Multiple Choice Questions Research Articles

Ten tips for effective use and quality assurance of multiple-choice questions in knowledge-based assessments.

Multiple-choice questions (MCQs) are the most popular type of items used in knowledge-based assessments in undergraduate and postgraduate healthcare education. MCQs allow assessment of candidates' knowledge on a broad range of knowledge-based learning outcomes in a single assessment. Single-best-answer (SBA) MCQs are the most versatile and commonly used format. Although writing MCQs may seem straight-forward, producing decent-quality MCQs is challenging and warrants a range of quality checks before an item is deemed suitable for inclusion in an assessment. Like all assessments, MCQ-based examinations must be aligned with the learning outcomes and learning opportunities provided to the students. This paper provides evidence-based guidance on the effective use of MCQs in student assessments, not only to make decisions regarding student progression but also to build an academic environment that promotes assessment as a driver for learning. Practical tips are provided to the readers to produce authentic MCQ items, along with appropriate pre- and post-assessment reviews, the use of standard setting and psychometric evaluation of assessments based on MCQs. Institutions need to develop an academic culture that fosters transparency, openness, equality and inclusivity. In line with contemporary educational principles, teamwork amongst teaching faculty, administrators and students is essential to establish effective learning and assessment practices.

Multimodal prediction of student performance: A fusion of signed graph neural networks and large language models

In online education platforms, accurately predicting student performance is essential for timely dropout prevention and interventions for at-risk students. This task is made difficult by the prevalent use of Multiple-Choice Questions (MCQs) in learnersourcing platforms, where noise in student-generated content and the limitations of existing unsigned graph-based models, specifically their inability to distinguish the semantic meaning between correct and incorrect responses, hinder accurate performance predictions. To address these issues, we introduce the Large Language Model enhanced Signed Bipartite graph Contrastive Learning (LLM-SBCL) model—a novel Multimodal Model utilizing Signed Graph Neural Networks (SGNNs) and a Large Language Model (LLM). Our model uses a signed bipartite graph to represent students’ answers, with positive and negative edges denoting correct and incorrect responses, respectively. To mitigate noise impact, we apply contrastive learning to the signed graphs, combined with knowledge point embeddings from the LLM to further enhance our model’s predictive performance. Upon evaluating our model on five real-world datasets, it demonstrates superior accuracy and stability, exhibiting an average F1 improvement of 3.7% over the best baseline models.

Interactive Learning: Online Audience Response System and Multiple Choice Questions Improve Student Participation in Lectures.

Background and Objective Multiple Choice Questions (MCQs) are commonly used in medical education for formative and summative assessment. However, the use of MCQs as a pedagogical tool in lectures is an area that is yet to be fully explored. This study aimed to gather feedback on including MCQs in lectures using an Online Audience Response System (OARS). Methods This quasi-experimental study involved 114 first professional MBBS students. A lecture with strategically integrated MCQs was delivered in a traditional classroom setting. Students answered the ten MCQs and provided feedback via OARS using their mobile phones. The feedback included eleven questions about student engagement, attentiveness, and critical thinking skills. Students' perception of the advantages and limitations of OARS in lectures was also collected. Data are presented as percentages and median with an interquartile range. Results Higher levels of engagement were reported by 80% of students; 81% felt improvement in understanding of the learning objectives and lecture content; 5% felt that mobile phone usage caused distractions; 79% reported increased focus and concentration; 84% reported that MCQs promoted their critical thinking skills; 75% reported enhanced overall learning experience without causing any discomfort; 69% believed that asking questions individually by the instructor is less effective than MCQs with OARS; 83% felt motivated to come prepared with study material when MCQs were included in the lecture; 67% preferred an even distribution of MCQs throughout the lecture; 53% preferred six to ten MCQs per lecture. Conclusion This study suggests that integrating MCQs in lectures using OARS can be a valuable pedagogical strategy in medical education and can potentially improve the learning experience by enhancing student engagement, attentiveness, and critical thinking skills. However, educators should also be aware of the potential limitations and take steps to mitigate them.

Manajemen Kegiatan Ekstrakulikuler dalam Seleksi Santri Berbasis Multiple Intelegence

The research aims to help the basic problems of students in determining their choice of majors in Extracurriculars by using Multiple Intelligence-based selection. The research was conducted at the Nurul Jadid Islamic Boarding School in the Jalaluddin Ar-rumi area using a qualitative approach with field research. Data obtained from interviews and observations. Data analysis was preceded by interviews with the Head of Santri Resources & Management, chairman of the Association for Organization Rumi (AFORMI), and the students. The results of the study show that the selection of students based on Multiple Intelligence (MI) is very helpful in determining the extracurricular class of the dormitory, it is even very helpful in procuring activities so that the administrators are able to carry out procurement according to the needs of the talents & interests of the students. The stages of the selection implementation consist of conducting questionnaire tests, giving briefings, and delivering material on Multiple Intelligence. The use of Multiple Choice questions was maintained because the administrators considered that the use of these questions was still effective for students. The Multiple Intelligence-based student selection model is very important to develop because Multiple Intelligence emphasizes the individual intelligence of students, which is considered suitable for the current approach to students.

A Novel Approach for Exam E-assessment Utilizing Image Processing

There is a brand-new feature called Exam (Infinity Exam) that supports paper-based exams and speeds up the entire process while maintaining all of their beneficial qualities and minimizing their drawbacks, notably in higher education. The method is very different from those employed in the earlier 10+ years, which were implemented in a way that prevented them from replicating and supplanting the conventional paper-based examination format. The article's core relies on the image processing flow, which is the most crucial component of the software. Multiple Choice Questions (MCQ) have been a more common method of testing someone's knowledge over time. The use of multiple choice questions in exams is becoming more widespread in the education sector (including in schools and colleges). It is employed even when conducting interviews. The current scenario involves either manually correcting the test or using OMR technology. Having OMR at all times in real time is rather challenging, and manually correcting it takes a lot of effort and could result in a mistake. We address this issue by applying a digital image processing technique in our proposed system to correct the response using multiple-choice questions written in Python. Here, we are processing data using Open-Source Computer Vision Library (OpenCV).

Effectiveness of Multiple-choice Questions (MCQS) Discussion as a Learning Enhancer in Conventional Lecture Class of Undergraduate Medical Students

Context: Multiple Choice Questions (MCQs) are one of the commonly used assessment tools in the field of medical education. A few initiatives have been made to use MCQ tests during classes as a mode of test-enhanced learning. However, there is a lack of enough scientific information regarding the use of MCQs as a learning enhancer. Aims: The present study has been undertaken to analyze the effectiveness of MCQ discussion as a learning tool. Methodology: The study was conducted in a Government Medical College in Kerala among Phase 2 MBBS students. They were divided into study and control groups of 61 students each. Online lecture classes were conducted for both groups on six topics of Pathology. For the study group, an additional MCQ discussion was done towards the end of the lecture class. Post-test was conducted on day 1 and after 2 weeks. A questionnaire was given to assess the attitude of students. Scores obtained in the post-tests were used to test the significance. Results: There was a significant difference in the mean score obtained between control and study groups in the post-test conducted on the same day suggesting that the MCQ discussion conducted at the end of the lecture class has added benefit (P value = 0.04). There was no significant improvement in academic performance when an assessment was made after 2 weeks (P value = 0.59). More than 80% of students showed a positive perception of this method. Conclusion: MCQ supplementation towards the end of the didactic lecture class is an effective teaching-learning method. Students also express a very positive attitude towards it. MCQs can be considered not only as an assessment tool but as a learning enhancer also.

A Simple and Sustainable Exercise to Enhance Student Self-Reflection on Error-Making, Focus Support, and Guide Curricular Design

Problem: Self-reflection is a critical component of professional development and clinical practice, but medical students’ ability to self-reflect is typically limited. While inadequate self-reflection impacts future clinical decision-making, it may also adversely impact current learning through an inability to identify learning-behavior deficits. This may be exacerbated by common use of multiple-choice questions (MCQ) where incorrect responses provide less insight than other measures for students, faculty, or academic support. To address this, an Error Reflection Method (ERM) was developed to help students focus on ‘why’ they got an MCQ wrong rather than ‘what’ they got wrong, thereby promoting self-reflection and a learning-focus on assessment. Understanding students’ learning-behavior deficits could also enrich engagement with academic support services and guide curricular design. Intervention: The ERM is a list of 10 common types of exam errors that were either ‘test-taking’ (unwitting) errors or ‘learning-behavior’ errors that reflected learning deficits. The ERM is simple, transferable, and sustainable, allowing longitudinal and regular monitoring of individual and collective error-making to focus support and guide curricular development. Context: Undergraduate medical students at the Virginia Tech Carilion School of Medicine, USA, used the ERM in formative assessment review sessions in pre-clinical years to select an error type that best described the cause of each incorrect response. Impact: Initial findings suggest the ERM is robust and associated with improved student performance and curricular development. Analysis of 3,775 student-identified errors showed the error types in the ERM described 96% of errors students made. Learning-behavior errors were more common (76%), but surprisingly, 19% were test-taking errors, allowing academic support to focus on test-taking skills in a population previously thought of as consummate test-takers. The most common error type reported was ‘the content looked familiar but I couldn’t answer the question’ (32%); which we suggest is consistent with shallow learning. This finding has helped steer recent curricular development toward active and applied learning techniques. Lessons Learned: By formally and regularly identifying learning deficits, students may be more capable of addressing them and improve summative exam performance. As well as focusing academic support, understanding common student errors has been useful in guiding curricular design and content delivery. Further potential of the ERM may be realized in faculty development and directing assessment culture toward a learning focus.

Multiple Choice Questions (MCQ) as Assessment Tool in Primary Schools: Perceived Influence on Students Writing Skills in Secondary Schools in Yaounde VI.

This study examines the influence of excess used MCQ as an assessment or test tool in senior primary schools on their writing competences in early secondary schools. The problem of this study emanates from the observed fall in writing skills among early primary school pupils in Yaounde V. Majority of the students who register in Form One do not know how to write well. The piloted and validated qualitative data analyses tool (interview guide) were used to collect data from 18 participants being 10 primary school class 5 and 6 teachers and 8 secondary school English teachers. The data was examined and the majority of the opinions sampled were considered and it indicated that: Junior secondary school students have unadorned writing challenges due to excess use of MCQ as assessment tools. They concede that it does not give the learner the ability to practice their own writing skills, thus, they turn to neglect it because it’s not much tested in the exams. We therefore recommend to the teachers that; they should employ a heterogeneous testing method in pupil’s exams.

Evaluation of assessment marks in the clinical years of an undergraduate medical training programme: Where are we and how can we improve?

Background. In high-stakes assessments, the accuracy and consistency of the decision to pass or fail a student is as important as the reliability of the assessment. Objective. To evaluate the reliability of results of high-stakes assessments in the clinical phase of the undergraduate medical programme at the University of the Free State, as a step to make recommendations for improving quality assessment. Methods. A cohort analytical study design was used. The final, end-of-block marks and the end-of-year assessment marks of both fourth-year and final-year medical students over 3 years were compared for decision reliability, test-retest reliability, stability and reproducibility. Results. 1 380 marks in 26 assessments were evaluated. The G-index of agreement for decision reliability ranged from 0.86 to 0.98. In 88.9% of assessments, the test-retest correlation coefficient was <0.7. Mean marks for end-of-block and end-of-year assessments were similar. However, the standard deviations of differences between end-of-block and end-of-year assessment marks were high. Multiple-choice questions (MCQs) and objective structured clinical examinations (OSCEs) yielded good reliability results. Conclusion. The reliability of pass/fail outcome decisions was good. The test reliability, as well as stability and reproducibility of individual student marks, could not be accurately replicated. The use of MCQs and OSCEs are practical examples of where the number of assessments can be increased to improve reliability. In order to increase the number of assessments and to reduce the stress of high-stake assessments, more workplace-based assessment with observed clinical cases is recommended.

The role of virtual interactive simulators in medical education: Exploring their integration as an assessment methodology in clinical years

IntroductionAssessment takes a fundamental role in medical education. In Portugal, the main assessment method used is multiple-choice questions (MCQ). The virtual interactive patients (VIP) as an assessment method can be indicated for the development of clinical reasoning for medical students of the clinical years. This study compared the use of MCQ and VIP as assessment methodologies in medical students. MethodsA randomized crossover study carried out, which 35 Portuguese medical students from clinical years evaluated by MCQ and VIP in four medical specialties. In each specialty, students randomized into two groups. One group started being evaluated by MCQ and finished with VIP, the other vice-versa. After the evaluation, students asked to complete a satisfaction questionnaire. ResultsThe final score was higher with the VIP than MCQ method (MCQ: 67.40±8.96 vs. VIP: 73.34±10.8; p<0.01) with significant differences higher in Clinical History and Physical Examination. The satisfaction quiz showed although both methodologies are adequate and satisfactory, but VIP method allows to improve knowledge. ConclusionVIP is more accurate for clinical reasoning, accepted by medical students and should be integrated into the pre-graduate medical curriculum with other methodologies already used. The COVID-19 pandemic allowed a digital transformation of medical education and VIP will have a crucial role as a knowledge and evaluation method.

The study of the impact of various question formats and their combination variability on test results

The aim of the research was to identify and examine the characteristics of the impact of four question types and their combination in polymorphism tests on the testing output indicators for secondary school students. Students in the 8th and 9th grades (N=164) participated in the study that was done in the field of studying the native (Ukrainian) language. It was possible to demonstrate the existence of the effect of factor "question form" on the test scores (impact power is up to 35%) and to identify the combination of question forms that is characterized by the highest diagnostic accuracy thanks to the tools (criterion-oriented author's tests, questionnaire), and methods of results' analysis (factor analysis, dispersion analysis, etc.) used in the research process. According to research findings, creating a polymorphic linguistic test is advised. If there is an even distribution of these questions throughout the test, the use of multiple-choice questions with one correct answer combined with text and constructed-response questions can guarantee the clarity of test output indicators (with the least impact of question formats on them). The study's drawbacks are identified, and the likelihood that they will be addressed by further research on the most effective test framework for empirical validity is supported.

Оценивание результатов обучения по дисциплине "Экономическая теория"

The article is devoted to the use of multiple choice questions and essay assignments as assessment tools during the economics training. The authors describe three levels of economics understanding. They provide examples of multiple-choice questions and essay assignments that can measure different levels of economics understanding. The authors conclude that multiple-choice questions can be used to measure some but not all levels of student understanding while higher levels cannot be accurately measured with this assessment tool. The essay has great potential to assess the higher levels of student understanding of economics.

Use of multiple choice questions to assess the knowledge and awareness about HIV/AIDS - A core competency, among first and second MBBS students of KBNIMS, Gulbarga

Multiple choice questions (MCQ’s) are a reliable and an efficient method of assessment. It is an objective type of assessment where, its efficiency depends on the item and the options given. A well framed MCQ is a very good tool for formative and summative assessment of the student. It is also been introduced in medical education, different entrance exams. The student can be assessed for the depth of the knowledge he/she has using MCQs. Also, the MCQs are less time consuming, cost effective and efficient and reliable. To compare the knowledge and awareness about HIV/AIDS in 1 and 2 MBBS students using MCQs as an assessment method The study comprised 100 1 MBBS and 100 2 MBBS students from KBNIMS, Gulbarga. Students were given 14 MCQs on HIV/AIDS and were asked to tick the correct answers. In our study we found that the performance of 2 MBBS students was better compared to 1 MBBS students in majority of the MCQs In this study we found that MCQs can be used as assessment tool to compare the knowledge and awareness about HIV/AIDS among UG medical students. We found that the knowledge and awareness of AIDS is better among 2 year students who have the topic as core competency to learn.

Teachers' Opinions about Using Multiple-choice Questions in the National Examination of Mathematics: A Survey in Vietnam

In Vietnam, after completing the high school education program, students who want to receive high school diploma, they must take a national examination including three compulsory subjects: Math, a foreign language, and Literature and electives in two groups: or Physics, Chemistry and Biology or History, Geography and Citizenship education. The method of exam-taking in the national examination greatly affected the teaching and learning style of teachers and students. In the process of teaching at high school, especially in the final year, the teachers studied how to ask types of question that have appeared in the national exam to focus on practising test-taking skills for students. Before 2017, the national examinations of Physics, Chemistry, Biology, Foreign Languages, Geography, History and Citizenship education consisted of all the multiple-choice questions; only in Math and Literature, essay questions were used. Since 2017, in national examinations of mathematics in Vietnam, there has been a significant change: the examinations with all essay questions with a time of 180 minutes (10 questions) were replaced by examinations with all multiple-choice questions (45 questions / 90 minutes). This change has met many conflicting opinions and has profoundly affected teachers' the method of teaching and reviewing knowledge and skills of mathematics for high school students in Vietnam. To better understand the perception of mathematics teachers - who directly teach - about this change, we conducted a survey of 50 mathematics teachers. The results showed that they were fully aware of the advantages of multiple-choice question-based examinations, and most of them wanted to maintain the use of multiple-choice questions for the following years.

Very Short Answer Questions: A Novel Approach To Summative Assessments In Pathology.

BackgroundA solid understanding of the science underpinning treatment is essential for all doctors. Pathology teaching and assessment are fundamental components of the undergraduate medicine curriculum. Assessment drives learning and the choice of assessments influences students’ learning behaviours. The use of multiple-choice questions is common but is associated with significant cueing and may promote “rote learning”. Essay-type questions and Objective Structured Clinical Examinations (OSCEs) are resource-intensive in terms of delivery and marking and do not allow adequate sampling of the curriculum. To address these limitations, we used a novel online tool to administer Very Short Answer questions (VSAQs) and evaluated the utility of the VSAQs in an undergraduate summative pathology assessment.MethodsA group of 285 medical students took the summative assessment, comprising 50 VSAQs, 50 single best answer questions (SBAQs), and 75 extended matching questions (EMQs). The VSAQs were machine-marked against pre-approved responses and subsequently reviewed by a panel of pathologists, with the software remembering all new marking judgements.ResultsThe total time taken to mark all 50 VSAQs for all 285 students was 5 hours, compared to 70 hours required to manually mark an equivalent number of questions in a paper-based pathology exam. The median percentage score for the VSAQs test (72%) was significantly lower than that of the SBAQs (80%) and EMQs (84%), p <0.0001. VSAQs had a higher Cronbach alpha (0.86) than SBAQs (0.76), and EMQs (0.77). VSAQs, SBAQs and EMQs had a mean point-biserial of 0.35, 0.30 and 0.28, respectively.ConclusionVSAQs are an acceptable, reliable and discriminatory method for assessing pathology, and may enhance students’ understanding of how pathology supports clinical decision-making and clinical care by changing learning behaviour.

Biology Teachers’ Perception of Multiple Choice Questions (MCQs) in the G.C.E Advanced Level Biology and Students’ Academic Performance in the South West Region of Cameroon

The study “Biology Teachers’ Perception of MCQs in the GCE Advanced Level Biology and Students’ Academic Performance in the South West Region” was carried out to examine the views of Advanced Level Biology teachers on the use of MCQs, an innovation in the GCE Examination since June, 2009. The study made use of the survey design. The research questions were as follows: (1) To what extent do Biology teachers master the construction of MCQs? (2) To what extent does the use of MCQs provide Biology teachers with a better means for assessing student learning than the use of short-answer structural questions? (3) How is the use of MCQs in Advanced Level Biology helpful in the attainment of instructional objectives? (4) What constraints do Biology teachers experience in constructing MCQs? (5) To what extent do Biology teachers feel adequately involved in the making of decisions with respect to the use of MCQs? The instrument used to collect the data was the questionnaire made up of 37 close-ended questions. A sample of 183 Advanced Level Biology teachers in the Fako, Manyu and Meme Divisions of the South West Region took part in the study. Out of the 183 copies of the questionnaire administered, 183 were returned, giving a percentage returned of 100%. Data were analyzed descriptively using simple and relative frequencies and percentages and also inferentially using the chi square. The results showed that Biology teachers have a positive perception of the use of MCQs. They have a mastery of the construction of MCQs, MCQs is better than short answer structural questions, MCQs at the Advanced level Biology is helpful in the attainment of instructional objectives, they do not experience constraints in constructing MCQs, and Biology Teachers partake in decision-making on matters that concern MCQs. This positive perception influences the students’ academic performance positively as seen in the steady rise of Advanced Level GCE Biology results from 2009 which is the year of inception of MCQs in the GCE Board examination to 2016. The sudden drop in the year 2017 is due to the unrest atmosphere witnessed in the English speaking Region of Cameroon as gotten from some purposively selected schools in the South West Region as shown in the appendix.

Do medical students who are multilingual have higher spatial and verbal intelligence and do they perform better in anatomy examinations?

We have already reported that medical students who have prior knowledge of classical Greek and Latin perform better in anatomy examinations. It has also been shown that fluency in more than one language can influence spatial and verbal intelligence and here we hypothesize that medical students who have linguistic skills develop higher spatial and verbal intelligence compared with monolingual students, that there are gender differences, and that there are positive effects on performance in anatomy examinations. One hundred and seventy-three second year medical students at Cardiff University responded to spatial and verbal intelligence questions that were adapted from the British MENSA website. This is a 63% response rate for the student cohort. The students were then categorized into different groups depending upon their linguistic knowledge and skills. Across all groups, no gender differences were discerned for either spatial or verbal intelligence. Students who were categorized as monolingual (with only skills in English) had lower spatial and verbal intelligence than those who were multilingual. Medical students who had fluency in English and non-European languages showed greater spatial and verbal intelligence than other groups. However, there was no significant improvement in their examination marks for anatomy, although the examination performance might be complicated by cultural considerations. A further finding from our study was that, where an anatomy test required spatial recognition using cadaveric specimens, students with low spatial intelligence had significantly poorer performances. Furthermore, where tests used multiple choice questions, the level of spatial and verbal intelligences had no influence. We would advocate that, when all newly-recruited medical students are tutored in medical terminologies to help them develop the extensive vocabulary required for their professional careers, they should also be made aware of any deficiencies in spatial and verbal skills that could affect their learning abilities. Given that we would expect students to benefit in their careers from developing spatial and verbal skills, we also recommend that examination tests in anatomy should avoid the exclusive use of multiple choice questions. Clin. Anat., 2018. © 2018 Wiley Periodicals, Inc.

Plus ça change, plus c’est pareil: Making a continued case for the use of MCQs in medical education

Despite the increased emphasis on the use of workplace-based assessment in competency-based education models, there is still an important role for the use of multiple choice questions (MCQs) in the assessment of health professionals. The challenge, however, is to ensure that MCQs are developed in a way to allow educators to derive meaningful information about examinees’ abilities. As educators’ needs for high-quality test items have evolved so has our approach to developing MCQs. This evolution has been reflected in a number of ways including: the use of different stimulus formats; the creation of novel response formats; the development of new approaches to problem conceptualization; and the incorporation of technology. The purpose of this narrative review is to provide the reader with an overview of how our understanding of the use of MCQs in the assessment of health professionals has evolved to better measure clinical reasoning and to improve both efficiency and item quality.

The Prevalence of Multiple-Choice Testing in Registered Nurse Licensure-Qualifying Nursing Education Programs in New York State.

The aim of this study was to describe nurse educators' use of multiple-choice questions (MCQs) in testing in registered nurse licensure-qualifying nursing education programs in New York State. This study was a descriptive correlational analysis of data obtained from surveying 1,559 nurse educators; 297 educators from 61 institutions responded (response rate [RR] = 19 percent), yielding a final cohort of 200. MCQs were reported to comprise a mean of 81 percent of questions on a typical test. Baccalaureate program respondents were equally likely to use MCQs as associate degree program respondents (p > .05) but were more likely to report using other methods of assessing student achievement to construct course grades (p < .01). Both groups reported little use of alternate format-type questions. Respondent educators reported substantial reliance upon the use of MCQs, corroborating the limited data quantifying the prevalence of use of MCQ tests in licensure-qualifying nursing education programs.

Suitability of Multiple-choice Questions in Evaluating the Objectives of Academic Educational Process of Accounting Specialization

This study attempt to investigate and explore the use of multiple-choice questions (MCQs) at Jordanian public and private universities, from the point view of faculty member who are teaching accounting specialization and use MCQs in examinations of accountancy subjects. The study seeks to identify the extent MCQs can achieve regarding level of knowledge among students, assess the deep understanding and In-depth learning to measure and evaluate the range of learning in subjects of accounting specialization. To achieve the objectives of the study, a questionnaire designed according to the requirements of Ministry of Education in Jordan and the International Accounting Education Standard Board (IAESB).Based on analyzing responses and examining hypotheses, results clarify that MCQs are not enough tools to achieve the educational objectives concerning accounting subjects, in both theoretically and practically aspects. It was clear that there are no significant differences between the opinions of faculty members, neither public nor private universities. The study recommends that there is a need to re-consider the use of MCQs as a tool to evaluate the students’ abilities and skills in subjects of accounting specialization.