MCQ Exam Research Articles

The Impact of Generative Artificial Intelligence on Education: A Comparative Study

Generative Artificial Intelligence (GAI), also known as creative AI, is a technology capable of independently producing original and creative content, such as text, images, videos, music, or code. It uses machine learning algorithms to analyze and learn from a vast amount of data to generate similar or even innovative content. This type of AI is used in several fields, such as health for the production of new drugs, design to generate adaptive designs, video games to create immersive environments, and artificial intelligence to generate training data for new networks. Seizing the opportunities offered by AI innovations in education is crucial. We cannot afford to let time pass without doing anything and without benefiting from this technique. It is time to enhance traditional learning methods and adopt more innovative approaches that take advantage of the potential of AI, like content and exam generation, and also improve students' learning experience by exploiting their closeness to the technology. This paper aims to answer the question of when and how to use artificial intelligence responsibly, considering it as a technology and limiting its excessive use. Additionally, we present the impact of GAI through a comparative study of student performance in three exam types: a classical MCQ exam written by the teacher, an adaptive MCQ exam generated by GAI from a topic, and a third MCQ exam generated by GAI based on the content of the provided documents. This study is conducted on students undergoing a software testing training program. By evaluating their performance in the AI-generated adaptive MCQ exams and the traditional MCQ exam, we can gain insights into the potential of this technology to enhance educational practices and provide personalized learning experiences. This research will contribute to the ongoing discussion on responsible and effective utilization of GAI in education, paving the way for future advancements in the field.

ONLINE LECTURE DURING THE COVID-19 PANDEMIC AFFECTS MEDICAL STUDENTS' LEARNING OUTCOMES BASED ON MCQ SCORE EXAMINATION

The COVID-19 pandemic has made major changes, one of which is the policy of implementing online lectures from home. This change in method can affect student learning outcomes that can be observed through the results of the MCQ exam at the end of the block. This study was conducted to determine the influence of online lectures during the COVID-19 pandemic on the results of the final MCQ test of student blocks. The retrospective cohort research design was carried out by collecting the MCQ scores at the end of the block at the time before the pandemic, the beginning of the pandemic and the sixth month of the pandemic from students of the 2017, 2018, and 2019 batches and data analysis was carried out by means of the Friedman Test, the Repeated Anova test, the Kruskal-Wallis test and the One-Way Anova. In the class of 2017, the average MCQ test results of the final block before, at the beginning, and in the sixth month of the pandemic were 50.72+8.02 consecutively; 51,39+6,8; 50.31+8.41, and there was no significant difference between them (p>0.001). Meanwhile, in other batches, a significant difference (p<0.001) was found with the results of the final MCQ exam in the 2018 batch of 45.29 + 7.87; 42.77 + 10.4 and 50.68 + 10.63 and for the class of 2019 it is 62.00 + 6.83; 46.8+10.92 and 52.32+10.02. At the beginning of the implementation of online lectures, the final MCQ score of the junior student block was most affected (p<0.001). The online lecture method affects the learning outcomes of first- and second-year students but not for third-year students.

English Language Learners’ Perception and Motivation Towards Exam Format: A Qualitative Study

The objective of this study is to explore the perceptions of students at the English department at Ibn Tofail University towards exam format, and the motivation behind their preferences. In order to achieve these objectives, a qualitative methodology was employed. The subjects of this study are sixteen undergraduate students currently enrolled at the English department at ITU, and three students who recently graduated from the same university. An open-ended questionnaire is used to glean the necessary information, and to allow the subjects to freely express their opinions and attitudes with minimal influence. The collected data is analyzed using thematic analysis. Findings show that ITU’s EFL students believe that MCQ exams cannot properly assess writing skills, or competency in certain courses. The MCQ exam preference is mainly motivated by the practicality of the format, while the written format preference is mainly motivated by the fact that it encourages deep learning, enables higher cognitive levels of thinking, and improves writing skills.

Approach to learning for assessment in economics

It is observed that in economics education research “there are few lines of research that lead to definitive conclusions” (Allgood & Schaur, 2019). This can be explained by the dominant use of the input–output model in economics education research studies. According to the Student Experience of Learning (SEL) framework, the links between learning inputs and output are mediated by students’ approaches to learning. This paper argues that to fully investigate learning in economics, the researcher should go beyond the input–output framework. While empirical studies confirm the theoretical association of approach and outcome based on the SEL framework, education researchers tend to focus on the overall approaches students utilise at the unit or course level. There is ample evidence that students utilise different learning approaches in different assessment regimes. If a study fails to investigate learning approaches students utilise in different assessment regimes, important assessment-specific relationships between approach and outcome may be missed. This paper documents the construction of an instrument for measuring learning activities in three traditional assessment regimes, viz. MCQ exam, essay assignment and essay exam in economics. The instrument was found to possess sound psychometric properties and ecological (post-dictive) validity, capable of discriminating approaches to learning for different assessments. Given the use of innovative assessments nowadays, researchers are encouraged to reword the instrument to make it more relevant to other assessment types.

Exam Questions from Essay to MCQ with Different Levels of Difficulty in Engineering Mechanics

In this paper some techniques to avoid students’ guessing of the correct answer in MCQ exams are proposed. An example from engineering mechanics course is introduced with different levels of difficulty. Random and adaptive selections of choices are introduced according to the physics and difficulty of the problem. The proposed techniques proved to be efficient in reducing guessing of correct answers based on partial or no knowledge of the subject matter. Also, a proposal of establishing online exam using pools of MCQ’s with different levels of difficulty is introduced. Five basic types of exams with different number of questions based on instructors’ desire are tabulated.

Does prior knowledge affect interaction dynamics and learning achievement in digital problem-based learning? A pilot study.

Previous research on problem-based learning (PBL) describes that videotaped observations develop meaningful insights into cognitive processes in tutorial groups. Analysis regarding the amount of prior knowledge on learning achievement has not been investigated in medical education so far, although both are key factors of PBL success. Thus, we intended to analyse videos of digital problem-based learning (dPBL) sessions, focusing on knowledge acquisition and interaction dynamics among groups with different levels of prior knowledge to reveal any distinctions. This study employed a pilot design by dividing 60 dental students into twelve subgroups with less or more prior knowledge, determined by a pre-semester multiple choice test (MCQ). The groups engaged in videotaped dPBL cases, which were examined regarding group interactions and tutor effectiveness. The learning achievement was assessed through a post-semester MCQ, an oral and practical exam. The video analysis showed that dPBL groups with less prior knowledge achieved significantly higher tutor effectiveness and group interaction utterances, but that the percentage of time in which utterances occurred was similar in both groups. Related to the MCQ results, the students with less prior knowledge learned four times more than those with profound previous abilities, but no significant difference was found in the results of the oral exam and practical exam. The interaction dynamics in dPBL depend on the group's amount of prior knowledge. Especially groups including participants with less prior knowledge seemed to benefit from dPBL in comparison to groups with more prior knowledge. The dPBL groups acquired knowledge in different ways during the courses but, finally, all students arrived at a similar level of knowledge.

Clinical Competence and Perception of Medical Students after Early Clinical Exposure through Pre-clerckship Education at an Ethiopian Medical School: A Cross-sectional Study.

Conventional medical curriculum is the mainstay in the long history of modern medical education. Innovative integrated medical curriculum attracted significant attention in improving conventional curriculum. In the integrated curriculum, basic sciences are incorporated horizontally with each other, and students are exposed early to clinical settings. This is expected to improve students' knowledge and skills in clinical medicine by the time they start their clerkship rotation. the study aims to make a baseline assessment on the overall knowledge and skills of medical students towards clinical medicine. An institution-based cross-sectional study was conducted from March to April of 2020 using 91 third year medical students (convenience sampling). A three-section self-administered survey instrument, short written MCQ exam, and practical (OSCE) students' examination were used for this survey. participants tend to exhibit better knowledge on basics of history taking and physical examination with an average score of 79%. Comparatively, the score for average physical examination skill was low (56.3%). Students' perception on ECE showed, over 50% of participants believe ECE increases burden on their overall workload. Even then, the majority (92.3%) still think that ECE has positive impact on their clerkship education. Taken together, it appears more hands-on interventions is needed to further improve skills of medical students in physical examination with particular emphasis on the clinical examination of breast, thyroid, musculoskeletal, and neurologic systems.

Making the most of ‘Clinical Optics’ by Elkington: A Brief Guide for FRCOphth Part 1

Having fortunately passed the FRCOphth part 1 in my first attempt, the aim herein is to detail exactly how I used one of the main foundational texts- ‘Clinical Optics’ by Elkington and Frank. Out of 174 multiple choice questions in the April 2022 FRCOphth Part 1 exam, 47 questions were dedicated to Optics; comprising the most number of questions of any topic. Optics generally has formed the majority of questions in previous sittings too, although since its conversion to a pure MCQ exam, there appears to be increasing emphasis in this area.

Evaluating the effectiveness of the response - after MCQ exam item analysis on students' performance in a Communication Skills exam

Evaluating the effectiveness of the response - after MCQ exam item analysis on students' performance in a Communication Skills exam

Item difficulty & item discrimination as quality indicators of physiology MCQ examinations at the Faculty of Medicine Khartoum University

Background:
 Item analysis is an essential tool used in the evaluation of the quality of MCQ examinations.
 Objectives:
 This study aimed at assessing the quality of items of physiology MCQ examinations at the
 Faculty of Medicine, Khartoum University, using post-examination item analysis.
 Methods:
 A descriptive, cross-sectional study was carried-out on test statistics reports using Remark-
 software which provides analysis of students’ responses in the form of Excel file formats. Ten physiology
 MCQ exams held from September 2015 - September 2016 were included in the study. Each exam paper
 consisted of 60-80 (five-option) items. Item Difficulty index [DIF I] and Discrimination Index (DI) (using
 point biserial correlation coefficient/
 r
 pbis
 ) were measured as quality indicators.
 Results:
 The mean DIF I of 645 items was (56.01±19.97) & the mean
 r
 pbis
 was (0.37±0.13). The majority
 of items (63.3%) were of acceptable difficulty; (25.5%) were easy & (11.1%) were difficult. The highest
 proportion (43.5%) of items were excellent; followed by (30%) good; (16.6%) acceptable and (9.6%) poor
 discriminating items. There were only 5 negatively discriminating items that constituted 0.8% out of total
 items. The relationship between difficulty and discrimination indices was dome-shaped with maximal
 discrimination at acceptable difficulty level.
 Conclusion:
 Generation of high quality examinations with average difficulty and good discrimination
 power is the outcome of careful construction and meticulous review of exams’ items by our expert staff
 members. Maximizing exam discrimination is achieved by reducing the number of the very difficult and
 very easy items.

Item Analysis and Decision Making for MCQ Exams

Background: MCQ exam analysis is an important part of the assessment cycle. We should pay more attention to item analysis to ensure test validity and reliability. We need to make decisions regarding the exam results like Mark moderation or removing questions from Question Bank. Aim of the work: 1. To highlight basic principles of MCQs item analysis and how to make maximum benefits from the data we got. 2. To make decisions regarding the exam like moderation of marks based on item analysis. Methods: An online presentation had been completed, where core principles of Item analysis had been reviewed. Then, an online survey had been completed to check the decisions of each participant when he will be faced by MCQ of aberrant discrimination or difficulty. Results:An online survey had been completed about the decision that each participant will take in different situations [e.g., low discrimination questions, or high difficulty questions]. Answers were reviewed and decision-making was clarified. As feedback, 80.0% of attendants reported that, they will increase their participation in the question bank-building process at Damietta Faculty of Medicine, and 65% requested more in-depth future events. Conclusion: Post examination item analysis is crucial for building a high-quality Question bank. Also, it supports informed decisions regarding the moderation of marks in MCQ exams.

Enhancement of students learning outcome by supplementing Self-directed learning with video clipping.

Objectives: Self-directed learning is an active learning strategy in which students are accountable for their own learning by taking initiative. It has been supplemented with other instructional strategies like interactive lecturing, in order to enhance student’s learning. The current study was carried out to determine the effect of supplementing self-directed learning with video clipping on students learning outcome. Study Design: Randomized Control study. Setting: Islamic International Medical College. Period: For one day at 16th September 2016. Material & Methods: This study was conducted among seventy six students of Islamic International Medical College. Students participated in a quiz competition whose syllabus of content and suggested reading materials were communicated to the students three week prior to the competition date. The registered students were than randomly divided into two Groups each comprising of 38 students. Self-directed learning of one group was supplemented with a video while the other group did not watch the video. Learning outcome of both groups was assessed by an MCQ exam. Results: The overall score of Self-directed learning group supplemented with a video was 65.17(7.68) while the score of self-directed learning group was 59.54 (8.54). A significant difference were found among the score of both groups (p = .042). Conclusion: Self-directed learning when supplemented with the instructional strategy of video clipping leads to the improvement in the learning outcome of the students.

The Reality of Using Virtual Reality: Understanding How Undergraduate Students use a Virtual Bell Ringer App to Study Anatomy

IntroductionWith the increasing number of online resources for anatomical education available to students, understanding why students use one resource over another is crucial for resource design. The benefit of using stereoscopic images in anatomical education has recently been demonstrated (Remmele et al, 2018; Cui et al, 2017). However, few resources utilize stereopsis in depicting anatomical dissections. Using images from the Stereoscopic Atlas of Human Anatomy (D.L. Bassett & W.B. Gruber, Stanford University, 1962), we developed a smartphone application that uses the Google cardboard platform to visualize stereoscopic images in an inexpensive and accessible manner. The app was implemented in a second year undergraduate anatomy and physiology course with 975 students. The app was designed to incorporate virtual pins in the 3D space that enabled it to be used as a self‐evaluated Objective Structured Practical Exam (OSPE) called the Virtual Reality Bell Ringer (VRBR). Questions and answers accompanying the images were provided on the online course management system. This allowed the use of the app to be correlated with course evaluation performance. The PURPOSE of this study was to track use of the app by students, get feedback to develop better online resources, and correlate the app use with final scores.Methods67 OSPE practice questions using stereo pairs from the Bassett collection were selected, 30 of which pertained to the first semester: 5 questions for each of 6 bi‐weekly labs. Questions related to each sets of images were available throughout the semester and students had unlimited attempts to successfully answer the questions. Upon submitting their response, students would see their answer, the correct answer, and the rationale behind the answer. Student participation and correlation between their VRBR score and multiple choice (MCQ) midterm exam were evaluated.ResultsBy mid‐semester, 89 of 975 students attempted the VRBR practice questions. Of those who participated in the VRBR quiz, 93% achieved above average scores on their MCQ midterm exam. Correlation of midterm scores with successful VRBR question responses for the lowest, middle, and highest third percentile‐scoring students resulted in Pearson correlation coefficients (r) of 0.39, 0.5, and 0.78, respectively. This indicates that there is a good correlation between success on the VRBR practice questions and success on different types of course evaluations. Although 91% of students did not use the app to prepare for the midterm MCQ exam, a qualitative mid‐semester survey of app use revealed that 51% planned to complete the VRBR questions prior to the final OSPE exam. Only 13% of students reported that they feel that the app would not prepare them for the final exam.ConclusionDespite less than 10% of students using the app at mid‐semester, a large proportion planned to use the app to prepare for the final exam. There was a correlation between success on the midterm MCQ and VRBR quizzes, however, this is probably correlative rather than causative. Evaluation of VRBR app use on the outcome of the final OSPE exam will occur at the end of the first and second semesters.This abstract is from the Experimental Biology 2019 Meeting. There is no full text article associated with this abstract published in The FASEB Journal.

CASE-BASED LEARNING MENINGKATKAN CLINICAL REASONING PADA BLOK PENYAKIT INFEKSI TROPIS MAHASISWA FAKULTAS KEDOKTERAN UNIVERSITAS ISLAM INDONESIA

Case-based learning (CBL) is a newer modality of teaching healthcare, combining medical theory and practices. Many researchs reported effectivity of CBL to improve student-centered learning. The aim of this study is to evaluate how CBL improved students’ clinical reasoning compared with other method such as problem-based learning. An interventional research involved 4th grade medical students receiving PBL and CBL methods supervised by a tutor or instructure. An alternative method combining CBL and structured role play (in an Integrated-Patient Management method) also performed using clinical scenario with medical practice setting. The result showed that student received CBL had better clinical reasoning indicated from better miniquizz result compared with PBL students (p < 0,05, Mann Whitney U test. Students exposed with CBL had final MCQ and essay exam grade compared with unexposed students. Students exposed with CBL also had better clinical skill indicated from Objective Structure Clinical Examination (OSCE) result (p < 0,05, independent t test). Students’ acceptance level on CBL method is 79, 4%. It is concluded that compared with PBL, CBL is more effective in improved students’ performance, learning outcomes and clinical reasoning and also had high acceptance among medical students.

P.085 Hot seat concept in neurosurgical exam simulation adopted by the Comprehensive Clinical Neurosurgery Review

Background: Neurosurgical education is one of the most exciting topics in contemporary neurosurgery. Passing the final boards is a real challenge. Methods: We conducted a prospective study of 48 candidates who attended the hot-seat sessions during CCN review over three years. Detailed statistical analysis was conducted. Those who attended the Hot seats (Group 1) and those who didn’t (Group 2). The neurosurgery exam simulation was conducted using both MCQ and Oral simulated exams with clinical cases led by world expert faculty in a lecture format for the MCQ and 15-minute mock oral sessions which was video-taped scoring candidates in a standardized fashion for their performance. Results: Group 1 had a better MCQ performance (83 %) compared to group 2 (61 %). Candidates were better in data gathering, differential diagnosis and management. They were worst in simulating surgical techniques and follow-up plans. Geographical characterization showed a big range of intra and inter variability in performances. Interestingly, candidates with excellent MCQ performance had moderate hot seat performance while those with moderate MCQ performance did much better during the hot seat session. Conclusions: Our preliminary results showed that simulation of board exams is a method that will help neurosurgery residents not only pass their board exams, but also achieve the best marks.

MULTI-CHOICE QUESTIONS AND THEIR PROBLEMS WHEN USED FOR ASSESSMENT OF AIRCRAFT ENGINEERS EDUCATION

Licensed aircraft engineers under the European Aviation Safety Agency, EASA, undertake academic training tocomplement their practical and type specific studies. These exams are mainly Multi-Choice Questions, MCQ, and four20-minute essays. The MCQ exams are as few as 16 questions to a maximum of 140 questions. A score of 75% is neededto pass each exam, and each question has three possible answers. This authors of this paper reviews the theory and designof the MCQ and asks if the assumptions are valid andthat it achieves the academic level assumed for engineers who willbe maintaining some of the most complex system in the world, and the safety of passengers. It will argue that there are failings and how this can be address, in particular, that repeated tests should have a higher pass level.

Case Based Metabolic Unit (CBMU): A Model for Better Understanding of Metabolic Pathways in the Second Year Extended Modular Program Medical Students

Projetamos uma unidade metabólicas baseada em casos (CBMU), o qual consiste em um modelo de aprendizagem híbrido usando aprendisagem baseada em caso e aulas interativas para estudantes do segundo ano de medicina (n:113) que realizaram o módulo “sangue” (modelo clássico de aprendizagem de metabolismo) e do módulo lomomotor (CBMU). Ao final de ambos módulos, avaliamos os estudantes por meio do exame MCQ e comparamos os resultados de ambos exames. A maioria dos estudantes (97-100%) opinaram que o modelo de aprendizagem baseada em casos da unidade metabólica foi interessante, moticadora e melhor que o modelo clássico na compreensão das vias metabólicas e tornando a aprendizagem efetiva. 99% dos respondentes demandaram a aplicação deste modelo de aprendizagem em futuros cursos de Bioquímica. A melhoria nas pontuações nas provas pelos estudantes assegura que eles obtiveram o conhecimento por meio deste modelo. Em conclusão, CBMU é uma ferramenta útil par aprendizagem das vias metabólicas usando casos humanos para auxiliar na conexão de teoria com a prática que é positivamente refletida no desempenho acadêmico dos estudates.

Validitas dan Reliabilitas Ujian SOCA (Students Oral Case Analysis): Studi di Salah Satu Fakultas Kedokteran di Indonesia

Background: Medical faculty has to make sure that the students meet the minimal competence needed using apropriate exam. While the exam itself should facilitate students to learn. Oral examination has known for its ability to facilitate students learn but low in validity and reliability. Medical faculty of Lampung University (FK Unila) apply the student oral case analysis (SOCA) exam as one of block assessment component, as with MCQ, tutorial, and laboratory exam. This study aimed to evaluate validity and reliability of SOCA examination at FK UnilaMethod: Video of 65 students doing SOCA examination and 28 question rubrics had taken when odd semester exam year 2014-2015 has been carying out at FK Unila. Video and question rubrics were assessed by 5 panelis and analysed using Lawshe's content validity ratio (CVR) to determinate its content validity. Students performance on the video were re-assessed by another assessor to see inter-rater reliability, than analysed using kappa Cohen. Two expert in medical education assessed the cognitive comlpexity of the question rubrics. Data of SOCA's student's mark from year II, III, and IV were analysed for construct valdity and internal consistency.Results: 93,7% of the overall question in 65 video were valid (CVR>99%) and 71,8% question number in 28 question rubrics also valid according to 5 panelis. SOCA cognitive complexity were at level of analyse, know how and 4a. Inter-rater reliability analysis showed 0,549 (moderate agreement) kappa value. Mann Whitney analysis for construct validity showed no significant difference of all year. Cronbach alpha analysis showed internal consistency at the point 0,575.Conclusion: FK Unila's SOCA of odd semester examination year 2014-2015 has sufficient content validity, sufficient cognitive complexity and sufficent inter-rater reliability but lack in construct validity and internal consistency. Keywords: SOCA, validity, reliability

Evaluation of pharmacology teaching-learning methods in a government medical college

Background: Pharmacology is a major subject in medical science and always changing, so according to it pharmacologists also need to reform their teaching method. Students’ opinion in this evaluation of teaching is a good option. Methods: A questionnaire based study was conducted in Burdwan Medical College on 230 students of two batches. Results: One forty eight students thought that the subject was interesting. Twenty eight percent students opined that Pharmacology class was not interesting. Students found interest mostly in prescription writing, drug interaction and therapeutic problems in practical section. Though 53.4% found pharmacy was less interest. They suggested some reforms in teaching, 61.3% voted for introduction of audio-visual aid and 74.3% response on clinical demonstration in hospital ward. According to 82.6% students, there should be more problem solving exercise during teaching pharmacology; giving hand-out before everyday class, class in clinical setting and more emphasis on clinical pharmacology were also needed. MCQ exam only (31%) and written viva both (51%) were good ways of evaluation according to them. Conclusions: This study helps in knowing students’ opinion and preferences in curriculum to make it more interesting and aids to modify the undergraduate Pharmacology teaching pattern.

The perfect MCQ exam

Aims: The Saudi Commission for Health Specialties' question banks for licensing international medical graduates and certifying Saudi trained residents are currently being upgraded. The process is briefly explained with the justification for the developments made to all the banks over the last 3 years. A process of quality assurance has been introduced to ensure the banks are maintained at the highest standards and procedures that are fit for purpose introduced for the management of test results. Results: An analysis of 16 undergraduate exams shows a sigmoid relationship between mean test scores (MTS) and numbers of functioning distractors. Sixteen computer-based Saudi Licencing Exam (SLE) banks were also analyzed to determine the mean MTS for all, 52.3 ± 7.88, which justifies the use of four option items and also validates the quality of the exams with a majority of items falling within the desired range of medium difficulty to obtain the highest reliability. Conclusion: The Saudi Commission believes that it is achieving increased quality in all aspects of test measurement and would hope that over the next 3 - 5 years achieve levels of excellence for all aspects of it licensing and certifying assessments.