Multiple Choice Question Assessment Research Articles

Efficacy of Faculty Development Training Workshops (FDTWs) on Writing High-Quality Multiple-Choice Questions at Northern Border University (NBU) in the Kingdom of Saudi Arabia (KSA).

A multiple-choice question (MCQ) is a frequently used assessment tool in medical education for both certification and competitive examinations. Well-constructed MCQs impact the utility of the assessment and, thus, the fate of the examinee. To analyze the basic science faculty perceptions of writing high-quality MCQs, to create awareness of item-writing flaws in constructing high-quality MCQs, and to determine the impact of faculty development training workshops (FDTWs) on MCQ writing skills. An online workshop was held over two weeks for basic science faculty to learn high-quality MCQ construction. Faculty-made MCQs were analyzed for flaws, and a questionnaire assessed the impact of the workshop on MCQ construction. Pre- and post-workshop responses were compared to evaluate the necessity of such workshops for improving faculty skills in MCQ assessments. A total of 47 (83.2%) of participating faculty believed the workshop could reduce MCQ construction errors. The participants agreed that a series of workshops were needed for lasting improvements in MCQ construction. One-day short-duration workshops, such as the current one alone, cannot achieve the objectives of training participants to write high-quality MCQs. To improve student assessment through high-quality MCQs, the faculty needs to be exposed to continuous and frequent sessions that will help them.

Evaluating the Effectiveness of Online Assessments and Their Parameters as Predictors of Academic Performance in Undergraduate Medical Education.

Background and objectives Considering the increasing utilization of online educational tools in medical education, it is essential to evaluate the reliability and validity of online assessments to accurately assess student proficiency and predict academic success. This study investigated the predictive efficacy of different online assessment methods in comparison to standard offline methods within the medical educational setting. Methods This study utilized a mixed-methods crossover design, involving 125 first-year medical students who were randomly assigned to either online or traditional examinations. The students then crossed over to the other type of assessment. The assessments consisted of multiple-choice questions (MCQs), viva voce, objective structured clinical examinations, and written theory examinations. Quantitative data on results, finishing times, and academic metrics were analyzed, along with qualitative data from student interviews exploring perceptions of each format. Results The online MCQs had the highest average scores and a moderately positive correlation with performance on the theory examination (r=0.326). Regression models indicated that online and offline MCQs were moderate positive predictors of theoretical marks (R2=0.106 and 0.107, respectively). Qualitative responses emphasized advantages such as flexibility and accessibility for online examinations but also concerns regarding technological challenges, interaction, and integrity compared to traditional formats. Conclusions Online MCQ assessments showed promise as indicators of medical student academic performance. However, additional online forms require improvement to match conventional assessments reliably. As medical education involves digital technologies, cautious implementation of online evaluations substantiated by further research is needed to preserve educational quality standards.

Giving partial credit during a multiple-choice question assessment reappraisal does not make the assessment process fairer.

Giving partial credit during a multiple-choice question assessment reappraisal does not make the assessment process fairer.

The impact of systematically repairing multiple choice questions with low discrimination on assessment reliability: an interrupted time series analysis

At our centre, we introduced a continuous quality improvement (CQI) initiative during academic year 2018-19 targeting for repair multiple choice question (MCQ) items with discrimination index (D) < 0.1. The purpose of this study was to assess the impact of this initiative on reliability/internal consistency of our assessments. Our participants were medical students during academic years 2015-16 to 2020-21 and our data were summative MCQ assessments during this time. Since the goal was to systematically review and improve summative assessments in our undergraduate program on an ongoing basis, we used interrupted time series analysis to assess the impact on reliability. Between 2015-16 and 2017-18 there was a significant negative trend in the mean alpha coefficient for MCQ exams (regression coefficient -0.027 [-0.008, -0.047], p = 0.024). In the academic year following the introduction of our initiative (2018-19) there was a significant increase in the mean alpha coefficient (regression coefficient 0.113 [0.063, 0.163], p = 0.010) which was then followed by a significant positive post-intervention trend (regression coefficient 0.056 [0.037, 0.075], p = 0.006). In conclusion, our CQI intervention resulted in an immediate and progressive improvement reliability of our MCQ assessments.

Video training of nursing home healthcare workers in palliative care

ObjectivesThe provision of palliative care in nursing homes (NHs) is of paramount importance, a realism underscored by the frailty and medical complexity of the residents. However, palliative care (PC) education...

ChatGPT and academia on accounting assessments

ChatGPT is considered a risk and an opportunity for academia. An area of threat in contemporary settings is whether it can become a student agent for assessments in academia. This study determines how ChatGPT can become a human agent for students on two financial accounting course units, multiple choice question assessments. The study provided five numerical-based and five narrative-based multiple choice questions. There were ten questions for the Introductory Financial Accounting and 10 for the Advanced Financial Accounting course units. ChatGPT received one question at a time requesting a solution. In the Introductory Financial Accounting section, ChatGPT produced incorrect answers because it incorrectly assumed the underlying assumptions contained in those questions. In Advanced Financial Accounting, ChatGPT presented incorrect answers because of the complexity of the task contained in those questions. ChatGPT demonstrated similar competencies in providing solutions to numerical-based and narrative-based questions. ChatGPT obtained the correct answers to sit in the 80th percentile in the Introductory Financial Accounting course unit assessment and the 50th percentile in the Advanced Financial course unit assessment. ChatGPT4 showed improved performance, with the 90th percentile for Introductory Financial Accounting and the 70th percentile for Advanced Financial Accounting. The findings indicate that the knowledge construct requires reflective thinking with ChatGPT in the ecosystem, and what is assumed and assessable knowledge must be revisited.

Associations between item characteristics and statistical performance for paediatric medical student multiple choice assessments.

Background: Multiple choice questions (MCQs) are commonly used in medical student assessments but often prepared by clinicians without formal education qualifications. This study aimed to inform the question writing process by investigating the association between MCQ characteristics and commonly used statistical measures of individual item quality for a paediatric medical term. Methods: Item characteristics and statistics for five consecutive annual barrier paediatric medical student assessments (each n=60 items) were examined retrospectively. Items were characterised according to format (single best answer vs. extended matching); stem and option length; vignette presence and whether required to answer the question, inclusion of images/tables; clinical skill assessed; paediatric speciality; clinical relevance/applicability; Bloom's taxonomy domain and item flaws. For each item, we recorded the facility (proportion of students answering correctly) and point biserial (discrimination). Results: Item characteristics significantly positively correlated (p<0.05) with facility were relevant vignette, diagnosis or application items, longer stem length and higher clinical relevance. Recall items (e.g., epidemiology items) were associated with lower facility. Characteristics significantly correlated with higher discrimination were extended matching question (EMQ) format, longer options, diagnostic and subspeciality items. Variation in item characteristics did not predict variation in the facility or point biserial (less than 10% variation explained). Conclusions: Our research supports the use of longer items, relevant vignettes, clinically-relevant content, EMQs and diagnostic items for optimising paediatric MCQ assessment quality. Variation in item characteristics explains a small amount of the observed variation in statistical measures of MCQ quality, highlighting the importance of clinical expertise in writing high quality assessments.

Testing anatomy: Dissecting spatial and non-spatial knowledge in multiple-choice question assessment.

Limited research has been conducted on the spatial ability of veterinary students and how this is evaluated within anatomy assessments. This study describes the creation and evaluation of a split design multiple-choice question (MCQ) assessment (totaling 30 questions divided into 15 non-spatial MCQs and 15 spatial MCQs). Two cohorts were tested, one cohort received a 2D teaching method in the academic year 2014/15 (male = 15/108, female 93/108), and the second a 3D teaching method in the academic year 2015/16 (male 14/98, female 84/98). The evaluation of the MCQ demonstrated strong reliability (KR-20 = 0.71 2D and 0.63 3D) meaning the MCQ consistently tests the same construct. Factor analysis of the MCQ provides evidence of validity of the split design of the assessment (RR = 1.11, p = 0.013). Neither cohort outperformed on the non-spatial questions (p > 0.05), however, the 3D cohort performed statistically significantly higher on the spatial questions (p = 0.013). The results of this research support the design of a new anatomy assessment aimed at testing both anatomy knowledge and the problem-solving aspects of anatomical spatial ability. Furthermore, a 3D teaching method was shown to increase students' performance on anatomy questions testing spatial ability.

A randomised control trial study on the efficacy of high-fidelity simulation in enhancing knowledge

Introduction: The practice of high-fidelity simulation-based medical education has become a popular small-group teaching modality across all spheres of clinical medicine. High-fidelity simulation (HFS) is now being increasingly used in the context of undergraduate medical education, but its superiority over traditional teaching methods is still not established. The main objective of this study was to analyse the effectiveness of HFS-based teaching over video-assisted lecture (VAL)-based teaching in the enhancement of knowledge for the management of tension pneumothorax among undergraduate medical students. Methods: A cohort of 111 final-year undergraduate medical students were randomised for this study. The efficacy of HFS-based teaching (intervention group) and VAL-based teaching (control group), on the acquisition of knowledge, was assessed by single-best answer multiple choice questions (MCQ) tests in the first and eighth week of their surgery posting. Mean and standard deviation (SD) for the total score of MCQ assessments were used as outcome measures. ANCOVA was used to determine the difference in post-test MCQ marks between groups. The intragroup comparison of the pre-test and post-test MCQ scores was done by using paired t-test. The P-value was set at 0.05. Results: The mean of post-test MCQ scores were significantly higher than the mean of pre-test MCQ scores in both groups. The mean pre-test and post-test MCQ scores in the intervention group were slightly more than those of the control group but not statistically significant. Conclusion: There was a statistically significant enhancement of knowledge in both groups but the difference in knowledge enhancement between the groups was insignificant.

Project CARPE Diem: Curriculum and Assessment for Residents on Pediatric Epilepsy

BackgroundPediatric residencies are not preparing trainees well to manage patients with seizures and epilepsy. To address this, we implemented a six-session curriculum using active learning techniques with the goals of improving the knowledge and attitudes needed to effectively identify and treat seizures and epilepsy, and increase residents’ comfort with counseling families about these topics. MethodsA structured curriculum was implemented over the course of one month for the pediatric resident learner using andragogical methods such as didactic presentation, small-group sessions, role play, and simulation. A 15 multiple-choice question (MCQ) assessment and 13-statement attitudes inventory (AI) using Likert scales were given before the start of the curriculum and after its completion. Pretest and post-test scores were compared for the residents who attended at least three of the six sessions. ResultsFifty residents completed the pretest, 34 residents completed the post-test, and 24 residents completed both. Of the 24, only 19 residents attended at least half the sessions. There was a significant improvement in scores for the MCQ assessment, for each grouped AI domain, and in the proportion of residents who reported comfort with 11 of 13 AI statements. ConclusionThe findings of this study demonstrate that given quality high-impact educational content, pediatric residents’ knowledge about seizures and epilepsy improves significantly. It is feasible to implement active learning techniques during structured didactic time. The importance of closing the knowledge gap and improving pediatric comfort with seizures and epilepsy is essential and may translate to the much needed improvement in access to care for the patients with these disorders.

The Development and Deployment of an Online Exam System: A Web Application

The rapid advancements in computer technology and the internet’s acceptance in every aspect of our lives, particularly in recent years, have made students and instructors vital in the teaching and learning sector. Web-based studies have also brought about advances in the education area, and numerous applications have become widespread in this field. In this paper, we suggested an online test multiple-choice question assessment system for students called the Online Exam System (OES). This system may be used by any university, college, or institution that has a computerized education system. The OES can be used by teachers to administer quizzes. The system will calculate the participant’s performance based on his response, and the following question will be created based on the participant’s performance. After the examination, the system will display the results and offer feedback based on the participant’s request. Administrative control over the entire system is available. A teacher has authority over the question bank and is responsible for creating test schedules. Therefore, the project will be very helpful for the beginner and mid-level programming learners. And also, will give a proper guideline to the students who are willing to learn programming and introduce the users with competitive programming and problem-solving skills.

Evaluating a dental sleep apnea mini-residency program using the Kirkpatrick model.

The aim of this retrospective study was to evaluate a Dental Sleep Medicine Mini-Residency (DSMMR) continuing education (CE) program using the Kirkpatrick model. After receiving ethical approval, data from participants in the 2019-2020 DSMMR CE course were included for the Kirkpatrick evaluation. The analysis was stratified and all the Kirkpatrick levels were integrated: level 1 (satisfaction) was assessed via Likert scale and open-ended questions; level 2 (learning) was evaluated using pretest and posttest knowledge data following Module 1 (M1) and an assessment of multiple-choice questions (MCQs) developed by participants; level 3 (behavior) was evaluated using Likert scale questions; and level 4 (results) was assessed via the percentage of participants who passed the American Board of Dental Sleep Medicine (ABDSM) examination on their first attempt. A total of 90 participants were included in the study. At least 83.1% of participants agreed/strongly agreed with positively worded statements about satisfaction. Knowledge scores significantly increased from pre-M1 to post-M1 (p<0.001); however, only 15.2% of MCQs were evaluated as well-formulated. At least 88.6% of participants agreed/strongly agreed with positively worded statements about transfer of knowledge/skills to their practice. 91.1% passed the ABDSM examination on their first attempt. The evaluation of the 2019-2020 DSMMR using the Kirkpatrick model suggests its overall positive impact as a training program. The Kirkpatrick model provided information that can be used to improve the quality of a program. Future studies should assess other dental CE programs using the Kirkpatrick model or another evaluation model.

Need for Training in Research Methodology Prior to Conducting Systematic Reviews and Meta-Analyses, and the Effectiveness of an Online Training Program: The Global Andrology Forum Model.

Systematic reviews and meta-analyses (SRMAs) are used to generate evidence-based guidelines. Although the number of SRMAs published in the literature has increased dramatically in the last decade, the training and the experience of the researchers performing the SRMAs are usually not explained in the SRMAs' methodology, and this may be a source of bias. Although some studies pointed out the need for quality control of SRMAs and training in proper statistical methods, to the best of our knowledge, no study has reported the importance of training the researchers that conduct the SRMAs. The aim of this study is to describe a training program designed to impart the essential knowledge and skills required for the conduct of an SRMA and to assess the need for, and outcome of, such a training. Researchers were trained for use of Scopus, study eligibility, assessment of the quality of evidence (QoE) through the Cambridge Quality Checklist for observational studies, as well as the Cochrane Risk of Bias tool, the Consolidated Standards of Reporting Trials (CONSORT) guidelines, and the Jadad score for randomized controlled trials (RCTs), Population, Intervention, Comparison, Outcome (PICO) questions and data extraction. A total of 35 of them were approved to join a planned SRMA. At the end of the SRMA, they were administered 43 multiple-choice questions (MCQs) on demographics, motivation for participation in the SRMA, self-perceived change in knowledge before and after conducting the SRMA, and self-assessment of performance. The senior researchers then revised the spreadsheet of the SRMA and, based on the mistakes found, organized a training focused on the correct assessment of the study design, where 43 researchers (9 joined midway) and 11 trainees with no experience in conducting SRMA attended. They all were tested through a 5 MCQ assessment that was administered before and after the training. Those scoring poorly were re-trained and re-tested, and only those scoring satisfactorily were admitted to further SRMAs. Approximately 54.3% of the participants were medical doctors (MD), 31.4% were urologists and 48.6% had previous experience with SRMAs. Joining an international collaborative study was the main motivation, chosen by 19.7% of researchers. The results of the self-perceived change in knowledge showed a significant improvement in the use of Scopus, checklists for QoE, PICO questions, data required to perform a meta-analysis, and critical reading of scientific articles. Also, the majority of the researchers ranked the quality of their work as high. The pre-test results of the 5 MCQ showed a low score, which was not different from that achieved by a group of fresh trainees (median, 2; IQR 1-3 vs. median, 1; IQR, 1-2; p=0.3). Post-training there was significant improvement in both groups (researchers: median, 4; IQR, 3-5 vs. median, 2; IQR, 1-3; p<0.001; trainees: median, 4; IQR, 3-4 vs. median, 1; IQR, 1-2; p=0.02). Out of the 44 researchers, 12 (27.3%) scored poorly (≤3). After re-training, all of them scored satisfactorily (>3) and were admitted to subsequent SRMAs. At the end of our model, 100% of researchers participating in this study were validated to be included in a meta-analysis. This validation required the involvement of the MT, two meetings, a self-evaluation survey, and one or two sets of objective tests with explanations and corrections. Our results indicate that even well-trained clinicians are naïve when it comes to the methodology of SRMA. All the researchers performing an SRMA need comprehensive training that must cover each aspect of the SRMA methodology. This paper provides a replicable training program that could be used by other investigators to train the researchers to perform high-quality SRMAs.

Assessment of multiple-choice questions by item analysis for medical students’ examinations

Background: Multiple-choice questions (MCQs) are a common assessment method, and it is crucial to design them carefully. Therefore, this study aimed to determine the item analysis of MCQ exams in clerkship tests for general medicine students. Methods: Following a cross-sectional study, a total of 1202 MCQs designed for fourth-year clerkship medical students in the second semester of 2019 were analyzed. Difficulty and discrimination indices of student scores and taxonomy levels were then computed. Furthermore, the prepared standard structural Millman checklist was utilized. Results: Of the 1202 MCQs, according to difficulty indices, most questions (666) were considered acceptable (55.39%). In terms of the discrimination index (DI), 530 (44.09%) questions had an average discrimination coefficient. Additionally, 215 (17.88%) had a negative or poor DI and required revision or elimination from the tests bank. Of the 1202 MCQs, 669 (50.7 %) were designed at a lower cognitive level (taxonomy I), 174 (14.5 %) belonged to taxonomy II, and 419 (34.8%) of the questions had taxonomy III. Moreover, according to the structural flaws of the Millman checklist, the most common structural flaw was a lack of negative choices for Stems 1127 (93.8 %), while vertical options 376 (31.3%) were the least common. Conclusion: Based on the results, it is recommended that easy questions and negative/poor DI of items, a high level of Bloom’s taxonomy type I, and questions with unstructured flaws be reviewed and reconstructed to improve the quality of the question banks. Holding training courses on designing test questions could effectively improve the quality of the questions.

Pediatric Chronic Tracheostomy Care: An Evaluation of an Innovative Competency-Based Education Program for Community Health Care Providers

ObjectiveTo evaluate the immediate and sustained knowledge retention and sense of self-efficacy of homecare nurses following completion of a standardized competency-based tracheostomy education course. Safe discharge of children requiring tracheostomy with or without ventilation relies on the competence of homecare nurses.Study DesignPragmatic, randomized controlled trial of 44 homecare nurses. Participants were randomized into the intervention group (n = 21), which received the tracheostomy course, or the control group (n = 23), which received an enterostomy and vascular access course. Multiple-choice question (MCQ) knowledge assessments and self-efficacy questionnaires were administered to both groups pre-course and post-course at 6 week, 3 month, 6 month, and 12 month follow-ups.ResultsTwenty participants in the intervention group and 19 in the control group were included. Four withdrew from the study and two crossed over from the control into the intervention arm. The change in mean self-efficacy scores (total score = 100) was significantly higher in the intervention group than in the control group at 6 weeks (intervention (mean ± SD): 18.6 ± 14.5; control: 6.6 ± 20.4; p = 0.04) and 3 months (intervention: 19.6 ± 14.2; control: 5.2 ± 17.0; p = 0.007), and trended higher at 6 months (intervention: 18.0 ± 14.5; control: 6.9 ± 24.1; p = 0.1). The change in mean MCQ assessment scores (total score = 20) trended higher in the intervention group than in the control group at 6 weeks (intervention (mean ± SD): 1.8 ± 2.2; control: 1.6, ± 2.9; p = 0.8).ConclusionsHomecare nurses who attended the tracheostomy course demonstrated a higher sense of self-efficacy at long-term follow-up.Clinical Trial Registrationwww.ClinicalTrials.gov, identifier: NCT04559932.

Comparison of Traditional Lecture-Based Learning versus Interactive Electronic Book Learning in Veterinary Student Comprehension of Inhalant Anesthetic Administration, Uptake, and Elimination.

The administration, uptake, and elimination of inhalant anesthetics is a challenging topic in the veterinary curriculum, and lecture-based learning is often insufficient to ensure that students understand these concepts. We hypothesized that the use of an interactive electronic book (e-book) would enhance student comprehension of the material. Two sequential Doctorate of Veterinary Medicine student cohorts participated in a prospective controlled study. The first cohort received traditional lecture-based learning while the second cohort was taught the topic using an interactive e-book. Student comprehension of the material was assessed twice during the course via multiple-choice questions: five questions in a midcourse quiz and seven within the final exam. At the end of the course, students also completed a Likert survey assessing their confidence regarding the topic. Averaged across assessment types, students taught using the interactive e-book scored higher than those taught via the traditional method (p < .001). Final exam scores were significantly higher in the e-book cohort compared with the lecture-based cohort (p < .001). However, there was no difference in quiz scores between groups (p = .109). No significant difference was found between groups in responses to the Likert survey. In conclusion, students using the interactive e-book had better comprehension of the material than students in the traditional lecture group as measured by their scores on multiple-choice question assessments. Future studies are needed to determine whether this advantage persists later in the curriculum when students apply these concepts in the clinical year.

I-Quiz: An Intelligent Assessment Tool for Non-Verbal Behaviour Detection

Electronic learning (e-learning) has become one of the widely used modes of pedagogy in higher education today due to the convenience and flexibility offered in comparison to traditional learning activities. Advancements in Information and Communication Technology have eased learner connectivity online and enabled access to an extensive range of learning materials on the World Wide Web. Post covid-19 pandemic, online learning has become the most essential and inevitable medium of learning in primary, secondary and higher education. In recent times, Massive Open Online Courses (MOOCs) have transformed the current education strategy by offering a technology-rich and flexible form of online learning. A key component to assess the learner’s progress and effectiveness of online teaching is the Multiple Choice Question (MCQ) assessment in most of the MOOC courses. Uncertainty exists on the reliability and validity of the assessment component as it raises a qualm whether the real knowledge acquisition level reflects upon the assessment score. This is due to the possibility of random and smart guesses, learners can attempt, as MCQ assessments are more vulnerable than essay type assessments. This paper presents the architecture, development, evaluation of the I-Quiz system, an intelligent assessment tool, which captures and analyses both the implicit and explicit non-verbal behaviour of learner and provides insights about the learner’s real knowledge acquisition level. The I-Quiz system uses an innovative way to analyse the learner non-verbal behaviour and trains the agent using machine learning techniques. The intelligent agent in the system evaluates and predicts the real knowledge acquisition level of learners. A total of 500 undergraduate engineering students were asked to attend an on-Screen MCQ assessment test using the I-Quiz system comprising 20 multiple choice questions related to advanced C programming. The non-verbal behaviour of the learner is recorded using a front-facing camera during the entire assessment period. The resultant dataset of non-verbal behaviour and question-answer scores is used to train the random forest classifier model to predict the real knowledge acquisition level of the learner. The trained model after hyperparameter tuning and cross validation achieved a normalized prediction accuracy of 85.68%.

Assessing Novice Preservice Physics Teachers’ Conceptual Knowledge of Mechanical Waves

Current research in physics shows that undergraduate students have poor conceptual knowledge of waves and optics. This research seeks to assess novice preservice physics teachers’ conceptual understanding of mechanical waves using qualitative and quantitative approaches. Year one semester two (Y1,2) preservice physics teachers in a university in Uganda were exposed to a standardized multiple choice question (MCQ) assessment test- Mechanical Waves Conceptual Survey 2 (MWCS2). The descriptive quantitative analysis of novice preservice physics teachers’(NPPT) responses to the MCQ assessment test was taken beyond the conventional responses of assessing the correct options by considering how their responses to the multiple choice were distributed. Interpretive qualitative analysis was used to interpret the responses to each question in order to determine what informed NPPT responses to each question. The interpretation of both qualitative and quantitative data was combined to make sense of scientific conceptions, alternative conceptions, and misconceptions upheld by the preservice teachers. The findings and the distributions showed that novice preservice physics teachers’ conceptual understanding and knowledge of mechanical waves are poor. Implications for teaching and learning mechanical waves amongst NPPT in view of their future professional practice were identified

Effectiveness of Innovation Basic Life Support Training Devices to Layperson: A Randomized Controlled Trial

Objectives: The use of a cardiopulmonary resuscitation feedback device and automated external defibrillator trainer is beneficial in basic life support (BLS) training. Nevertheless, Thailand lacks these devices in BLS support training. This study aimed to compare the efficacy of the Chiang Mai BLS training devices with conventional training devices in BLS training for laypeople.Material and Methods: A randomized controlled trial was conducted to compare the efficacy of the Chiang Mai device group with the conventional device group, by assessing the theory and practical examination scores of the participants; who were adult, laypeople attending the BLS provider course endorsed by the Thai Resuscitation Council. Evaluating instructors were blinded from both groups of participants.Results: A total of 60 adult, laypeople participants were divided into two groups: 32 and 28 participants of the Chiang Mai device group and conventional device group, respectively. Overall examination scores of included participants were very high. The participants in the Chiang Mai device group had a higher median score of multiple-choice question assessment [9.0/9.0 (8.5-9.0) vs 8.5/9.0 (8.0-9.0) points, p-value=0.134] as well as a higher median score of practical examination [26.0/26.0 (24.3-26.0) vs 25.0/26.0 (24.0-26.0) points, p-value=0.278] when compared to those using conventional BLS training devices. However, there was no statistical significance between both groups.Conclusion: The effectiveness of the Chiang Mai BLS training device in basic life support training for adult laypeople is comparable to conventional BLS training devices.

Assessment of Global Health Education: The Role of Multiple-Choice Questions.

Introduction: The standardization of global health education and assessment remains a significant issue among global health educators. This paper explores the role of multiple choice questions (MCQs) in global health education: whether MCQs are appropriate in written assessment of what may be perceived to be a broad curriculum packed with fewer facts than biomedical science curricula; what form the MCQs might take; what we want to test; how to select the most appropriate question format; the challenge of quality item-writing; and, which aspects of the curriculum MCQs may be used to assess.Materials and Methods: The Medical School for International Health (MSIH) global health curriculum was blue-printed by content experts and course teachers. A 30-question, 1-h examination was produced after exhaustive item writing and revision by teachers of the course. Reliability, difficulty index and discrimination were calculated and examination results were analyzed using SPSS software.Results: Twenty-nine students sat the 1-h examination. All students passed (scores above 67% - in accordance with University criteria). Twenty-three (77%) questions were found to be easy, 4 (14%) of moderate difficulty, and 3 (9%) difficult (using examinations department difficulty index calculations). Eight questions (27%) were considered discriminatory and 20 (67%) were non-discriminatory according to examinations department calculations and criteria. The reliability score was 0.27.Discussion: Our experience shows that there may be a role for single-best-option (SBO) MCQ assessment in global health education. MCQs may be written that cover the majority of the curriculum. Aspects of the curriculum may be better addressed by non-SBO format MCQs. MCQ assessment might usefully complement other forms of assessment that assess skills, attitude and behavior. Preparation of effective MCQs is an exhaustive process, but high quality MCQs in global health may serve as an important driver of learning.