Multiple-choice Assessment Research Articles

Examining the concurrent impact of teaching and assessment format on undergraduate students' academic performance in marketing

PurposeBy focusing on a Greek traditional learning university, during and post Covid-19 restrictions, the study aims at examining the concurrent effects of teaching and assessment format on students' academic performance. The inclusion of case studies in course assessment post Covid-19 restrictions is also expected to give a rough insight into students' employability skills and workplace readiness.Design/methodology/approachThe academic performance of 489 undergraduate students, as determined by the grades they earned, was measured in the final exams of business-to-business marketing and integrated marketing communication courses, held in January 2021 and 2022 at a Greek public university. The primary predictor variable or interest was “teaching and assessment format”, and took two values: (1) online teaching with multiple-choice assessment format (during Covid-19 restrictions) and (2) traditional classroom teaching with in-person case study and open-ended assessment format (post Covid-19 restrictions).FindingsGrades were found to be affected by the participants' year of study, the type of marketing course, in which they were examined, and the teaching and assessment format employed. Either in whole or by gender, students appear to perform significantly worse in the traditional teaching with in-person case study and open-ended questions assessment format.Practical implicationsGood pedagogical practice in the use of digital technology is advised to incorporate diverse teaching tools and assessment methods.Originality/valueExamination of the concurrent effects of teaching and assessment on academic performance unveils significant variation in students' academic performance under different formats, which may be attributable to multiple reasons.

Knowledge Retention Using "Relay": A Novel Active-Learning Technique.

Many residency programs utilize passive didactic lectures despite mixed evidence for knowledge retention. This prospective study investigated the efficacy of "relay," an active-learning technique, as measured by residents' knowledge retention and attitudes compared to traditional format. Faculty presented lectures to four family medicine residency programs on a randomized schedule. Lectures were followed by a 15-minute question-and-answer (Q&A) session or relay session. A relay is a collaborative, question-based game. The primary outcome was knowledge retention at 3 months, comparing Q&A to relay sessions as measured by a multiple-choice assessment. Responses were only included if a given resident completed knowledge assessments for both Q&A and relay sessions, to allow for intraresident adjustments, in addition to program, training year, and lecturer/topic. Secondary outcomes included residents' self-perceived knowledge and engagement as surveyed by an ordinal scale immediately following the learning session. The primary analysis included 51 responses from 18 unique residents. The adjusted mean knowledge assessment score at 3 months was not statistically different after the relay sessions compared to Q&A (67% vs 60%, respectively; 7% difference, 95% CI: -4 to 18%, P=.20). For the secondary outcomes of learner attitudes (n=143 responses), learners reported greater engagement after the relay sessions compared to Q&A (51% vs 28% "very engaged"; overall P=.003), but self-perceived knowledge was not significantly different (overall P=.05, rounded down). The relay technique did not show significant difference in 3-month knowledge retention, nor immediate self-perceived knowledge, despite greater learner self-perceived engagement.

Aligning multiple choice assessments with active learning instruction: More accurate and equitable ways to measure student learning

In response to geoscience education research and calls for the need to reform STEM teaching in higher education, many geoscience faculty are adopting research-based pedagogies to improve student learning in their courses. Our own effort to reform our introductory geology course included daily student-centered activities and other reforms we expected would result in improved student performance, but did not. This paper describes our evaluation of student performance on multiple choice questions (MCQs), which are the largest component of student grades in the course. Student scores were highest on MCQs that were aligned with the style in which students learned material and that required students to interact with the question (active questions) rather than just regurgitating information (passive questions). We also examined whether the style of MCQs impacted student scores equitably and found that the grades of all students would have improved if we had asked MCQs that were aligned and active, and the improvements were equitable, but final grades remain biased.

Motivations for using the item response theory nominal response model to rank responses to multiple-choice items

New insights on multiple-choice assessments based on similarities between the item response theory nominal response model and the probability function of the canonical ensemble with degenerate energy states.

Simulation-Based Training to Improve Clinical Pharmacist Self-Efficacy in the Management of a Rapidly Decompensating Patient.

Background: The optimal training method to prepare pharmacists as an integral rapid response team or cardiopulmonary arrest responders is poorly described. This study assessed the utility of simulation-based training (SBT) as a training technique for clinical pharmacists. Objective: This study aimed to determine if attending SBT is associated with an improvement in self-efficacy. Methods: This single-center, prospective, interventional cohort study offered three simulations to clinical pharmacists over the course of seven months at a 957-bed quaternary care academic medical center. Pharmacists who participated in at least one simulation were categorized in the intervention group and were compared to pharmacists who did not attend a simulation. All participants were asked to complete a 19-question self-efficacy survey in the form of a 100-point scale, a 15-question multiple-choice knowledge assessment, and a perception survey in the form of 4-point Likert scale administered at baseline and following the conclusion of the SBT. Results: Forty-four clinical pharmacists participated; 20 in the intervention group and 24 in the control group. Median change in self-efficacy score improved significantly in the intervention group compared to the control group (14.3 vs 2.3, P = .009). Median change in perception score improved significantly (2 vs 0, P = .046). Knowledge score did not change significantly from baseline. Conclusion: Simulation-based training improved clinical pharmacist self-efficacy and perceptions in the care of rapidly decompensating patients. These findings support SBT as a viable modality of training clinical pharmacists for the management of rapidly decompensating patients.

Conceptual Understanding of Students Towards COVID-19 Pandemic: Basis for Inclusion in Science, Technology and Society (STS) Curriculum

The study has utilized a multi-methodology research design with assessment tests and a structured interview guide as instruments in gathering the data. In class 2020, there are 72 students and 8 Teachers of Science, Technology and Society (STS) who served as participants in the study. The assessment test was validated by three experts, Science, Technology and Society teachers with sciences as specialization and a professional education teacher. There are two types of instruments were used to gather data, these are (a) an assessment test and (b) structured interview guide. The assessment test is compose of two parts (I) the traditional multiple choice assessment test that is composed of 20 questions and (II) close-ended essay which consists of five questions. The structured interview guide is composed of one question “In your perspective, should ‘COVID-19 Pandemic’ be included in a STS curriculum? Why?” The results of the study have concluded: level of understanding of the students based from the traditional and essay assessment tests are both fairly satisfactory; level of conceptual understanding based from the relationship between the assessment tests has moderate positive correlation; the perspectives of the teachers regarding the inclusion of COVID-19 Pandemic concept in the STS curriculum, seven themes have emerged. The themes focused on addressing COVID disparities experience across multiple dimension and use it for curriculum formulation and empowerment of STS literacy. The study has recommended; heighten and strengthen the students’ conceptual understanding toward COVID 19 Pandemic through series of topics and teaching approaches; strategically include the concept of COVID-19 pandemic in the curriculum.

Comparison of Traditional Lecture-Based Learning versus Interactive Electronic Book Learning in Veterinary Student Comprehension of Inhalant Anesthetic Administration, Uptake, and Elimination.

The administration, uptake, and elimination of inhalant anesthetics is a challenging topic in the veterinary curriculum, and lecture-based learning is often insufficient to ensure that students understand these concepts. We hypothesized that the use of an interactive electronic book (e-book) would enhance student comprehension of the material. Two sequential Doctorate of Veterinary Medicine student cohorts participated in a prospective controlled study. The first cohort received traditional lecture-based learning while the second cohort was taught the topic using an interactive e-book. Student comprehension of the material was assessed twice during the course via multiple-choice questions: five questions in a midcourse quiz and seven within the final exam. At the end of the course, students also completed a Likert survey assessing their confidence regarding the topic. Averaged across assessment types, students taught using the interactive e-book scored higher than those taught via the traditional method (p < .001). Final exam scores were significantly higher in the e-book cohort compared with the lecture-based cohort (p < .001). However, there was no difference in quiz scores between groups (p = .109). No significant difference was found between groups in responses to the Likert survey. In conclusion, students using the interactive e-book had better comprehension of the material than students in the traditional lecture group as measured by their scores on multiple-choice question assessments. Future studies are needed to determine whether this advantage persists later in the curriculum when students apply these concepts in the clinical year.

Analisis Tingkat Kesiapan Pengguna Sistem Informasi Koreksi Essay Otomatis Berbasis Web Menggunakan Model Technology Readiness Index (TRI)

During the pandemic, the learning model directly impacts both students and teachers, especially in evaluating and correcting assignments to measure student learning. Evaluation and correction of the learning process at both the high school and high school and university levels are generally carried out multiple-choice examinations during this pandemic. Many tools can be used to carry out multiple-choice assessments. The essay assessment is usually done manually and takes a long time because the lecturer has to make corrections one by one. On the other hand, information retrieval is a concept that can be used to build an automatic essay correction information system so that the correction of essay questions can be done quickly and efficiently, as in multiple-choice works. The automatic essay correction information system is a relatively new technology, especially for its plural use in this pandemic. A new technology that will be applied in a plural manner requires a measurement. One of the measurements that can be used is the Technology Readiness Index (TRI). TRI is used because it can measure the readiness and tendency of users to implement an information system. The case study in this research was conducted at the Faculty of Informatics (FIF) Telkom Institute of Technology Purwokerto (ITTP). The TRI measurement of the automatic essay correction information system carried out at FIF produces a TRI of 3.27. This value is between 2.90 and 3.51 (2.90 ≤ TRI ≤ 3.51). Consisted of an optimism value of 1.01, and innovativeness value of 0.52, a discomfort value of 0.55, and an insecurity value of 1.19.

The self-management abilities test (SMAT): a tool to identify the self-management abilities of adults with bronchiectasis

Bronchiectasis is an increasingly common chronic respiratory disease which requires a high level of patient engagement in self-management. Whilst the need for self-management has been recognised, the knowledge and skills needed to do so— and the extent to which patients possess these—has not been well-specified. On one hand, understanding the gaps in people’s knowledge and skills can enable better targeting of self-management supports. On the other, clarity about what they do know can increase patients’ confidence to self-manage. This study aims to develop an assessment of patients’ ability to self-manage effectively, through a consensus-building process with patients, clinicians and policymakers. The study employs a modified, online three-round Delphi to solicit the opinions of patients, clinicians, and policymakers (N = 30) with experience of bronchiectasis. The first round seeks consensus on the content domains for an assessment of bronchiectasis self-management ability. Subsequent rounds propose and refine multiple-choice assessment items to address the agreed domains. A group of ten clinicians, ten patients and ten policymakers provide both qualitative and quantitative feedback. Consensus is determined using content validity ratios. Qualitative feedback is analysed using the summative content analysis method. Overarching domains are General Health Knowledge, Bronchiectasis-Specific Knowledge, Symptom Management, Communication, and Addressing Deterioration, each with two sub-domains. A final assessment tool of 20 items contains two items addressing each sub-domain. This study establishes that there is broad consensus about the knowledge and skills required to self-manage bronchiectasis effectively, across stakeholder groups. The output of the study is an assessment tool that can be used by patients and their healthcare providers to guide the provision of self-management education, opportunities, and support.

Multifaceted Approach to Evaluation in a Pediatric and Adolescent Gynecology Rotation for Medical Students

We evaluated whether and to what extent a novel medical student rotation in pediatric and adolescent gynecology (PAG) increases clinical knowledge and skills and meets student needs and expectations. Constructivist prospective pre-post study and post-rotation student survey SETTING: Academic medical center PARTICIPANTS: Pilot study of 9 medical students, which represents the entire population of those who completed the rotation. Four-week clinical rotation in PAG MAIN OUTCOME MEASURES: Changes in clinical knowledge were measured by a pre- and post-intervention multiple-choice assessment, and clinical skills were assessed before and after the intervention using entrustable professional activities (EPAs); these data were analyzed with paired Student's t tests. Student evaluations of the rotation were measured through an anonymous, end-of-rotation, closed- and open-ended survey and were analyzed using descriptive statistics. A statistically significant increase in clinical knowledge was observed post-rotation, with a mean pretest score of 67.0% (standard deviation [SD] 1.7%) and a mean posttest score of 75.2% (SD 3.2%, P=0.02). Statistically significant increases were observed for all EPAs between the first and final day of the rotation. Eight students who completed the post-rotation survey rated the rotation favorably (5 on a scale from 1 to 5). A multipronged evaluation showed that a new PAG clinical rotation significantly increased medical students' clinical skills and knowledge. This multifaceted evaluation method provides valuable insights to educators on how best to tailor a rotation to individual learners' levels of clinical skills and knowledge. If comparable rotations could be instituted and similarly evaluated in other medical schools, a noticeable knowledge/skill gap among trainees might be addressed.

Effectiveness of Innovation Basic Life Support Training Devices to Layperson: A Randomized Controlled Trial

Objectives: The use of a cardiopulmonary resuscitation feedback device and automated external defibrillator trainer is beneficial in basic life support (BLS) training. Nevertheless, Thailand lacks these devices in BLS support training. This study aimed to compare the efficacy of the Chiang Mai BLS training devices with conventional training devices in BLS training for laypeople.Material and Methods: A randomized controlled trial was conducted to compare the efficacy of the Chiang Mai device group with the conventional device group, by assessing the theory and practical examination scores of the participants; who were adult, laypeople attending the BLS provider course endorsed by the Thai Resuscitation Council. Evaluating instructors were blinded from both groups of participants.Results: A total of 60 adult, laypeople participants were divided into two groups: 32 and 28 participants of the Chiang Mai device group and conventional device group, respectively. Overall examination scores of included participants were very high. The participants in the Chiang Mai device group had a higher median score of multiple-choice question assessment [9.0/9.0 (8.5-9.0) vs 8.5/9.0 (8.0-9.0) points, p-value=0.134] as well as a higher median score of practical examination [26.0/26.0 (24.3-26.0) vs 25.0/26.0 (24.0-26.0) points, p-value=0.278] when compared to those using conventional BLS training devices. However, there was no statistical significance between both groups.Conclusion: The effectiveness of the Chiang Mai BLS training device in basic life support training for adult laypeople is comparable to conventional BLS training devices.

Abstract 11394: Comparison Between Advanced Cardiovascular Life Support Course and a New Emergency Cardiovascular Care Course for Developing Countries

Introduction: Training and cardiovascular emergency care (CEC) is different in developing countries from developed countries. Emergency medicine specialty in developing countries is not established without a large number of specialists and the Advanced Cardiovascular Life Support course (ACLS) is a very common course. Objective: To compare the ACLS with the Advanced Cardiovascular Emergency Training course (TECA) developed in Brazil according to the needs of local doctors. Both courses with 16 hours of training but the TECA adding more skills training time in Arrhythmia, Stroke, Acute Coronary Care (ACC) and Acute Heart Failure (AHF) in addition to the traditional training in cardiac arrest management (CAM). Methods: The study included the participation of 119 senior medical students divided into 2 groups: 56 students submitted to TECA training and 63 students submitted to the ACLS course. Both groups performed a mannequin simulated CAM skills evaluation care before and after participating in each course and a multiple-choice test after training. Performance in simulated scenario was compared through structured assessment score, time to onset of chest compression and time to perform defibrillation after collapse, as well as the number of correct answers in the multiple-choice assessment. Results: TECA group presented statistical higher grade than the ACLS group in the evaluation of the simulated case attendance (10,0 points (9,00-10,00) vs 9,0 (8,00-10,00) points; p&lt;0,001). There was no statistical difference between the groups in time to onset of chest compression (19,0 sec (16,25-23,00) vs 19,5 (15,75-25,25) sec; p = 0,500) and time to defibrillation (48,0 (39,00-53,00) sec vs 48,0 (39,00-55,00) sec; p = 0,740). TECA students had a better performance in questions related to AHF treatment (4,0 (3,00-4,00) vs 3,0 (2,00-3,00); p &lt; 0,001). There was no significant difference in the number of correct answers on arrhythmia, CAM, ACC and stroke. Conclusions: TECA course provided greater adherence to CAM care protocol, without prejudice to the time to start chest compression or time to defibrillation. In addition, it added training in AHF management, suggesting to be a good option for cardiac emergency care training in developing countries.

Choosing Non-Malignant Hematologic Care Wisely: An Educational Pilot Study for Internal Medicine Residents

Introduction: Choosing Wisely ® (CW ®)is an initiative of the American Board of Internal Medicine Foundation created to guide the selection of care that is 1) supported by evidence, 2) not duplicative, 3) free from harm, and 4) truly necessary. Between 2013-14, the American Society of Hematology (ASH) published 10 recommendations in accordance with CW ® principles relevant to hematologic care. Previous studies have demonstrated that clinical exposure to non-malignant hematology (NMH) improves trainee understanding of evidence-based, cost effective care as outlined by ASH CW ®. However, dedicated clinical rotations in NMH for internal medicine (IM) residents are not consistently available. Therefore, we created a condensed educational curricular experience using a small group, case-based structure designed to teach the fundamentals of ASH CW ® in NMH to first year IM residents. With a pilot of 6 sessions, we investigated resident baseline knowledge, evaluated the curricular session, and assessed knowledge retention.Methods: The educational intervention focused on 3 content areas in ASH CW ®: venous thromboembolism (VTE), heparin-induced thrombocytopenia (HIT), and sickle cell disease (SCD) (Table 1). Participants included 21 first year IM residents at a large academic medical center. A 1-hour small group teaching session was scheduled monthly as an assigned didactic for the 2020-21 academic year. A total of 6 sessions were provided, and each session had 2-4 residents assigned. The first 4 sessions were in-person and the final 2 were virtual due to the COVID-19 pandemic. The first author was the instructor at all sessions.To assess baseline knowledge of the 3 content areas, participants completed an online assessment with 5 case-based multiple-choice questions at the beginning of the session. The instructor then guided participants to again complete the questionnaire together, now using internet access via a personal computer and a recommended list of online resources, including ASH Clinical Practice Guidelines and Pocket Guides. The instructor then led discussion on how each correct answer or guideline recommendation achieves the 4 CW ® principles. At the conclusion of the session, participants completed an online survey to evaluate the educational intervention using a modified Likert scale. To assess knowledge retention, participants received the original online multiple-choice assessment by email 3 months later.Results: All participants (21/21, 100%) completed the baseline knowledge assessment. The average number of questions correct out of 5 total was 3.3 (67%), with a range of correct answers from 1 to 5. Table 2 includes the content area of each question and the number of correct responses. The question with the lowest total correct (9/21, 43%) addressed the use of transfusion in an uncomplicated pain crisis in SCD.Seventeen participants (81%) completed the curricular evaluation. All respondents (17/17, 100%) either agreed or strongly agreed 1) that the session filled a gap in their NMH training and 2) that they learned something that would change their clinical practice. Only 1 participant (5%) reported completing a rotation in NMH prior to the session.Six participants (29%) completed the repeat knowledge assessment at 3 months. All respondents (100%) achieved a perfect score on the multiple choice questions. When asked if the knowledge gained had influenced their clinical practice, 3 (50%) strongly agreed, 2 (33%) agreed, 1 (17%) was neutral, and none disagreed or strongly disagreed.Conclusion: Our results demonstrate a successful educational pilot to improve the knowledge of ASH CW ® initiatives in NMH for first year IM residents using small group interactive case-based learning. Participants were overwhelmingly receptive to this intervention, expressed high satisfaction and confirmed that the session positively influenced their clinical practice. Although participation in the repeat assessment of knowledge was limited, those that did participate demonstrated high knowledge retention. We intend to expand this pilot initiative by providing the educational session for all incoming IM residents at our institution. We then plan to assess its impact on clinical practice (i.e. use of transfusion in SCD, use of thrombophilia testing, documentation of 4Ts score calculation) to apply the principles of ASH CW ® for improved patient care. [Display omitted] DisclosuresNo relevant conflicts of interest to declare.

Is a Video Worth a Thousand Words? Educating Preclinical Medical Students on Sterile Scrubbing, Gowning, and Gloving Virtually and In-Person.

IntroductionPrograms that offer early exposure to surgery for medical students foster interest in and positive perceptions of surgery. The COVID-19 pandemic led to suspension of these activities at our institution, the University of Kansas School of Medicine. In response to the lack of virtual alternatives, a pilot virtual surgery enrichment program was implemented for first-year students in place of in-person surgical exposure. The aim of this study was to compare the efficacy of in-person and virtual-based surgical education programs to expose preclinical medical students about the surgical realm of medicine.MethodsFirst-year medical students participated in either a virtual (Group A) or in-person (Group B) week-long surgical enrichment program. Group assignments were dictated by COVID restrictions on each of our three medical school campuses: Salina, Wichita, and Kansas City. Pre- and post-surveys with a 14-question multiple-choice assessment of surgical knowledge were distributed to participants. Paired Wilcoxon Signed Rank tests and Mann-Whitney-U tests were used for statistical analysis.ResultsThere were 14 participants in Group A and 7 participants in Group B. Both groups improved significantly from pre- to post-assessment score. (Group A, p = 0.01; Group B, p = 0.04). There was no difference between groups in the magnitude of score improvement from pre- to post-assessment (p = 0.59).ConclusionsThis pilot program demonstrated that virtual platforms can be a method to provide meaningful clinical experiences in surgery to preclinical medical students restricted from clinical activities. Further development of mentorship in virtual surgical programs and assessment of subjective experience is needed.

Considerations and strategies for effective online assessment with a focus on the biomedical sciences.

The COVID‐19 pandemic in 2020 caused many universities to rapidly transition into online learning and assessment. For many this created a marked shift in design of assessments in an attempt to counteract the lack of invigilation of examinations conducted online. While disruptive for both staff and students, this sudden change provided a much needed reconsideration of the purpose of assessment. This review considers the implications of transitioning to online assessment providing practical strategies for achieving authentic assessment of students online, while ensuring standards and accountability against professional accrediting body requirements. The case study presented demonstrates that an online multiple choice assessment provides similar rigor in assessment to invigilated examination of the same concepts in human physiology. Online assessment has the added benefit of enabling rapid and specific feedback to large cohorts of students on their personal performance, allowing students to target their weaker areas for remediation. This has implications for improving both pedagogy and efficiency in assessment of large cohorts where the default is often to assess basic recall knowledge in a multiple choice assessment. This review examines the key elements for implementation of online assessments including consideration of the role of assessment in teaching and learning, the rationale for online delivery, accessibility of the assessment from both a technical and equity perspective, academic integrity as well as the authenticity and structure of the assessment.

Describing the Development of the Assessment of Biological Reasoning (ABR)

Assessments of scientific reasoning that capture the intertwining aspects of conceptual, procedural and epistemic knowledge are often associated with intensive qualitative analyses of student responses to open-ended questions, work products, interviews, discourse and classroom observations. While such analyses provide evaluations of students’ reasoning skills, they are not scalable. The purpose of this study is to develop a three-tiered multiple-choice assessment to measure students’ reasoning about biological phenomena and to understand the affordances and limitations of such an assessment. To validate the assessment and to understand what the assessment measures, qualitative and quantitative data were collected and analyzed, including read-aloud, focus group interviews and analysis of large sample data sets. These data served to validate our three-tiered assessment called the Assessment of Biological Reasoning (ABR) consisting of 10 question sets focused on core biological concepts. Further examination of our data suggests that students’ reasoning is intertwined in such a way that procedural and epistemic knowledge is reliant on and given meaning by conceptual knowledge, an idea that pushes against the conceptualization that the latter forms of knowledge construction are more broadly applicable across disciplines.

Patterns of Scientific Reasoning Skills among Pre-Service Science Teachers: A Latent Class Analysis

We investigated the scientific reasoning competencies of pre-service science teachers (PSTs) using a multiple-choice assessment. This assessment targeted seven reasoning skills commonly associated with scientific investigation and scientific modeling. The sample consisted of 112 PSTs enrolled in a secondary teacher education program. A latent class (LC) analysis was conducted to evaluate if there are subgroups with distinct patterns of reasoning skills. The analysis revealed two subgroups, where LC1 (73% of the PSTs) had a statistically higher probability of solving reasoning tasks than LC2. Specific patterns of reasoning emerged within each subgroup. Within LC1, tasks involving analyzing data and drawing conclusions were answered correctly more often than tasks involving formulating research questions and generating hypotheses. Related to modeling, tasks on testing models were solved more often than those requiring judgment on the purpose of models. This study illustrates the benefits of applying person-centered statistical analyses, such as LC analysis, to identify subgroups with distinct patterns of scientific reasoning skills in a larger sample. The findings also suggest that highlighting specific skills in teacher education, such as: formulating research questions, generating hypotheses, and judging the purposes of models, would better enhance the full complement of PSTs’ scientific reasoning competencies.

968 Delivering Human Factors and Non-Technical Skills Training Using Interactive Online Platforms in the COVID Era

Abstract Aim To evaluate the impact and effectiveness of an interactive online human factors (HF) and non-technical skills (NTS) course delivered to Core Surgical Trainees during the COVID-19 pandemic Method A 1-day HF and NTS course was conducted online, using the Zoom platform, to Core Surgical Trainees in the North West. The course consisted of interactive lectures, small group teaching sessions, and self-directed learning with written reflections. Pre- and post-course surveys were administered, evaluating the participants’ awareness, knowledge and skills using a 5-part Likert scale, along with a multiple-choice assessment of knowledge. Statistical analysis was undertaken with significance considered at p &amp;lt; 0.05 Results The course was attended by 63 CT1/2 participants, representing all surgical specialties. In the post-course evaluation, participants’ self-rating of awareness and knowledge for both HF and patient safety increased by between 10-20%. There was a significant increase in the mean post-course test score from 7.54 (SD ± 1.7) to 8.65 (SD ± 1.2) out of 10 (p &amp;lt; 0.0001). The course overall was rated relevant and useful (weighted averages 4.4 and 4.5 respectively) Conclusions To our knowledge, this is the first time a video conferencing platform has been used to deliver a live HF/NTS course. This study provides evidence supporting the use of such interactive online platforms in postgraduate surgical education. Training and professional development must continue, despite the evolving pressures from COVID-19. Embracing new methods of education delivery is required, with ongoing reporting and evaluation of education practice, sharing lessons learned and informing the evidence base in postgraduate surgical training during this time

Multiple-choice questionnaire assessments: do they have a role in assessing law students?

ABSTRACT In the UK there has been a resistance to using multiple-choice questionnaire (MCQ) assessments to assess undergraduate students. This resistance is based on traditional legal pedagogy and the needs of the legal profession. However, the use of MCQ-style assessments by the Solicitors Regulation Authority and other shifts in legal education make it necessary to reflect on the role that MCQs could play in assessing law students. This article seeks to contribute to this discussion by providing a pedagogical analysis of the assessment strategy of the “Public Law I – Sources of Power” module at the University of Dundee. First, the article sets out Bloom’s Revised Taxonomy and its applicability to legal education. The article then applies this taxonomy to identify the complexity of the module’s learning outcomes and its assessments, as well as considering the extent to which they align with each other. The article finds that while MCQs cannot contribute to all of a module’s learning outcomes, they do provide some pedagogical value in assessing law students, particularly when implemented in parallel with other assessment methods. The article concludes by reflecting on the broader position of MCQs and their role in assessing undergraduate students.

Analyzing Cognitive Demands of a Scientific Reasoning Test Using the Linear Logistic Test Model (LLTM)

The development and evaluation of valid assessments of scientific reasoning are an integral part of research in science education. In the present study, we used the linear logistic test model (LLTM) to analyze how item features related to text complexity and the presence of visual representations influence the overall item difficulty of an established, multiple-choice, scientific reasoning competencies assessment instrument. This study used data from n = 243 pre-service science teachers from Australia, Canada, and the UK. The findings revealed that text complexity and the presence of visual representations increased item difficulty and, in total, contributed to 32% of the variance in item difficulty. These findings suggest that the multiple-choice items contain the following cognitive demands: encoding, processing, and combining of textually presented information from different parts of the items and encoding, processing, and combining information that is presented in both the text and images. The present study adds to our knowledge of which cognitive demands are imposed upon by multiple-choice assessment instruments and whether these demands are relevant for the construct under investigation—in this case, scientific reasoning competencies. The findings are discussed and related to the relevant science education literature.