Exam Delivery Research Articles

Students’ preferences for teaching and exam delivery modes in accounting education post-COVID-19 pandemic

ABSTRACT The COVID-19 pandemic has sparked a revolution in the delivery of modules in higher education. This paper aims to answer the research question: What are the preferences of undergraduate accounting and finance students regarding teaching delivery and exam modes following the COVID-19 pandemic lockdown? Specifically, we focus on campus, online synchronous, and hybrid synchronous teaching delivery and exam modes. To address this research question, we conducted an online questionnaire surveying students at a U. K. university. Our data show that modules involving calculations, such as financial accounting, are preferred to be taught on campus, whereas theoretical modules like business law are preferred to be taught online. Additionally, the data reveal reasons for these preferences, including community learning, isolation, concentration, and access to recordings. This research contributes valuable insights into optimising accounting education. We advocate for flexibility in both teaching and exam delivery within accounting education, recognising the diverse needs of students.

A profiling-based algorithm for exams’ scheduling problem

<span lang="EN-US">Typically, the problem of scheduling exams for universities aims to determine a schedule that satisfies logistics constraints, including the number of available exam rooms and the exam delivery mode (online or paper-based). The objective of this problem varies according to the university’s requirements. For example, some universities may seek to minimize operational costs, while others may work to minimize the schedule's length. Consequently, the objective imposed by the university affects the complexity of the problem. In this study, we present a grouping-based approach designed to address the problem of scheduling the exam timetable. The approach begins by profiling the courses’ exams based on their requirements, grouping exams with similar requirements to be scheduled at the same time. Then, an insertion strategy is used to obtain the exam schedule while satisfying the imposed constraints of the targeted university. We applied this approach to the problem of exam scheduling at Al-Hussein Bin Talal University in Jordan and achieved a balanced exam schedule that met all the imposed constraints.</span>

How Does the Sequence of the American Board of Surgery Examinations Impact Pass/Fail Outcomes?

Historically, the American Board of Surgery required surgeons to pass the qualifying examination (QE) before taking the certifying examination (CE). However, in the 2020-2021 academic year, with mitigating circumstances related to COVID-19, the ABS removed this sequencing requirement to facilitate the certification process for those candidates who were negatively impacted by a QE delivery failure. This decoupling of the traditional order of exam delivery has provided a natural comparator to the traditional route and an analysis of the impact of examination sequencing on candidate performance. All candidates who applied for the canceled July 2020 QE were allowed to take the CE before passing the QE. The sample was then reduced to include only first-time candidates to ensure comparable groups for performance outcomes. Logistic regression was used to analyze the relationship between the order of taking the QE and the CE, controlling for other examination performance, international medical graduate status, and gender. Only first-time candidates who took both examinations were compared (n=947). Examination sequence was not a significant predictor of QE pass/fail outcomes, OR=0.54; 95% CI, 0.19-1.61, P =0.26. However, examination sequence was a significant predictor of CE pass/fail outcomes, OR=2.54; 95% CI, 1.46-4.68, P =0.002. This important study suggests that preparation for the QE increases the probability of passing the CE and provides evidence that knowledge may be foundational for clinical judgment. The ABS will consider these findings for examination sequencing moving forward.

High-Stakes Online Exams

COVID-19 has affected university assessment procedures on a large scale. This empirical study aims to understand the types of high-stakes exams delivered online at the Lucerne School of Business in Switzerland during the “Corona Semesters” of 2020 and 2021 and the decision-making factors that influenced their implementation. To do so, the authors conducted semi-structured interviews with eight faculty members across a variety of disciplines. Requirements from the exam workflow (preparation, proctoring, grading) were identified and analyzed by course type. Four factors emerged that significantly impacted design and delivery for high-stakes exams online: 1) Digital exam formats significantly impact the nature of exams for procedural subjects such as mathematics; 2) “Group Exams” are not the answer to preventing student collusion on online exams; 3) interrater reliability and low answer variance are considered a central factor for exam quality assurance; 4) second-order effects such as stable wifi and device compatibility will continue to hinder widescale adoption of digital exams. The findings suggest that online exam delivery significantly affects institutional exam practice beyond mere consideration of learning outcomes. The authors conclude by speculating that similar dynamics may have impacted other business schools during their Corona semesters and invite future research on whether the findings from this article can spark discussion and reflection for policy makers in other institutions on post-pandemic legacies.

A Student Learning Style Auto-Detection Model in a Learning Management System

Learning style plays an important role in enabling students to have an efficient learning process. This paper proposes an auto-detection model of student learning styles in learning management systems based on student learning activities. A literature review was conducted to investigate the components of online learning activities. The search terms used were "online learning activities", "learning management systems", and "Felder-Silverman Learning Style Model (FSLSM)." A combination of the search terms above was also executed to enhance the search process. Based on the results of the review, eleven classes of online learning activities were identified, namely forum, chat, mail, reading materials, exam delivery time, exercises, access to examples, answer changes, learning materials, exam results, and information access. The online learning activities identified were then mapped to the Felder-Silverman model based on four model dimensions: processing, perception, input, and understanding. The proposed model shows the attributes of the online learning activities based on the dimensions in the FSLSM. The proposed model can assist educators to improve learning content according to the suitability of students and recommend appropriate learning materials to students based on their characteristics and preferences. Future studies include the use of machine learning algorithms such as decision trees to auto-detect student learning styles in learning management systems.

An example of social interaction: Spatial contagion effect in exams

In the last decades the study of collective phenomena has produced a great interest in the field of Statistical Physics within the framework of Complex Systems, being a paradigmatic example flocking, the collective motion of self-propelled organisms. Such studies have been more recently extended to collective human behavior, where social interactions are important and concepts such as ’social force’ have arisen. In this work we want to explore the possible existence of a ’social field’ in a very controlled human environment: a classroom where the students take an exam. Since the students are seated in individual tables while working in their corresponding exams, the only possible interaction occurs when the students finish the exam and deliver it to the teacher. We conjecture that the existence of social interactions could lead to a contagion effect among the students, so that a given student who delivers the exam may influence another close student to do the same, and as a result the exams are not randomly delivered in the space. In this sense, each classroom can be seen as a complex system, where there exist interactions between the different elements, the students. To show the existence of this contagion effect, we use experimental data registered in 10 high-school classrooms during different exams, and for each student we record the exam delivery time and the spatial location of the student in the classroom while taking the exam. We use the distances between students who finish the exam consecutively and compare these distances with the random expectation in the corresponding classroom using Monte Carlo simulations. We observe a significant nonrandom behavior of the experimental data, and show the existence of a clustering effect in space, supporting the existence of a contagion effect as a consequence of an underlying ’social field’. Finally, to quantify this contagion effect, we propose a probabilistic distance-driven contagion model, according to which a given student who delivers the exam may influence another student closer than a given distance to do the same with certain contagion probability. By comparing the results of the model with the experimental data, we obtain a global contagion probability of around 1/6.

Strategies and System Implementations for Secure Electronic Written Exams

The pandemic has accelerated the process of digitization in many fields, highlighting some critical issues that have slowed the development of digital technologies in many areas. One of these is undoubtedly that of skills assessment, which on a digital platform is a more streamlined, reliable and faster process and allows some individuals from disadvantaged groups to be able to carry out the exam more easily. We present some strategies for enabling online remote secure written exams and their implementation through the open-source system LibreEOL. The system can be used for university exams (in presence or from remote locations) and training courses of all types to evaluate the effectiveness of learning in itinere, i.e., before, during and after a course, to provide quantitative data on training effectiveness. We outline the architecture of the services underpinning the system and the logical organisation of the various user roles and content relating to questions and exam delivery. We describe the algorithms that have allowed us to develop an environment that appears to be an essential aid both for teachers and students in light of the questionnaires conducted on students. The system is successfully used in the EchemTest, promoted by the European Chemistry Thematic Network, and allows comparison of chemistry training outcomes in different European countries. Despite the importance, it has taken on in the last two years due to the pandemic, such a system makes it possible to carry out a valuable skills assessment.

Effect of remote online exam delivery on student experience and performance in applied knowledge tests

BackgroundThe use of remote online delivery of summative assessments has been underexplored in medical education. Due to the COVID-19 pandemic, all end of year applied knowledge multiple choice question (MCQ) tests at one UK medical school were switched from on campus to remote assessments.MethodsWe conducted an online survey of student experience with remote exam delivery and compared test performance in remote versus invigilated campus-based forms of similar assessments for Year 4 and 5 students across two academic years.ResultsVery few students experienced technical or practical problems in completing their exam remotely. Test anxiety was reduced for some students but increased for others. The majority of students preferred the traditional setting of invigilated exams in a computer lab, feeling this ensured an even playing field for all candidates. Mean score was higher for Year 4 students in the remotely-delivered versus campus-based form of the same exam (76.53% [SD 6.57] vs. 72.81% [6.64]; t438.38 = 5.94, p = 0.001; d = 0.56), whereas candidate performance was equivalent across both forms for Year 5 students.ConclusionsRemote online MCQ exam delivery is an effective and generally acceptable approach to summative assessment, and could be used again in future without detriment to students if onsite delivery is not possible.

Florida International University Herbert Wertheim College of Medicine.

Florida International University Herbert Wertheim College of Medicine.

Testing in the Time of COVID-19: A Sudden Transition to Unproctored Online Exams

The swift conversion of chemistry courses to various online formats has presented challenges for students and instructors alike, with one of the most significant challenges being the logistics concerning online testing. At our institution, instructors of general, organic, and physical chemistry courses sought to balance issues of exam access, exam security, and continuity with a traditional test format when switching to unproctored online exams. Different approaches were utilized for exam delivery with varying success. Common challenges arose in several courses, such as the need to train students to use new technology, and the threat of online information for compromising exam security. The most successful attempts to meet these challenges, such as use of a test-bank, are consistent with reported best practices and suggest a way to maximize access, security, and continuity, while working to diminish technological and ethical pitfalls that accompany unproctored online exams.

ANYWHERE-ANYTIME ENGINEERING EDUCATION IN A COMPLETE UNDERGRADUATE PROGRAM

Increasingly, many STEM schools are interested in offering their educational programs online. They struggle with several difficulties associated with online education: the structure of programs and individual courses, communication among students and teachers, delivery of learning material, delivery of exams, accreditation, equity between on-campus and off-campus students, and especially the delivery of practical and laboratory training. This paper presents a longitudinal case study of engineering distance education and online learning in Australia. In the early 1990s, Deakin University faced these same challenges when it commenced teaching undergraduate engineering by distance education. It offers a full bachelor of engineering degree in both on-campus and off-campus modes. Student cohorts are approximately 70% on campus, 30% off campus. Fully accredited and part of the Washington Accord, the programs have adapted to advances in communications technology and changes in educational design. The success of the off-campus program was measured by five key performance indicators: enrollments, graduations, attrition, graduate employment, and graduate satisfaction. Data on these measures are presented from 1992 to 2016. The future direction of the school includes an emphasis on design- and project-based learning.

Comparing single-best-answer and very-short-answer questions for the assessment of applied medical knowledge in 20 UK medical schools: Cross-sectional study

ObjectivesThe study aimed to compare candidate performance between traditional best-of-five single-best-answer (SBA) questions and very-short-answer (VSA) questions, in which candidates must generate their own answers of between one and five...

Assessment of First-Year Veterinary Students' Clinical Skills Using Objective Structured Clinical Examinations

The DVM program at the University of Calgary offers a Clinical Skills course each year for the first three years. The course is designed to teach students the procedural skills required for entry-level general veterinary practice. Objective Structured Clinical Examinations (OSCEs) were used to assess students' performance on these procedural skills. A series of three OSCEs were developed for the first year. Content was determined by an exam blueprint, exam scoring sheets were created, rater training was provided, a mock OSCE was performed with faculty and staff, and the criterion-referencing Ebel method was used to set cut scores for each station using two content experts. Each station and the overall exam were graded as pass or fail. Thirty first-year DVM students were assessed. Content validity was ensured by the exam blueprint and expert review. Reliability (coefficient α) of the stations from the three OSCE exams ranged from 0.0 to 0.71. The three exam reliabilities (Generalizability Theory) were, for OSCE 1, G=0.56; OSCE 2, G=0.37; and OSCE 3, G=0.32. Preliminary analysis has suggested that the OSCEs demonstrate face and content validity, and certain stations demonstrated adequate reliability. Overall exam reliability was low, which reflects issues with first-time exam delivery. Because this year was the first that this course was taught and this exam format was used, work continues in the program on the teaching of the procedural skills and the development and revision of OSCE stations and scoring checklists.

Sustainable IT Budgeting: A Method to Determine Not to Exceed Values for Annual Infrastructure Purchases

Picture-archiving and communication systems are complex entities, but at core they consist of compute processors that are networked together to store and retrieve objects. Therein lay fundamental aspects of both performance benchmarking and predicting future costs, provided one can accurately predict trends in both exam volumes and sizes. Hence, determining the correct amount of capital to reserve annually for the information technology infrastructure can be a difficult process for the administrator of a medical center. Both exam volumes and sizes tend to increase over time. In addition, users demand more compute-intensive applications and expect exam delivery to the desktop to be ever timelier despite the increase in size. Against this, storage, compute, and networking costs tend to decrease over time for the same performance level. At the end of the day, the question of whether to budget more or less capital for next year's infrastructure is not trivial. This paper develops a methodology that uses current baseline data to predict the "ampleness" of a budget to meet future needs.

AGRICULTURE STUDENTS’ COMPUTER SKILLS AND ELECTRONIC EXAMS

The purpose of this study was to determine if a statistical relationship existed between academic achievement and exam delivery method for students enrolled in the AgEE 62-Computer Applications in Agriculture course during spring semester 1999. Significant, positive, moderate associations resulted between academic achievement and exam delivery method; agriculture students ( n = 45) who took the quiz using the paper-and-pencil method scored significantly higher than did students who took the quiz in an electronic version only. A significant, moderate relationship existed between academic achievement and learning style for the quiz; field-independent students achieved significantly higher quiz scores than did field-dependent learners. Independent learners had significantly more computer enjoyment than did dependent learners. Field-independent learners had significantly lower levels of computer anxiety after the midterm than did field-dependent learners. AgEE 62 students agreed that taking computer skills exams electronically was not as easy as taking the exams in a more traditional paper-and-pencil fashion. Early identification of field-dependent learners allows the instructor, teaching assistants, and/or cooperative learning teams an opportunity to provide additional assistance for students who find learning computer skills an academic challenge.