Online Assessment Research Articles

Fundamental Considerations in Technology Mediated Language Assessment (Book Review)

Computer-based testing has been widely used across many testing environments, serving both high-stakes and low-stakes contexts. Its global use calls for the language assessment community’s attention to reevaluate some fundamental issues, in particular the extent to which these test scores are reliable, consistent, and meaningful. In light of these needs, Sadeghi and Douglas’s recently published edited volume Fundamental Considerations in Technology-Mediated Language Assessment provides a timely treatment on this topic, exploring not only critical issues in computer-based testing but also topics related to the implementation of technology in other aspects of language assessment. This book will appeal to a broader audience beyond just professionals in the field of language assessment because it offers research-based and theoretical-based knowledge for professionals, teachers, and policymakers interested in exploring and developing computer-based tests. [First Paragraph of Review]

Online Assessment of Spontaneous Mental Fatigue in Construction Workers Considering Data Quality: Improved Online Sequential Extreme Learning Machine

Online Assessment of Spontaneous Mental Fatigue in Construction Workers Considering Data Quality: Improved Online Sequential Extreme Learning Machine

Efficacy of the CARES® Dementia 5-Step Method for Hospitals™ Online Training and Certification Program for hospital staff

The main objective of the current study was to ascertain the efficacy of a newly developed online, video-based dementia training and certification program for hospital staff, the CARES® Dementia 5-Step Method for Hospitals™ Online Training and Certification Program. A parallel randomized waitlist control design was utilized. Participants (N = 272) completed online pre-evaluation measures. Forty-five days after completion of CARES® (treatment) or pre-evaluation measures (control), participants completed a post-evaluation online assessment. Analyses of variance found that participants in the treatment condition indicated greater and statistically significant (p < .001) increases in sense of competence, approaches to dementia, and patient care. The results implied that the CARES® Dementia 5-Step Method for Hospitals™ program could serve as a dynamic resource for hospitals nationwide to update training and share experiences, perspectives, and resources. The inherent scalability of the program regarding its acceptability, feasibility, and potential to incorporate seamlessly into routine workflows and staff training suggests high implementation potential.

Arabic Question Generation Using Transformers

After the increased reliance on online education, online assessment became an essential tool for educators to remotely monitor and evaluate students’ understanding in order to assist them properly. However, the laborious process of creating exam questions is a challenge for most teachers. Thus, automated Question Generation aims to assist teachers by generating questions from given data. Limited research has been conducted to tackle this issue in the Arabic Language due to the complexity of the language and the limited amount of available Arabic data. This paper explores different implementations of the transformer models, that demonstrated their superiority in natural language processing. Three approaches were introduced to tackle this problem with Arabic data using an Arabic-based transformer, an English-based transformer, and a multilingual-based transformer. Each of the fine-tuned models was trained using ARCD, XGLUE, DialectBench, and ArabicQA data sets and evaluated on automatic and manual metrics. Two of the proposed models achieve state-of-the-art results on the Arabic question generation task. The English transformer obtained a ROUGE score of 0.59 on XGLUE, while the Arabic transformer model achieves 0.49 on ARCD. Both of these models demonstrate excellent quality of questions through human-conducted evaluations by achieving low WER and high GC, U, and A scores.

A comprehensive approach for mitigating impersonation in online assessment: integrity policy and random authentication

A comprehensive approach for mitigating impersonation in online assessment: integrity policy and random authentication

Frailty and quality of life in italian older adults: implications for public health interventions

Abstract Background The increasing focus on frailty in older adults underscores the significance of a timely multidimensional assessments and early detection to avert adverse outcomes. The primary aim is to integrate digital technologies, specifically online assessment tools, for the identification of frailty and the assessment of Quality of Life (QoL) in elderly individuals. Methods A nationwide multicentric study was conducted. Seven Italian centers and regions evaluated frailty in older adults residing in the community, through a prospective observational cohort study. The chosen questionnaires, Sunfrail+ and SF-12, were implemented through an easily accessible online tool, swiftly generating final scores for each test. The cross-sectional analysis included 503 older adults. The cut off to define a person “frail” is 3 positive alerts in Sunfrail baseline items, according to the literature. Results In this sample an average of 2.31 positive alerts were generated. Significant associations were established between the number of positive warnings and SF-12 physical and mental indexes, indicated by X2=53.1 (p &amp;lt; 0.001) and X2=25.1 (p &amp;lt; 0.001), respectively. The difference between two groups (frail and not frail) in relation to QoL is statistical significantly (U di Mann Whithney p &amp;lt; 0.001). Mean QoL mental and physical indexes are 46.3 and 39.3 in frailty group, versus 51.7 and 46.5 in non frailty group, respectively. The correlation is conceptually consistent: as frailty increases, QoL decreases. Conclusions The frailty assessment associated with the QoL assessment provides relevant information to monitor the impact of prevention interventions at community level. The assessment of frailty leads the prevention intervention whose impact could be assessed also through QoL tools. This study underlines a meaningful association between assessment tools, substantiated by conceptual correlations. Key messages • Sunfrail and SF-12 are easily accessible online tool useful for identification of frailty and the assessment of Quality of Life (QoL) in elderly individuals. • The older population in Europe is increasing a timely assessment of frailty must be a public health priority.

Social Comparison, Belongingness, Self-Doubt, and Stress: The Case of Hispanic Students at Hispanic Majority Institutions

Social comparison processes can affect academic and interpersonal outcomes for minoritized college students. However, the literature has not addressed how these processes play out for Hispanic students who attend Hispanic Majority Institutions, in which Hispanic students comprise more than 50% of the population. In this study, such students ( N = 191) completed online assessments for social comparison orientation (SCO), belongingness, academic self-concept, and perceived stress. Results showed that, regarding belongingness, SCO was positively correlated to rejection/exclusion, but not acceptance/inclusion. Regarding academic self-concept, SCO was positively correlated to self-doubt, but not self-confidence. Mediation analyses showed that perceived stress fully mediated the relationship between SCO and rejection/exclusion, and partially mediated the relationship between SCO and academic self-doubt. In other words, SCO was associated with increased stress, which was then associated with increased feelings of rejection/exclusion and self-doubt about academic abilities. These findings offer additional insight into Hispanic college students’ experience.

Exploration of Blended Teaching Mode for Ideological and Political Courses in Universities

In the era of “Internet plus,” the hybrid teaching model has injected new vitality into the reform of ideological and political education in colleges and universities, promoted the connection between online and offline teaching, built an air cloud classroom, promoted the sharing of high-quality educational resources, and enabled the reform of ideological and political education, which is conducive to stimulating students’ interest in learning ideological and political courses and improving the quality of ideological and political education. College ideological and political education teachers should actively learn the concept of blended learning, create micro lessons based on teaching content, introduce current political news, and use micro lessons to guide students’ pre-class preparation. Universities can carry out online live teaching, conduct interactive sessions, group discussions, and online testing to improve the quality of online teaching, collect online teaching data, conduct precise offline teaching, and optimize the connection between in-class and out-of-class teaching, utilizing big data to conduct teaching evaluation, collect student teaching suggestions, and comprehensively improve the quality of blended learning in ideological and political courses.

Perspectives of Czech High School Students on Cheating in Remote Education

This study explores the perspectives of Czech high school students on using digital technologies for cheating during remote education, a phenomenon encouraged by the abrupt transition to online learning due to the COVID-19 pandemic. Through a mixed-methods research design, this study collected data via an online questionnaire with 24 questions completed by 316 students from secondary schools and conducted individual interviews with seven teachers. In their study, the authors focus on three questions: How much do high school students know which digital technology they can use for cheating in remote online education? What are their experiences with cheating in remote online education? What are the preventive solutions recommended by students and teachers to eliminate online cheating? The research findings indicate that students perceive online cheating as justifiable, primarily due to their disengagement with the course material and lack of interest in learning under the pandemic's extraordinary circumstances. Students frequently used technical means for cheating, with mobile phones and screen-sharing during online tests being the most popular methods. Additionally, the use of internet resources to find answers was the most commonly reported form of online cheating. To address this issue, students outlined various anti-cheating strategies, categorised into three perspectives: pessimistic, practical, and critical-realistic. The pessimistic perspective indicates scepticism about the feasibility of effectively combating cheating, whereas the practical perspective offers tangible, immediate solutions. The critical-realistic view provides a thoughtful and pragmatic approach to the problem, suggesting a balance between understanding the complexities of cheating and implementing realistic measures. The study highlights educators' critical role and emphasises the importance of aligning assessment methods with students' learning needs and expectations to reduce cheating tendencies naturally. The rise of AI technologies has introduced new dimensions to the problem of online cheating, as AI tools can facilitate academic dishonesty. However, AI can help detect and prevent cheating. The authors suggest using formative assessments and creating an engaging, supportive online learning environment to deter dishonesty and promote academic integrity.

Using Continuous Online Assessment of Learning Outcomes for Grading: A Case Study

In recent years, the problem of the number of students enrolled in STEM courses falling below the minimum required to carry out the courses has worsened. This situation, which used to occur primarily in degree programs at the end of their life cycle, is increasingly happening in active STEM degree programs as well. The declining attractiveness of traditional STEM degree programs and negative demographic trends are seen as the main causes of this problem. To assist students and their teachers in this situation, a novel approach to grading student work based on the level at which students achieve learning outcomes in an online system was explored. Since the focus was on STEM students, math-based tasks were used to assess student knowledge within the online system. In addition to the accuracy of the final results of the tasks, the accuracy of the intermediate results was also monitored. Each result was linked to the corresponding concepts, which in turn were linked to the corresponding learning outcomes. By entering correct intermediate and final results, the online system was able to monitor student success in mastering concepts and learning outcomes. The implemented online system utilizes fuzzy inference to calculate the levels at which students have achieved the learning outcomes. These levels are continuously recalculated during the semester and presented to the students through generated recommendations after each solved task. The proposed approach has positively impacted students’ motivation to learn during the semester, as confirmed by an anonymous questionnaire. Additionally, the results have shown that the final level at which students have achieved the learning outcomes within the online system, calculated at the end of the semester, can be used for grading and is comparable to the grades students would achieve in traditional midterm and final exams, thus helping teachers with their workload.

Assessment Technologies for Enhancing Civil Engineering Education: A Comprehensive Review

This review paper provides a comprehensive analysis of assessment technologies used to enhance civil engineering education. Drawing from a selection of recent research papers published between 2020 and 2024, the paper synthesizes insights into various assessment methods and their impact on student learning outcomes in civil engineering programs. The review covers diverse topics such as project-based learning, virtual laboratories, online assessments, mobile learning, gamification, and augmented reality. Each paper is analysed for its methodology, key findings, and implications for civil engineering education. Through this review, the paper aims to offer educators and policymakers valuable insights into the effectiveness of assessment technologies in fostering student engagement and improving learning experiences in civil engineering programs. Keywords: Assessment technologies, Civil engineering education, Project-based learning, Online assessments, Mobile learning

A computer-based evaluation system: Its design, implementation and results obtained on a general chemistry course

Using multiple evaluation methods and systems give a comprehensive assessment. A computer-based multiple-choice assessment system was designed, implemented, posted online, and used to assess students as part of their final evaluation marks for a discipline. The online system of evaluation was intended to be used multiple times for evaluating the assimilation degree of a specific course at the end of the course. The data recorded for the period 2017–2023 with about 1400 distinct users were used to analyze the performance of the evaluation system. The system worked fine and a slight modification of it served well on remote evaluation during COVID-19 period. However, the upturn of mobile phone applications requires the creation of a system adapted to the new virtual reality.

Online unsupervised performance-based cognitive testing: A feasible and reliable approach to scalable cognitive phenotyping of Parkinson's patients

IntroductionA better understanding of the heterogeneity in the cognitive and mood symptoms of Parkinson's disease will require research conducted in large samples of patients. Fully online and remote research assessments present interesting opportunities for scaling up research but the feasibility and reliability of remote and fully unsupervised performance-based cognitive testing in individuals with Parkinson's disease is unknown. This study aims to establish the feasibility and reliability of this testing modality in Parkinson's patients. MethodsSixty-seven Parkinson's patients and 36 older adults completed two sessions of an at-home, online battery of five cognitive tasks and three self-report questionnaires. Feasibility was established by examining completion rates and data quality. Test-retest reliability was evaluated using the Intraclass Correlation Coefficient (ICC (2,1)). ResultsOverall completion rates and data quality were high with few participant exclusions across tasks. With regards to test-retest reliability, intraclass correlation coefficients were quite variable across measures extracted from a task as well as across tasks, but at least one standard measure from each task achieved moderate to good reliability levels. Self-report questionnaires achieved a higher test-retest reliability than cognitive tasks. Feasibility and reliability were similar between Parkinson's patients and older adults. ConclusionThese results demonstrate that remote and unsupervised testing is a feasible and reliable method of measuring cognition and mood in Parkinson's patients that achieves levels of test-retest reliability that are comparable to those reported for standard in-person testing.

Pedagogical technologies for teaching environmental engineering in the online learning system of higher education institution

The article outlines the pedagogical technologies for teaching environmental engineering in the online learning system of higher education institutions. There are proposed ways of developing educational technologies for teaching environmental engineering: active and collaborative learning, individualisation of training, use of video and audio materials, using electronic tools for evaluation of educational results, development of digital literacy, implementation of interactive online courses and professional communication and mentoring. There are presented the use of these technologies in the different aspects of environmental engineering, such as analysing water quality parameters, soil quality assessment, studying waste management practices, and studying the integration of renewable energy into the practice through virtual labs, online tests, video lectures, and collaborative projects. The proposed pedagogical technologies for teaching environmental engineering in the online learning system of higher education institutions were tested in the educational process. The Student's t-test was used for statistical calculation of the obtained experimental data. At the end of the experiment, there is a statistically significant difference between the mean values of the experimental and control groups. It can be claimed that the use of pedagogical technologies for teaching environmental engineering in the online learning system of higher education institutions is perspective.

Prevalence and associated factors of ECG abnormality patterns indicative of cardiac channelopathies among adult general population of Tehran, Iran: a report from the Tehran Cohort Study (TeCS)

BackgroundThe characteristics of electrocardiogram (ECG) abnormalities related to cardiac channelopathies potentially linked to sudden cardiac death (SCD) are not widely recognized in Iran. We examined the prevalence of such ECG patterns and their related factors among adult residents of Tehran, Iran.MethodsThe clinical characteristics and 12-lead ECGs of Tehran Cohort Study participants were examined. Long QT intervals, short QT intervals, Brugada syndrome (BrS) patterns, and early repolarization (ER) were evaluated using computer-based assessment software validated by cardiologists. Logistic regression models were employed to identify the factors associated with the prevalence of different ECG patterns.ResultsOut of 7678 available ECGs, 7350 were included in this analysis. Long QT interval, ER pattern, BrS patterns, and short QT interval were found in 3.08%, 1.43%, 0.31%, and 0.03% of participants, respectively. The prevalence of long QT interval increased with age, opium consumption, and presence of hypertension. Younger age, lower body mass index (BMI), alcohol use and male sex were independently linked to an elevated prevalence of ER pattern. Most individuals with BrS patterns were men (95%) and had lower BMI, high- and low-density lipoprotein, and total cholesterol compared to those without the BrS pattern. At a mean follow-up of 30.2 ± 5.5 months, all-cause mortality in the group exhibiting abnormal ECG patterns (6.3%) was approximately twice as high as that in the group without such patterns (2.96%).ConclusionAbnormal ECG patterns corresponding to channelopathies were relatively rare among adult residents of the Tehran population, and their prevalence was influenced by various factors.Clinical trial numberNot applicable.

The legality and appropriateness of keeping Korean Medical Licensing Examination items confidential: a comparative analysis and review of court rulings.

This study examines the legality and appropriateness of keeping the multiple-choice question items of the Korean Medical Licensing Examination (KMLE) confidential. Through an analysis of cases from the United States, Canada, and Australia, where medical licensing exams are conducted using item banks and computer-based testing, we found that exam items are kept confidential to ensure fairness and prevent cheating. In Korea, the Korea Health Personnel Licensing Examination Institute (KHPLEI) has been disclosing KMLE questions despite concerns over exam integrity. Korean courts have consistently ruled that multiple-choice question items prepared by public institutions are non-public information under Article 9(1)(v) of the Korea Official Information Disclosure Act (KOIDA), which exempts disclosure if it significantly hinders the fairness of exams or research and development. The Constitutional Court of Korea has upheld this provision. Given the time and cost involved in developing high-quality items and the need to accurately assess examinees' abilities, there are compelling reasons to keep KMLE items confidential. As a public institution responsible for selecting qualified medical practitioners, KHPLEI should establish its disclosure policy based on a balanced assessment of public interest, without influence from specific groups. We conclude that KMLE questions qualify as non-public information under KOIDA, and KHPLEI may choose to maintain their confidentiality to ensure exam fairness and efficiency.

Differing effectiveness of transcranial random noise stimulation and transcranial direct current stimulation for enhancing working memory in healthy individuals: a randomized controlled trial

BackgroundTranscranial direct current stimulation (tDCS) applied to the left dorsolateral prefrontal cortex (DLPFC) is a promising technique for enhancing working memory (WM) performance in healthy and psychiatric populations. However, limited information is available about the effectiveness of transcranial random noise stimulation (tRNS) applied to the left DLPFC on WM. This study investigated the effectiveness of tRNS on WM compared with that of tDCS, which has established functional evidence.MethodsThis randomized, double-blind, sham-controlled trial enrolled 120 healthy right-handed adults who were randomly allocated to four stimulation groups: tRNS + direct current (DC) offset, tRNS, tDCS, or sham. Each stimulus was placed over the left DLPFC and had a current intensity of 2 mA applied for 20 min during the dual n-back task. The dual n-back task was repeated thrice: pre-stimulation, during stimulation, and post-stimulation. The d-prime scores, and response times were calculated as the main outcome measures. A linear mixed model was created to identify the main effects and interactions between the groups and times, with the group and time as fixed effects, and baseline performance and the subject as a covariate and random effect, respectively. The relationships between the benefit of each stimulus and baseline WM performance were also examined.ResultsFor the d-prime score during stimulation, the tRNS group significantly performed better than the sham group at online assessment (β = 0.310, p = 0.001). In the relationships between the benefit of each stimulus and baseline WM performance, the tRNS group had significantly larger negative line slopes than the sham group for the d-prime score (β = −0.233, p = 0.038).ConclusionstRNS applied to the left DLPFC significantly improved WM performance and generated greater benefits for healthy individuals with lower WM performance. These findings highlight the potential utility of tRNS for enhancing WM performance in individuals with lower WM performance and contribute evidence for clinical application to patients with cognitive decline.Trial RegistrationThis study was registered in the University Hospital Medical Information Network Clinical Trial Registry in Japan (UMIN000047365) on April 1, 2022; https://center6.umin.ac.jp/cgi-open-bin/ctr/ctr_view.cgi?recptno=R000054021.

Empirical Analysis of Widely Used Website Automated Testing Tools

In today's software development, achieving product quality while minimising cost and time is critical. Automated testing is crucial to attaining these goals by lowering inspection efforts and discovering faults more effectively. This paper compares widely used automated testing tools, such as Selenium, Appium, Java Unit (JUnit), Test Next Generation (TestNG), Jenkins, Cucumber, LoadRunner, Katalon Studio, Simple Object Access Protocol User Interface (SoapUI), and TestComplete, based on functionality, ease of use, platform compatibility, and integration capabilities. Our findings show that no single tool is inherently superior, with each excelling in certain areas such as online, mobile, Application Programming Interface (API), or performance testing. While Selenium and Appium are the dominant online and mobile testing frameworks, TestComplete and Katalon Studio offer complete, user-friendly cross-platform testing solutions. Despite the benefits of automation, obstacles such as tool maintenance, scalability, and cost issues remain. The report finishes with advice for picking the best tool for the project and offers potential approaches for enhancing testing frameworks, such as AI-driven optimisation, cloud-based testing, and greater Continuous Integration/ Continuous Deployment (CI/CD) integration. This study offers useful information for developers and testers looking to optimise their testing methods and increase software quality.

Analisis Kemampuan Mahasiswa Matematika FMIPA Unimed dalam Menyelesaikan Pertidaksamaan Nilai Mutlak dengan Berbantuan Python

This research aims to analyze the ability of Unimed FMIPA Mathematics students in solving absolute value inequalities with the help of the Python program. The problem faced is that students often have difficulty solving inequality problems manually. The purpose of this research is to see the effectiveness of using Python in helping to solve absolute value inequalities and how it affects understanding of mathematical concepts. The method used was qualitative research with a sample of 20 fifth semester students of the Mathematics Study Program, FMIPA, Medan State University. Students are given absolute value inequality problems to solve manually and with the help of Python. Data was collected through online tests and questionnaires using Google Form. The research results show that the majority of students feel that using Python is very helpful in solving absolute value inequalities. As many as 95% of students consider Python to be effective in making it easier to solve mathematical problems and increasing understanding of the concept of absolute value.

Enhancing Integration of Technology in Authentic Assessment for Education: A Structured Review

In the ever-evolving landscape of education, the seamless integration of technology into authentic assessment practices has become a transformative force, reshaping how students are both evaluated and engaged. Authentic assessment, characterized by its emphasis on the real-world application of knowledge and skills, has gained prominence for its ability to gauge students' comprehension and foster deeper learning experiences. As technology continues to infiltrate the educational field, a pressing need arises to assess the effectiveness of these digital tools in promoting genuine understanding and critical thinking among students. This structured review of the PRISMA approach was applied to find the primary data based on keywords such as Technology, Science, and Authentic Assessment. Based on advanced searching on SCOPUS and Web of Science, we found (n=55). Expert scholar decided to develop 3 themes, which are (1) Online Assessment and Technology- Enhanced Learning, (2) Educational Practices and Student Engagement, and (3) Assessment and Curriculum Development. The findings underscore technology's immense potential in elevating authentic assessments for students, offering new avenues for engagement and skill development. It is evident that when wielded thoughtfully and integrated into pedagogy, technology can empower young learners to explore, inquire, and excel in the fascinating world of education. This study not only bridges existing gaps in the literature but also serves as a valuable resource for educators and policymakers alike, providing insights into harnessing technology's power to enhance authentic assessment practices and, consequently, the overall educational experience.