Online Assessment Research Articles

Online assessment in higher education: a mapping review and narrative synthesis

Online assessment takes many forms. While there have been reviews on a particular online assessment approach (e.g., online examinations or tests), there has not been a knowledge synthesis that considers online assessment research as a whole. A holistic understanding of online assessment research is important, as it recognizes differences in assessment approaches and the role of technologies in assessment. This understanding helps researchers navigate the heterogeneous body of research and allows educators to make research-informed improvements. To establish such an understanding, we analyzed 235 articles, following a mapping review and a narrative synthesis method. The findings revealed tests, assignments and skills assessments are major online assessment approaches. While research into tests reported using online technologies mainly to substitute or augment existing assessment, research into assignments and skills assessments was more likely to report using online technologies to modify or redefine assessment. Further, we identified disparities across the three approaches regarding academic misconduct, assessment validity, and reliability. We also identified a dearth of comparative research and a reliance on overall satisfaction and short-term self-reported impact measures as indicators of success. We discuss the implications of this review to provide insights for institutions and educators seeking to improve online assessment practices.

In silico design and evaluation of a multiepitope vaccine targeting the nucleoprotein of Puumala orthohantavirus.

The Puumala orthohantavirus is present in the body of the bank vole (Myodes glareolus). Humans infected with this virus may develop hemorrhagic fever accompanying renal syndrome. In addition, the infection may further lead to the failure of an immune system completely. The present study aimed to propose a possible vaccine by employing bioinformatics techniques to identify B and T-cell antigens. The best multi-epitope of potential immunogenicity was generated by combining epitopes. Additionally, the linkers EAAAK, AAY, and GPGPG were utilized in order to link the epitopes successfully. Further, C-ImmSim was used to perform in silico immunological simulations upon the vaccine. For the purpose of conducting expression tests in Escherichia coli, the chimeric protein construct was cloned using Snapgene into the pET-9c vector. The designed vaccine showed adequate results, evidenced by the global population coverage and favorable immune response. The developed vaccine was found to be highly effective and to have excellent population coverage in a number of computer-based assessments. This work is fully dependent on the development of nucleoprotein-based vaccines, which would constitute a significant step forward if our findings were used in developing a global vaccination to combat the Puumala virus.

An online intervention for carers to manage behavioral symptoms in motor neuron disease (MiNDToolkit): a randomized parallel multi-center feasibility trial

Background Evidence on management of behavioral symptoms in motor neuron disease (MND) is lacking. The MiNDToolkit, an online psychoeducational platform, supports carers dealing with behavioral symptoms (BehSymp). The study objectives were to ascertain recruitment and retention rates, carer and healthcare professional (HCP) use of the platform, and completion of online assessments, to inform a full-scale trial. Design: Randomized, parallel, multi-center, feasibility trial. Setting England and Wales, across diverse MND services; recruitment from July/21 to November/22; last participant follow-up in March/23. Participants Carers of people with motor neuron disease (PwMND) with BehSymp, recruited through MND services. After confirming eligibility, participants completed screening and baseline assessments online via the MiNDToolkit platform and were randomized centrally in a 1:1 ratio to MiNDToolkit or control. Intervention MiNDToolkit offered tailored modules to carers for the 3-month study period. Carers in the intervention group could receive additional support from MiNDToolkit trained HCPs. The control group was offered access to the intervention at the end of the study. Data were collected on platform usage and psychosocial variables. Main outcomes One hundred and fifty-one carers from 11 sites were invited to join the study (letter, face-to-face); 30 were screened; 29 were randomized. Fifteen people were allocated to the control arm; 14 to intervention. Carers were mostly female; median age for was 62.5 (IQR: 58, 68; intervention) and 57 (IQR: 56, 70; controls). Study retention was high (24/29 = 82.76%); carers engaged with the platform on average 14 times (median (IQR):14.0 (10.0, 18.5)) during the study period. Conclusion The MiNDToolkit study was feasible and well accepted by carers and trained HCPs. A definitive trial is warranted.

Does Online Testing Make the Grade? Evidence from a Randomized Controlled Trial

Abstract As a result of the increase in online course offerings at colleges and universities, the prevalence of online and computerized examinations has increased over the last two decades. Beyond academia, online examinations have been used to assess competence for certifications by both governments and private firms. Despite this importance, a randomized controlled trial that randomizes individual test takers into online and in-person examinations has not previously been conducted. In this paper, we rectify this by conducting this experiment in a large economics class at a large research university in the American southeast. We supplement our experiment results with a novel methodology of using screening surveys administered to potential volunteers which solicit preference information. We find that online testing reduces examination grades by over a tenth of a standard deviation, relative to in-person assessment. Additional exploratory exercises provide evidence that this effect is driven by students who prefer in-person examinations.

A level playing field? Evaluation of the virtual Objective Structured Clinical Examination in Psychiatry and Addiction Medicine: A mixed methods study.

The Objective Structured Clinical Examination (OSCE) has been used for clinical assessment of a broad range of medical student competencies in Psychiatry and Addiction Medicine. However, there has been little research into online assessments. We investigated the virtual OSCE (v-OSCE) from the user perspective to better understand its acceptability, usefulness, benefits, challenges and potential improvements. At the conclusion of the v-OSCE, all participants (medical students, examiners and simulated patients) were invited to participate in a brief online survey, based on the Technology and Acceptance Model. Freeform qualitative feedback was also obtained to explore participants' experiences and attitudes. Participants reported the v-OSCE was acceptable, efficient, convenient and easy to use. It was perceived as useful for demonstrating students' interviewing skills and interacting with the simulated patient. Benefits included greater convenience, reduced stress and travel time. Challenges were similar to those experienced in 'real world' telepsychiatry, primarily related to assessment of non-verbal cues and emotional prosody. Our findings inform recommendations for improving online examinations. These include increased preparation, practice and professionalism, to better simulate the in-person experience. Study credibility was strengthened by the triangulation of qualitative, quantitative and psychometric data.

ANALYSIS OF PARTICULAR SYSTEMS AND APPLICATIONS IN THE MODEL OF BLENDED LEARNING IN SECONDARY EDUCATION INSTITUTIONS

ANALYSIS OF PARTICULAR SYSTEMS AND APPLICATIONS IN THE MODEL OF BLENDED LEARNING IN SECONDARY EDUCATION INSTITUTIONS

Online Exam Portal

An online test is one that is conducted via the internet or an intranet utilizing a computer system. It is a web-based exam system. It works well as a mass education evaluation solution. Our system for online exams is built on a Browser/Server architecture. The multiple choice questions that are fed into the system are examined and automatically graded by the system. Even while it is comfortable, the classic pen and paper exam format has drawbacks in the current digital learning environment. IT students find these limits even more noticeable because they are used to using internet resources and platforms. In response, this proposal suggests creating an efficient, user-friendly, and safe online assessment system especially for IT students. This project goes beyond practicality. It increases assessment quality with interesting formats, empowers students with freedom, and promotes efficiency with quicker administration and grading. Teachers receive insightful data-driven information for bettering their curricula. In the end, it welcomes the digital age and uses cutting-edge evaluation techniques to get IT students ready for success

Early identification of macrophage activation syndrome secondary to systemic lupus erythematosus with machine learning

ObjectiveThe macrophage activation syndrome (MAS) secondary to systemic lupus erythematosus (SLE) is a severe and life-threatening complication. Early diagnosis of MAS is particularly challenging. In this study, machine learning models and diagnostic scoring card were developed to aid in clinical decision-making using clinical characteristics.MethodsWe retrospectively collected clinical data from 188 patients with either SLE or the MAS secondary to SLE. 13 significant clinical predictor variables were filtered out using the Least Absolute Shrinkage and Selection Operator (LASSO). These variables were subsequently utilized as inputs in five machine learning models. The performance of the models was evaluated using the area under the receiver operating characteristic curve (ROC-AUC), F1 score, and F2 score. To enhance clinical usability, we developed a diagnostic scoring card based on logistic regression (LR) analysis and Chi-Square binning, establishing probability thresholds and stratification for the card. Additionally, this study collected data from four other domestic hospitals for external validation.ResultsAmong all the machine learning models, the LR model demonstrates the highest level of performance in internal validation, achieving a ROC-AUC of 0.998, an F1 score of 0.96, and an F2 score of 0.952. The score card we constructed identifies the probability threshold at a score of 49, achieving a ROC-AUC of 0.994 and an F2 score of 0.936. The score results were categorized into five groups based on diagnostic probability: extremely low (below 5%), low (5–25%), normal (25–75%), high (75–95%), and extremely high (above 95%). During external validation, the performance evaluation revealed that the Support Vector Machine (SVM) model outperformed other models with an AUC value of 0.947, and the scorecard model has an AUC of 0.915. Additionally, we have established an online assessment system for early identification of MAS secondary to SLE.ConclusionMachine learning models can significantly improve the diagnostic accuracy of MAS secondary to SLE, and the diagnostic scorecard model can facilitate personalized probabilistic predictions of disease occurrence in clinical environments.

A Robust Examination of Cheating on Unproctored Online Exams

The rapid growth of online education, especially since the pandemic, is presenting educators with numerous challenges. Chief among these is concern about academic dishonesty, especially on unproctored online exams. Students cheating on exams is not a new phenomenon. The topic has been discussed and debated within institutions of higher learning, and significant levels of cheating have been reported in the academic literature for over sixty years. Much of this literature, however, has focused on student behavior in a classroom utilizing proctored, in-class exams. Grades on exams usually determine most of a student’s final grade in a course, and GPAs are used by employers and graduate schools to indicate a student’s subject matter mastery. As more conventional colleges and universities expand their online course offerings it is natural to wonder if academic dishonesty is more prevalent in online classes than in face-to-face classes. In particular, are students more likely to cheat when no one is watching (i.e., on unproctored assessment assignments) than they do when someone is watching (i.e., on proctored assessment assignments)? The purpose of this study is to investigate whether students cheat more on unproctored online exams than they do on proctored in-classroom exams, and if so, is there any pattern to their cheating behavior. Our findings are derived from careful empirical analysis of 741 undergraduate students who completed three unproctored online exams, several collaboration-encouraged assignments, and a proctored in-class comprehensive final exam in the same course with the same instructor. Additionally, we collected demographic and human capital data for every student. Using bivariate and regression analysis, we find significant evidence of more cheating on unproctored online exams than on proctored in-class exams even though students were given stern honor code violation warnings. Moreover, we discover that student cheating increased with each unproctored online exam, implying that students learn how to cheat as they become more familiar with taking online assessment assignments. Finally, we find that students with certain demographic and human capital characteristics tend to cheat more than others. This research strongly supports the use of proctoring for all evaluation assignments in online classes to ensure that grades in these classes properly reflect student aptitude as opposed to merely reflecting their ability to cheat.

Eating Disorder Treatment Access in the United States: Perceived Inequities Among Treatment Seekers.

Although eating disorders are associated with high rates of psychological and physical impairments and mortality, only about 20% of individuals with eating disorders receive treatment. No study has comprehensively assessed treatment access for those with these disorders in the United States. The authors examined access to eating disorder treatments and how it might vary among some populations. Seekers of treatment for eating disorders (N=1,995) completed an online assessment of clinical demographic and anthropometric characteristics, barriers to eating disorder treatment access, and eating disorder symptomatology. Analyses were conducted to identify treatment access barriers, compare barriers to treatment access across demographic groups, and investigate relationships between barriers to treatment access and eating disorder symptoms. Financial barriers (e.g., lack of insurance coverage) were the most frequently reported barrier to treatment access. Participants with historically underrepresented identities and with a diagnosis of other specified feeding or eating disorder (OSFED) reported more barriers related to financial challenges, geographic location, eating disorder identification, sociocultural factors, and treatment quality compared with those with historically represented identities (e.g., White and cisgender persons). Higher frequencies of reported barriers to treatment access were associated with more severe eating disorder symptoms and poorer illness trajectories. Financial barriers were the most significant impediment to accessing treatment among individuals seeking eating disorder treatment. Barriers to treatment access disproportionally affected underrepresented groups and those with an OSFED diagnosis.

Using technology to prevent fraud in high stakes national school examinations: Evidence from Indonesia

Cheating reduces the signaling value of examinations. It also shifts the focus of teachers and students away from learning. Combating widespread cheating is difficult as students, teachers, and bureaucrats all benefit from high reported grades. We evaluate the impact of computer-based testing (CBT), an at-scale policy implemented by the Indonesian government to reduce widespread cheating in the national examinations. Exploiting the phased roll-out of the program from 2015 to 2019, we find that test scores declined dramatically, by 0.5 standard deviations, after the introduction of CBT. Schools with response patterns that indicated cheating prior to CBT adoption experienced a steeper decline. The effect is similar between schools with and without access to a computer lab, indicating that the reduction in the opportunity to cheat is the main reason for the test score decline. In districts with high adoption of CBT, schools that still used paper-based exams cheated less and scored lower, indicating spillovers of CBT. The results highlight the potential role of technology in improving the effectiveness in efforts to overcome collusive behavior in the education sector.

PSYCHOACOUSTICS-WEB: A free online tool for the estimation of auditory thresholds

PSYCHOACOUSTICS-WEB is an online tool written in JavaScript and PHP that enables the estimation of auditory sensory thresholds via adaptive threshold tracking. The toolbox implements the transformed up-down methods proposed by Levitt (Journal of the Acoustical Society of America, 49, 467-477, (1971) for a set of classic psychoacoustical tasks: frequency, intensity, and duration discrimination of pure tones; duration discrimination and gap detection of noise; and amplitude modulation detection with noise carriers. The toolbox can be used through a common web browser; it works with both fixed and mobile devices, and requires no programming skills. PSYCHOACOUSTICS-WEB is suitable for laboratory, classroom, and online testing and is designed for two main types of users: an occasional user and, above all, an experimenter using the toolbox for their own research. This latter user can create a personal account, customise existing experiments, and share them in the form of direct links to further users (e.g., the participants of a hypothetical experiment). Finally, because data storage is centralised, the toolbox offers the potential for creating a database of auditory skills.

Development of an assessment tool for designated medical institutions in China--Based on the application of an online assessment system.

Due to the expanding coverage of medical insurance and the growth of medical expenses, the ability to assess the performance of designated medical institutions (DMIs) in supporting the delivery of high-quality patient care and the standardized use of funds represents a priority in China. Despite such interest, there has yet to be an operable standard and labor-saving method for assessing DMIs in China. The main objectives include two aspects: (1) establishing an evaluation index system for DMIs based on contracts; (2) designing and developing an online evaluation platform. A group of 20 experts with theoretical and practical expertise in medical insurance regulation and performance evaluation were invited to select available indicators. A combination weighting method based on analytic hierarchy process and entropy method was used to determine the weight coefficient. Shanghai was taken as the sample area, and 760 DMIs were included in the empirical research. The test-retest reliability method and criterion-related validity method was used to test the reliability and the validity of the evaluation result. An assessment index system that included 6 domains and 56 indicators was established in this study. Furthermore, we developed an online platform to assist in the implementation of the assessment. The results showed that the average score of assessment was 94.39, the median was 96.92. The test-retest reliability value was 0.96 (P ≤ 0.01), which indicated high stability of the assessment. In addition, there was a significant negative relationship between assessment score and the penalty amount of DMIs (R = -0.133, P < 0.001). After adjusting for the basic characteristics of medical institutions, the number of visits and revenue, the negative relationship was still significant (B = -0.080, P < 0.05). These results are consistent with expectations, indicating that the assessment had good criterion-related validity. This study established an operable assessment measure and developed an online platform to assess the performance of DMIs. The results showed good feasibility and reliability in empirical research. Our research findings provided an operable Chinese solution for DMI assessment that saves manpower and time, which would have good enlightening significance in other regions of China and in low-income and middle-income countries internationally.

Exploring the influences of online assessments on teaching practice at a language school in Vietnam

Exploring the influences of online assessments on teaching practice at a language school in Vietnam

The Influence of the Teacher’s Psycho-Physiological States on the Development of Professional Competencies in the Course of the Military Aggression

The aim of the article is to determine the specifics of the impact of the correctional programme of psycho-physiological conditions on the development of the professional competence of general secondary school teachers. The Diagnostics of Mental States of the National Academy of Sciences and the Impact of Event Scale - Revised (IES-R) were used to obtain reliable data on the psycho-physiological state of the individual. A specialized online test for teachers was used to obtain data on the teachers’ competence aimed at determining informational competence, testing to determine the teacher’s subject methodical competence, observation of communicative competence. A formative experiment was planned and conducted. It was established that average indicators of psycho-physiological state prevail among teachers. The positive impact of the developed corrective programme on the teachers’ activity and mood was proved. It was concluded that the change in the psycho-physiological states of “invasion” and “avoidance” significantly depends on current social conditions, in particular, on the stability of the safety component. It was found that the correctional programme levelled the connections between the teachers’ subject methodical competence and the current psycho-physiological state. It was established that the teachers’ communicative competence continues to correlate after the completion of the programme with psycho-physiological comfort and general emotional background. In general, we can talk about the relative effectiveness of the developed corrective programme. The obtained results can be used to optimize programmes of continuous postgraduate education in the context of the development of resistance to the stresses of wartime. The prospects for further research are expanding the arsenal of methods for studying the teachers’ psycho-physiological state and clarifying gender differences in the teachers’ experience of stress. Received: 2 February 2024 / Accepted: 25 April 2024 / Published: 5 May 2024

Rapid online assessment of reading and phonological awareness (ROAR-PA)

Phonological awareness (PA) is at the foundation of reading development: PA is introduced before formal reading instruction, predicts reading development, is a target for early intervention, and is a core mechanism in dyslexia. Conventional approaches to assessing PA are time-consuming and resource intensive: assessments are individually administered and scoring verbal responses is challenging and subjective. Therefore, we introduce a rapid, automated, online measure of PA—The Rapid Online Assessment of Reading—Phonological Awareness—that can be implemented at scale without a test administrator. We explored whether this gamified, online task is an accurate and reliable measure of PA and predicts reading development. We found high correlations with standardized measures of PA (CTOPP-2, r = .80) for children from Pre-K through fourth grade and exceptional reliability (α = .96). Validation in 50 first and second grade classrooms showed reliable implementation in a public school setting with predictive value of future reading development.

Online assessment practices during the pandemic in secondary schools in Catalonia

ABSTRACT During the COVID-19 pandemic, teachers had to face different teaching and assessment challenges. This study aims to analyse the relationship between the assessment practices of teachers (related to their assessment conceptions) with their self-perceived level of teachers’ digital competence (TDC) and their received TDC and assessment training courses. For this, the authors applied an online questionnaire that analysed the case of secondary school teachers in Catalonia (Spain) (n = 278). Results show that participating teachers who have the highest self-perception of TDC are the ones who perceive themselves as integrating mainly formative assessment. Significant correlations were also found between recent teacher training and the kind of assessment practices carried out by participants. Official certification of TDC also has a statistically significant association with different assessment characteristics and purposes. Finally, the implications of these findings for pre-service and in-service teacher training are discussed, both regarding the TDC and the assessment competence.

Online Assessment in the Spotlight: A Case Study in a Spanish University

Objective. This exploratory case study aims to analyze the attitudes, practices and preferences of higher education faculty and students in relation to online assessment. Methodology. A quantitative methodology based on a questionnaire was used, and a descriptive and correlational analysis was conducted. Participants included students and faculty members from a Spanish on-site university. Results. Certain similarities and differences in attitudes, practices and preferences between students and teachers are presented. Some relationships, albeit weak, were also identified between the online assessment practices used and some characteristics of the teachers’ profile or teaching (teaching experience, teaching load and class size). Conclusions. The findings show that a large part of the teaching staff shows some reluctance in the design and implementation of online assessment strategies beyond traditional exams, as well as a great concern about student control and the danger of fraud. On the other hand, students show generally more positive attitudes towards online assessment than the teaching staff. The recommendations point to the consideration of the role of institutional support and teacher training units for the improvement of teachers’ digital competence in relation to online assessment, the importance of updating security systems in technical infrastructures, and adjusting the effects of teaching load and class size when online assessment is applied.

Comparing the Performance of ChatGPT-4 and Medical Students on MCQs at Varied Levels of Bloom's Taxonomy.

This research investigated the capabilities of ChatGPT-4 compared to medical students in answering MCQs using the revised Bloom's Taxonomy as a benchmark. A cross-sectional study was conducted at The University of the West Indies, Barbados. ChatGPT-4 and medical students were assessed on MCQs from various medical courses using computer-based testing. The study included 304 MCQs. Students demonstrated good knowledge, with 78% correctly answering at least 90% of the questions. However, ChatGPT-4 achieved a higher overall score (73.7%) compared to students (66.7%). Course type significantly affected ChatGPT-4's performance, but revised Bloom's Taxonomy levels did not. A detailed association check between program levels and Bloom's taxonomy levels for correct answers by ChatGPT-4 showed a highly significant correlation (p<0.001), reflecting a concentration of "remember-level" questions in preclinical and "evaluate-level" questions in clinical courses. The study highlights ChatGPT-4's proficiency in standardized tests but indicates limitations in clinical reasoning and practical skills. This performance discrepancy suggests that the effectiveness of artificial intelligence (AI) varies based on course content. While ChatGPT-4 shows promise as an educational tool, its role should be supplementary, with strategic integration into medical education to leverage its strengths and address limitations. Further research is needed to explore AI's impact on medical education and student performance across educational levels and courses.

PROPHETA: A NEW MORPHOMETRIC ANALYSIS SOFTWARE FOR INDUSTRIAL PRODUCT DESIGN

ABSTRACT: This paper introduces ProPheTA (Product Phenetics-based Typology Assessment) as a new morphometric analysis software for industrial, phenetics-based product design (PP). Phenetics is used in Biology to determine similarities between species by comparing their formal external attributes. This way, PP replicates this procedure and, by doing a morphological analysis of different models from the same product family, makes it possible to relate their morphology with the user's emotional, perceptive, hedonic and buying intention response. All this requires working with CAD models, aligning, scaling, selecting them along the X, Y and Z axes, editing images, obtaining dissimilarity indexes, and correlating them with the user's response. Various CAD and morphometric computation tools not explicitly developed for these procedures are used to fulfil these tasks. File compatibility problems and excessive manual work and working time must be circumvented. ProPheTA has been developed in Matlab to solve these problems using a single platform. Its performance in a previous PP study, which related shape to buying intention, has been evaluated to demonstrate its reliability and speed. Results show a ten-time speed improvement in the computation with identical dissimilarity results. In addition, ProPheTA offers several advantages, such as ease of use, robust batch-processing capabilities, multiple output formats and improved compatibility. These functionalities, together with its ability to analyze the outline of 3D shapes without manual selection of landmark points, make it a versatile software application which is flexible, reliable and much faster. Thus, ProPheta is arguably valuable for helping speed up the application of PP in industrial design. Keywords: phenetics, product design, morphology, computer-based assessment