Online Assessment System Research Articles

Effects of familiarity with digital assessment in online education

This study analyses changes in students’ perceptions of online examinations during the transition from a face-to-face to a fully online assessment system. We compare data from a survey administered to two samples of students at a distance learning university at the end of two consecutive academic years in which a new online examination system was introduced. The survey focused on students’ experiences and opinions of online examinations and the digital requirements associated with them. Analyses of variance were conducted on the data from the two student samples, finding significant evidence of changes in students’ perceptions as they became more familiar with online examinations. The results suggest that familiarity with digital tools positively influences preference for online over face-to-face exams, as well as positive perceptions of the procedures associated with online exams, particularly webcam proctoring and the use of a digital interface to answer exam questions.

초등학생용 수행형 디지털 리터러시 온라인 평가도구 개발

Objectives The purpose of this study is to develop a performance-based online assessment tool to measure the level of digital literacy of primary school students. Methods To achieve this, test items were developed based on several scenarios that reflected the context of schools, families, and communities that primary school students encounter in their daily lives, and an online as-sessment system was developed in the form of several sub-tasks being connected to perform one integrated task. The developed assessment tool consisted of 4 test modules and a total of 28 sub-task items. A preliminary test was conducted on 462 primary school students to verify the reliability and validity of the developed online as-sessment system. Results As a result of analyzing the preliminary test results, the test reliability was found to be good. As a result of correlation analysis with the multiple-choice assessment tool, it was found to be distinguished from the exist-ing tool as a performance-based assessment tool. In addition, as a result of item analysis, the overall discrim-ination and difficulty level was found to be at an appropriate level except for a few items. Conclusions This study is significant in that it developed a performance-based online assessment tool beyond self-reporting or knowledge-based multiple-choice assessment, and provided primary data for future research on measuring students’ digital literacy level using a performance-based online assessment tool based on the pre-liminary test results.

A study on the design of a deep learning model for classroom based on user participation and game chemistry processes

In gamified learning, user engagement has always been an important concern in the field of education. For mass learning, the integration of classroom learning and gamification can make users well-stimulated and help them to improve and cultivate their comprehensive abilities. With the characteristics of efficiency, fairness, and objectivity, online assessment systems (OJ systems) have become an important tool for learning. However, the following limitations still exist in current OJ systems: First, most OJ systems cannot perceive and quantify the user’s learning level at the knowledge point level. This can lead to a mismatch between the difficulty of study questions adapted by the system to the user and the user’s learning ability; second, most OJ systems lack research and design of motivation methods. These limitations can negatively affect the enhancement of students’ motivation and the consolidation and improvement of their learning ability. Given this, the article designs and implements a gamified online assessment system (Game_OJ) based on user participation and the game chemistry process. The online assessment system uses a modified Bayesian knowledge tracking model (CC-BKT). The application of the Bayesian knowledge tracking model can accurately perceive and quantify students’ learning of each knowledge point. The online assessment system introduces the gamification idea to design the motivation methods in the OJ system, including the design of gamification elements and the design of the challenge process of gamified learning, aiming to enhance students’ motivation to learn. It promotes students’ deep learning in the classroom and outside the classroom. The CC-KPT model proposed in this paper has an average increase of 2.15 % in AUG value and an average increase of 3.25 % in accuracy compared to the same type of model, and the speed of response and the median speed of processing demand of this system is within 1 s when 500 users use the system at the same time, which meets the needs of on-campus teaching scenarios.

Online Exam Portal

An online test is one that is conducted via the internet or an intranet utilizing a computer system. It is a web-based exam system. It works well as a mass education evaluation solution. Our system for online exams is built on a Browser/Server architecture. The multiple choice questions that are fed into the system are examined and automatically graded by the system. Even while it is comfortable, the classic pen and paper exam format has drawbacks in the current digital learning environment. IT students find these limits even more noticeable because they are used to using internet resources and platforms. In response, this proposal suggests creating an efficient, user-friendly, and safe online assessment system especially for IT students. This project goes beyond practicality. It increases assessment quality with interesting formats, empowers students with freedom, and promotes efficiency with quicker administration and grading. Teachers receive insightful data-driven information for bettering their curricula. In the end, it welcomes the digital age and uses cutting-edge evaluation techniques to get IT students ready for success

Early identification of macrophage activation syndrome secondary to systemic lupus erythematosus with machine learning

ObjectiveThe macrophage activation syndrome (MAS) secondary to systemic lupus erythematosus (SLE) is a severe and life-threatening complication. Early diagnosis of MAS is particularly challenging. In this study, machine learning models and diagnostic scoring card were developed to aid in clinical decision-making using clinical characteristics.MethodsWe retrospectively collected clinical data from 188 patients with either SLE or the MAS secondary to SLE. 13 significant clinical predictor variables were filtered out using the Least Absolute Shrinkage and Selection Operator (LASSO). These variables were subsequently utilized as inputs in five machine learning models. The performance of the models was evaluated using the area under the receiver operating characteristic curve (ROC-AUC), F1 score, and F2 score. To enhance clinical usability, we developed a diagnostic scoring card based on logistic regression (LR) analysis and Chi-Square binning, establishing probability thresholds and stratification for the card. Additionally, this study collected data from four other domestic hospitals for external validation.ResultsAmong all the machine learning models, the LR model demonstrates the highest level of performance in internal validation, achieving a ROC-AUC of 0.998, an F1 score of 0.96, and an F2 score of 0.952. The score card we constructed identifies the probability threshold at a score of 49, achieving a ROC-AUC of 0.994 and an F2 score of 0.936. The score results were categorized into five groups based on diagnostic probability: extremely low (below 5%), low (5–25%), normal (25–75%), high (75–95%), and extremely high (above 95%). During external validation, the performance evaluation revealed that the Support Vector Machine (SVM) model outperformed other models with an AUC value of 0.947, and the scorecard model has an AUC of 0.915. Additionally, we have established an online assessment system for early identification of MAS secondary to SLE.ConclusionMachine learning models can significantly improve the diagnostic accuracy of MAS secondary to SLE, and the diagnostic scorecard model can facilitate personalized probabilistic predictions of disease occurrence in clinical environments.

Development of an assessment tool for designated medical institutions in China--Based on the application of an online assessment system.

Due to the expanding coverage of medical insurance and the growth of medical expenses, the ability to assess the performance of designated medical institutions (DMIs) in supporting the delivery of high-quality patient care and the standardized use of funds represents a priority in China. Despite such interest, there has yet to be an operable standard and labor-saving method for assessing DMIs in China. The main objectives include two aspects: (1) establishing an evaluation index system for DMIs based on contracts; (2) designing and developing an online evaluation platform. A group of 20 experts with theoretical and practical expertise in medical insurance regulation and performance evaluation were invited to select available indicators. A combination weighting method based on analytic hierarchy process and entropy method was used to determine the weight coefficient. Shanghai was taken as the sample area, and 760 DMIs were included in the empirical research. The test-retest reliability method and criterion-related validity method was used to test the reliability and the validity of the evaluation result. An assessment index system that included 6 domains and 56 indicators was established in this study. Furthermore, we developed an online platform to assist in the implementation of the assessment. The results showed that the average score of assessment was 94.39, the median was 96.92. The test-retest reliability value was 0.96 (P ≤ 0.01), which indicated high stability of the assessment. In addition, there was a significant negative relationship between assessment score and the penalty amount of DMIs (R = -0.133, P < 0.001). After adjusting for the basic characteristics of medical institutions, the number of visits and revenue, the negative relationship was still significant (B = -0.080, P < 0.05). These results are consistent with expectations, indicating that the assessment had good criterion-related validity. This study established an operable assessment measure and developed an online platform to assess the performance of DMIs. The results showed good feasibility and reliability in empirical research. Our research findings provided an operable Chinese solution for DMI assessment that saves manpower and time, which would have good enlightening significance in other regions of China and in low-income and middle-income countries internationally.

AssessChain: A hybrid blockchain-based system for transparent and reliable online assessment

AssessChain: A hybrid blockchain-based system for transparent and reliable online assessment

Development and application of safety culture analysis program: First online safety culture quantitative analysis and assessment system in China

To promote safety culture assessment, this paper developed China's first online safety culture analysis and evaluation system—Safety Culture Analysis Program (SCAP). SCAP is based on the 32 safety culture elements in the 24Model and provides two login methods: web version and WeChat mini program. It can meet the needs of individuals, groups, and enterprises to conduct safety culture assessments anytime and anywhere and provide convenience for universities to conduct safety culture experiments. The application of SCAP showed that (1) SCAP has rich functions and is highly scientific and universal. (2) The SCAP can output assessment results according to categories, such as total, personnel, safety culture elements, and individual employee requirements, which can be compared with the highest and lowest values stored in the system. This makes it easy for intuitively understanding the current evaluation status. (3) The suggestion report generated by SCAP can help assessment units improve safety culture in a targeted manner and form a dynamic mechanism cycle of “construction-evaluation-construction.” (4) In the future, SCAP will be employed in designing more targeted questions based on the characteristics of different industries and fields to balance the adaptability and specificity of the industry.

A systematic review of educational online peer-review and assessment systems: charting the landscape

Over the past two decades, there has been an explosion of innovation in software tools that encapsulate and expand the capabilities of the widely used student peer assessment. While the affordances and pedagogical impacts of traditional in-person, “paper-and-pencil” peer assessment have been studied extensively and are relatively well understood, computerized (online) peer assessment introduced not only shifts in scalability and efficiency, but also entirely new capabilities and forms of social learning interactions, instructor leverage, and distributed cognition, that still need to be researched and systematized. Despite the ample research on traditional peer assessment and evidence of its efficacy, common vocabulary and shared understanding of online peer-assessment system design, including the variety of methods, techniques, and implementations, is still missing. We present key findings of a comprehensive survey based on a systematic research framework for examining and generalizing affordances and constraints of online peer-assessment systems. This framework (a) provides a foundation of a design-science metatheory of online peer assessment, (b) helps structure the discussion of user needs and design options, and (c) informs educators and system design practitioners. We identified two major themes in existing and potential research—orientation towards scaffolded learning vs. exploratory learning and system maturity. We also outlined an agenda for future studies.

Exploring the Efficacy and Implementation of an Online Examination System: A Comprehensive Study on eXamPro in Modern Education

Abstract: Online assessments play a crucial role in evaluating the academic capabilities of students, particularly those enrolling in computer courses or technology-related programs. This model outlines an online placement assessment system, considering the perspectives of the examination process, end-users, and software development. The online assessment system is a digital platform where evaluations are conducted over the internet or intranet using computer systems. It presents an effective solution for large-scale educational assessments. This online educational system incorporates features such as test processing and electronic record-keeping. In this framework, instructors create course-specific questions, forming the basis for online assignments. Users engaging with the platform can access the electronic information they provided and utilize various functions within the online educational system to participate in online assessments. Participants have the opportunity to take multimedia enhanced quizzes related to their courses and submit electronic responses. Upon completing the test duration, users receive grades or marks secured in their assessments. The system performs assessments and auto-grading for multiple-choice questions, streamlining the evaluation process. This online assessment system not only facilitates efficient evaluation but also provides a dynamic and interactive educational experience for both instructors and students in the digital learning environment.

Emergency department staff views of NHS 111 First: qualitative interview study in England

BackgroundNHS 111 is a phone and online urgent care triage and assessment system that aims to reduce UK ED demand. In 2020, 111 First was introduced to triage patients before...

Perception, attitude, and preference of trainees towards workplace-based assessment in dental education at a tertiary care institute in Multan, Pakistan.

Modern medical education has flourished with an emergent interest in a variety of assessment approaches that entails direct observation of performance and provision of feedback. Workplace-based assessment (WPBA) has been embraced into specialty training in the United Kingdom (UK) within National Health Service (NHS). In Pakistan, this educational framework is incorporated to gauge the clinical proficiency of trainees at the postgraduate and undergraduate levels. The present study was done to identify how WPBA is perceived by dental field postgraduate trainees in Pakistan. The study explored and ascertained trainees' perceptions, attitudes, and preferences considering their experiences with the systematic organization and execution of WPBA in dental educational settings. This cross-sectional study design involved the clinical trainees of various cadres in a tertiary care hospital in Multan. A structured and validated questionnaire previously used for another analysis was administered among 90 trainees at the institute. The results were analyzed and tabulated using SPSS-21. Influential statistics (Pearson's Chi-Square Test) was performed and the confidence interval was set at 95% (P≤0.05). The response rate was 88%. Opinions were more positive compared to the former surveys. The majority of the respondents (65.8%) have shown a positive attitude towards WPBA and preferred it to be a valuable assessment system as it fosters reflective practices in educational settings. In addition, it was shown that WPBA has the potential to improve clinical training (72.2%) and aid in the effective implementation of clinical practice (68.4%). As the system of digital education has been introduced ever since the pandemic, the highest percentage of participants (56%) preferred a combination of on-paper and online assessment systems. The study reported that WPBA proposes the opportunity to associate teaching, learning, and assessment. The faculty training program is a significant input to upholding the quality of WPBA. Empirical research on WPBA is essential to be carried out to overcome the inadequacies thus endorsing its application universally.

Online Self-Assessment in Mathematics at the University of Bern

In the Swiss education system the cantons oversee educational matters, practically meaning there are 26 different school systems. This is also reflected in the graduation exam, which is organized locally in every high school. Based on this assessment and their preference, students are admitted to university without any additional entrance exam. This motivated us to create an online self-assessment. Its two mathematical parts serve a different purpose: one evaluates basic mathematical skills, and the other one deals with abstract reasoning that gives students a flavor of university mathematics. Additionally, the online self-assessment contains a set of metacognition questions regarding motivation, expectations and attitude towards mathematics. Our goal is that students to identify their strengths and help them make a more informed study choice.&#x0D; The technical tool used for the implementation is STACK, an open-source online assessment system in mathematics. Concerning this use case of STACK, we are currently collecting students’ feedback. We will present our first conclusions on this matter, which serve as a basis for further adjustments of the material.

Integrity of university online assessment: Towards developing a function model

This world is changing and the old education operation system is no longer compatible with these vicissitudes. Online assessment, that has been enforced by the COVID 19 pandemic and has become a norm in many universities world-wide, is failing to embrace all leaners, thus the need to address it. This content analysis study examined the integrity of online university assessments based on 2020 and 2021 online studies purposively selected from the internet. The researcher used Motivation Theory, Technology Acceptance Model and Self-efficacy Theory whose concepts are key in the changes that this study calls for. Findings revealed two basic operational types of online university assessments: the digitally proctored examinations and the open-access assessment. While affluent nations have smoother transitions and more benefits, developing nations are challenged by students online cheating and resource unavailability. The study concluded that some universities are hesitant to adopt online assessment systems due to lack of awareness, resistant to change and fear of losing examination integrity. The study therefore recommends universities to hold online assessment workshops to equip both the lecturers and the learners. Detecting cheating software can be used to detect students cheating and expenses can be averted by using free online examination software. They can also use Chromebooks for student assessment which can be setup to disable access to browse the web and the external storage during an examination. Above all, a reliable, authentic and cost-effective online assessment model with minimal disruptions, which ensures that no student is left behind, is a necessity.

Redesign of online proctored exams for STEM learners in HEIs: proposal for incorporating higher-order thinking skills and democratic pedagogy via OPERHOT platform.

In response to the challenges presented by the COVID-19 pandemic, this paper proposes an exam system for STEM students that effectively test higher-order thinking skills, such as the ability to apply, create, evaluate, and analyse. Higher education institutions (HEIs) must incorporate all three strands of democratic STEM pedagogy, namely student voice, shared and transformational authority, and STEM criticality, according to empathy interviews with educators. It is imperative that these strands are reflected in the assessments of students within HEIs. During the pandemic, it became evident that the lack of a suitable proctored online examination system impeded the accurate and fair evaluation of students. This circumstance was in stark contrast to the offline mode of examination, which permitted monitoring and control. Notably, many students struggled to answer questions requiring higher-order thinking, with responses frequently identical in unproctored online examinations or settings with insufficient proctoring mechanisms, especially in contexts with limited resources. Consequently, there is an urgent need to redesign the online proctored test system in accordance with STEM criticality and student voice. To address this need, this paper proposes the creation of OPERHOT, a proctored online exam system integrated with random questions requiring higher-order thinking. This system employs a timed assessment methodology and requires students to keep their cameras on throughout the examination. By implementing OPERHOT, a proper and secure online evaluation of students can be made possible. In addition, this system will strengthen the online education and assessment systems in higher education institutions, thereby promoting integrity and enhancing the learning experience. In addition, the adoption of OPERHOT will also improve the assessment process and contribute to the enhancement of online education and assessment in HEIs as a whole. This reimagined system will catalyse the development of a robust and inclusive learning environment that encourages students' voices and equips educators to foster the growth of critical thinking and problem-solving skills. This perspective aims to address the pressing need for an updated and effective proctored assessment system in the online educational landscape while promoting STEM critical thinking and student voice. The OPERHOT platform is at the ideation stage, and once developed, it may be tested in a few HEIs. Based on the feedback from educators and other stakeholders it can be further modified to make it more inclusive and accessible and can be offered to other HEIs for wider piloting.

Hyperspectral imaging for assessment of total fat in salmon fillets: A comparison between benchtop and snapshot systems

Portable hyperspectral Imaging (HSI) based on snapshot sensors is ideal for on-line applications, with very fast data acquisition, small camera and flexibility for implementation. The objective of the study was to assess a snapshot HSI system for on-line assessment of salmon by bench marking it against a conventional hyperspectral system. Two different HSI systems: (a) linescan (550–1700 nm) and (b) snapshot (Vis: 470–630nm and Vis-NIR: 670–950 nm) were investigated. For each sample, one image was collected using linescan while two images per camera were collected using each snapshot followed by chemical analysis of samples to measure total fat content. Partial Least Squares Regression (PLSR) was successfully utilized to develop fat prediction models. Two strategies were followed: (1) Models for each dataset including stitched data from two snapshots and (2) HSI technology vs spectral range: new datasets were generated in such a way that both linescan and the device being compared (e.g., linescan vs Vis snapshot) had the same number of wavelengths in the similar wavelength range. For strategy (1) all models showed good prediction accuracy, yielding R p 2 in the range of 0.78–0.94 and SEP in the range of 1.58–3.10 except for Vis snapshot. For strategy (2) all models performed well, yielding R p 2 in the range of 0.77–0.94 and SEP in the range of 1.74–3.19 except for linescan vs Vis snapshot. Additionally, the stitched data model was used to generate fat prediction map of a salmon fillet and compared to its reference fat map. Results showed that snapshot HSI can be successfully used to predict fat in salmon fillets and have great potential for on-line applications. • • Hyperspectral linescan and snapshot cameras were compared. • • Both cameras successfully predicted total fat in salmon fillets via PLSR. • • Relevance of wavelength range was shown in predicting fat. • • PLSR model was used to generate fat prediction map of salmon fillets. • • Supporting evidence for developing hyperspectral system for industry is reported.

The Practices of Online Assessment in a Digital Device in the Context of University Training: The Case of Hassan II University

This research presents online assessment in a digital device in the context of university training, aimed at improving their practices with emerging technologies based on an experiment with students from Hassan II University. Or, online assessment is a systematic process that helps measure the knowledge and skills of learners through multiple technological tools in a digital device. Indeed, digital devices intend to revolutionize higher education with the use of Information and Communication Technologies (ICT). Nevertheless, digital devices pose the problem of student identity verification during online assessment. In reality, automated online assessment systems are extremely vulnerable to cheating. So, our aim of this research is to explore, firstly, the types of online assessment that could be implemented in a digital device and secondly, how to verify the identity of the student during an online course on a digital device? The sample of our experiment consists of (N = 108) students from the Hassan II University of Casablanca, divided into two classes of the ITEF and MIMPA Masters and, our study was based on an online questionnaire for (N = 37) teachers at Hassan II University in Casablanca. The results obtained is to put into practice in digital devices diagnostic evaluations and formative evaluations using biometric methods for identity verification with a limited number. However, biometrics is inapplicable in summative assessments due to the problem of massiveness and hindrances in the online exam. For this reason, measures must be put in place to promote the smooth running of the online assessment.

A Review of the Studies Conducted on Online Exams in Turkey from the Millennium to the Coronavirus Period

With removing the time and place restrictions, online exams have an essential place in meeting the needs of the digital age. It has become a much-needed structure for measuring individuals' competence remotely, especially in emergencies such as during the coronavirus epidemic. However, there are problems in Turkish online assessment system. The researcher revealed online exams' status and identified their deficiencies in Turkey, and made suggestions. In the research, the articles and theses were examined made for online exams in Turkey between the years 2000-2020. Google academic engine, Dergi Park, TÜBİTAK ULAKBİM, Ebsco, Eric, and the YÖK thesis database were used to reach the publications. A total of 41 articles and 8 theses were identified on online exams in Turkey. The findings highlighted that the studies had not reached the point where they are needed. The studies examined many different variables, and among them the opinions about the online exams prepared within the researches were collected the most. The studies were generally carried out with the survey design instead of experimental. The purposeful sampling method was mostly used, and the sample group consisted of undergraduate students in general. The results of the study point to essential directions for the development of online exam systems and provide implications for future efforts.

Online Tests and Predictive Analytics

We present a framework that estimates students’ exam performance in the context of mathematics lectures. For the prediction, we apply a multi-variate nonlinear regression implemented in MATLAB based on the results of five tests acting as independent data. The electronic tests were implemented by means of online assessment system STACK. The tests cover all major topics and are evenly distributed throughout the lecture period. The performance of the approach is quantitatively tested on student groups. In the future, the insights gained in this way will serve as a starting point for prespective analytics.

The Imperative of Technical Skilled Staff in Legacy Information System Management

Technical skilled personnel and information systems are important for the success of contemporary firms as they face intense rivalry, globalization, and time-to-market pressures. They are an essential element that plays a critical role in an organisation’s ability to sense environmental change and respond efficiently and effectively to that change. Besides the technology components of an information system and the fast pace of technological innovation, the people involved in the design, operation and maintenance are essential elements of information systems. Most of the research work on deciding whether or not to continue with legacy information systems has been centred mainly on intrinsic agility factors. Extrinsic agility factors like technical skilled personnel have largely been overlooked. But, as information systems age or in the case of highly rigid processes, extrinsic agility factors like technical skilled personnel will start playing an important role in deciding whether to prolong the life cycle of the information system. However, human skills obsolescence is a growing problem for organizations that must support legacy systems. The technical skilled staff are either ageing out or upgrading their skills to a non-procedural programming paradigm due to market demand. It is, therefore, imperative to regularly assess the technical skilled personnel competency level of a legacy information system. This study evaluates and classifies the technical knowledge of skilled personnel using a developed online assessment system with test questions based on the legacy information system application program. The result obtained indicates that the technical skilled staff can still operate and maintain the case study information system.