Online Assessment Research Articles

An examination of sociodemographic and clinical factors influencing help-seeking attitudes and behaviors among adolescents with mental health problems.

This study investigated sociodemographic and clinical factors influencing help-seeking attitudes and behavior among adolescents with mental health problems. As part of the ProHEAD ("Promoting Help-seeking using E-technology for ADolescents") consortium a school-based, online assessment of sociodemographic information, psychopathology, risk-taking and self-harming behavior, help-seeking attitudes and behavior, and barriers to help-seeking was conducted in adolescents aged ≥ 12 years recruited from randomly selected schools in five regions of Germany. Linear regression analyses with the LMG formula were performed to explore predictors of help-seeking attitudes and behavior and assess their relative importance. Nine thousand five hundred and nine participants (95.5%) completed the online assessment (mean age: 15.1 years, 58.6% female). In total, 1606 participants (16.9%) showed relevant mental health problems (e.g., depressive and eating disorder symptoms, alcohol problems, and thoughts of self-harming behavior). Among them, 895 (55.7%) reported having sought help (lifetime), with higher rates for informal (n = 842, 52.4%) compared to professional (n = 380, 23.7%) sources. High help-seeking propensity emerged as the most important factor contributing to professional help-seeking, followed by elevated levels of psychopathology and perceived barriers, with sociodemographic factors being less impactful. Psychopathological severity also outweighed sociodemographic factors in predicting negative help-seeking attitudes. These findings indicate that attitudes towards seeking professional help, perceived barriers, and psychopathology severity critically influence limited adolescent help-seeking behavior. This emphasizes the need for initiatives that promote help-seeking, reduce negative attitudes, and address structural barriers in adolescent mental health care.

A qualitative study exploring how vocational rehabilitation for people with multiple sclerosis can be integrated within existing healthcare services in the United Kingdom

BackgroundTo explore how a vocational rehabilitation (VR) intervention can be integrated within existing healthcare services for people with multiple sclerosis (MS) in the United Kingdom (UK) National Health Service (NHS).MethodsData from 37 semi-structured interviews with 22 people with MS, eight employers, and seven healthcare professionals were analysed using a framework method informed by the Consolidated Framework for Implementation Research and an intervention logic model.ResultsFour themes were identified relating to the structure of current NHS services, how to improve access to and awareness of VR services, the collaboration between internal and external networks, and the benefits of integrating VR within the NHS services. Participants identified several implementation barriers such as poor links with external organisations, staffing issues, and lack of funding. To overcome these barriers, participants suggested enablers such as technology (e.g., apps or online assessments) and collaboration with third-sector organisations to reduce the pressure on the NHS.ConclusionSignificant organisational changes are required to ensure a successful implementation of a VR intervention within current NHS services. Despite this, the NHS was seen as a trustworthy organisation to offer support that can optimise the health and professional lives of people with MS.

On-line and in-line quality assessment across all scale levels of 3D concrete printing

3D Concrete printing requires much more elaborate quality control procedures compared to conventional concrete processing. Due to the various process steps, and the corresponding variation in material behaviour, time-, and length-scales, a single quality indicator and measurement technique (similar to the ‘slump test’ for traditional construction) cannot be selected. Instead, three families of quality indicators have been established: homogeneity during material production and deposition (quality variations), material evolution during printing (transient material behaviour), and macroscopic features and geometric conformity during printing and of the final object (geometry). For each family, quality assessment techniques which have been proven in other fields or for different applications, have been successfully transferred and adapted to the 3DCP process. In some cases, completely new methods have been developed. This paper aims to provide the state-of-the-art in such quality assessment methods, indicating high potential methods and research gaps across all scale levels of 3D concrete printing processes.

Integrating group awareness into team‐based learning

AbstractBackgroundTeam‐based learning (TBL) is a widely recognised pedagogical strategy used in diverse fields and courses. Despite the success of TBL, certain concerns have arisen regarding social loafing within group collaboration contexts; this loafing can reduce the overall effectiveness of learning.ObjectivesThis study addressed the aforementioned concerns by integrating group awareness (GA) information into each stage of TBL, resulting in a TBL + GA system designed to enhance TBL instruction. During online collaborative learning, GA information provides insights into the activities completed by peers and thus the peers' preparedness and achievement; this knowledge fosters motivation, participation, and reflection.MethodsA quasiexperiment was performed with students from two freshman classes (40 and 41 students, respectively) in a Taiwanese university; the control and experimental classes used the TBL only and TBL + GA systems, respectively. Student learning effort was evaluated in terms of preclass preparation, including the use of learning materials and online test scores. Learning achievement was evaluated by assessing the quality of students' assignments. Student feedback, including peer evaluations and students' perceptions of their TBL experience, was collected and compared for the two classes.Results and ConclusionsThe TBL + GA system significantly enhanced the preclass preparation efforts of the participants, mitigating social loafing. This improvement manifested as higher practice engagement and better online assessment scores. The TBL + GA system also improved assignment quality and elicited more constructive and positive peer feedback.ImplicationsIntegrating GA information into each stage of TBL may positively affect the effectiveness of instruction. The proposed TBL + GA system can serve as a practical model for instructors seeking to enhance TBL implementation and foster a more engaging and constructive learning environment.

Preadolescents' executive functions and resilience development: A cascade model of resilience resources and resilient functioning.

Executive functions (EF), encompassing inhibition, updating, and shifting, are widely acknowledged as cognitive factors that promote resilience. However, prior research examining the association between EF and resilience has been hampered by inconsistent conceptualizations of resilience and an overreliance on cross-sectional designs. We embraced a process-oriented conceptualization of resilience and employed a longitudinal approach to investigate how EF components interplay with the dynamic processes of resilience resources and resilient functioning. A total of 144 Chinese preadolescents (aged 10-12, 53.5% male) completed computer-based EF assessments at baseline and self-reported their resilience at three intervals during 2019-2020. The resilience evaluations encompassed resilience resources at individual, familial, and social levels, as measured by the Resilience Scale for Chinese Adolescents, and resilient functioning, operationalized as the residuals of socioemotional difficulties after accounting for stressful life events. The findings emphasized the dynamic nature of resilience, unveiling a developmental cascade from resilience resources to resilient functioning and back to resilience resources. Furthermore, distinct effects of EF components on resilience development were found. Specifically, inhibition was associated with both concurrent and long-term resilience resources and functioning, while updating predicted long-term resilience resources, and shifting predicted long-term resilient functioning. These results underscored the pivotal role of EF as a cognitive foundation in comprehending the dynamic resilience processes during preadolescence.

Task-irrelevant visual distractions and mindful self-regulated learning in a low-stakes computer-based assessment

Task-irrelevant visual distractions and mindful self-regulated learning in a low-stakes computer-based assessment

Feasibility and reliability of online vs in-person cognitive testing in healthy older people.

Early evidence in using online cognitive assessments show that they could offer a feasible and resource-efficient alternative to in-person clinical assessments in evaluating cognitive performance, yet there is currently little understanding about how these assessments relate to traditional, in-person cognitive tests. In this preliminary study, we assess the feasibility and reliability of NeurOn, a novel online cognitive assessment tool. NeurOn measures various cognitive domains including processing speed, executive functioning, spatial working memory, episodic memory, attentional control, visuospatial functioning, and spatial orientation. Thirty-two participants (mean age: 70.19) completed two testing sessions, unsupervised online and in-person, one-week apart. Participants were randomised in the order of testing appointments. For both sessions, participants completed questionnaires prior to a cognitive assessment. Test-retest reliability and concurrent validity of the online cognitive battery was assessed using intraclass correlation coefficients (ICCs) and correlational analysis, respectively. This was conducted by comparing performance in repeated tasks across testing sessions as well as with traditional, in-person cognitive tests. Global cognition in the NeurOn battery moderately validated against MoCA performance, and the battery demonstrated moderate test-retest reliability. Concurrent validity was found only between the online and paper versions of the Trail Making Test -A, as well as global cognitive performance between online and in-person testing sessions. The NeurOn cognitive battery provides a promising tool for measuring cognitive performance online both longitudinally and across short retesting intervals within healthy older adults. When considering cost-effectiveness, flexible administration, and improved accessibility for wider populations, online cognitive assessments show promise for future screening of neurodegenerative diseases.

Utilizing Technology to Assess English Learning Outcomes of Students Based on a Competency-based Approach

This study addresses the research gap in the integration of technology in evaluating English language learning outcomes, aiming to improve efficiency, accuracy, and flexibility in competency-based assessments. The research employs a mixed-methods approach, using quantitative data from online assessments. Participants included educators and learners engaged in English language programs across various educational institutions. The findings indicate that adaptive learning technologies significantly enhance accessibility, scalability, and student engagement in English language assessments. However, challenges such as technological limitations, data privacy issues, and the need for comprehensive teacher training are highlighted. This research underscores the potential of technology to transform competency-based English language proficiency examinations while offering practical insights for educational institutions, policymakers, and ed-tech companies to optimize these advancements effectively.

Cognitive Outcomes in Young Adults with Primary Arterial Hypertension: The Role of Cardiovascular Risk Factors and Hypertension-Mediated Organ Damage.

Background and Objectives: We aimed to explore the association between cognitive performance and markers of hypertension-mediated organ damage (HMOD) in young adults with hypertension. Materials and Methods: A group of adults aged 16 to 45 years with primary arterial hypertension completed a battery of paper-pencil as well as computer-based neuropsychological tests across all major cognitive domains. They also underwent office and ambulatory 24 h blood pressure, intima-media thickness measurements, heart ultrasound, and laboratory analysis of their lipid profile, blood uric acid concentration, and urine albumin-creatinine ratio. Associations between cognitive test results and markers of HMOD were explored through correlation analysis and age-, sex-, and body mass index-adjusted linear regression modeling. Results: Seventy-six individuals (62, 81.6% male) aged 36.5 years (interquartile range 18.4 to 42.0 years) were enrolled. After adjusting for age, sex, and body mass index in linear regression models, worse categorical fluency was associated with higher left ventricular mass (βst = -0.264, p = 0.043) and worse performance in a task of sustained attention-with higher left ventricular mass index (βst = -0. 304, p = 0.016). Worse phonemic fluency was related to higher pulse pressure (βst = -0.241, p = 0.049) in the respective model. Better strategy use in the task of spatial working memory was linked to higher daytime mean diastolic blood pressure (βst = -0.343 p = 0.017). Conclusions: Performance among young adults with hypertension across selected cognitive domains was inversely associated with pulse pressure, markers of left ventricular damage, and directly associated with daytime diastolic blood pressure. Our study suggests that the previously reported relationship between cognitive and cardiovascular markers in hypertension exists earlier than in middle or late adulthood.

Online assessment in the age of artificial intelligence

Online education, while not a new phenomenon, underwent a monumental shift during the COVID-19 pandemic, pushing educators and students alike into the uncharted waters of full-time digital learning. With this shift came renewed concerns about the integrity of online assessments. Amidst a landscape rapidly being reshaped by online exam/homework assistance platforms, which witnessed soaring stocks as students availed its questionable exam assistance, and the emergence of sophisticated artificial intelligence tools like ChatGPT, the traditional methods of assessment faced unprecedented challenges. This paper presents the results of an observational study, using data from an introductory statistics course taught every semester by the author, and delves into the proliferation of cheating methods. Analyzing exam score results from the pre and post introduction of ChatGPT periods, the research unpacks the extent of cheating and provides strategies to counteract this trend. The findings starkly illustrate significant increases in exam scores from when exams of similar difficulty were administered in person (pre-Covid) versus online. The format, difficulty, and grading of the exams was the same throughout. Although randomized controlled experiments are generally more effective than observational studies, we will indicate when we present the data why experiments would not be feasible for this research. In addition to presenting experimental findings, the paper offers some insights, based on the author's extensive experience, to guide educators in crafting more secure online assessments in this new era, both for courses at the introductory level and more advances courses The results and findings are relevant to introductory courses that can use multiple choice exams in any subject but the recommendations for upper-level courses will be relevant primarily to STEM subjects. The research underscores the pressing need for reinventing assessment techniques to uphold the sanctity of online education.

The Impact of Online Testing Mode on Business Assessment Test Performance Before and During COVID

We examined the impact of the COVID-19 on a Southern California public university’s the Business Assessment Test (BAT) scores. The analysis analyzed over 4,000 observations from Spring 2017 to Spring 2022. Results demonstrated a significant performance improvement during COVID, with the average BAT score increasing from 51.35 to 55.47. Furthermore, demographic factors like major, ethnicity, enrollment, gender, and English proficiency also influenced BAT scores. Finally, interaction effects, especially for quantitative test questions were significant between COVID and cs major, work, and language. The study provides insights for educators and policymakers regarding the implementation of online assessment format in the future.

An IMU-based machine learning approach for daily behavior pattern recognition in dairy cows

Technological advancements have revolutionized livestock farming, notably in health monitoring. Traditional methods, which have been criticized for subjectivity and treatment delays, can be replaced with efficient health monitoring systems, thereby reducing costs and workload. Implementing cow behavior recognition allows for effective dairy cow health monitoring. In this research, we propose an integrated system using inertial measurement unit (IMU) devices and machine learning techniques for dairy cow behavior recognition. Six main dairy cow behaviors were studied: lying, standing, walking, drinking, feeding, and ruminating. All behavior types were manually labeled into the IMU data by reviewing the recorded footage. The labeled IMU data underwent four processing steps: selecting different window sizes, feature extraction, feature selection, and normalization. These processed data were then used to build the behavior recognition model. Various model structures, including SVM, Random Forest, and XGBoost, were tested. The top-performing model, XGBoost, with its proposed 58 features achieved an F1-score of 0.87, with specific scores of 0.93 for lying, 0.85 for walking, 0.94 for ruminating, 0.89 for feeding, 0.86 for standing, 0.93 for drinking, and 0.59 for other activities. During our online testing, we observed similar patterns for each healthy cow. The cumulative time spent on each behavior also matched the statistics from previous surveys. Additionally, our backend optimization approach resulted in a final overall percentage error of 1.55 % per day during online testing. In conclusion, our study presents an IMU-based system capable of accurately recognizing dairy cow behavior. Feature design and appropriate models are proposed herein. A functional optimization method was introduced indicating that our system has the potential with applications for estrus detection and other reproductive management practices in the dairy industry.

Paper assessment or online assessment: exploring the impact of assessment modes on EFL students’ language learning outcomes and personal development

This research explores how paper and online assessments (OA) can affect EFL learners’ personal development variables such as self-esteem, mindfulness, demotivation, and language learning development. Sixty intermediate English as a foreign language (EFL) students participated in the current research and were randomly grouped into the experimental group (EG) and the control group (CG). Before the intervention, the data collection tools assessing language learners’ self-esteem, mindfulness, demotivation, and language learning development were administered as pre-tests to all the participants. Subsequently, the EG was taught all lessons of the coursebook, Top Notch 2, by utilizing OA while the CG was taught the identical lessons using a conventional (pen and paper) assessment. In the post-treatment phase, the two groups completed the post-tests on self-esteem mindfulness, demotivation, and language learning progress. The study demonstrated that the EG group outperformed the CG group on the four post-tests. The findings indicate that the intervention of OA significantly enhanced language learning outcomes and psychological well-being regarding the measured personal development variables. The enhanced efficacy in the EG group can be ascribed to the utilization of OA during the treatment phase, which has a beneficial effect on academic and personal development. This study also posits that integrating OA augments psychological engagement and language development among EFL learners. It highlights the implications of the results associating teaching EFL via OA with the observed personal development aspects and attributes them to effects of positive psychology.

Saudi EFL Universities Students’ Perceptions of Taking Computer-based Tests and Paper-based Tests

Saudi EFL Universities Students’ Perceptions of Taking Computer-based Tests and Paper-based Tests

Effective learning and testing of bird identification skills

ABSTRACT Biology education worldwide at different levels involves learning to identify plants and animals in nature. The aim of this study was to investigate the effectiveness of a teaching and learning method for bird identification that combines active outdoor fieldwork with active online learning, testing the validity of our instrument. Altogether, 798 of 853 students with different cultural and socio-political backgrounds from nine countries participated in 2010–2018. The research method was an online bird identification test at the beginning of the study, an online test and a field test at the end of the field teaching period, and an online exam at the end of the study. The results showed great progression from the beginning to the exam at the end of the study, and a positive correlation between initial skills and exam skills. Finally, the field test showed a strong positive correlation with the online test taken simultaneously, thus indicating that functional competence can be measured in the online exam. We recommend our instrument for teaching and testing of students’ skills in species identification worldwide and for all taxa, but further testing of validity and teaching effects is recommended.

COVERT SIMULTANEOUS POST-EDITING IN ONLINE ASSESSMENT OF STUDENTS’ SIGHT TRANSLATION

This first attempt aims to determine the extent of students’ covert use of machine translation (MT) in the online assessment of their sight translation, the strategies of such use, and its signs. The study is based on the analysis of target texts (TT) of specialised online sight translation from Ukrainian into English by 13 BA and 10 MA students. The procedure involved the comparison of the students' TTs with their MT counterparts. Signs of covert MT were found in 46% of the BA and 30% of the MA students’ translations. The main method of this covert MT use is "simultaneous post-editing", i.e., the immediate oral post-editing of the MT text generated by the students on their screens and hidden from the assessor, while they deliver their supposedly original TTs. Simultaneous post-editing strategies range from replacing individual lexemes with their synonyms, adding and deleting elements, changing the syntactic functions of words or phrases, rearranging sentence fragments, transforming their structure, to applying several of these strategies simultaneously. Other methods of concealment include alternating MT systems in translating the same source text, as well as artificially slowing down the process of reading the TT from the screen, accompanied by pauses in the relevant text fragments to perform certain mental operations. In order to increase objectivity, the author recommends a delayed assessment of students' online interpreting recordings. The research perspective is to study the didactic potential of simultaneous post-editing as a procedure for developing general interpreting skills.

Investigating the effects of online assessment on students’ learning experience

During the pandemic outbreak teaching and learning process has changed into an online mode and thus online evaluation became a requirement. Therefore, the current paper attempts to investigate the effects of online assessment on students’ learning behavior and teachers' views. It also includes the results of a questionnaire from 60 students followed by an interview with 07 EFL teachers from the Department of English in the University. The results from the questionnaire indicated that students have positive attitudes towards the administration of online assessment. Furthermore, the results from the interviews showed that there are four factors for valuable effects which are showing flexibility in assessment, enhancing evaluation adaptability, and increasing teachers' and students' awareness of the importance of evaluation by enlightening students’ autonomous learning abilities. Keywords: Online assessment; online learning experience; student attitudes; teachers’ views.

Automatic tuning of radio stations based on listener’s preference using Software Defined Radio and MATLAB

This work introduces a real-time system to automate the selection of radio stations based on the listener’s preference (either speech/music) by analyzing the incoming audio signals using a speech/music classifier (SMC) using machine learning approaches. Radio Frequency data from different Frequency Modulated (FM) stations are directly read in MATLAB using the Communications Toolbox Support Package for Register Transfer Level-Software Defined Radio (RTL-SDR). Further, the work is divided into two phases. Initially, the efficiency of different state-of-the-art features for designing a speech/music classifier is studied on new speech/music corpora developed for Indian radio stations. Box plots and Region of Convergence (ROC) plots were used to study feature importance. The learning from the experiments was used to select optimum sets of features for efficient working of the real-time model. The models were tested for both offline and online test data. The best-performing features (Mel-Frequency Cepstral Coefficients (MFCC) + Variance Spectral Roll-off) vectors were then concatenated to obtain the best classification accuracy of 93.06% and 83.91% for offline and real-time data, respectively, using Gaussian Mixture Model (GMM) classifier. We also studied the efficiency of recently proposed Empirical Mode decomposition (EMD)-based statistical and Hilbert Spectrum-based features on both standard (Slaney, GTZAN, Musan dataset) and newly created datasets. We achieved an overall accuracy of 94.91%, 93.20%, 91.72%, and 95.40% for SS, GTZAN, MUSAN, and BITM datasets respectively. However, concatenating several features increased the latency of the algorithm. This work also proposes a new speech/music corpus based on recordings from Indian radio stations in the Hindi language.

Teleassessment using five times sit-to-stand test in older adults with and without Parkinson's disease: Guidelines and barriers.

Assessing lower limb strength, balance, and fall risk are crucial components of rehabilitation, especially for the older adult population. With the growing interest in telehealth, teleassessment has been investigated as an alternative when in-person assessments are not possible. The Five Times Sit-to-Stand test (5TSTS) provides a quick measure of balance during chair transfers, muscle power, endurance, and the hability to change and maintain body position, and is highly recommended by guidelines. However, the literature is unclear about the viability and safety of teleassessment using the 5TSTS in older adultswith and without Parkinson's disease (PD). This study aimed to evaluate the reliability of teleassessment using the 5TSTS and to determine its feasibility and safety for older adults with and without PD. This cross-sectional study included older adults with and without PD who were evaluated remotely through a videoconference platform. To ensure effective and comprehensive instructions for the test, we developed a guideline called OMPEPE (an acronym for: Objective; Materials; Position-Start; Execution; Position-End; Environment). We assessed the 5TSTS intra- and inter-rater reliability by comparing scores obtained from the same examiner and from different examiners, respectively. Participants and examiners completed online surveys to provide information about feasibility and safety. Twelve older adults with PD and 17 older adultswithout PD were included in this study (mean ages 69.0 and 67.6years, respectively). Based on the participants' perspectives and the absence of adverse effects, teleassessment using the 5TSTS is feasible and safe for older adults with and without PD. Excellent intra- and inter-rater reliability (intraclass correlation coefficient >0.90) was found for all measurements of the 5TSTS. This study demonstrated the feasibility, safety, and reliability of teleassessment using the 5TSTS. The guidelines developed may help health professionals minimize barriers and safely conduct an online assessment that includes a physical test such as the 5TSTS in older adults with or without PD. In addition to addressing technological barriers, the OMPEPE guideline might ensure the optimal execution of evaluations. Teleassessment using the 5TSTS for older adults with and without PD is feasible and safe. Both synchronous (i.e., live) and asynchronous (i.e., recorded) online 5TSTS tests demonstrate excellent intra- and inter-rate reliability.

Development of Computational Mathematic Thinking Test Instruments Based On Computered Based Test

This reseacrh is a type of development reseacrh. The aims of this research is to develop an instrument for mathematical computational thinking tests based on Computer Based Testing (CBT) using the Quizizz application, evaluate the validity of the developed test instrument, and analyze the mathematical computational thinking abilities of students. The ADDIE model (Analysis, Design, Development, Implementation, and Evaluation) is adopted as the method in this research. A total of 52 students from classes X and XI of SMA Muhammadiyah 1 Surakarta were involved in this study as research subjects. Data analysis was conducted using the Rasch model with the assistance of Winsteps Rasch software. The results of this research indicate that 20 items are categorized as valid. The overall test reliability is 0.62, classified as sufficient, while the reliability of the test items is 0.88, classified as good. Test analysis of the level of difficulty of test items reveals that there are 3 items categorized as very difficult, 7 items categorized as difficult, 5 items categorized as moderate, and 5 items categorized as easy. In general that the computational thinking skills test does not show a tendency to be biased in any question item, because all probability values for the question items exceed 0.05. Items are considered unbiased if they do not give an unfair advantage to any one individual. Based on the experiment in this study, the average mathematical computational thinking ability of class X students is 75.53, while for class XI students it is 79.37.