Automated Scoring System Research Articles

Development of an Automatic Functional Movement Screening System with Inertial Measurement Unit Sensors

Background: In this study, an automatic scoring system for the functional movement screen (FMS) was developed. Methods: Thirty healthy adults fitted with full-body inertial measurement unit sensors completed six FMS exercises. The system recorded kinematics data, and a professional athletic trainer graded each participant. To reduce the number of input variables for the predictive model, ordinal logistic regression was used for subset feature selection. The ensemble learning algorithm AdaBoost.M1 was used to construct classifiers. Accuracy and F score were used for classification model evaluation. The consistency between automatic and manual scoring was assessed using a weighted kappa statistic. Results: When all the features were used, the predict model presented moderate to high accuracy, with kappa values between fair to very good agreement. After feature selection, model accuracy decreased about 10%, with kappa values between poor to moderate agreement. Conclusions: The results indicate that higher prediction accuracy was achieved using the full feature set compared with using the reduced feature set.

A Study on Development of Automatic Dementia Evaluation System by Measurement of Hand Motor Function

Recently, the number of dementia patients has been increasing due to the aging society. Therefore, it is necessary to develop a system that can automatically judge the degree of dementia progression, not to burden the doctor. The authors developed the puzzle game. We found features during the puzzle game to evaluate dementia progression.

RETRACTED: Research on automatic evaluation method of Mandarin Chinese pronunciation based on 5G network and FPGA

In the automatic evaluation system, need to learn the standard mandarin of scoring method for teaching in native Chinese pronunciation. The most pronounced goal protocols focus on the context in which native speakers are unnatural. The new Hidden Markov Model (HMM) algorithm based on the traditional algorithm likely algorithm for Chinese syllables, whose final initial period is found in the area where evidence for the measurement of weight control has been found. Experiments have also shown that this algorithm is more effective than the traditional posterior recording algorithm of the Mandarin learning method. Force Hidden Markov Model- HMM Align alignment identification for each syllable and associated recording probability for speech evaluation via race-based reliability system applications. These processes could then be formalized as a linear combination after the overall assessment functions: phonics, tone, intensity, and rhythm. Because both linear and non-linear parameters are involved in the overall evaluation functions. Incorporates variation in pronunciation to generate structure through a novel approach that incorporates tons of sub-tones that represent the missing automatic sound models. The word level assessment achieved through the pronunciation is similar to that which in the future showed the singing ability being realized by the evaluation system in full-length pronunciation as a method.

Performance of an Automated Polysomnography Scoring Using Noxturnal Program versus Manual Scoring

Background and Objectives A lack of investigators for polysomnography has risen due to increased demand since health insurance started to cover the cost of the test. We examined the reliability of the automated scoring of polysomnography, which has been deployed to improve this imbalance.Subjects and Method We analyzed the data of 20 patients who underwent level 1 polysomnography from April 1 to July 27, 2019. The software from Noxturnal (Nox Medical) was used for the scoring of the Polysomnography data. Each of the polysomnographic data was scored both by the automated scoring system and by a skilled technician.Results Twenty patients were analyzed. There was no significant difference between automated scoring and manual scoring in sleep latency, apnea index, and rapid eye movement sleep stage ratio. However, the concordance rate of the sleep stage by epoch was 83.32%, and there was a significant difference with regards to apnea-hypoapnea index (AHI) and respiratory disturbance index (RDI). Two obvious errors were noted in the automated scoring that could be easily fixed; the failure to recognize wakefulness during sleep and the fragmentation of respiratory events. When two errors were corrected, many differences in polysomnography parameters, including AHI and RDI, were eliminated.Conclusion It showed 80% coincidence of epoch in the sleep stage between the automated scoring and manual scoring. However, there was no difference in AHI and RDI when the fragmented respiratory events of the automated scoring were adjusted. Therefore, automated scoring is considered to be useful if only a little modification could be made.

Design and Implementation of Automatic Exercises Evaluation System for Programming Courses

The evaluation of students’ exercise process is an important part of the formative evaluation for programming courses. However, it is difficult for teachers to complete the evaluation of each student’s exercise process manually. This paper proposes a design of an automatic exercises evaluation system, which can automatically analyse the amount of students’ exercise codes and help teachers to complete the evaluation. The system uses Python’s Pyhook3 library to monitor the keyboard input of the student’s computer. After text segmentation is performed on the student’s input code, the amount of the student’s exercise codes is statistically analysed, and the result can be used as an important indicator of the formative evaluation. Through the trial, it is found that there is a certain proportional relationship between the evaluation given by the system and the final test result of the students, which indicates the system can give a relatively objective evaluation of the students’ learning process.

Glass-Adhesive-Steel Joint Inspection Using Mechanic and High Frequency Electromagnetic Waves.

The utilization of a glass–adhesive–steel joint in modern machine and vehicle production is constantly growing. Despite the technological regime during the production process, various defects in the adhesive joints may occur. One of the major problems is lack of adhesive between joined materials. Due to the widespread use of non-transparent layers increasing adhesion, it is frequently not possible to conduct simple visual inspections. Hence, it is important to develop a non-destructive adhesive path continuity examination procedure. In that process, the different physical properties of the joint materials must be taken into account. Therefore, in this paper various testing methods were used, including ultrasound, thermographic and electromagnetic methods operating in the microwave and terahertz frequency range. Different physical phenomena of the methods allowed for examination of the joint in a wide context. In order to verify the information brought by each method, the results were transferred into common coordinate space, compared and finally integrated. Various opinion pooling strategies were conducted to fuse data, which allowed us to emphasize convergent and complementary information on adhesive continuity. The obtained results are promising and constitute the basis for further work on an integrated system for automatic evaluation of a wide range of possible defects in glass–adhesive–steel joints.

Standards and scoring to increase transparency for archived public opinion data

Faced with increased diversification of methodologies in the polling industry, the Roper Center for Public Opinion Research Center is embarking on a major initiative aimed at increasing methodological transparency across the field of public opinion survey research by increasing minimum disclosure requirements and providing users with transparency scoring for new submissions to the archive. Roper Center, the world’s largest archive of public opinion survey data, has long enforced disclosure requirements for archival submissions based on transparency standards developed by professional organizations in the polling industry, particularly the American Association for Public Opinion Research (AAPOR). Roper Center’s new requirements and scoring mechanism expand longstanding policies and procedures to better meet the challenges of today’s research environment.In this paper, Roper Center’s new standards will be described in the context of the historical development of transparency expectations in the polling community. The paper presentation will also detail the implementation process, providing an account of how standards were translated into actionable DDI-based metadata to drive an automatic scoring system, how new workflows were developed with input from data providers to facilitate maximum disclosure, and how the display of the user interface was designed to ensure the transparency information can be easily viewed and understood.

Unsupervised Parameterization for Optimal Segmentation of Agricultural Parcels from Satellite Images in Different Agricultural Landscapes

Image segmentation is a cost-effective way to obtain information about the sizes and structural composition of agricultural parcels in an area. To accurately obtain such information, the parameters of the segmentation algorithm ought to be optimized using supervised or unsupervised methods. The difficulty in obtaining reference data makes unsupervised methods indispensable. In this study, we evaluated an existing unsupervised evaluation metric that minimizes a global score (GS), which is computed by summing up the intra-segment uniformity and inter-segment dissimilarity within a segmentation output. We modified this metric and proposed a new metric that uses absolute difference to compute the GS. We compared this proposed metric with the existing metric in two optimization approaches based on the Multiresolution Segmentation (MRS) algorithm to optimally delineate agricultural parcels from Sentinel-2 images in Lower Saxony, Germany. The first approach searches for optimal scale while keeping shape and compactness constant, while the second approach uses Bayesian optimization to optimize the three main parameters of the MRS algorithm. Based on a reference data of agricultural parcels, the optimal segmentation result of each optimization approach was evaluated by calculating the quality rate, over-segmentation, and under-segmentation. For both approaches, our proposed metric outperformed the existing metric in different agricultural landscapes. The proposed metric identified optimal segmentations that were less under-segmented compared to the existing metric. A comparison of the optimal segmentation results obtained in this study to existing benchmark results generated via supervised optimization showed that the unsupervised Bayesian optimization approach based on our proposed metric can potentially be used as an alternative to supervised optimization, particularly in geographic regions where reference data is unavailable or an automated evaluation system is sought.

Automatic evaluation system for bulk defects in optics

In the high frequency laser system, the internal defects of optics will affect their damage threshold and downstream modulation, and even lead to subsequent surface damage. In order to efficiently and reliably evaluate the internal defects in optics, an automatic evaluation method of scattering defects in optics is proposed, and the corresponding three-dimensional (3D) precision measurement system is established. The detection principle, image acquisition, image processing and bulk defects (BDs) information extraction methods of the system are studied. Based on a 3D dark-field microscopic imaging (DFMI) technology, samples are scanned with multi-dimensional motion devices. Meanwhile, the scattered beam from the BD was collected by a detector, and finally combined with the digital image processing technology, the acquired images are processed in real time. Through real-time focusing and improved 3D reconstruction algorithm, the images with full aperture BD features are obtained, and the position and size information of defects are extracted. The system has detected the BD inside the fused silica samples. The results show that our system can distinguish the 612.7 lp/mm. Furthermore, the 3D distributions of the BD were also reconstructed.

Development and Testing of a Methodology for the Assessment of Acceptability of LKA Systems

In recent years, driving simulators have been widely used by automotive manufacturers and researchers in human-in-the-loop experiments, because they can reduce time and prototyping costs, and provide unlimited parametrization, more safety, and higher repeatability. Simulators play an important role in studies about driver behavior in operating conditions or with unstable vehicles. The aim of the research is to study the effects that the force feedback (f.f.b.), provided to steering wheel by a lane-keeping-assist (LKA) system, has on a driver’s response in simulators. The steering’s force feedback system is tested by reproducing the conditions of criticality of the LKA system in order to minimize the distance required to recover the driving stability as a function of set f.f.b. intensity and speed. The results, obtained in three specific criticality conditions, show that the behaviour of the LKA system, reproduced in the simulator, is not immediately understood by the driver and, sometimes, it is in opposition with the interventions performed by the driver to ensure driving safety. The results also compare the performance of the subjects, either overall and classified into subgroups, with reference to the perception of the LKA system, evaluated by means of a questionnaire. The proposed experimental methodology is to be regarded as a contribution for the integration of acceptance tests in the evaluation of automation systems.

Automatic body condition scoring system for dairy cows based on depth-image analysis

Body condition score (BCS) is an important management tool in the modern dairy industry, and one of the basic techniques for animal welfare and precision dairy farming. The objective of this study was to use a vision system to evaluate the fat cover on the back of cows and to automatically determine BCS. A 3D camera was used to capture the depth images of the back of cows twice a day as each cow passed beneath the camera. Through background subtraction, the back area of the cow was extracted from the depth image. The thurl, sacral ligament, hook bone, and pin bone were located via depth image analysis and evaluated by calculating their visibility and curvature, and those four anatomical features were used to measure fatness. A dataset containing 4820 depth images of cows with 7 BCS levels was built, among which 952 images were used as training data. Taking four anatomical features as input and BCS as output, decision tree learning, linear regression, and BP network were calibrated on the training dataset and tested on the entire dataset. On average, the BP network model scored each cow within 0.25 BCS points compared to their manual scores during the study period. The measured values of visibility and curvature used in this study have strong correlations with BCS and can be used to automatically assess BCS with high accuracy. This study demonstrates that the automatic body condition scoring system has the possibility of being more accurate than human scoring. Keywords: body condition score, depth-image processing, curvature analysis, machine learning, precision dairy farming DOI: 10.25165/j.ijabe.20201304.5655 Citation: Zhao K X, Shelley A N, Lau D L, Dolecheck K A, Bewley J M. Automatic body condition scoring system for dairy cows based on depth-image analysis. Int J Agric & Biol Eng, 2020; 13(4): 45–54.

An Arduino based automatic pressure evaluation system to quantify growth of non‐model anaerobes in culture

AbstractMeasuring the growth rate of non‐model anaerobic microbes typically requires the use of time‐consuming and often destructive manual measurements. Here, an Arduino based automatic pressure evaluation system (A‐APES) was developed to automatically measure the rate of fermentation gas production as a proxy for microbial growth in anaerobic systems. The A‐APES system measures accumulated gas pressure in sealed cultures accurately at high‐resolution, while venting the system at programmed intervals to prevent over pressurization. The utility of A‐APES is demonstrated in this study by quantifying the growth rate and phases of a biomass‐degrading anaerobic gut fungus, which cannot be otherwise measured via conventional techniques due to its association with particulate substrates. Given the utility of the A‐APES approach, we provide a complete construction guide to fabricate the device, which is three times less expensive compared to existing commercial alternatives.

The Development and Application of Automated Scoring System for Constructed-Response Assessment of 5th Grade Science in Elementary Schools Using Recurrent Neural Network

The Development and Application of Automated Scoring System for Constructed-Response Assessment of 5th Grade Science in Elementary Schools Using Recurrent Neural Network

The Efficacy of Periodic Complete Blood Count Tests in Evaluation of the Health Status of Radiation Workers in Iran: A Systematic Review

Periodic medical examinations of radiation workers are routinely conducted in many countries. Although low dose radiation (LDR) is not expected to cause a significant effect on blood count, the periodic examination usually includes reviewing the work history, general medical history, a physical examination and collecting a blood sample. Despite lymphocytes are the most sensitive cells to radiation, their counts do not show any significant change as long as the radiation level is less than a few hundreds of millisievert (mSv). In spite of this, in Iran, radiation workers, even those who work in diagnostic radiology departments, are regularly examined for blood count changes. After a detailed search in PubMed, ISI, Scopus, SID and Google Scholar, only 12 out of 650 articles matched our criteria. A review of these 12 reports was conducted. The full texts were fully reviewed by the authors. The complete blood count (CBC) test has a very low efficacy in evaluation of the adverse health effects of ionizing radiation in radiation workers. Therefore, finding alternative methods with a higher efficacy is recommended. CBC tests cannot be introduced as valid markers of potential radiation effects in most occupational exposures. Given this consideration, in periodic tests of radiation workers, cytogenetic tests can be the gold-standard method. In particular, due to its relatively low cost and good sensitivity and specificity, the dicentric assay can be promising. Moreover, the simple and rapid evaluation of micronuclei by fast automated scoring systems can be a good alternative for current low efficacy CBC tests.

0441 An Automatic Sleep Scoring System Based on Ensemble Convolutional Neural Network and Spectrogram of Sleep Physiological Signal

Abstract Introduction Manual sleep stage scoring is time consuming and subjective. Therefore, several studies focused on developing automated sleep scoring algorithms. The previously reported the automatic sleep scoring have been develop usually using small dataset, which less than 100 subjects. In this study, an automatic sleep scoring system based on ensemble convolutional neural network (ensemble-CNN) and spectrogram of sleep physiological signal was proposed and evaluated using a large dataset with sleep disorder. Methods The spectrograms were computed from each 30-s EEG and EOG of 994 subjects from PhysioNet 2018 challenge dataset, using the continuous wavelet transform, which were fed into an ensemble-CNN classification for training. The ensemble-CNN contained five pretrained models, ResNet-101, Inception-v4, DenseNet-201, Xception, and NASNet models, because these models’ architectures are different which can learn different features from the spectrograms to obtain high accuracy. The probabilities of five models were averaged to decide the sleep stage for each spectrogram. After classifying sleep stage, a smoothing process was used for sleep continuity. Moreover, the total 80% data from PhysioNet dataset were randomly assigned to the training set, and the remaining data were assigned to the testing set. Results To validate the robustness of the proposed system, the validation procedure was repeated five times. The performance measures were averaged over the five runs. The overall agreement and kappa coefficient of the proposed method are 82% and 0.73, respectively. The sensitivity of the sleep stages of Wake, N1, N2, N3, and REM are 90.0%, 48.6%, 84.9%, 84.2%, and 81.9%, respectively. Conclusion The performance of the proposed method was achieved expert level, and it was noted that the ensemble-CNN is a promising solution for automatic sleep stage scoring. This method can assist clinical staff in reducing the time required for sleep stage scoring in the future. Support This work was supported by the Ministry of Science and Technology, Taiwan. (MOST 106-2218-E-035-013-MY2, 108-2221-E-035-064, and 108-2634-F-006-012).

Concept verification of a Remote Automatic Scoring System for Evaluating Knee Function after Total Knee Arthroplasty.

The Knee Society Score (KSS) is the most commonly used scale for evaluating postoperative pain and physical function after total knee arthroplasty (TKA). However, this scale requires clinic visiting, which is not quite convenient. Our concept verification study demonstrated a remote automatic system for evaluating knee function after TKA using the KSS. The remote scoring system consists of two modules for data acquisition, an application for patients, a cloud server, and an application for doctors. The kinematic data are collected by the data acquisition modules and transmitted to the patient application via Bluetooth. The data acquisition module contains a motion sensor, a microcontroller unit, a power supply, and a Bluetooth module. The motion sensor consists of an accelerometer, a gyroscope, and a geomagnetic sensor, all of which are three-axis instruments. Using the nine-axis data, the three-dimensional (3D) angles are calculated according to the theory of attitude and heading reference system. The KSS score is calculated using a scoring algorithm in the patient application and transmitted to the doctor application through the cloud server. The knee function of 10 patients treated with unilateral TKA was evaluated by both a doctor and the remote scoring system. The consistency in KSS between the doctor and the system was analyzed using the paired t-test. The remote scoring system successfully recorded knee function data and transmitted the scores from the patient application to the doctor application through the cloud server. There was no significant difference in the KSS scores evaluated by the doctor and that by the system (p = 0.326). This remote automatic scoring system provides a reliable and convenient method for evaluating knee function after TKA at home.

EMKLAS: A New Automatic Scoring System for Early and Mild Keratoconus Detection.

PurposeCreate a unique predictive model based on a set of demographic, optical, and geometric variables with two objectives: classifying keratoconus (KC) in its first clinical manifestation stages and establishing the probability of having correctly classified each case.MethodsWe selected 178 eyes of 178 subjects (115 males; 64.6%; 63 females, 35.4%). Of these, 74 were healthy control subjects, and 104 suffered from KC according to the RETICS grading system (61 early KC, 43 mild KC). Only one eye from each patient was selected, and 27 different parameters were studied (demographic, clinical, pachymetric, and geometric). The data obtained were used in an ordinal logistic regression model programmed as a web application capable of using new patient data for real-time predictions.ResultsEMKLAS, an early and mild KC classifier, showed good training performance figures, with 73% global accuracy and a 95% confidence interval of 65% to 79%. This classifier is particularly accurate when validated by an independent sample for the control (79%) and mild KC (80%) groups. The accuracy of the early KC group was remarkably lower (69%). The variables included in the model were age, gender, corrected distance visual acuity, 8-mm corneal diameter, and posterior minimum thickness point deviation.ConclusionsOur web application allows fast, objective, and quantitative assessment of early and mild KC in detection and classification terms and assists ophthalmology professionals in diagnosing this disease.Translational RelevanceNo single gold standard exists for detecting and classifying preclinical KC, but the use of our web application and EMKLAS score may aid the decision-making process of doctors.

Evaluations of Automated Scoring Systems in Practice

This research report provides a description of the processes of evaluating the “deployability” of automated scoring (AS) systems from the perspective of large‐scale educational assessments in operational settings. It discusses a comprehensive psychometric evaluation that entails analyses that take into consideration the specific purpose of AS, the test design, the quality of human scores, the data collection design needed to train and evaluate the AS model, and the application of statistics and evaluation criteria. Finally, it notes that an effective evaluation of an AS system requires professional judgment coupled with statistical and psychometric knowledge and understanding of the risk assessment and business metrics.

Detection of Respiratory Events by Respiratory Effort and Oxygen Desaturation

PurposeSleep respiratory events are scored based on the reduction of airflow measured by a thermistor or nasal pressure cannula, together with oxygen desaturation and arousal criteria for hypopneas. We investigated whether automatic scoring can be performed without using the uncomfortable oronasal sensors and developed an automatic scoring system that is compatible with level III home sleep apnea testing devices.MethodsWe developed a respiratory event detection algorithm, based on SpO2 and respiratory effort signal measured from respiratory inductance plethysmograph (uncalibrated RIPsum), that outputs the time and duration of detected events and calculates an apnea–hypopnea-index (AHI) based on total recording time. The algorithm was tested on 98 polysomnography (PSG) recordings of patients, 77 with suspected sleep apnea and 21 without. The results were compared to annotations provided by the PSG systems where PSG AHI was computed using the total sleep time. The predicted AHI was evaluated for correlation and agreement with the PSG AHI using the intra-class correlation coefficient (ICC). Severity classification was performed and evaluated using the following categories: normal (< 5), mild (5–15), moderate (15–30), and severe (ge 30).ResultsThe ICC between predicted AHI and PSG AHI scored r = 0.96 (0.95–0.97, p < 0.001). The algorithm correctly predicted the severity for 74 recordings, overestimated 16, and underestimated 8. There was no misclassification by more than one severity level.ConclusionUsing respiratory effort and SpO2, our algorithm was able to detect respiratory events with high correlation and agreement compared to full PSG-based detection.

Effect of Pigai.org on English Majors’ Writing Self-efficacy and Writing Performance

With the deepening of college English teaching reform, modern educational technology and network teaching platform are gradually applied to college English teaching. As to the English writing, using the automatic scoring system to automatically evaluate the students’ composition has become a new trend. Pigai.org, as one of the many automatic scoring systems in China, has received great attention at home and abroad due to its advantages of timely feedback, high working efficiency and wide application. The thesis conducted a three-month English writing experiment to study the effect of Pigai.org. This experiment used 39 sophomores of English major as the research subject and adopted English writing self-efficacy questionnaires, English writing tests and interview as the research methods to further discusses the influence of Pigai.org on writing self-efficacy and writing achievement of the subjects, so as to effectively play the role of the Pigai.org in the teaching of English writing. The results of this research were as follows: firstly, the English writing self-efficacy and the English writing achievement of the subjects were improved after using Pigai.org by and large. Secondly, using Pigai.org had different influence on the self-efficacy and the writing achievement of students in different levels. The experimental results showed that the effect of Pigai.org on low-level students was more significant than that of high-level students.