Automated Assessment System Research Articles

Fully automatic model-based calcium segmentation and scoring in coronary CT angiography

The paper presents new methods for automatic coronary calcium detection, segmentation and scoring in coronary CT angiography (cCTA) studies. Calcium detection and segmentation are performed by modeling image intensity profiles of coronary arteries. The scoring algorithm is based on a simulated unenhanced calcium score (CS) CT image, constructed by virtually removing the contrast media from cCTA. The methods are implemented as part of a fully automatic system for CS assessment from cCTA. The system was tested in two independent clinical trials on 263 studies and demonstrated 0.95/0.91 correlation between the CS computed from cCTA and the standard Agatston score derived from unenhanced CS CT. The mean absolute percent difference (MAPD) of 36/39 % between the two scores lies within the error range of the standard CS CT (15-65 %). High diagnostic performance, combined with the benefits of the fully automatic solution, suggests that the proposed technique can be used to eliminate the need in a separate CS CT scan as part of the cCTA examination, thus reducing the radiation exposure and simplifying the procedure.

Digital Analysis of Changes in Chronic Wounds through Image Processing

Measurement of wound healing status is very important for monitoring and evaluation of progress in an individual patient. Thus wound classification is a vital step in the development of an automatic measurement system for wound healing assessment. In the existing system a RGB (Red, Green and Blue) histogram distributions of pixel values from wound colour images is being used in the new tissue classification protocol [1],[4] . This protocol has been carried out using the KNN classifier and results show that the in the proposed system the existing system protocol is integrated with another protocol i.e., fuzzy classification for maximum probability. Thus the maximum probable result is considered as final classification of the resultant wound type depending whether the wound categorizes under the acute or the chronic wound and denotes the healing period.

Cost-sensitive ordinal regression for fully automatic facial beauty assessment

In this paper, we propose a new cost-sensitive ordinal regression (CSOR) approach for fully automatic facial beauty assessment. While there have been several facial beauty assessment methods in the literature, most of them require an accurate set of manual landmarks and are not fully automatic. In many real-world applications, face images are usually captured in unconstrained environments and hence it is desirable to develop fully automatic facial beauty assessment systems for these practical applications. To this end, we develop a fully automatic facial beauty assessment system that behaves like human beings in assessing the concept of facial beauty. To achieve this goal, we propose a new CSOR method to predict the beauty information of face images captured in unconstrained environments. Our method is motivated by the fact that the relative order information of the beauty label information in a face dataset and the costs (losses) of different mis-assessments are usually different. To make better use of these information, we learn a regression model by uncovering the nonlinear structure and preserving the ordinal information of facial images simultaneously. Experimental results on our dataset are presented to demonstrate the efficacy of the proposed approach.

METHODOLOGICAL ASPECTS OF AUTOMATIC PILOT FLIGHT ASSESSMENT SYSTEM

This article deals with the various aspects of automatic pilot flight assessment system methods. Different types of flight corridors are examined: circling, left and right superelevation drawing (number eight), and others. The advantages and disadvantages of each of them are discussed. Limit line calculations of the corridor turns are presented. The aim is to develop new flight assessment methods.

Design and Implementation of an Automatic Scoring Subjective Question System Based on Domain Ontology

Automated assessment technology for subjective tests is one of the key techniques of exam systems. A model based on domain ontology is proposed in this paper, which can be used in exam systems to estimate subjective tests. After analysing the present research status of subjective automated assessment technology, the paper makes a study on the construction method of domain ontology by taking software engeering domain as an example. Semantic similarity calculation based on domain ontology is used for automatic assessment in this paper. The automatic assessment system can divide a sentence into a series of phrases by using the natural language processing technology and get the score by evaluating the semantic similarity of the student's answer. The experiments show that the results of the system which has certain valuable feasibility and applicability are credible and the scoring errors are acceptable.

Student perception and usage of an automated programming assessment tool

Automated assessment systems are gaining popularity within computer programming courses. In this paper we perform an empirical evaluation of Mooshak, an online judge that verifies program correctness, in order to determine its usefulness in classroom settings. In particular, we provide a detailed study on how students use the tool, analyze their opinions and critiques about it, and measure other features like its capability to reduce dropout rates. The experience was carried out within a course on algorithm design and analysis where we collected information through several questionnaires and data generated by the tool during the course. Among the main findings we highlight: (1) the usage of the tool was adequate in relation to students’ own testing; (2) its feedback needs to be richer in order to improve its acceptance among students; and (3) there was no statistical evidence to claim Mooshak reduced the dropout rate.

Automatic Detection and Assessment System of Water Turbidity based on Image Processing

The realization of automatic control system of releasing flocculating agent is one of the bottlenecks in the automation process of waterworks. For various reasons, most of waterworks in China decide the dosing amount just by artificial methods, which may lead to great subjectivity. It can result in direct economic losses and threaten residents’ drinking water safety. To solve the problem, based on digital image processing and pattern recognition technology, the paper presents an advanced and practical automation scheme, trying to control the releasing of flocculating agent. According to the characteristics of alum images captured from waterworks, by comparing the effects of many vision algorithms, the paper selects a suitable combination and tries to implement it with EmguCV. On next step, the paper implements feature selection and classification with LibSVM. The experiments show that, the system designed in this paper is reasonable and feasible. DOI: http://dx.doi.org/10.11591/telkomnika.v11i3.2299

Test My Code

We describe an automated assessment system called Test My Code TMC that is designed to support instructors' and students' work in programming courses. From the students' point of view, TMC is a transparent assessment service that is integrated to an industry-standard programming environment. TMC is used to provide scaffolding of students' learning during the working process, and to retrieve and update exercises as the students work on them, without causing additional overhead to the learning process. From the instructors' perspective, TMC makes collaborative crafting of exercises easier, supports building exercises with smaller goals that combine into bigger programs, gathers snapshots from students' programming process, collects feedback from students, and has the capability to export course submission data into external systems. TMC has been used in massive open online courses on programming as well as in courses on web-development with hundreds of students.

Automated assessment of footpad dermatitis in broiler chickens at the slaughter-line: Evaluation and correspondence with human expert scores

Footpad dermatitis is increasingly used as an indicator of decreased broiler welfare, and automation of dermatitis monitoring potentially reduces the effort needed to monitor commercial flocks. In this study we evaluated a prototype system for the automatic assessment of footpad dermatitis in broiler chickens by comparing the automatic assessment with a human expert assessment. The expert aimed at selecting 2 times (different period) 20 broilers per footpad dermatitis category (5 categories in total), from 2 different flocks of 38-d-old broilers on an experimental farm. Two days later these broilers were transported to the slaughterhouse, where footpad dermatitis was assessed by the automatic system. Subsequently the footpads were reassessed by the same expert that had selected the birds. Automatic scores were only weakly correlated with scores given by the expert on-farm (r = 0.54) and at the slaughterhouse (r = 0.59). Manual evaluation of the photographs on which the automatic system based its scores revealed several errors. For 41.1% of the birds, the automatic system assessed only one of the footpads, whereas for 15.2% neither footpad was assessed. For 49.4% of the birds, scores were based on partially incorrectly identified areas. When data from such incomplete and obviously incorrect assessments were discarded, stronger correlations between automatic and expert scores were found (r = 0.68 and r = 0.74 for expert scores given on-farm and at-slaughter, respectively). Footpads that were missed by the automatic system were more likely to receive a high expert score at slaughter (P = 0.02). However, average flock scores did not differ greatly between automatic and expert scores. The prototype system for automatic dermatitis assessment needs to be improved on several points if it is to replace expert assessment of footpad dermatitis.

Categorizing variations of student-implemented sorting algorithms

In this study, we examined freshmen students' sorting algorithm implementations in data structures and algorithms' course in two phases: at the beginning of the course before the students received any instruction on sorting algorithms, and after taking a lecture on sorting algorithms. The analysis revealed that many students have insufficient understanding of implementing sorting algorithms. For example, they include unnecessary swaps in their Insertion or Selection sort implementations resulting in more complicated and inefficient code. Based on the data, we present a categorization of these types of variations and discuss the implications of the results. In addition, we introduce an instrument to recognize these algorithms automatically. This is done in terms of white-box testing. Our aim is to develop an automatic assessment system to help teachers in the burden of marking students' assignments and give feedback to the students on their algorithmic solutions. We outline how the presented results can be used to develop the instrument further.

A new quality assessment and improvement system for print media

Print media collections of considerable size are held by cultural heritage organizations and will soon be subject to digitization activities. However, technical content quality management in digitization workflows strongly relies on human monitoring. This heavy human intervention is cost intensive and time consuming, which makes automization mandatory. In this article, a new automatic quality assessment and improvement system is proposed. The digitized source image and color reference target are extracted from the raw digitized images by an automatic segmentation process. The target is evaluated by a reference-based algorithm. No-reference quality metrics are applied to the source image. Experimental results are provided to illustrate the performance of the proposed system. We show that it features a good performance in the extraction as well as in the quality assessment step compared to the state-of-the-art. The impact of efficient and dedicated quality assessors on the optimization step is extensively documented.

Text-independent pronunciation quality automatic assessment system for English retelling test

An automatic grading system for spoken English retelling test is presented in this paper. Speech recognition technology is used in the system to evaluate the quality of retelling according to the pre-defined scoring rubric which includes speech fluency, pronunciation accuracy and content integrity. Scoring features for these quality aspects are firstly extracted by applying LVCSR, keyword spotting, forced alignment and confidence measurements. And then, these features are mapped to a score by using SVM model which is pre-trained on human rated test items. According to the experimental results the correlation coefficient between machine scores and expert scores is 0.729, which means that the system can be used in real examination to replace human scores. This work is partially supported by the National Natural Science Foundation of China (No. 10925419, 90920302, 10874203, 60875014, 61072124, 11074275, 11161140319).

Automatic Assessment of Programming assignment

In today’s world study of computer’s language is more important. Effective and good programming skills are need full all computer science students. They can be master in programming, only through intensive exercise practices. Due to day by day increasing number of students in the class, the assessment of programming exercises leads to extensive workload for teacher/instructor, particularly if it has to be carried out manually. In this paper, we propose an automatic assessment system for programming assignments, using verification program with random inputs. One of the most important properties of a program is that, it carries out its intended function. The intended function of a program or part of a program can be verified by using inverse function’s verification program. For checking intended functionality and evaluation of a program, we have used verification program. This assessment system has been tested on basic C programming courses, and results shows that it can work well in basic programming exercises, with some initial promising results.

A Forced Alignment Based Approach for English Passage Reading Assessment

This paper presents our investigation into improving the performance of our previous automatic reading quality assessment system. The method of the baseline system is calculating the average value of the Phone Log-Posterior Probability (PLPP) of all phones in the voice to be assessed, and the average value is used as the reading quality assessment feature. In this paper, we presents three improvements. First, we cluster the triphones, and then calculate the average value of the normalized PLPP for each classification separately, and use this average values as the multi-dimensional assessment features instead of the original one-dimensional assessment feature. This method is simple but effective, which made the score difference of the machine scoring and manual scoring decrease by 30.2% relatively. Second, in order to assess the reading rhythm, we train Gaussian Mixture Models (GMM), which contain the information of each triphone's relative duration under standard pronunciation. Using the GMM, we can calculate the probability that the relative duration of each phone is conform to the standard pronunciation, and the average value of the probabilities is added to the assessment feature vector as a dimension of feature, which decreased the score difference between the machine scoring and manual scoring by 9.7% relatively. Third, we detect Filled Pauses (FP) by analyzing the formant curve, and then calculate the relative duration of FP, and add the relative duration of FP to the assessment feature vector as a dimension of feature. This method made the score difference between the machine scoring and manual scoring be further decreased by 10.2% relatively. Finally, when the feature vector extracted by the three methods are used together, the score difference between the machine scoring and manual scoring was decreased by 43.9% relatively compared to the baseline system.

Automated optical inspection system for digital TV sets

This article proposes a real-time test and verification system for full-reference automatic image quality assessment and verification of digital TV sets. Digital camera is used for acquisition of the TV screen content in order to ensure quality assessment of the content as perceived by the user. Test has been executed in three steps: image acquisition by camera, TV screen content extraction and full-reference image quality assessment. The TV screen content is extracted from the captured image in two steps: detection of the TV screen edge and transformation of the TV screen content to dimensions of the reference image. Three image comparison methods are incorporated to perform full-reference image quality assessment. Reference image for quality assessment is obtained either by grabbing the image from TV set or by capturing the TV screen content on the golden sample. Digital camera was later replaced with DSP-based camera for image acquisition and algorithm execution which brought significant performance improvements. The comparison methods were tested under constant and variable illumination conditions. The proposed system is used to automate the verification step on the final production line of digital TV sets. The time required for verification step decreased by a factor of 5 when using the proposed system on the final production line instead of a manual one.

Cognitive Abilities of Alzheimers Disease Transgenic Mice are Modulated by Social Context and Circadian Rhythm

In the present study, we used a new training paradigm in the intelliCage automatic behavioral assessment system to investigate cognitive functions of the transgenic mice harboring London mutation of the human amyloid precursor protein (APP.V717I). Three groups of animals: 5-, 12- and 18-24-month old were subjected to both Water Maze training and the IntelliCage-based appetitive conditioning. The spatial memory deficit was observed in all three groups of transgenic mice in both behavioral paradigms. However, the APP mice were capable to learn normally when co-housed with the wild-type (WT) littermates, in contrast to clearly impaired learning observed when the transgenic mice were housed alone. Furthermore, in the transgenic mice kept in the Intellicage alone, the cognitive deficit of the young animals was modulated by the circadian rhythm, namely was prominent only during the active phase of the day. The novel approach to study the transgenic mice cognitive abilities presented in this paper offers new insight into cognitive dysfunctions of the Alzheimer's disease mouse model.

Automated Assessment in a Programming Tools Course

Automated assessment systems can be useful for both students and instructors. Ranking and immediate feedback can have a strongly positive effect on student learning. This paper presents an experience using automatic assessment in a programming tools course. The proposal aims at extending the traditional use of an online judging system with a series of assignments related to programming tools. Some empirical results on how students use an automated assessment system in a CS2 course are presented. Research suggested that automated assessment systems promoted the students' interest and produced statistically significant differences in the scores between experimental and control groups.

Automatic video monitoring system for assessment of Alzheimer’s Disease symptoms

In order to fully capture the complexity of the behavioural, functioning and cognitive disturbances in Alzheimer Disease (AD) and related disorders information and communication techniques (ICT), could be of interest. This article presents using 3 clinical cases the feasibility results of an automatic video monitoring system aiming to assess subjects involved in a clinical scenario. The study was conducted in an observation room equipped with everyday objects for use in activities of daily living. The overall aim of the clinical scenario was to enable the participants to undertake a set of daily tasks that could realistically be achieved in the setting of the observation room. The scenario was divided in three steps covering basic to more complex activities: (1) Directed activities, (2) Semi-directed activities, (3) Undirected (“free”) activities. The assessment of each participant of the study was done with an automatic video monitoring system composed of a vision component and an event recognition component. The feasibility study involved three participants: two AD patients and one elderly control participant. The first result of the study was to demonstrate the feasibility of this new assessment method from both the patient and the technical points of view. During the first step the control participant performed all these activities faster than the two AD participants. During the second step of the scenario AD participants were not able to follow the correct order of the tasks and even omitted some of them. Finally during the last step of the scenario devoted to free activities the control participant chose one of the proposed activities (reading) and undertook this activity for almost the entire duration. In contrast, the two AD participants had more difficulties choosing one of the suggested activities and were not able to undertake any one activity in a sustained manner. The automatic video monitoring system presented here analyzes human behaviours and looks for changes in activity through the detection of the presence of people and their movements in real time. Once the technique has been standardized, it could significantly enhance the assessment of AD patients in both clinical and clinical trial settings as well as providing further information regarding patient frailty that could enhance their safety and ease caregiver burden.

Bone Age Assessment in Young Children Using Automatic Carpal Bone Feature Extraction and Support Vector Regression

Boundary extraction of carpal bone images is a critical operation of the automatic bone age assessment system, since the contrast between the bony structure and soft tissue are very poor. In this paper, we present an edge following technique for boundary extraction in carpal bone images and apply it to assess bone age in young children. Our proposed technique can detect the boundaries of carpal bones in X-ray images by using the information from the vector image model and the edge map. Feature analysis of the carpal bones can reveal the important information for bone age assessment. Five features for bone age assessment are calculated from the boundary extraction result of each carpal bone. All features are taken as input into the support vector regression (SVR) that assesses the bone age. We compare the SVR with the neural network regression (NNR). We use 180 images of carpal bone from a digital hand atlas to assess the bone age of young children from 0 to 6 years old. Leave-one-out cross validation is used for testing the efficiency of the techniques. The opinions of the skilled radiologists provided in the atlas are used as the ground truth in bone age assessment. The SVR is able to provide more accurate bone age assessment results than the NNR. The experimental results from SVR are very close to the bone age assessment by skilled radiologists.

Electronic System for Data Record and Automatic Diagnosis Assessment in the Temporomandibular Joint Disorders

Background: The research goal of the Dental segment of the Centre of Biomedical Informatics is focused on the electronic health record (EHR) development for dentistry. Objectives: At the beginning there has been constructed an electronic dental cross DentCross, which was representing patients dental data in the graphical form. It has been completed with the system of the automatic speech recognition (ASR) and voice synthesis module (TTS). Methods: The main goal of this work was to reach the high entirety of the system and its automatization. For this reason it has been completed with the special record medium for the temporomandibular disorders (TMD). Results: Concerning the experience with the old version the knowledge database (KB) for TMD has been structured differently. A classification diagnostic schema by the American Academy of Orofacial Pain (AAOP) has been used. The KB has been created in the MUDR KB Editor application. On this basis a relational database has been constructed and a user interface for data collection based on MUDR and MUDRLite EHR systems was developed. Conclusions: The main advantage of this system is determination of probable diagnosis of the disease (AAOP) by the system (custom component). It is based on the characteristic data, which have been recorded in the electronic form after the investigation. For the creation of the component of its alone MS Visual Studio.NET 2003 development tool has been used. The whole component is programmed in C] language.