Adaptive Assessment System Research Articles

Temporal Learning Analytics for Adaptive Assessment

Accurate and early predictions of students’ performance could significantly affect interventions during teaching and assessment, which gradually could lead to improved learning outcomes. In our research work, we seek to identify and formalize temporal parameters as predictors of performance (“temporal learning analytics”-TLA) and examine students' temporal behaviour (i.e. in terms of time-spent) during testing. The goal is to specify a functional set of parameters that will be embedded in an adaptive assessment system in order to contribute to personalization of feedback services. We adopted the Partial Least-Squares (PLS) analysis method for formulating the causal dependencies between latent variables and the relations to their indicators. In this paper we present the motivation and rationale of our work, along with the followed methodology, initial results and contributions so far, and our plans on future work.

T2FS-Based Adaptive Linguistic Assessment System for Semantic Analysis and Human Performance Evaluation on Game of Go

The game of Go is a board game with a long history that is much more complex than chess. The uncertainties of this game will be higher when the board size gets bigger. For evaluating the human performance on Go games, one human could be advanced to a higher rank based on the number of winning games via a formal human against human competition. However, a human Go player's performance could be influenced by factors such as the on-the-spot environment, as well as physical and mental situations of the day, which causes difficulty and uncertainty in certificating the human's rank. Thanks to a sample of one player's games, evaluating his/her strength by classical models such as the Bradley-Terry model is possible. However, due to inhomogeneous game conditions and limited access to archives of games, such estimates can be imprecise. In addition, classical rankings (1 Dan, 2 Dan, ...) are integers, which lead to a rather imprecise estimate of the opponent's strengths. Therefore, we propose to use a sample of games played against a computer to estimate the human's strength. In order to increase the precision, the strength of the computer is adapted from one move to the next by increasing or decreasing the computational power based on the current situation and the result of games. The human can decide some specific conditions, such as komi and board size. In this paper, we use type-2 fuzzy sets (T2FSs) with parameters optimized by a genetic algorithm for estimating the rank in a stable manner, independently of board size. More precisely, an adaptive Monte Carlo tree search (MCTS) estimates the number of simulations, corresponding to the strength of its opponents. Next, the T2FS-based adaptive linguistic assessment system infers the human performance and presents the results using the linguistic description. The experimental results show that the proposed approach is feasible for application to the adaptive linguistic assessment on a human Go player's performance.

From ABLE to TAPAS: A New Generation of Personality Tests to Support Military Selection and Classification Decisions

This article discusses 3 modern forced-choice personality tests developed for the U.S Armed Services to provide resistance to faking and other forms of response distortion: the Assessment of Individual Motivation, the Navy Computerized Adaptive Personality Scales, and the Tailored Adaptive Personality Assessment System. These tests represent the transition from Likert to forced-choice formats and from static to computerized adaptive item selection to meet the challenges of large-scale, high-stakes testing environments. For each test, we briefly describe the personality constructs that are assessed, the response format and scoring methods, and selected ongoing research and development efforts. We also highlight the potential of these tests for personnel selection, classification, and diagnostic screening purposes.

Prediction of preschool functional abilities after early complex cardiac surgery.

It is important to identify early predictors of functional limitations in children after congenital heart surgery to optimise their independence as they prepare for school. The purpose of this study is to determine potentially modifiable predictor variables of functional abilities in pre-school children who underwent complex cardiac surgery at 6 weeks of age or earlier. This prospective inception cohort study comprised a sample of 165 survivors (63% boys) who had complex cardiac surgery (75% biventricular repairs) at Stollery Children's Hospital, Edmonton, Alberta. We excluded children with chromosomal abnormalities. When children were 4-5 years of age, the parents completed the Adaptive Behavioral Assessment System II. Regression analysis was used to assess the association between multiple risk factors and each of the four continuous composite scores. The mean scores for the practical domain and general adaptive composite score of the Adaptive Behavioural Assessment System were lower than the conceptual and social domains, with 13.3% of the children having a delay in the practical domain. There was a significant association between the general adaptive (p=0.003; 0.012), conceptual (p=0.0004; 0.042), social (p=0.0007; 0.028), and the practical (p=0.046; 0.003) domain composite scores with the mother's education and preoperative plasma lactate, respectively. Maternal education may be a marker for the social context of children, and warrants societal attention to improve functional outcomes. Preoperative lactate as a potentially modifiable variable may warrant increased attention to early diagnosis and aggressive resuscitation of young infants with congenital heart disease.

Personality Assessment Questionnaire as a pre-accession screen for risk of mental disorders and early attrition in U. S. Army recruits.

Personality assessment tools have been studied as predictors of performance in civilian and military work settings. The Tailored Adaptive Personality Assessment System (TAPAS) was developed to improve selection of new military recruits by predicting motivational outcomes such as job effort, physical fitness, and drive to perform at high standards. The purpose of this study is to examine the utility of TAPAS as a predictor of psychiatric morbidity and early discharge in a sample of 15,082 Army, active duty, enlisted, nonprior service recruits. Associations between TAPAS personality dimension score quintiles and mental disorder diagnoses, attrition, and health care utilization in United States Army recruits who took TAPAS in the fiscal year 2010 were analyzed using multivariate logistic regression and log-linear modeling. TAPAS physical conditioning dimension scores were predictive of mental disorder diagnosis and attrition, with TAPAS scorers in the lowest quintile at increased odds of early discharge (odds ratio [OR]: 2.08, 95% CI 1.73, 2.51), mental disorder diagnosis (OR: 1.41, 95% CI 1.20, 1.66) and greater mental health care utilization (1.61, 95% CI 1.46, 1.78) compared with TAPAS scorers in the highest quintile. Results indicated that TAPAS may have an important use as a mental health fitness screening tool for those who wish to serve in the military by identifying a limited high risk group of applicants for mental health diagnostic evaluation. TAPAS may augment current cognitive and educational screens and potentially reduce the burden of mental disorders and premature attrition.

A Bayesian Classification Approach for Handling Uncertainty in Adaptive E-Assessment

New technologies have provided the educational field with innovations that allow significant improvement in the teaching learning process. The development of information technology has increased the popularity of Web based education, ; particularly E-assessment This paper provides an overview of Bayesian Network and its application to handle uncertainty in adaptive E-assessment. An adaptive assessment system was realized in PHP and MYSQL. The empirical data was collected and the relationship between the response time and the assessment grading was established. The classifier accuracy obtained using Bayesian classifier was compared with C4.5 Decision tree algorithm and Naive Bayes classifier. The superiority of Bayesian algorithm over the well known algorithms in handling uncertainty of adaptive assessment data is established in this paper.

Usage of graph patterns for concept map extension

Abstract In concept map-based assessment an expert’s concept map can be expanded using graph patterns to add hidden and inverse relations. This helps to avoid forcing a learner to use certain structures and names. Graph patterns are subgraphs that describe combinations of concept map elements, from which extra relations can be inferred. In this paper an enriched set of graph patterns is described along with their respective IF...THEN rules which can be used for automated knowledge assessment. Some of them are already implemented in the intelligent and adaptive knowledge assessment system IKAS.

Architecture And Design Process Of The Individualized Assesment System Integrable To Distance Education Softwares

Assessment is one of the methods used for evaluation of the learning outputs. Nowadays, use of adaptive assessment systems estimating ability level and abilities of the students is becoming widespread instead of traditional assessment systems. Adaptive assessment system evaluates students not only according to their marks that they take in test exams but also according to their ability levels. In this study, we dealt with theoretical background of the adaptive assessment systems. Study covers the structure, characteristics and architecture of the computerized adaptive test systems in detail. It dwells on the Item Response theory which is used for improving computerized adaptive test systems and the models with which this theory is used. Besides, processing steps which are required to realize a computerized adaptive test system and algorithms used for the validation of the test systems are introduced in detail. Briefly, study introduces the structure and the improvement processes of the computerized adaptive testing systems which can be integrated to distance education software.

A study of Electrical Security Risk Assessment System based on Electricity Regulation

With the expansion of China's power system, there are more and more factors contributing to Large-area Power Outages. The snow disaster that hit Southern China in 2008 caused serious damages to the power systems, which made Chinese government realize that they cannot solely depend on power enterprises to effectively control the risk involved in power system. The Chinese government needs to collaborate with power enterprises in strengthening Electricity Regulation and taking countermeasures to reduce the risk of Large-area Power Outages. This paper first proposes an objective, practical and adaptive Electrical Security Risk Assessment System based on Electricity Regulation. The system consists of an assessment process with 278 indices, which are used to calculate the risk of Large-area Power Outages by the use of Analytic Hierarchy Process and the Delphi Method. Then, an example of Guangdong power system serves to illustrate the detailed implementation procedures of the proposed assessment system and the results show that Guangdong power system faces a moderate risk of Large-area Power Outages. Finally, some countermeasures are proposed to overcome the defects in the existing power system.

Research on Vulnerability Assessment System of Complicated Power System

The factors which lead to large-area power outages are more than ever, when the complexity of the expansion power system is increasing. Therefore, it is important to assess the vulnerability of the complicated power system. This paper first proposes an objective, practical and adaptive vulnerability assessment system of complicated power system by use of Analytic Hierarchy Process, which based on multi-factors analysis of large-area outages. This system can find out the vulnerability point of the power system，and take advantages of the government, who can over-all plan, correspond and organize the social resource to reduce the vulnerability of power system and prevent large-area power outages, and the power enterprises who can maintaining the stability of power system by technology. Finally, the paper gives an assessment example of provincial power system.

Functional outcomes after neonatal open cardiac surgery: comparison of survivors of the Norwood staged procedure and the arterial switch operation

Improvements in long-term survival of children undergoing the Norwood staged procedure and the arterial switch operation have resulted in the need to prepare these at-risk children for each stage of their developmental trajectory, including school readiness. This study describes and compares functional outcomes following the Norwood staged procedure and arterial switch operations. This prospective inception cohort study comprised a sample of 73 children (71% boys) who had the Norwood staged procedure (n = 28) or the arterial switch operation (n = 45) at the age of 6 weeks or younger at the Stollery Children’s Hospital, Edmonton, Alberta, between 2002 and 2005. We excluded children who had chromosomal abnormalities or cerebral palsy. When children were 18-24 months of age, parents completed the Adaptive Behavioral Assessment System II. Standard scores for the domains are mean 100, standard deviation (15); skill area scaled scores, 10 (3). Student’s t-test with Bonferonni correction was used to compare groups. This population has greater than four times the number of children delayed on the General Adaptive Composite than the normative group. Functional outcomes were similar in the two groups other than those of home living (Norwood: 8.8 (2.8) compared with arterial switch: 11.2 (3.1), t = 3.389, p = 0.001) and self-care (Norwood: 5.9 (3.5) versus arterial switch: 8.1 (2.6), t = 3.140, p = 0.002). These survivors are at increased risk for delayed functional abilities. Self-care, necessary for independence and confidence as children reach school age, was particularly low in the Norwood group. Reasons for low self-care abilities require further study.

Evolution of the Concept Map Based Adaptive Knowledge Assessment System: Implementation and Evaluation Results

Evolution of the Concept Map Based Adaptive Knowledge Assessment System: Implementation and Evaluation ResultsThe paper represents the concept map based adaptive knowledge assessment system. Advantages of concept maps are analyzed emphasizing that the approach offers a reasonable balance between requirements to assess higher levels of knowledge according to Bloom's taxonomy and complexity of a system. Concept maps allow revealing of student's knowledge structure, promote system thinking and support process oriented learning where a study course is divided into stages in each of which knowledge assessment is carried out. The developed knowledge assessment system consists from a teacher's, learner's and administrator's modules and is implemented as a multiagent system. Four prototypes of the system developed within four projects are described. The first prototype supports only fill-in-the-map tasks where a learner must put given concepts in correct places. The second prototype provides changing the degree of task difficulty, thus, performing adaptation to a learner's knowledge level. The set of tasks are also extended by construct-the-map tasks. Improvements implemented in the third prototype allow using of directed arcs and standard relationships in concept maps. The three-tier architecture used in the fourth prototype is chosen to rise the security level of the system. Besides that learner's support is considerably expanded giving help and tutoring to a learner. Results of evaluation of the developed system's prototypes in different study courses are presented. The paper concludes with the comparison of all four prototypes using all main characteristics of the developed knowledge assessment system.

An Adaptive Quality Assessment System—Aspect of Human Factor and Measurement Uncertainty

In this paper, we discuss a model of quality that makes use of the fuzzily defined variable approach to better understand the concept and, thus, enables the further development of this variable. We propose a general method that may estimate a quality index (QI) that handles both qualitative and quantitative issues. The system further uses a neural network since the system learns how to integrate human factors into a quantitative QI. In our case study, we have examined the measurement of image quality and proposed a theoretical model of pulp quality.

Usage of Graph Patterns for Knowledge Assessment Based on Concept Maps

Usage of Graph Patterns for Knowledge Assessment Based on Concept MapsThe paper discusses application of concepts maps (CMs) for knowledge assessment. CMs are graphs which nodes represent concepts and arcs represent relationships between them. CMs reveal learners' knowledge structure and allow assessing their knowledge level. Step-by-step construction and use of CMs is easy. However, mere comparison of expert constructed and learners' completed CMs forces students to construct their knowledge exactly in the same way as experts. At the same time it is known that individuals construct their knowledge structures in different ways. The developed adaptive knowledge assessment system which is implemented as multiagent system includes the knowledge evaluation agent which carries out the comparison of CMs. The paper presents a novel approach to comparison of CMs using graph patterns. Graph patterns are subgraphs, i.e., paths with limited length. Graph patterns are given for both fill-in-the-map tasks where CM structure is predefined and construct-the-map tasks. The corresponding production rules of graph patterns allow to expand the expert's constructed CM and in this way to promote more flexible and adaptive knowledge assessment.

Time-estimation skill assessment: a demostration of adaptive assessment techniques.

The general applicability of adaptive measurement techniques to human skill and capability assessment is demonstrated in terms of the adaptive assessment of a representative cognitive (judgmental) skill. Adaptive assessment techniques are defined, and an adaptive time-estimation assessment system that automatically adjusts task difficulty on the basis of operator performance is described. Data are presented to demonstrate the sensitivity and economy of adaptive assessment systems. The advantages of adaptive assessment systems are detailed and guidelines for the development of such systems are suggested.