Predicting Training Performance Research Articles

From classroom to kitchen: Predictors of training performance and transfer of culinary skills

With growing concerns over the workforce “skills gap,” additional insight is needed on the training and transfer of professional skills, particularly surrounding the transition from education to employment. This study leveraged latent change score modeling to examine the role of the Big Five personality traits, general mental ability (GMA), and training self-efficacy for predicting training performance and transfer, operationalized as change in performance ratings across two transition periods. Data were collected from Midwestern United States culinary students (N = 239) completing a 6-week capstone course and two consecutive 6-week externships during which they were evaluated on professional and culinary skills. Findings demonstrated complex relations with training performance, initial transfer, and subsequent transfer, such that GMA, conscientiousness, and openness/intellect (negatively) predicted performance in the training program, whereas GMA, neuroticism, and openness/intellect (positively) predicted change in performance between training and initial transfer. Implications for theory, practice, and the “skills gap” are discussed.

Do Personality Traits Predict Training Performance?

Abstract. Most test batteries for the selection of ab initio air traffic controllers (ATCOs) include a personality questionnaire. However, the literature suggests that the relationship between personality and job performance is not straightforward: Correlations are found only for a limited set of personality factors, depend on job characteristics, and are at best small to moderate in size. Moreover, the relationship between personality scores and performance can be curvilinear rather than linear. This research investigates whether a personality test used for the selection of ATCOs predicts subsequent training performance. Training performance was measured in terms of instructor ratings ( N = 262) and training outcome ( N = 431). The rated training performance correlated positively with general self-efficacy and team orientation, and negatively with negativity and vulnerability. For some traits (i.e., general self-efficacy and rebelliousness), curvilinear relationships were found. A comparison of trait scores between students who had successfully completed and those who had failed the training showed significant differences for 10 of the 17 measured traits, all of which were in the expected direction.

A new training method for leg explosive power in taekwondo and its data-driven predictive models

BACKGROUND: Kicking is the major way to score in a Taekwondo competition, which makes athletes’ leg power a key quality. However, the characteristics of leg power are very complex and it is difficult to generate physical models to predict training performance. OBJECTIVE: To study training programmes of leg power for Taekwondo using data-driven techniques in correlation analyses and modelling. METHODS: An 8-week program for back squat training was performed using two devices, a Cormax training system and a conventional barbell. Data analysis was conducted to identify the factors affecting the explosive power training. Finally, a data-driven modelling paradigm employing fuzzy rule-based systems was developed to predict the training performance. RESULTS: The Cormax system performed better in improving athletes’ maximum power and velocity. Maximum leg power was best correlated with athletes’ height. The developed predictive models showed good accuracy despite possession of limited training data. CONCLUSIONS: This study demonstrated some new training devices which could greatly improve power training. Moreover, a state-of-the-art modelling strategy was able to construct accurate models for training and exercise performance. The predictive models will likely enhance the anticipation of training outcome in advance which may assist in formulating and improving the training programmes.

Predicting Attrition and Performance in Navy and Marine Corps Air Traffic Control School

Attrition rates due to poor academic performance are particularly high for the Navy’s air traffic control training program. The school relies on scores from the Armed Services Vocational Aptitude Battery (ASVAB) for its selection criteria which do not seem to be sufficient by themselves in predicting who will succeed in training. We hypothesized that including cognitive assessments designed to assess skills specific to air traffic control would improve the ability to predict training performance. The model results showed that cognitive abilities were related to training performance beyond ASVAB scores, however, they were more successful at classifying academic success ( M = .92) than academic attrition ( M = .23). Importantly, the results highlight the importance of additional screening materials that can be used to help reduce attrition and the cost of training

More than g: Evidence for the Incremental Validity of Performance‐Based Assessments for Predicting Training Performance

Past research has consistently shown that tests measuring specific cognitive abilities provide little if any incremental validity over tests of general mental ability when predicting performance on the job. In this study, we suggest that the seeming lack of incremental validity may have been due to the type of content that has traditionally been assessed. Therefore, we hypothesised that incremental validity can be obtained using specific cognitive abilities that are less highly correlated with g and are matched to the tasks performed on the job. To test this, we examined a recently developed performance‐based measure that assesses a number of cognitive abilities related to training performance. In a sample of 310 US Navy student pilots, results indicated that performance‐based scores added sizeable incremental validity to a measure of g. The significant increases in R2 ranged from .08 to .10 across criteria. Similar results were obtained after correcting correlations for range restriction, though the magnitude of incremental validity was slightly smaller (ΔR2 ranged from .05 to .07).

Do Individual Differences Predict Change in Cognitive Training Performance? A Latent Growth Curve Modeling Approach

Cognitive training interventions have become increasingly popular as a potential means to cost-efficiently stabilize or enhance cognitive functioning across the lifespan. Large training improvements have been consistently reported on the group level, with, however, large differences on the individual level. Identifying the factors contributing to these individual differences could allow for developing individually tailored interventions to boost training gains. In this study, we therefore examined a range of individual differences variables that had been discussed in the literature to potentially predict training performance. To estimate and predict individual differences in the training trajectories, we applied Latent Growth Curve models to existing data from three working memory training interventions with younger and older adults. However, we found that individual differences in demographic variables, real-world cognition, motivation, cognition-related beliefs, personality, leisure activities, and computer literacy and training experience were largely unrelated to change in training performance. Solely baseline cognitive performance was substantially related to change in training performance and particularly so in young adults, with individuals with higher baseline performance showing the largest gains. Thus, our results conform to magnification accounts of cognitive change.

Prediction of expected performance for a genetic programming classifier

The estimation of problem difficulty is an open issue in genetic programming (GP). The goal of this work is to generate models that predict the expected performance of a GP-based classifier when it is applied to an unseen task. Classification problems are described using domain-specific features, some of which are proposed in this work, and these features are given as input to the predictive models. These models are referred to as predictors of expected performance. We extend this approach by using an ensemble of specialized predictors (SPEP), dividing classification problems into groups and choosing the corresponding SPEP. The proposed predictors are trained using 2D synthetic classification problems with balanced datasets. The models are then used to predict the performance of the GP classifier on unseen real-world datasets that are multidimensional and imbalanced. This work is the first to provide a performance prediction of a GP system on test data, while previous works focused on predicting training performance. Accurate predictive models are generated by posing a symbolic regression task and solving it with GP. These results are achieved by using highly descriptive features and including a dimensionality reduction stage that simplifies the learning and testing process. The proposed approach could be extended to other classification algorithms and used as the basis of an expert system for algorithm selection.

Validity of the Medical College Admission Test for predicting MD-PhD student outcomes.

The Medical College Admission Test (MCAT) is a quantitative metric used by MD and MD–PhD programs to evaluate applicants for admission. This study assessed the validity of the MCAT in predicting training performance measures and career outcomes for MD–PhD students at a single institution. The study population consisted of 153 graduates of the Vanderbilt Medical Scientist Training Program (combined MD–PhD program) who matriculated between 1963 and 2003 and completed dual-degree training. This population was divided into three cohorts corresponding to the version of the MCAT taken at the time of application. Multivariable regression (logistic for binary outcomes and linear for continuous outcomes) was used to analyze factors associated with outcome measures. The MCAT score and undergraduate GPA (uGPA) were treated as independent variables; medical and graduate school grades, time-to-PhD defense, USMLE scores, publication number, and career outcome were dependent variables. For cohort 1 (1963–1977), MCAT score was not associated with any assessed outcome, although uGPA was associated with medical school preclinical GPA and graduate school GPA (gsGPA). For cohort 2 (1978–1991), MCAT score was associated with USMLE Step II score and inversely correlated with publication number, and uGPA was associated with preclinical GPA (mspGPA) and clinical GPA (mscGPA). For cohort 3 (1992–2003), the MCAT score was associated with mscGPA, and uGPA was associated with gsGPA. Overall, MCAT score and uGPA were inconsistent or weak predictors of training metrics and career outcomes for this population of MD–PhD students.

Something more than g: Meaningful Memory uniquely predicts training performance

Past research has shown that mental ability tests predict training and job performance by virtue of measuring general mental ability (g). After removing the effects of g, mental ability tests no longer predict performance, with the exception of perceptual speed and spatial abilities for certain occupations. In our study, we found evidence for a third exception: Meaningful Memory, the ability to learn and recall information that has meaningfully-related content predicts training performance (ρ=.511), even after removing the effects of g. Therefore, Meaningful Memory tests show promise in enhancing the effectiveness of mental ability tests to predict training performance.

Consistency of the Relations of Cognitive Ability and Personality Traits to Pilot Training Performance

The predictive validity of cognitive ability and personality traits was examined in large samples of U.S. Air Force pilot trainees. Criterion data were collected between 1995 and 2008 from 4 training bases across 3 training tracks. Analyses also examined consistency in pilot aptitude and training outcomes. Results were consistent with previous research indicating cognitive ability is the best predictor of pilot training performance. There were few differences across training tracks, bases, and years, and none was large. Overall, results illustrated the consistency of the quality of pilot trainees as assessed by cognitive ability and personality trait measures, and the consistency of these measures in predicting training performance over time. This consistency results in a more stable training system, enabling greater efficiency and effectiveness.

Which Aspects of a Semistructured Interview, Besides Cognitive Ability Tests, Contribute Incrementally to Predicting the Training Success of Air Traffic Controller Trainees?

This study examined whether a semistructured interview is valid in predicting training performance in theoretical and simulation examinations and overall training success; which aspects of the interview show incremental validity over cognitive ability tests; and whether additional variables enhance the prediction from the interview. The sample involved data from 337 trainees of the German Air Navigation Service Provider selected by the German Aerospace Center and 657 (interview) and 6,885 (cognitive ability) candidates being the corresponding reference samples. Findings indicate incremental validity of the interview over the cognitive ability tests. Additional variables contributed only to the prediction of theoretical examination performance. Attention, general motivation, and social competence predicted the criteria best, particularly the theoretical examination performance and overall training success.

Multidimensional Training System Evaluation using the Revised Bloom's Taxonomy

Many organizations employ training systems to reduce work-related low back disorders. However, the evaluation of training programs is often not satisfactory, mainly due to the complexity of training systems, the high costs involved, heavy reliance on trainee reactions, and little integration of outcomes-based validation. In this work, we used the Revised Bloom's Taxonomy to classify the learning objectives of a training program. These classified objectives were then examined to predict training performance, and were further compared with trainee reactions using multiple regression and correlation analyses accordingly. Results indicated that the classified learning objectives were better predictors of training performance than trainee reactions. Practical implications of the results are discussed.

Stress Appraisals and Training Performance on a Complex Laboratory Task

We investigated whether training performance on a complex laboratory task differs for trainees whose stress appraisals denote challenge or threat. Past research outside a training context found better performance on tasks when stress appraisals denoted challenge (in which perceived situational demands are commensurate with perceived resources) as opposed to threat (in which perceived coping resources fall short of the demands of the stressor). College students performed Space Fortress during 80 3-min practice trials, 20 tests, and posttests (retention, transfer, secondary task interference). Stress appraisals were measured with a two-item scale (Experiments 1-3) or an eight-item scale (Experiment 3) with (Study 1) or without (Experiments 2 and 3) brief hands-on experience. In all experiments, training improved performance and challenged trainees outperformed threatened trainees throughout training as well as on some baseline and posttraining tests. These studies are the first to document that stress appraisals predict training performance. They suggest that little information about a task is needed for appraisals to account for a significant amount of variance (11%) in training performance. Investigating the dynamic interplay of stress appraisals and training will increase the understanding of stress appraisals and of training. Stress appraisals may improve training if used as a screening tool and/or by implementing interventions aimed at changing appraisals from threat to challenge.

Examination of the predictive validity of Big Five personality dimensions across training performance criteria

This study examined relationships among Big Five personality dimensions, cognitive ability, and training performance in a sample of 370 Marines attending Marine Corps’ recruiter training. Several Big Five dimensions (Extraversion, Openness to Experience, Conscientiousness) significantly predicted performance on simulation-based training criteria but did not predict training performance on paper and pencil exams. The results indicated the importance of assessing personality against multiple criteria to determine the utility of personality in predicting training performance. Given that the recruiting of Marines has become particularly challenging, understanding and assessing characteristics that increase the likelihood of successful learning in the training environment is becoming more critical.

Using Measures Of Specific Abilities To Predict Training Performance in Canadian Forces Operator Occupations

This study examined the predictive validity of information processing measures—Speed of Closure, Flexibility of Closure, Perceptual Speed, and Auditory Attention—on training performance. The incremental validity of these validity measures beyond general cognitive ability was examined in a military sample of Canadian Forces personnel in the Operator Family. Trainees engaged in Qualification Level 3 training (n = 122) completed 5 information processing ability measures, the Canadian Forces General Classification Test Form 3 Revised (n = 72), and the Canadian Forces Aptitude Test (n = 98). The criterion was a global percentage grade taken from course evaluation reports. Information processing improved the predictive validity of selection against training performance when used with measures of general cognitive ability (ΔR2 = .06). The only specific ability measure that significantly predicted training performance was the X-A2 battery, a measure of auditory attention (β = .21, p < .05). The information processing measures did not reach significance but still contributed to the predictive equation. These results are comparable with past research where specific abilities provided small yet statistically significant increases in incremental validity beyond g.

U.S. Air Force pilot selection and training methods.

U.S. Air Force (USAF) pilot selection and training procedures have changed substantially in the last several years. Pilot selection procedures were augmented with the introduction of computer-based testing to assess pilot aptitude. Training procedures have changed in an effort to modernize the training fleet and provide better, more specialized training earlier in the training process. Despite several studies showing the utility of USAF pilot aptitude tests for predicting training performance, results of a recent policy capturing study suggest that this information is often ignored by pilot candidate selection boards. The two largest sources of USAF pilot trainees relied heavily on measures of officership when making selection decisions. Cumulative research findings suggest that USAF pilot selection decisions could be improved by making better use of currently available personnel attribute data. Further improvements could be expected from the addition of a structured selection interview and measures of personality.

Pilot selection batteries: shortcomings and perspectives.

This article presents a critical examination of pilot selection batteries. The first part of the article focuses on two problems. First, the vast majority of pilot selection batteries predict training performance rather than operational performance; second, the batteries have low correlations between the predictors and the criterion. The second part of the article examines why these two problems occur. Last, a number of suggestions for improving the predictive validity of the selection batteries are offered.

Beyond cognitive ability: Improving the prediction of performance

New measures of practical intelligence and personality traits were assessed for their ability to improve the prediction of performance beyond cognitive ability. Measures of practical intelligence from the newly developed Constructive Thinking Inventory and traits measuring emotional stability and expedience were used as independent variables in regression analyses predicting academic performance, training performance and leadership ratings of 86 student leaders. The constructive thinking scales added significantly to the prediction of academic performance, beyond measures of cognitive ability. Constructive thinking also accounted for unique variance predicting training performance and leadership ratings. Personality traits were unrelated to performance or leadership.