Measuring Knowledge Organization as a Method for Assessing Learning during Training

Abstract

This paper applies the conceptual work of K. Kraiger, J. K. Ford, and E. Salas (1993) to the evaluation of two training programs. A method known as structural assessment (SA) was described and adapted for use in the evaluation of a training program for computer programming and a PC-based simulation of a naval decision-making task. SA represents and evaluates pairwise judgments of relatedness of concepts drawn from the training content domain. In the first study, SA scores of students (determined by similarity to an expert solution) were significantly higher after training than before but did not predict performance on a take-home exam 12 weeks later. In the second study, we manipulated training content by providing half the students with the goals and objectives of the transfer task (an advance organizer) before training and providing the other half with the same information after training. As hypothesized, SA scores were higher for those receiving the organizers before training; SA scores were also more strongly related to performance on the criterion task for this group. Implications of the results for training evaluation are discussed.