Assessment of Structural Knowledge as a Training Outcome in Process Control Environments

Abstract

The objective of the present studies was to apply a novel method for structural knowledge assessment to process control to assess the potential of its measures as a training outcome. Traditionally, knowledge is assessed by verbal achievement tests on the subject matter. However, traditional methods are regarded as limited in their ability to assess higher-order learning or understanding. Two experiments (Experiment 1, N= 41; Experiment 2, N= 50) were conducted in which participants were given a 4-hr training session on a simulated process control task. At a later testing session, participants worked on the task for 70 min and completed knowledge tests on declarative, procedural, and structural knowledge. Structural knowledge was measured with the computer-based Association Structure Test (AST), which combines an association task and Pathfinder network on the basis of relatedness ratings. In both studies, structural knowledge was significantly related to diagnostic performance, and evidence was found for internal consistency as well as convergent and predictive validity. Findings indicate that structural assessment with the AST shows promise as a training outcome in process control. Potential applications of this research include the improvement of training design, delivery, and evaluation.