Advancing Performance Measurement in Cognitive Engineering: The Abstraction Hierarchy as a Framework for Dynamical Systems Analysis

Abstract

This paper describes a novel approach to performance measurement that is based on the abstraction hierarchy (AH) and dynamical systems theory (DST). Each level in the AH provides a systematic way of identifying a state space that can be used to conduct complementary DST analyses. This approach was applied to data from a longitudinal experiment that measured subjects' performance in interacting with a process control microworld simulation on a quasi-daily basis over a period of six months. The variability in trajectories at each level of the AH was examined over successive blocks of trials. The analyses at different levels revealed complementary insights into subjects' behavior. Collectively, the results provided objective, quantitative evidence that highly experienced and very proficient subjects were actually performing the task using very different strategies. Thus, integration of the AH and DST provides a novel measurement approach that can reveal unique and important insights into performance.