Using Cognitive Work Analysis to Develop a Capability for the Evaluation of Future Systems

Abstract

Evaluating complex human-machine systems is a multidimensional problem that requires the analyst to specify the characteristics of the system to be evaluated and to specify the method of evaluation to be used in determining its effectiveness. Whereas current approaches using task analysis techniques are used widely for evolutionary systems, other approaches, such as those from within the Cognitive Work Analysis (CWA) framework, may be better suited to revolutionary systems. This paper presents ongoing work directed at developing a human-in-the-loop simulation capability for revolutionary, or novel, systems that aims to evaluate the impact of a subsystem's performance (e.g. radar detection range) on the overall system's purpose (e.g. the safe transportation of troops). Two aspects of the development of this capability are presented: 1) the use of Work Domain Analysis and Control Task Analysis to represent systems for the purpose of providing measures of performance for human-in-the-loop experiments performed in support of system evaluation and 2) criteria for judging the utility of a CWA approach for system evaluation compared to other more traditional techniques.