Unified Vision-series of military exercise - the methodological struggles in conducting evaluation of Human Factors and Systems Interaction

Abstract

The complexity and practical constraints of sociotechnical systems demand that researchers develop better understanding and methods for testing and providing input on critical Human Factor (HF) and Systems Interaction problems. This article takes a concrete approach to this problem describing the NATO HFM-276 Task Group methodological struggles in conducting evaluation of a military exercise. To analyze the duality; theory and practice, in the area an overall Socio-Technical Multiteam (STM) model was used to identify critical Human Factors (HF) and Systems Interaction in order to survey the gap between simulation and best practice. The Systems Interaction and the methodological struggles in conducting the evaluation of HF and Systems Interaction One dilemma that occurs in these events is the one between being a training arena for military units and operational versus being an arena for operational testing of systems and new capabilities, may lead to some challenges for HF methods. This speaks to the aspect of multiplicity and uncertainty. The behavior of the distributed cooperative systems and operators during the preparation phase and execution are made up of multiple, highly interconnected individuals who influence one another formally and informally. A particular aspect of the UV exercises is the ongoing modifications of systems (e.g., Command and control systems and information systems) and their use during the trial itself. Operators and Supervisors are to analyze data seeking to identify targets across Intelligence, Surveillance, and Reconnaissance (ISR) operations. Evaluating these operations included Human Factors and Systems Interaction (pre- and post-exercise) surveys, observational, archival, and document study. We discuss the utility of these methods using recent work in the HF and ergonomics literature on how to match HF methods to core aspects of the complexity of sociotechnical systems. We also discuss how the methods used could capture complexity, and how to improve HF evaluations of training and test practices in the military and other field settings taking into account some aspects of the complexity of ISR operations. The example of mapping organizational structure using a problem space, as well as experimental designs could be avenues for accommodating complexity in future HF evaluations of military exercises.