Integrating Verification and Validation with the Design of Complex Man-Machine Systems

Abstract

Human factors evaluation is a critical process for ensuring the success of man-machine systems. Meister (1987a, b) describes more traditional methods for systems design, development and testing, and system effectiveness testing. However, recent advances in man-machine interface system (M-MIS) technology (i.e., sensors, processors, displays and controls, and the overall control room configuration) and the man-machine systems that they control have created the need for more sophisticated approaches to evaluation. In particular, special attention is needed to address (a) cognitive issues related to how operators develop and maintain awareness of the state of the controlled system and (b) crew coordination issues related to the ways that individuals interact with each other and the rest of the man-machine system to accomplish operational goals. These issues may be difficult and expensive to resolve if evaluation is left until late in the design process and tested using a production prototype or a full-scope, full-scale simulator. This paper proposes an alternative approach in which human performance issues are evaluated earlier in the design process using lower fidelity testbeds (e.g., part-task simulators) in addition to more traditional means. This approach requires a systematic evaluation framework for defining issues and specifying required attributes of testbed fidelity.