Developing and Validating Cockpit Interventions based on Cognitive Modeling

Abstract

Developing and Validating Cockpit Interventions based on Cognitive Modeling Deborah A. Boehm-Davis (dbdavis@gmu.edu) Robert W. Holt (bholt@gmu.edu) Melanie Diez (mdiez@gmu.edu) Jeffrey T. Hansberger (jhansber@gmu.edu) Department of Psychology, George Mason University 4400 University Dr., Fairfax, VA 22030 USA Aviation accidents are a rare event. However, when they do occur, the cause is attributed to “human error” over 60% of the time (National Transportation Safety Board, 1994). This suggests that the greatest increments in safety can be gained by improving human performance. Indeed, the typical response to an accident investigation is changes to operating procedures that pilots follow in the cockpit. However, in these situations, the changes are made in response to one specific event, which does not allow researchers to pinpoint the more general causes of errors. Further, this approach is not suited to understanding the process of pilot-system interaction that results in the errors. This makes it impossible to know how to design interventions such as training (Boehm-Davis, Holt, Hansberger, & Seamster, 1999), how to redesign instruments, displays, or software, or how to assess the effects of the intervention. In this research project, we took an alternative approach by developing a computational model of the cognitive processes underlying pilot performance while flying a descent in an automated cockpit. The computational model was built from a cognitive task analysis coupled with empirical performance data. The cognitive task analysis of these phases was developed using NGOMSL (Natural Language GOMS, see Kieras, 1997). This information was combined with eye tracking data taken from pilots interacting with a low-fidelity desktop simulator of a 747- 400 aircraft cockpit (Diez et al., 2001) to inform our design decisions about what information pilots are acquiring from the flight deck while working with automated systems. It also formed the basis of a working computational cognitive model, built using the ACT-R cognitive architecture (Anderson & Lebiere, 1998). The computational model was used to fly the same descent that our pilots had flown on the desktop simulator. Observations of the problems encountered by the model in flying the simulator suggested a number of interventions that might mitigate error in the cockpit. Two of these interventions were selected for empirical testing. First, model runs and eye track data both suggested that the pilots/model were often unaware of changes in automation mode that were driven by the software rather than the pilot (i.e., uncommanded mode changes). A potential intervention developed for this problem is a chime that rings in the cockpit to indicate that the flight management system has autonomously changed the flight mode. We believe that this intervention will draw attention to mode changes that can then be diagnosed and understood. Second, when the model was interrupted, it often was unable to remember the goal that it was trying to achieve; thus, the model was unable to continue flying. For this problem, new annunciations have been developed for display in the cockpit to capture the goal the automated flight system is trying to achieve. We believe that this goal- oriented display will provide guidance to the pilot about what the flight management system is doing, which can help pilots reconstruct their interrupted goal. Empirical data collected from commercial pilots using the modified flight management system on the desktop simulator suggests that these interventions will be useful in reducing these specific errors in the cockpit. Further work remains to determine the more general benefits of these interventions. Acknowledgments This research was supported by grant NAG 2-1289 from the NASA, and 99-G-010 from the FAA. References Anderson, J.R. & Lebiere, C. (1998). The Atomic components of thought. Mahwah, NJ: Erlbaum. Boehm-Davis, D. A., Holt, R. W., and Seamster, T. (2001). Airline resource management programs. In E. Salas, C. A. Bowers, and E. Edens (Eds.), Improving Teamwork in Organizations: Applications of Resource Management Training, NJ: Lawrence Erlbaum Associates. Diez, M., Boehm-Davis, D. A., Holt, R. W., Pinney, M. E., Hansberger, J. T., Schoppek, W. (2001, March 5-8). Tracking pilot interactions with flight management systems through eye movements. Paper presented at the 11th International Symposium on Aviation Psychology, Columbus, Ohio. Kieras, D.E. (1997). A guide to GOMS model usability evaluation using NGOMSL. In M. Helander, T. K. Landauer & P. Prabhu (Eds.), The handbook of human- computer interaction. (Second Edition). Amsterdam: North-Holland, 733-766. NTSB (1994). Safety Study: A Review of Flightcrew- Involved, Major Accidents of U.S. Air Carriers, 1978 through (PB94-917001 NTSB.SS-94/01). Washington DC: National Transportation Safety Board.