Intelligent Multimodal Signal Adaptation System (IMSAS)

Abstract

As cockpits become more complex, pilots must monitor more data in the normal operation of an aircraft including the states of automated systems. Our objective was to demonstrate that pilots can achieve a greater level of situational awareness when key information is presented in an untaxed modality at the proper time. We modeled a portion of a flight and developed predictor values that we could use to calculate whether or not a pilot noticed an alert. We first recorded some baseline measures of these times, and then used MIDA (Multimodal Interface Design Advisor) to suggest optimal modalities for the signals. We then tested a proposed system that would anticipate a violation (i.e., an air speed violation) and present the appropriate signal with some lead-time. IMSAS (Intelligent Multimodal Signal Adaptation System) continuously assesses the cockpit environment, forecasts airspeed state into the future, and, if it recognizes a need to communicate to the pilot situational information, determines the proper modality in which to present a signal. This study showed an improvement in the reaction time to a signal of over 50% over the baseline. Although further research is needed, this study shows that IMSAS has the potential to improve pilots' response time to in-cockpit alerts such as airspeed violations.