Adaptive Neuro-Fuzzy Fusion of Multi-Sensor Data for Monitoring a Pilot's Workload Condition.

Xia Zhang,Zhifan Qiu,Junping Bao,Yanjun Zhang,Youchao Sun

doi:10.3390/s19163629

Abstract

To realize an early warning of unbalanced workload in the aircraft cockpit, it is required to monitor the pilot’s real-time workload condition. For the purpose of building the mapping relationship from physiological and flight data to workload, a multi-source data fusion model is proposed based on a fuzzy neural network, mainly structured using a principal components extraction layer, fuzzification layer, fuzzy rules matching layer, and normalization layer. Aiming at the high coupling characteristic variables contributing to workload, principal component analysis reconstructs the feature data by reducing its dimension. Considering the uncertainty for a single variable to reflect overall workload, a fuzzy membership function and fuzzy control rules are defined to abstract the inference process. An error feedforward algorithm based on gradient descent is utilized for parameter learning. Convergence speed and accuracy can be adjusted by controlling the gradient descent rate and error tolerance threshold. Combined with takeoff and initial climbing tasks of a Boeing 737–800 aircraft, crucial performance indicators—including pitch angle, heading, and airspeed—as well as physiological indicators—including electrocardiogram (ECG), respiration, and eye movements—were featured. The mapping relationship between multi-source data and the comprehensive workload level synthesized using the NASA task load index was established. Experimental results revealed that the predicted workload corresponding to different flight phases and difficulty levels showed clear distinctions, thereby proving the validity of data fusion.

Highlights

The rapid development of the air transportation industry puts forward higher requirements on aircraft performance and safety than ever before
The same holds true for ground controllers of unmanned aerial vehicles (UAV), in which case, workload still cannot be neglected, the Sensors 2019, 19, 3629; doi:10.3390/s19163629
In contrast to the aforementioned methods, the model proposed in this paper aimed to assess a pilot’s real-time workload condition quantitatively by fusing multi-source physiological and flight data based on a fuzzy neural network

Summary

Introduction

The rapid development of the air transportation industry puts forward higher requirements on aircraft performance and safety than ever before. By means of structural optimization, system integration, and redundant design, airborne systems are being highly complicated and coupled. The cockpit is the only interface between pilot and aircraft, integrating all the human–machine interaction equipment required to perform flight tasks. Novel features constantly arise during pilot–aircraft interaction, including large-scale information, simplex interactive mode, multiple interactive nodes, and high real-time requirements. It is highly possible for pilots to be caught in an unbalanced workload condition. Future advanced flight support systems are expected to monitor and evaluate the pilot’s real-time workload with multi-source data, which could obtain a timely and effective warning of overload status. The same holds true for ground controllers of unmanned aerial vehicles (UAV), in which case, workload still cannot be neglected, the Sensors 2019, 19, 3629; doi:10.3390/s19163629 www.mdpi.com/journal/sensors

Objectives

Findings

Discussion

Conclusion