Predicting decision accuracy and certainty in complex brain-machine interactions

Abstract

A promising application of brain machine interfaces (BMIs) is predicting user cognitive state, particularly in complex and demanding scenarios, so that automation can dynamically and adaptively adjust task parameters to optimize joint human-machine performance. In this paper we analyze neural, physiological and behavioral data recorded during a complex two-person “crew station” task and investigate whether these measures provide information for inferring user decision state. Specifically, we investigate how measures of EEG, pupil dilation, heart rate and response time, can be fused to infer decision confidence and accuracy in two side-tasks occurring throughout a three hour experimental session. One side-task is an auditory task, the other a visual task, both occurring within the context of the crew station scenario (auditory alert and a visual satellite map N-back task). We find that the best prediction performance always fuses EEG and pupil dilation measures, with results yielding between 70%–75% accuracy with respect to whether the subject(s) will skip making the decision (i.e. have high uncertainty) or whether he/she makes an error. Interestingly, the results suggest a possible mechanistic explanation for the utility of the fused measures, specifically the interaction between the locus coeruleus (LC), whose activity is linked to arousal state and can be inferred from pupil dilation, and the anterior cingulate (ACC), which has been linked to decision formation and monitoring and whose activity is typically measured via EEG. In general, our results demonstrate the potential in using fused neuro/physio measures to infer and track human operator decision uncertainty during demanding complex tasks, possibly enabling BMIs to eventually be employed as “cognitive orthotics” for improving man-machine interaction and performance.