Auditory attention decoding from electroencephalography based on long short-term memory networks

Abstract

Available researches have confirmed that the target speech stream of human listener's selective attention in a multi-speaker scenario can be decoded from ongoing electroencephalography (EEG) signals. Due to its great potential application for the neuro-steered hearing devices, auditory attention decoding has recently received increasing attention. In practice, development of high-accuracy decoding performance and use of EEG signals with as short duration as possible are urgently needed for auditory attention decoding in real-time scenarios. However, these are still great challenges for the existing decoding methods to achieve such performances. In this paper, we propose a nonlinear modeling approach between EEG signals and speech envelopes by exploiting long short-term memory networks (LSTM), a special structure of recurrent neural networks (RNN), to decipher selective auditory attention from single-trial EEG signals, which is capable of learning long-term dependencies between EEG responses and continuous auditory stimuli. Unlike the existing linear modeling approaches, the LSTM RNN-based auditory attention decoding method is an end-to-end nonlinear decoding framework that subsumes the explicit similarity determination as part of feature learning in deep neural networks. The proposed decoding method is evaluated using the real EEG data collected from 21 subjects. Experimental results show that our proposed decoding method can achieve the state-of-the-art performance to identify the target speech in a competing two-talker scenario. When used the EEG signals with 1 s duration, the average accuracy for the 21 subjects' attention decoding is reached to 96.12%. The results show great potentials for the real-time auditory attention detection in neuro-steered hearing aids.