Auditory attention tracking states in a cocktail party environment can be decoded by deep convolutional neural networks

Abstract

Objective. A deep convolutional neural network (CNN) is a method for deep learning (DL). It has a powerful ability to automatically extract features and is widely used in classification tasks with scalp electroencephalogram (EEG) signals. However, the small number of samples and low signal-to-noise ratio involved in scalp EEG with low spatial resolution constitute a limitation that might restrict potential brain-computer interface (BCI) applications that are based on the CNN model. In the present study, a novel CNN model with source-spatial feature images (SSFIs) as the input is proposed to decode auditory attention tracking states in a cocktail party environment. Approach. We first extract SSFIs using rhythm entropy and weighted minimum norm estimation. Next, we develop a CNN model with three convolutional layers. Furthermore, we estimate the performance of the proposed model via generalized performance, alternative models that deleted or replaced a model’s component, and loss curves. Finally, we use a deep transfer model with fine-tuning for a low (poor) behavioral performance group (L-group). Main results. Based on cortical activity reconstructions from the scalp EEGs, the classification accuracy (CA) of the proposed model is 80.4% (chance level: 52.5%), which is superior to that achieved by scalp EEG. Additionally, the performance of the proposed model is more stable when compared to alternative models that delete or replace specific model components. The proposed model identifies the difference between two auditory attention tracking states (successful versus unsuccessful) at an early stage with a short time window (250 ms after target offset). Furthermore, we propose a deep transfer learning model to improve the classification for the L-group. With this model, the CA of the L-group significantly increase by 5.3%. Significance. Our proposed model improves the performance of a decoder for auditory attention tracking, which could be suitable for relieving the difficulty with the attentional modulation of individual’s neural responses. It provides a novel communication channel with auditory cognitive BCI for patients with attention and hearing impairment.