Deep-Learning-Based BCI for Automatic Imagined Speech Recognition Using SPWVD

Abstract

The electroencephalogram (EEG)-based brain–computer interface (BCI) has potential applications in neuroscience and rehabilitation. It benefits a person with neurological impairment to communicate their thoughts to the external world without using any appendages. Decoding imagined speech had limited success, mainly because neural signals are weak and more variable than overt speech, hence challenging to decode by machine-learning (ML)-based algorithms. In recent years, deep learning (DL) with convolutional neural networks (CNNs) has transformed computer vision and can perform pattern recognition better than the traditional ML-based algorithms. The objective of this article is to design a smoothed pseudo-Wigner–Ville distribution (SPWVD) and CNN-based automatic imagined speech recognition (AISR) system to recognize imagined words. This article uses a publically available 64-channel EEG dataset, collected from 15 healthy subjects for three categories: long words, short words, and vowels. The EEG signals were transformed into time–frequency representation (TFR) using SPWVD, which are used as an input to CNN such that the EEG dataset was identified and classified into binary and multiclass categories. In addition, the CNN model was optimized using a recently developed Keras-tuner library to achieve optimal performance. The performance of the SPWVD-CNN-driven AISR system is evaluated using seven performance evaluation metrics: accuracy (ACC), recall (REC), precision (PREC), Mathew’s correlation coefficient (MCC), Cohen’s kappa ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\kappa $ </tex-math></inline-formula> ), F1-score, and area under the curve (AUC). It is found that the proposed system achieved the maximum classification ACCs of 94.82%, 94.26%, 94.68%, and 84.50% for long words, short–long words, short words, and vowels, respectively. The proposed AISR strengthens the possibility of using imagined speech recognition as a future BCI application.