A 2D CNN-LSTM hybrid algorithm using time series segments of EEG data for motor imagery classification

Abstract

Motor imagery-based brain–computer interaction (MI-BCI) converts human neural activity into computational information, often used as commands, by recognizing electroencephalogram (EEG) patterns in different MI tasks. Owing to the poor accuracy and stability of classification algorithms, MI systems remain challenging to employ in practice. Therefore, we proposed a two-dimensional convolutional neural network-long short-term memory (2D CNN-LSTM) hybrid algorithm to classify EEG in MI tasks. We converted EEG signals into time series segments, and then extracted the connectivity features between the different EEG channels in each segment through 2D CNN, finally sent the feature vectors to LSTM network for training. The experimental results showed that the 2D CNN-LSTM algorithm had an average accuracy of 98.5% in the validation set, and a 93.3% average accuracy in the test set, and compared with existing algorithm, such as 1D CNN-LSTM algorithm in the test set, the average accuracy of our algorithm was improved by 3.1% and the F1-score by 0.04, while the loss of the verification set decreased by more than 0.1 in multi-person modeling (general model) in the test set, which was the largest decrease among the algorithms being compared.