The unilateral upper limb classification from fMRI-weighted EEG signals using convolutional neural network

Abstract

BackgroundUnilateral upper limb multitasking brings essential improvements to stroke rehabilitation and prosthetic control. However, the influence and recognition of multiple tasks are core issues in the Motor Imagery (MI) system. MethodsFirst, we design the unilateral upper limb MI experimental paradigm and acquire asynchronous functional Magnetic Resonance Imaging (fMRI) and Electroencephalogram based on MI (MI-EEG) data. Then, Brain activation areas for each task are statistically analyzed by fMRI data. A novel fMRI-weighted Convolutional Neural Network (CNN) is designed to reassign each channel’s weight based on brain activation areas to improve classification accuracy. Finally, the EEG data is classified by the fMRI-weighted CNN. ResultsThe four classes of MI tasks reflect significant activation in brain motor areas. The average classification accuracy of fMRI-weighted CNN is 47.0%. The visualization results show the similarity between fMRI and EEG activation areas in the same task. ConclusionsThe distinguishability of multiple tasks and the influence on motor areas of the brain are confirmed by fMRI experiments. The classification accuracy of multitasks is improved according to the fMRI-weighted CNN. This paper provides a reference for further research into asynchronous fMRI-EEG modeling and multitask MI classification.