Optimized Motor Imagery Paradigm via Multimodal Stimulation and Explainable LSTM Model in fNIRS-based BCI

Abstract

Motor imagery-based brain–computer interface (MI-BCI) has been applied in the field of motor function rehabilitation for the improvement of patients’ motor function. Functional near-infrared spectroscopy (fNIRS) can be used to study the working mechanism of the brain by monitoring hemodynamic responses related to the activation of cortical neurons. Hence, a novel MI-fNIRS-BCI system with multimodal stimulation is proposed. The multimodal stimulation paradigms include “visual-auditory stimulation,” “electrical stimulation + proprioceptive stimulation,” and “visual-auditory stimulation + electrical stimulation + proprioceptive stimulation.” We explored the optimization of the combination of various stimulations to enhance motor imagery patterns. Furthermore, an explainable long short-term memory (e-LSTM) model was designed to decode cortical activation. The motor imagery task was classified using an LSTM network and the relationship between the classification results and cortical activation was analyzed using an explanation module. Comparative experiments were conducted with eight healthy subjects. The results demonstrated that classification accuracies were significantly improved in the multimodal stimulation paradigms. The proposed MI-fNIRS-BCI system can improve the motor imagery patterns and enhance cortical activation during motor imagery.