BandFocusNet: A Lightweight Model for Motor Imagery Classification of a Supernumerary Thumb in Virtual Reality.

Human movement augmentation is a rising field of research. A promising control strategy for augmented effectors involves utilizing electroencephalography through motor imagery (MI) functions. However, performing MI of a supernumerary effector is challenging, to which MI training is one potential solution. In this study, we investigate the validity of a virtual reality (VR) environment as a medium for eliciting MI neural activations for a supernumerary thumb. Specifically, we assess whether it is possible to induce a distinct neural signature for MI of a supernumerary thumb in VR. Twenty participants underwent a two-fold experiment in which they observed movements of natural and supernumerary thumbs, then engaged in MI of the observed movements. Spectral power and event related desynchronization (ERD) analyses at the group level showed that the MI signature associated with the supernumerary thumb was indeed distinct, significantly different from both the baseline and the MI signature associated with the natural thumb, while single-trial classification showed that it is distinguishable with a 78% and 69% classification accuracy, respectively. Furthermore, spectral power and ERD analyses at the group level showed that the MI signatures associated with directional movement of the supernumerary thumb, flexion and extension, were also significantly different, and single-trial classification demonstrated that these movements could be distinguished with 60% accuracy. Fine-tuning the models further increased the respective classification accuracies, indicating the potential presence of personalized features across subjects.

Brain activity is composed of oscillatory and broadband arrhythmic components; however, there is more focus on oscillatory sensorimotor rhythms to study movement, but temporal dynamics of broadband arrhythmic electroencephalography (EEG) remain unexplored. We have previously demonstrated that broadband arrhythmic EEG contains both short- and long-range temporal correlations that change significantly during movement. In this study, we build upon our previous work to gain a deeper understanding of these changes in the long-range temporal correlation (LRTC) in broadband EEG and contrast them with the well-known LRTC in alpha oscillation amplitude typically found in the literature. We investigate and validate changes in LRTCs during five different types of movements and motor imagery tasks using two independent EEG datasets recorded with two different paradigms—our finger tapping dataset with single self-initiated asynchronous finger taps and publicly available EEG dataset containing cued continuous movement and motor imagery of fists and feet. We quantified instantaneous changes in broadband LRTCs by detrended fluctuation analysis on single trial 2 s EEG sliding windows. The broadband LRTC increased significantly (p < 0.05) during all motor tasks as compared to the resting state. In contrast, the alpha oscillation LRTC, which had to be computed on longer stitched EEG segments, decreased significantly (p < 0.05) consistently with the literature. This suggests the complementarity of underlying fast and slow neuronal scale-free dynamics during movement and motor imagery. The single trial broadband LRTC gave high average binary classification accuracy in the range of 70.54±10.03% to 76.07±6.40% for all motor execution and imagery tasks and hence can be used in brain–computer interface (BCI). Thus, we demonstrate generalizability, robustness, and reproducibility of novel motor neural correlate, the single trial broadband LRTC, during different motor execution and imagery tasks in single asynchronous and cued continuous motor-BCI paradigms and its contrasting behavior with LRTC in alpha oscillation amplitude.

Introduction: Motor imagery (MI) and virtual reality (VR) are used in the upper limb amputee’s (ULA) rehabilitation. ULAs using an advanced technical prosthesis are more and more numerous. Our aim was to better understand the motor working of ULAs, wearing prosthesis and using MI and VR. Method: A systematic research using Pubmed, Cochrane, ProQuest, Web of Science, CINAHL, and PSYCHINFO was conducted to analyse the consequences of such therapies on myoelectric and cerebral activities, motor performance and subjective data related to movement. Inclusion criteria for articles were: motor control of ULAs practicing MI or VR was investigated. Exclusion criteria were: only healthy subjects were tested or other therapies except MI and VR were practiced. Results: Initial search provided 2,635 articles, in which 10 have been included in the analysis. ULAs and healthy subjects do not have the same neurological way of working when practicing MI or VR. The number of possible controlled movements and their accuracy are different. Some aspects, such as visual content of VR systems, may have an influence. If such therapies are interesting to be used for some ULAs, they have to be used as a supplement, but not as a substitute for classical therapies. Discussion: Supplemental search such as RCTs have to be conducted to provide strong evidences, that MI or VR are appropriate to be used for ULAs rehabilitation. MI and VR procedures have to be defined more clearly. These techniques must absolutely be tested with ULAs.

With the emergence of brain-computer interface (BCI) technology and virtual reality (VR), how to improve the quality of motor imagery (MI) electroencephalogram (EEG) signal has become a key issue for MI BCI applications under VR. In this paper, we proposed to enhance the quality of MI EEG signal by using haptic stimulation training. We designed a first-person perspective and a third-person perspective scene under VR, and the experimental results showed that the left- and right-hand MI EEG quality of the participants improved significantly compared with that before training, and the mean differentiation of the left- and right-hand MI tasks was improved by 21.8% and 15.7%, respectively. We implemented a BCI application system in VR and developed a game based on MI EEG for control of ball movement, in which the average classification accuracy by the participants after training in the first-person perspective reached 93.5%, which was a significant improvement over existing study.

Introduction: Parkinson’s disease is a neuro-degenerative disorder, characterized by degeneration of dopaminergic neurons. Motor imagery practice uses the imagery of a motor act to learn and Improve outcome. Virtual reality can provide patients with sensory stimulation, immersive environment, feedback during motor task, reflecting motor learning and neuroplasticity. The purpose is to investigate effectiveness of motor imagery and virtual reality on motor functions in individuals with Parkinson’s disease. Methods: 10 articles were included with 287 participants. 9 were RCT and one was case study. A search of literature was conducted by using following keywords: Parkinson’s disease, Virtual reality, motor imagery. A search was undertaken via google scholar, PubMed and research gate. Out of 287 articles, 28 studies were reviewed. 10 studies met the inclusion criteria. UPDRS was the outcome measure used to assess motor functions. These articles were analyzed using PRISMA 2020 checklist and PEDro. Cochrane risk of bias tool was used to check the bias. Three articles studied the combined effect of virtual reality and motor imagery, two motor imagery alone and five virtual realities alone. Result: Combining VR with MI showed statistically significant difference. Motor imagery alone showed significant difference in one article and no difference in the other. Articles about VR alone showed statistically significant difference. Conclusion: Virtual reality and motor imagery are effective rehabilitation tool for improving motor function in individuals with Parkinson’s disease. Implications: Virtual reality and motor imagery can be used in designing more effective and engaging rehabilitation program leading to improved outcomes and quality of life.

Motor imagery (MI) and virtual reality (VR) are two evolving therapeutic approaches that make use of cognitive function to study and enhance movement, in particular, balance and mobility of people with Parkinson's disease (PD). This review examines the literature on the use of VR and MI in the assessment of mobility and as a therapeutic intervention to improve balance and gait in patients with PD. A study was eligible for inclusion if MI or VR were used to assess motor or cognitive function to improve gait, balance, or mobility in patients with PD. Data were extracted on the following categories: participants; study design; intervention (type, duration, and frequency); and outcomes. Intervention studies were evaluated for quality using the Physiotherapy Evidence Database scale. Sixteen studies were identified; 4 articles used MI and 12 used VR for assessment and treatment of gait impairments in PD. The studies included small samples and were diverse in terms of methodology. Quality of the intervention trials varied from fair for VR to good for MI. The benefits of using MI and VR for assessment and treatment were noted. Encouraging findings on the potential benefits of using MI and VR in PD were found, although further good-quality research is still needed. Questions remain on the optimal use, content of interventions, and generalizability of findings across the different stages of the disease. The possible mechanisms underlying MI and VR and recommendations for future research and therapy are also presented.

ObjectiveTo investigate the effects of using motor imagery (MI) in combination with a virtual reality (VR) program on healthy volunteers and stroke patients. In addition, this study investigated whether task variability within the VR-guided MI programs would influence corticomotor excitability.MethodsThe present study included 15 stroke patients and 15 healthy right-handed volunteers who were presented with four different conditions in a random order: rest, MI alone, VR-guided MI, and VR-guided MI with task variability. The corticomotor excitability of each participant was assessed before, during, and after each condition by measuring changes in the various parameters of motor-evoked potentials (MEPs) of the extensor carpi radials (ECR). Changes in intracortical inhibition (ICI) and intracortical facilitation (ICF) were calculated after each condition as percentages of inhibition (%INH) and facilitation (%FAC) at rest.ResultsIn both groups, the increases in MEP amplitudes were greater during the two VR-guided MI conditions than during MI alone. Additionally, the reductions in ECR %INH in both groups were greater under the condition involving VR-guided MI with task variability than under that involving VR-guided MI with regular interval.ConclusionThe corticomotor excitability elicited by MI using a VR avatar representation was greater than that elicited by MI with real body observations. Furthermore, the use of task variability in a VR program may enhance neural regeneration after stroke by reducing ICI. The present findings support the use of various VR programs as well as the concept of combining MI with VR programs for neurorehabilitation.

Motor imagery (MI) is the mental execution of actions without overt movements that depends on the ability to imagine. We explored whether this ability could be related to the cortical activity of the brain areas involved in the MI network. To this goal, brain activity was recorded using high-density electroencephalography (hdEEG) in nineteen healthy adults while visually imagining walking on a straight path. We extracted Event-Related Desynchronizations (ERDs) in the β band, and we measured MI ability via (i) the Kinesthetic and Visual Imagery Questionnaire (KVIQ), (ii) the Vividness of Movement Imagery Questionnaire-2 (VMIQ), and (iii) the Imagery Ability (IA) score. We then used Pearson’s and Spearman’s coefficients to correlate MI ability scores and average ERD power (avgERD). VMIQ was positively correlated with avgERD of frontal and cingulate areas, whereas IA SCORE was positively correlated with avgERD of left inferior frontal and superior temporal regions. Stronger activation of the MI network was related to better scores of MI ability evaluations, supporting the importance of testing MI ability during MI protocols. This result will help to understand MI mechanisms and develop personalized MI treatments for patients with neurological dysfunctions.

Motor imagery (MI) training-based Brain-Computer Interfaces (BCI) improved individuals' motor function by inducing direct alterations in the sensorimotor area. Virtual Reality (VR)-based MI training has been identified as a promising technique for achieving high performance. However, the physical limb position should be considered when designing a better training task. This paper investigated the effect of induced MI activities when the virtual arms were at normal and shifted down position. The paradigm used of Virtual Reality (VR) to simulate the situation of having realistic and unrealistic arms position in an immersive virtual environment. Analyses of electroencephalograms (EEGs) revealed significant differences in MI activity levels between two positions on both the left and right sides. During shifted arms MI, the negative power regions were found in beta and gamma bands on the contralateral hemisphere in time-frequency analysis. Resting vs. normal position arms MI and resting vs. shifted position arms MI classification accuracy reached 80 % and 63 %, respectively. Overall, these findings suggested that taking into account the realistic physical position of virtual limbs is critical for optimizing MI training performance.

Motor imagery (MI) is widely employed in stroke rehabilitation due to the event-related desynchronization (ERD) phenomenon in sensorimotor cortex induced by MI is similar to actual movement. However, the traditional BCI paradigm, in which the patient imagines the movement of affected hand (AH-MI) with a weak ERD caused by the damaged brain regions, retards motor relearning process. In this work, we applied a novel MI paradigm based on the "sixth-finger" (SF-MI) in stroke patients and systematically uncovered the ERD pattern enhancement of novel MI paradigm compared to traditional MI paradigm. Twenty stroke patients were recruited for this experiment. Event-related spectral perturbation was adopted to supply details about ERD. Brain activation region, intensity and functional connectivity were compared between SF-MI and AH-MI to reveal the ERD enhancement performance of novel MI paradigm. A "wider range, stronger intensity, greater connection" ERD activation pattern was induced in stroke patients by novel SF-MI paradigm compared to traditional AH-MI paradigm. The bilateral sensorimotor and prefrontal modulation was found in SF-MI, which was different in AH-MI only weak sensorimotor modulation was exhibited. The ERD enhancement is mainly concentrated in mu rhythm. More synchronized and intimate neural activity between different brain regions was found during SF-MI tasks compared to AH-MI tasks. Classification results (>80% in SF-MI vs. REST) also indicated the feasibility of applying novel MI paradigm to clinical stroke rehabilitation. This work provides a novel MI paradigm and demonstrates its neural activation-enhancing performance, helping to develop more effective MI-based BCI system for stroke rehabilitation.

Virtual reality (VR)-guided motor imagery (MI) is a widely used approach for motor rehabilitation, especially for patients with severe motor impairments. Most approaches provide visual guidance from the first-person perspective (1PP). MI training with visual guidance from the third-person perspective (3PP) remains largely unexplored. We argue that 3PP MI training has its own advantages and can supplement 1PP MI. For some movements beyond the view of 1PP, such as shoulder shrugging and other axial movements, MI are suitable performed under 3PP. However, the efficiency of existing paradigms for 3PP MI is unsatisfactory. We speculate that the absence of sense of body ownership (SOO) from 3PP could be one possible factor and hypothesize that 3PP MI could be enhanced by eliciting SOO over a 3PP avatar. Based on our hypothesis, a novel paradigm was proposed to enhance 3PP MI by inducing full-body illusion (FBI) from 3PP, which is similar to the so-called out-of-body experience (OBE), using synchronous visuo-tactile stimulus with VR. The event-related Electroencephalograph (EEG) desynchronization (ERD) at motor-related regions from 31 healthy participants were calculated and compared with a control paradigm without "OBE" FBI induction. This study attempts to enhance 3PP MI with FBI induction. It offers an opportunity to perform MI guided by action observation from 3PP with elicited SOO to the observed avatar. We believe that 3PP MI could provide more possibilities for effective rehabilitation training, when SOO could be elicited to a virtual avatar and the present work demonstrates its viability and effectiveness.

BackgroundMotor impairment is a common consequence of stroke causing difficulty in independent movement. The first month of post-stroke rehabilitation is the most effective period for recovery. Movement imagination, known as motor imagery, in combination with virtual reality may provide a way for stroke patients with severe motor disabilities to begin rehabilitation.MethodsThe aim of this study is to verify whether motor imagery and virtual reality help to activate stroke patients’ motor cortex. 16 acute/subacute (< 6 months) stroke patients participated in this study. All participants performed motor imagery of basketball shooting which involved the following tasks: listening to audio instruction only, watching a basketball shooting animation in 3D with audio, and also performing motor imagery afterwards. Electroencephalogram (EEG) was recorded for analysis of motor-related features of the brain such as power spectral analysis in the alpha and beta frequency bands and spectral entropy. 18 EEG channels over the motor cortex were used for all stroke patients.ResultsAll results are normalised relative to all tasks for each participant. The power spectral densities peak near the alpha band for all participants and also the beta band for some participants. Tasks with instructions during motor imagery generally show greater power spectral peaks. The p-values of the Wilcoxon signed-rank test for band power comparison from the 18 EEG channels between different pairs of tasks show a 0.01 significance of rejecting the band powers being the same for most tasks done by stroke subjects. The motor cortex of most stroke patients is more active when virtual reality is involved during motor imagery as indicated by their respective scalp maps of band power and spectral entropy.ConclusionThe resulting activation of stroke patient’s motor cortices in this study reveals evidence that it is induced by imagination of movement and virtual reality supports motor imagery. The framework of the current study also provides an efficient way to investigate motor imagery and virtual reality during post-stroke rehabilitation.

Adaptive transfer learning-based multiscale feature fused deep convolutional neural network for EEG MI multiclassification in brain–computer interface

The motor imagery (MI)-based brain computer interface (BCI) is an intuitive interface that enables users to communicate with external environments through their minds. However, current MI-BCI systems ask naïve subjects to perform unfamiliar MI tasks with simple textual instruction or a visual/auditory cue. The unclear instruction for MI execution not only results in large inter-subject variability in the measured EEG patterns but also causes the difficulty of grouping cross-subject data for big-data training. In this study, we designed an BCI training method in a virtual reality (VR) environment. Subjects wore a head-mounted device (HMD) and executed action observation (AO) concurrently with MI (i.e., AO + MI) in VR environments. EEG signals recorded in AO + MI task were used to train an initial model, and the initial model was continually improved by the provision of EEG data in the following BCI training sessions. We recruited five healthy subjects, and each subject was requested to participate in three kinds of tasks, including an AO + MI task, an MI task, and the task of MI with visual feedback (MI-FB) three times. This study adopted a transformer- based spatial-temporal network (TSTN) to decode the user’s MI intentions. In contrast to other convolutional neural network (CNN) or recurrent neural network (RNN) approaches, the TSTN extracts spatial and temporal features, and applies attention mechanisms along spatial and temporal dimensions to perceive the global dependencies. The mean detection accuracies of TSTN were 0.63, 0.68, 0.75, and 0.77 in the MI, first MI-FB, second MI-FB, and third MI-FB sessions, respectively. This study demonstrated the AO + MI gave an easier way for subjects to conform their imagery actions, and the BCI performance was improved with the continual learning of the MI-FB training process.

Attention-based CNN model for motor imagery classification from nonlinear EEG signals