Motion Intention Research Articles

Neural Control for Human-Robot Interaction with Human Motion Intention Estimation

Neural Control for Human-Robot Interaction with Human Motion Intention Estimation

Integrating Motion Intention and Impedance Control in Teleoperated Robotic Rehabilitation for Upper Extremity Disorders

Abstract Due to the global pandemic in 2020, rehabilitation services had to adapt to virtual care to meet the needs of patients with a diversity of upper extremity disorders and injuries, from simple fingertip injuries to replanted extremities, while mitigating the spread of COVID-19. For patients who require occupational therapy in their rehabilitation, teleoperated robotic care can fulfill this need. Moreover, some clinical studies show that incorporating the active intention of patients into rehabilitation training enhances the effectiveness of therapies. This research aims to develop a bilateral impedance controller with a Franka Emika 7 DOF (degree-of-freedom) articulated robot using MATLAB, facilitating upper limb occupational therapy. Through this setup, the Franka robot will lead patients through various therapeutic exercises based on their motion intention, such as a pick-and-place exercise. The exercises’ efficacy is assessed through evaluations of torque, muscle strength, and jerking performance. During patients’ rehabilitation exercises with the Franka Robot, we anticipate observing decreased muscle activity alongside a smooth, assist-as-needed trajectory. This method will allow a patient to perform complex movements essential for daily activities quicker than traditional occupational therapy.

Decoding multi-limb movements from two-photon calcium imaging of neuronal activity using deep learning

Objective.Brain-machine interfaces (BMIs) aim to restore sensorimotor function to individuals suffering from neural injury and disease. A critical step in implementing a BMI is to decode movement intention from recorded neural activity patterns in sensorimotor areas. Optical imaging, including two-photon (2p) calcium imaging, is an attractive approach for recording large-scale neural activity with high spatial resolution using a minimally-invasive technique. However, relating slow two-photon calcium imaging data to fast behaviors is challenging due to the relatively low optical imaging sampling rates. Nevertheless, neural activity recorded with 2p calcium imaging has been used to decode information about stereotyped single-limb movements and to control BMIs. Here, we expand upon prior work by applying deep learning to decode multi-limb movements of running mice from 2p calcium imaging data.Approach.We developed a recurrent encoder-decoder network (LSTM-encdec) in which the output is longer than the input.Main results.LSTM-encdec could accurately decode information about all four limbs (contralateral and ipsilateral front and hind limbs) from calcium imaging data recorded in a single cortical hemisphere.Significance.Our approach provides interpretability measures to validate decoding accuracy and expands the utility of BMIs by establishing the groundwork for control of multiple limbs. Our work contributes to the advancement of neural decoding techniques and the development of next-generation optical BMIs.

Temporal attention fusion network with custom loss function for EEG--fNIRS classification.

Objective.Methods that can detect brain activities accurately are crucial owing to the increasing prevalence of neurological disorders. In this context, a combination of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) offers a powerful approach to understanding normal and pathological brain functions, thereby overcoming the limitations of each modality, such as susceptibility to artefacts of EEG and limited temporal resolution of fNIRS. However, challenges such as class imbalance and inter-class variability within multisubject data hinder their full potential.Approach.To address this issue, we propose a novel temporal attention fusion network (TAFN) with a custom loss function. The TAFN model incorporates attention mechanisms to its long short-term memory and temporal convolutional layers to accurately capture spatial and temporal dependencies in the EEG--fNIRS data. The custom loss function combines class weights and asymmetric loss terms to ensure the precise classification of cognitive and motor intentions, along with addressing class imbalance issues.Main resultsRigorous testing demonstrated the exceptional cross-subject accuracy of the TAFN, exceeding 99% for cognitive tasks and 97% for motor imagery (MI) tasks. Additionally, the ability of the model to detect subtle differences in epilepsy was analyzed using scalp topography in MI tasks.Significance.This study presents a technique that outperforms traditional methods for detecting high-precision brain activity with subtle differences in the associated patterns. This makes it a promising tool for applications such as epilepsy and seizure detection, in which discerning subtle pattern differences is of paramount importance.

Domain Generalization through Latent Distribution Exploration for Motor Imagery EEG Classification

Electroencephalography (EEG)-based Motor Imagery (MI) brain-computer interface (BCI) systems play essential roles in motor function rehabilitation for patients with post-stroke. Existing neural networks for decoding MI EEG face challenges due to nonstationary characteristics and subject-specific variations of EEG data. To address these challenges and improve generalization performance, this study proposes a domain generalization (DG) model that eliminates the need for user-specific calibration in real-life applications. Specifically, the proposed model comprises two branches: the first branch applies several independent decision-making networks to decode and classify subjects’ motor intentions, while the second branch adaptively assigns weights to classification results and fuses them into a comprehensive decision. Both branches utilize EEGNet and ShallowConvNet to extract time-frequency-spatial features. By implementing multiple classification networks, the model can learn a broad range of data distributions from source subjects, which contributes to improved generalization performance on target subjects. The proposed EEG-DG framework was evaluated on BCI Competition IV Dataset 2a, 2b and PhysioNet. Results show that the proposed framework significantly enhances the classification performance of MI EEG, outperforming several state-of-the-art models on all three datasets, underlining its superior efficacy in real-world scenarios and exceptional generalization performance. The source code can be accessed at https://github.com/DrugLover/Multibranch-DG-EEG.

How Can We Reconstruct Mental Imagery from Brain Activities?

Advances in machine learning and artificial intelligence (AI) have enabled the extraction of information such as perception and motor intention from neural activities, which is called neural decoding. A recent study demonstrated that mental imagery can be reconstructed from brain activity measured using functional magnetic resonance imaging. This article describes the method of mental image reconstruction and the underlying technologies that support it.

Measuring instability in chronic human intracortical neural recordings towards stable, long-term brain-computer interfaces.

Intracortical brain-computer interfaces (iBCIs) enable people with tetraplegia to gain intuitive cursor control from movement intentions. To translate to practical use, iBCIs should provide reliable performance for extended periods of time. However, performance begins to degrade as the relationship between kinematic intention and recorded neural activity shifts compared to when the decoder was initially trained. In addition to developing decoders to better handle long-term instability, identifying when to recalibrate will also optimize performance. We propose a method, "MINDFUL", to measure instabilities in neural data for useful long-term iBCI,without needing labels of user intentions. Longitudinal data were analyzed from two BrainGate2 participants with tetraplegia as they used fixed decoders to control a computer cursor spanning 142 days and 28 days, respectively. We demonstrate a measure of instability that correlates with changes in closed-loop cursor performance solely based on the recorded neural activity (Pearson r = 0.93 and 0.72, respectively). This result suggests a strategy to infer online iBCI performance from neural data alone and to determine when recalibration should take place for practical long-term use.

A Rigid–Flexible Supernumerary Robotic Arm/Leg: Design, Modeling, and Control

As humans’ additional arms or legs, supernumerary robotic limbs (SRLs) have gained great application prospects in many fields. However, current SRLs lack both rigidity/flexibility adaptability and arm/leg function conversion. Inspired by the muscular hydrostat characteristics of octopus tentacles, fiber-reinforced actuators (FRAs) were employed to develop SRLs simultaneously realizing flexible operation and stable support. In this paper, an SRL with FRAs was designed and implemented. The analytic model of the FRA was established to formulate the movement trajectory and stiffness profile of the SRL. A hierarchical hidden Markov model (HHMM) was proposed to recognize the wearer’s motion intention and control the SRL to complete the specific working mode and motion type. Experiments were conducted to exhibit the feasibility and superiority of the proposed robot.

Estimation of Lower Limb Joint Angles Using sEMG Signals and RGB-D Camera.

Estimating human joint angles is a crucial task in motion analysis, gesture recognition, and motion intention prediction. This paper presents a novel model-based approach for generating reliable and accurate human joint angle estimation using a dual-branch network. The proposed network leverages combined features derived from encoded sEMG signals and RGB-D image data. To ensure the accuracy and reliability of the estimation algorithm, the proposed network employs a convolutional autoencoder to generate a high-level compression of sEMG features aimed at motion prediction. Considering the variability in the distribution of sEMG signals, the proposed network introduces a vision-based joint regression network to maintain the stability of combined features. Taking into account latency, occlusion, and shading issues with vision data acquisition, the feature fusion network utilizes high-frequency sEMG features as weights for specific features extracted from image data. The proposed method achieves effective human body joint angle estimation for motion analysis and motion intention prediction by mitigating the effects of non-stationary sEMG signals.

Feasibility of submillimeter functional quantitative susceptibility mapping using 3D echo planar imaging at 7 T.

Quantitative susceptibility mapping (QSM) is a tool for mapping tissue susceptibility. Using QSM for functional brain mapping, it is possible to directly quantify blood-oxygen-level-dependent (BOLD) susceptibility changes. This study presents a submillimeter functional QSM (fQSM) approach compared to BOLD fMRI from data acquired with 3D gradient-echo echo planar imaging (EPI) at ultra-high field. Complex EPI data were acquired in nine healthy subjects with varying temporal and spatial resolutions and used for BOLD fMRI and for fQSM. Right-hand finger tapping experiments were performed as well as one measurement with intentional subject movement. Susceptibility maps were computed using 3D path-based unwrapping, the variable-kernel sophisticated harmonic artifact reduction for phase data, and the streaking artifact reduction for QSM algorithm. Functional data analysis included general linear modeling and computation of z-scores. Submillimeter data were denoised using NOise reduction with DIstribution Corrected (NORDIC), which improved z-scores in the motor cortex for fQSM and fMRI. An expected increase in BOLD fMRI signal and corresponding decrease in magnetic susceptibility was observed in sensorimotor areas during active periods. For all experiments, fQSM showed smaller activation regions compared with fMRI. The percentage of high negative t-values localized in the cortex was higher for fQSM (52%) than for positive or negative t-values for fMRI (45%). For the scans with intentional motion, movement exceeded the size of a voxel, but paradigm dependent signal evolution could be recovered using motion correction. In conclusion, this study demonstrates the feasibility of submillimeter whole-brain fQSM with voxel volume of 0.53 μL. In comparison to traditional BOLD fMRI, fQSM provided improved localization of brain activation within the cortex, especially in submillimeter 3D EPI sequences.

Multi-branch deep learning neural network prediction model for the development of angular biosensors based on sEMG.

Human gait motion intention recognition is very important for the lower extremity exoskeleton robot to accurately synchronize and respond to the user's natural motion. And motion intention recognition is generally performed through sEMG. Deep learning neural networks perform well in dealing with high-dimensional data and nonlinear relationships such as sEMG, but different deep learning neural networks have their own advantages in dealing with different types of data. Therefore, a multi-branch deep learning neural network, which enables different neural networks to process different feature items, could achieve more accurate and efficient motion intention recognition. The purpose of this study is to 1) Establish a multi-branch deep learning neural network model to achieve accurate gait recognition and effective estimation of joint angles. 2) Quantify the performance of the multi-branch deep learning neural network model in gait recognition and joint angle prediction using sEMG. This study involved the collection of sEMG and plantar pressure data during walking in human subjects. Firstly, the collected signals are filtered and denoised to ensure the quality and reliability of the data. Calculate the time domain features and the frequency domain features to capture the key information of gait. Then, using the sensitivity difference of different structural neural networks to different feature data, a multi-branch deep learning neural network model is developed, in which the extracted features are used as the input of the model. The output of the model includes gait cycle and joint angle, so as to realize the accurate recognition of human gait and the effective estimation of joint angle. The results show that the proposed method has high accuracy in identifying human gait and estimating joint angles. The multi-branch neural network model successfully integrates time-domain and frequency-domain features and provides reliable prediction of gait cycle and joint angle. The highest accuracy of gait recognition is 95.42%, the lowest is 90.11%, and the average is 92.16%. The average error of joint angle estimation is 3.19. This study designed a human walking gait recognition and joint angle prediction model to achieve accurate human lower limb motion intention recognition.The model can be integrated into the sEMG sensor to design a angular biosensors, which can predict the human joint angle in real time.

Neuro-Musculoskeletal Modeling for Online Estimation of Continuous Wrist Movements from Motor Unit Activities.

Decoding movement intentions from motor unit (MU) activities remains an ongoing challenge, which restricts our comprehension of the intricate transition mechanism from microscopic neural drive to macroscopic movements. This study presents an innovative neuro-musculoskeletal (NMS) model driven by MU activities for online estimation of continuous wrist movements. The proposed model employs a physiological and comprehensive utilization of MU firings and waveforms, thus facilitating the localization of MUs to muscle-tendon units (MTU) as well as the computation of MU-specific neural excitation. Subsequently, the MU-specific neural excitation was integrated to form the MTU-specific neural excitation, which were then inputted into a musculoskeletal model to accomplish the joint angle estimation. To assess the effectiveness of this model, high-density surface electromyography and angular data were collected from the forearms of eight subjects during their performance of wrist flexion-extension task. Two pieces of 8 × 8 electrode arrays and a motion capture system were employed for data acquisition. Following offline model calibration with a global optimization algorithm, online angle estimation results demonstrated a significant superiority of the proposed model over the state-of-the-art NMS models (p < 0.05), yielding the lowest normalized root mean square error (0.10 ± 0.02) and the highest determination coefficient (0.87 ± 0.06). This study provides a novel idea for the decoding of joint movements from MU activities. The research findings hold the potential to advance the development of NMS models towards the control of multiple degrees of freedom, with promising applications in the fields of motor control, biomechanics, and neuro-rehabilitation engineering.

Spike-based computational primitives to reproduce neural dynamics in the parietal cortex during motor preparation

Abstract Biologically plausible spiking neural network models of sensory cortices can be instrumental in understanding and validating their principles of computation.&amp;#xD; Models based on Cortical Computational Primitives (CCPs), such as Hebbian plasticity and Winner-Take-All (WTA) networks, have already been successful in this approach.&amp;#xD; However, the specific nature and roles of CCPs in sensorimotor cortices during cognitive tasks are yet to be fully deciphered.&amp;#xD; The evolution of motor intention in the Posterior Parietal Cortex (PPC) before arm-reaching movements is a well-suited cognitive process to assess the effectiveness of different CCPs.&amp;#xD; To this end, we propose a biologically plausible model composed of heterogeneous spiking neurons which implements and combines multiple CCPs, such as multi-timescale learning and soft WTA modules.&amp;#xD; By training the model to replicate the dynamics of in-vivo recordings from non-human primates, we show how it is effective in generating meaningful representations from unbalanced input data, and in faithfully reproducing the transition from motor planning to action selection.&amp;#xD; Our findings elucidate the importance of distributing spike-based plasticity across multi-timescales, and provide an explanation for the role of different CCPs in models of frontoparietal cortical networks for performing multisensory integration to efficiently inform action execution.&amp;#xD;

Gait self-learning control based on reference trajectory generation online for an asymmetric limb rehabilitation exoskeleton

Lower limb exoskeleton (LEX) are widely used to assist stoke survivors with walking dysfunction, which is lack of a more flexible trajectory and fails to address the control challenge posed by gait variability and asymmetry in rehabilitation training. This paper introduces an asymmetric self-learning lower exoskeleton (AS-LEX) based on reference trajectory generation for the affected side. Motor intent of the unaffected limb based on thresholds was identified to classify the gait phase of stance and swing. A parameterized gait trajectory was generated online, namely a combination of circular trajectory in the stance phase and an elliptical trajectory in the swing phase. Gait self-learning control is presented to make the affected limb adaptively learn the gait parameters generated by the unaffected limb. Feasibility of the AS-LEX is demonstrated experimentally using three healthy subjects. Resuls demonstrate that overground walking in a more natural speed (with a stride length 600 mm and 700 mm) make subjects more actively learn gait of the affected side from the unaffected side. Additionally, experiments of the fatigue level of the affected limb and human-robot interaction torques were carried out, and the results indicate a more natural gait and reduced interaction forces with the AS-LEX.

A new approach for neural decoding by inspiring of hyperdimensional computing for implantable intra-cortical BMIs

In the field of Brain Machine Interface (BMI), the process of translating motor intention into a machine command is denoted as decoding. However, despite recent advancements, decoding remains a formidable challenge within BMI. The utilization of current decoding algorithms in the field of BMI often involves computational complexity and requires the use of computers. This is primarily due to the reliance on mathematical models to address the decoding issue and perform subsequent output calculations. Unfortunately, computers are not feasible for implantable BMI systems due to their size and power consumption. To address this predicament, this study proposes a pioneering approach inspired by hyperdimensional computing. This approach first involves identifying the pattern of each stimulus by considering the normal firing rate distribution of each neuron. Subsequently, the newly observed firing pattern for each input is compared with the patterns detected at each moment for each neuron. The algorithm, which shares similarities with hyperdimensional computing, identifies the most similar pattern as the final output. This approach reduces the dependence on mathematical models. The efficacy of this method is assessed through the utilization of an authentic dataset acquired from the Frontal Eye Field (FEF) of two male rhesus monkeys. The output space encompasses eight possible angles. The results demonstrate an accuracy rate of 51.5% while exhibiting significantly low computational complexity, involving a mere 2050 adder operators. Furthermore, the proposed algorithm is implemented on a field-programmable gate array (FPGA) and as an ASIC designe in a standard CMOS 180 nm technology, underscoring its suitability for real-time implantable BMI applications. The implementation required only 2.3 Kbytes of RAM, occupied an area of 2.2 mm2, and consumed 9.32 µW at a 1.8 V power supply. Consequently, the proposed solution represents an accurate, low computational complexity, hardware-friendly, and real-time approach.

A multimodal multistream multilevel fusion network for finger joint angle estimation with hybrid sEMG and FMG sensing

Finger joint angle (FJA) estimation, as a dynamic and fine-grained decoding mode, can support intuitive and natural human–machine interactions. This study is pioneering work dedicated to achieving accurate and clinically friendly FJA estimation. In this context, a novel surface electromyography and force myography-based multimodal multistream multilevel fusion network (MMMFNet) is proposed. MMMFNet utilises gate-recurrent units (GRU) networks to capture spatial dependencies among features and adopts a multilevel fusion strategy to extract more comprehensive representations from information sources than a single-level fusion. A cross-stream interaction (CSI) block is added between the GRU networks to focus on important information in the prediction and optimise the learning process and representation capability by facilitating information interaction among branch networks. The experimental results suggest that the MMMFNet has higher estimation accuracy (RMSE: 6.225±0.276 vs. 6.953±0.282, R2: 0.879±0.019 vs 0.846±0.021), shorter training time (4.146±0.074 min vs. 16.536±0.430 min), and lower computational complexity (5.4×106 FLOPs vs 8.8×106 FLOPs) than state-of-the-art fusion methods. Moreover, MMMFNet, which quantifies feature importance using attention weights, is the first explainable multimodal model for hand movement intention decoding. In summary, compared with state-of-the-art methods, MMMFNet can not only estimate FJAs more accurately but is also more clinically friendly.

A narrative review of velocity-based training best practice: The importance of contraction intent vs. movement speed.

Explosive movements requiring high force and power outputs are integral to many sports, posing distinct challenges for the neuromuscular system. Traditional resistance training can improve muscle strength, power, endurance, and range of motion; however, evidence regarding its effects on athletic performance, such as sprint speed, agility, and jump height, remains conflicting. The specificity of resistance training movements, including velocity, contraction type, and joint angles affects performance outcomes, demonstrates advantages when matching training modalities with targeted sports activities. However, independent of movement speed, the intent to contract explosively (ballistic) has also demonstrated high velocity-specific training adaptations. The purpose of this narrative review was to assess the impact of explosive or ballistic contraction intent on velocity-specific training adaptations. Such movement intent may predominantly elicit motor efferent neural adaptations, including motor unit recruitment and rate coding enhancements. Plyometrics, which utilize rapid stretch-shortening cycle (SSC) movements may augment high-speed movement efficiency and muscle activation, possibly leading to improved motor control through adaptations like faster eccentric force absorption, reduced amortization periods, and quicker transitions to explosive concentric contractions. An optimal training paradigm for power and performance enhancement might involve a combination of maximal explosive intent training with heavier loads and plyometric exercises with lighter loads at high velocities. This narrative review synthesizes key literature to answer whether contraction intent or movement speed is more critical for athletic performance enhancement, ultimately advocating for an integrative approach to resistance training tailored for sports-specific explosive action.

Long-term Outcomes of Complete Tears of the Distal Biceps Tendon: An Analysis of Surgical Management at a Median Follow-up of 14.7 Years.

Surgical repair of full thickness biceps tears has demonstrated adequate outcomes in short and mid-term studies. However, data on the long-term outcomes of full thickness distal biceps injuries and their treatment are currently lacking. The purpose of this study was to report on patient demographics, injury characteristics, and long-term outcomes for patients with full-thickness distal biceps tears. It was hypothesized that complete distal biceps tears managed operatively would demonstrate robust clinical success at long-term follow-up. Case series; Level of evidence, 4. Patients with magnetic resonance imaging-confirmed, complete distal biceps tendon rupture sustained between 1996 and 2016 were identified. Patients were cross-referenced with a regional geographic database. A total of 66 patients (3 female, 63 male) with a median age of 50.8 years (IQR, 41.5-60.4) and a median clinical follow-up of 14.7 years (IQR, 9.6-17.9 years) were included. Patients who sustained a full-thickness distal biceps tendon tear were likely to be in their early 50s, male, right-hand dominant, current/former smokers, and laborers with a history of traumatic injury during an intentional movement. Most of these patients had pain and supination weakness but no loss of range of motion. All included tears were treated operatively. At final follow-up, patients maintained a majority of near-normal range of motion (median total arc of flexion/extension 140° and supination/pronation 80°), excellent elbow flexion strength (91% of patients had full strength), and adequate elbow supination strength (76% of patients had full strength). The overall complication rate was 24%, with 16 out of 66 patients experiencing some type of complication between infection, rerupture, heterotopic ossification, reoperation, and nerve complications. Overall return to work was 98%, and 85% of those who returned to work did so without restrictions. Complete tears of the distal biceps were most common in patients 50 years of age, male sex, right-hand dominant, and current/former smokers. The most common profession was laborer, and injuries were primarily traumatic in nature during intentional activity. Patients managed operatively demonstrated high rates of success at long-term follow-up with respect to elbow function and clinical outcomes.

Upper Limb Rehabilitation Robots Training Analysis based on Multi-sensor Trajectory Data and Human-computer Interaction

Upper limb rehabilitation robots have important practical significance in helping patients recover their motor function, but traditional training methods often lack real-time and accurate evaluation of patient movement status and intention and have low interactive participation. Therefore, the study proposes a training system for upper limb rehabilitation robots based on multi-sensor trajectory data and human-computer interaction. The system is designed from three aspects: inertial sensor trajectory tracking, Kinect sensor image restoration, and interaction system design. A polynomial joint zero value constraint algorithm is introduced to correct errors, and variable parameter pixel filtering combined with a weighted average moving algorithm is used to improve image quality. The training results showed that the robot trajectory tracking accuracy significantly improved, and the improvement in motion trajectory error was greater than 20%. Compared with traditional training methods, interactive training had a recognition accuracy of over 85% in rehabilitation actions, and the system had better stability. This rehabilitation robot can effectively meet the feasibility and effectiveness of improving the interaction system, providing new technological means for intelligent medical services.

Examining Tongue Movement Intentions in EEG-Based BCI with Machine and Deep Learning: An Approach for Dysphagia Rehabilitation

Abstract Dysphagia, a common swallowing disorder particularly prevalent among older adults and often associated with neurological conditions, significantly affects individuals’ quality of life by negatively impacting their eating habits, physical health, and social interactions. This study investigates the potential of brain-computer interface (BCI) technologies in dysphagia rehabilitation, focusing specifically on motor imagery paradigms based on EEG signals and integration with machine learning and deep learning methods for tongue movement. Traditional machine learning classifiers, such as K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Decision Tree, Naive Bayes, Random Forest, AdaBoost, Bagging, and Kernel were employed in discrimination of rest and imagination phases of EEG signals obtained from 30 healthy subjects. Scalogram images obtained using continuous wavelet transform of EEG signals corresponding to the rest and imagination phases of the experiment were used as the input images to the CNN architecture. As a result, KNN (79.4%) and SVM (63.4%) exhibited lower accuracy rates compared to ensemble methods like AdaBoost, Bagging, and Random Forest, all achieving high accuracy rates of 99.8%. These ensemble techniques proved to be highly effective in handling complex EEG datasets, particularly in distinguishing between rest and imagination phases. Furthermore, the deep learning approach, utilizing CNN and Continuous Wavelet Transform (CWT), achieved an accuracy of 83%, highlighting its potential in analyzing motor imagery data. Overall, this study demonstrates the promising role of BCI technologies and advanced machine learning techniques, especially ensemble and deep learning methods, in improving outcomes for dysphagia rehabilitation.