Motion Intention Research Articles

Dynamical Differential Covariance Based Brain Network for Motor Intent Recognition

Dynamical Differential Covariance Based Brain Network for Motor Intent Recognition

Towards unlocking motor control in spinal cord injured by applying an online EEG-based framework to decode motor intention, trajectory and error processing

Brain-computer interfaces (BCIs) can translate brain signals directly into commands for external devices. Electroencephalography (EEG)-based BCIs mostly rely on the classification of discrete mental states, leading to unintuitive control. The ERC-funded project "Feel Your Reach" aimed to establish a novel framework based on continuous decoding of hand/arm movement intention, for a more natural and intuitive control. Over the years, we investigated various aspects of natural control, however, the individual components had not yet been integrated. Here, we present a first implementation of the framework in a comprehensive online study, combining (i) goal-directed movement intention, (ii) trajectory decoding, and (iii) error processing in a unique closed-loop control paradigm. Testing involved twelve able-bodied volunteers, performing attempted movements, and one spinal cord injured (SCI) participant. Similar movement-related cortical potentials and error potentials to previous studies were revealed, and the attempted movement trajectories were overall reconstructed. Source analysis confirmed the involvement of sensorimotor and posterior parietal areas for goal-directed movement intention and trajectory decoding. The increased experiment complexity and duration led to a decreased performance than each single BCI. Nevertheless, the study contributes to understanding natural motor control, providing insights for more intuitive strategies for individuals with motor impairments.

Motion intention recognition of the affected hand based on the sEMG and improved DenseNet network

The key to sEMG (surface electromyography)-based control of robotic hands is the utilization of sEMG signals from the affected hand of amputees to infer their motion intentions. With the advancements in deep learning, researchers have successfully developed viable solutions for CNN (Convolutional Neural Network)-based gesture recognition. However, most studies have primarily concentrated on utilizing sEMG data from the hands of healthy subjects, often relying on high-dimensional feature vectors obtained from a substantial number of electrodes. This approach has yielded high-performing sEMG recognition systems but has failed to consider the considerable inconvenience that the abundance of electrodes poses to the daily lives and work of patients. In this paper, we focused on transradial amputees and used sEMG data from the Ninapro DB3 database as our dataset. Firstly, we introduce a STFT (Short-Time Fourier Transform)-based time-frequency feature fusion map for sEMG. This map includes both time-frequency features and the time-frequency localization of sEMG signals. Secondly, we propose an Improved DenseNet (Dense Convolutional Network) model for recognizing motion intentions in the affected hand of amputees based on their sEMG signals. Finally, addressing the issue of optimizing the number of electrodes carried by amputees, we introduce the PCMIRR (Pearson Correlation and Motion Intention Recognition Rate) algorithm. This algorithm optimizes the number of channels by considering the Pearson correlation between the sEMG channels of amputees and the recognition rate of motion intentions in the affected hand based on single-channel sEMG data. The experimental results reveal that the recognition accuracy, recall, and F1 score achieved by the Improved DenseNet model were 93.82%, 93.61%, and 93.65%, respectively. When the number of electrodes was optimized to 8, the recognition accuracy reached 94.50%. In summary, this paper ultimately attained precise recognition of motion intentions in amputees' affected hands while utilizing the minimum number of sEMG channels. This method offers a novel approach to sEMG-based control of bionic robotic hands.

Optimized FFNN with multichannel CSP-ICA framework of EEG signal for BCI

The electroencephalogram (EEG) of the patient is used to identify their motor intention, which is then converted into a control signal through a brain-computer interface (BCI) based on motor imagery. Whenever gathering features from EEG signals, making a BCI is difficult in part because of the enormous dimensionality of the data. Three stages make up the suggested methodology: pre-processing, extraction of features, selection, and categorization. To remove unwanted artifacts, the EEG signals are filtered by a fifth-order Butterworth multichannel band-pass filter. This decreases execution time and memory use, both of which improve system performance. Then a novel multichannel optimized CSP-ICA feature extraction technique is used to separate and eliminate non-discriminative information from discriminative information in the EEG channels. Furthermore, CSP uses the concept of an Artificial Bee Colony (ABC) algorithm to automatically identify the simultaneous global ideal frequency band and time interval combination for the extraction and classification of common spatial pattern characteristics. Finally, a Tunable optimized feed-forward neural network (FFNN) classifier is utilized to extract and categorize the temporal and frequency domain features, which employs an FFNN classifier with Tunable-Q wavelet transform. The proposed framework, therefore optimizes signal processing, enabling enhanced EEG signal classification for BCI applications. The result shows that the models that use Tunable optimized FFNN produce higher classification accuracy of more than 20% when compared to the existing models.

Decentralized variable impedance control of modular robot manipulators with physical human–robot interaction using Gaussian process-based motion intention estimation

Decentralized variable impedance control of modular robot manipulators with physical human–robot interaction using Gaussian process-based motion intention estimation

Human-robot planar co-manipulation of extended objects: data-driven models and control from human-human dyads.

Human teams are able to easily perform collaborative manipulation tasks. However, simultaneously manipulating a large extended object for a robot and human is a difficult task due to the inherent ambiguity in the desired motion. Our approach in this paper is to leverage data from human-human dyad experiments to determine motion intent for a physical human-robot co-manipulation task. We do this by showing that the human-human dyad data exhibits distinct torque triggers for a lateral movement. As an alternative intent estimation method, we also develop a deep neural network based on motion data from human-human trials to predict future trajectories based on past object motion. We then show how force and motion data can be used to determine robot control in a human-robot dyad. Finally, we compare human-human dyad performance to the performance of two controllers that we developed for human-robot co-manipulation. We evaluate these controllers in three-degree-of-freedom planar motion where determining if the task involves rotation or translation is ambiguous.

Simulation of a control method for active kinesiotherapy with an upper extremity rehabilitation exoskeleton without force sensor

Exoskeleton-aided active rehabilitation is a process that requires sensing and acting upon the motion intentions of the user. Typically, force sensors are used for this. However, they increase the weight and cost of these wearable devices. This paper presents the methodology for detecting users’ intentions only with encoders integrated with the drives. It is unique compared to other algorithms, as enables active kinesiotherapy while adding no sensory systems. The method is based on comparing the measured motion with the one computed with the idealised model of the multibody system. The investigation assesses the method’s performance and its robustness to model and measurement inaccuracies, as well as patients’ unintended motions. Moreover, the PID parameters are selected to provide the optimal regulation based on the dynamics requirements. The research proves the presented concept of the control approach. For all the tests with the final settings, the system reacts to a change in the user’s intention below one second and minimises the changes in proportion between the system’s acceleration and the generated user’s joint torque. The results are comparable to those obtained by EMG-based systems and significantly better than low-cost force sensors.

Compliant-Control-Based Assisted Walking with Mobile Manipulator.

In this paper, a new approach involving the use of a mobile manipulator to assist humans with mobility impairments to walk is proposed. First, in order to achieve flexible interaction between humans and mobile manipulators, we propose a variable admittance controller that can adaptively regulate the virtual mass and damping parameters based on the interaction forces and the human motion intention predicted using the fuzzy theory. Moreover, a feedforward velocity compensator based on a designed state observer is proposed to decrease the inertia resistance of the manipulator, effectively enhancing the compliance of the human-robot interaction. Then, the configuration of the mobile manipulator is optimized based on a null-space approach by considering the singularity, force capacity, and deformation induced by gravity. Finally, the proposed assisted walking approach for the mobile manipulator is implemented using the human-robot interaction controller and the null-space controller. The validity of the proposed controllers and the feasibility of assisted human walking are verified by conducting a set of tests involving different human volunteers.

Improving motor imagery through a mirror box for BCI users.

The aim of this study was to evaluate mirror visual feedback (MVF) as a training tool for brain-computer interface (BCI) users. This is because approximately 20-30% of subjects require more training to operate a BCI system using motor imagery. Electroencephalograms (EEGs) were recorded from 18 healthy subjects, using event-related desynchronization (ERD) to observe the responses during the movement or movement intention of the hand for the conditions of control, imagination, and the MVF with the mirror box. We constituted two groups: group 1: control, imagination, and MVF; group 2: control, MVF, and imagination. There were significant differences in imagination conditions between groups using MVF before or after imagination (right-hand, P = 0.0403; left-hand, P = 0.00939). The illusion of movement through MVF is not possible in all subjects, but even in those cases, we found an increase in imagination when the subject used the MVF previously. The increase in the r2s of imagination in the right and left hands suggests cross-learning. The increase in motor imagery recorded with EEG after MVF suggests that the mirror box made it easier to imagine movements. Our results provide evidence that the MVF could be used as a training tool to improve motor imagery.NEW & NOTEWORTHY The increase in motor imagery recorded with EEG after MVF (mirror visual feedback) suggests that the mirror box made it easier to imagine movements. Our results demonstrate that MVF could be used as a training tool to improve motor imagery.

Automatic Detection of Faults in Simulated Race Walking from a Fixed Smartphone Camera

Abstract Automatic fault detection is a major challenge in many sports. In race walking, judges visually detect faults according to the rules. Hence, automatic fault detection systems will help a training of race walking without experts’ visual judgement. Some studies have attempted to use sensors and machine learning to automatically detect faults. However, there are problems associated with sensor attachments and equipment such as a high-speed camera, which conflict with the visual judgement of judges, and the interpretability of the fault detection models. In this study, we proposed an automatic fault detection system for non-contact measurement. We used pose estimation and machine learning models trained based on the judgements of multiple qualified judges to realize fair fault judgement. We verified them using smartphone videos of normal race walking and walking with intentional faults in several athletes including the medalist of the Tokyo Olympics. The results show that the proposed system detected faults with an average accuracy of over 90%. We also revealed that the machine learning model detects faults according to the rules. In addition, the intentional faulty walking movement of the medalist was different from that of other walkers. This finding informs realization of a more general fault detection model.

High Performance Wearable Ultrasound as a Human-Machine Interface for Wrist and Hand Kinematic Tracking.

Non-invasive human machine interfaces (HMIs) have high potential in medical, entertainment, and industrial applications. Traditionally, surface electromyography (sEMG) has been used to track muscular activity and infer motor intention. Ultrasound (US) has received increasing attention as an alternative to sEMG-based HMIs. Here, we developed a portable US armband system with 24 channels and a multiple receiver approach, and compared it with existing sEMG- and US-based HMIs on movement intention decoding. US and motion capture data was recorded while participants performed wrist and hand movements of four degrees of freedom (DoFs) and their combinations. A linear regression model was used to offline predict hand kinematics from the US (or sEMG, for comparison) features. The method was further validated in real-time for a 3-DoF target reaching task. In the offline analysis, the wearable US system achieved an average [Formula: see text] of 0.94 in the prediction of four DoFs of the wrist and hand while sEMG reached a performance of [Formula: see text]= 0.60. In online control, the participants achieved an average 93% completion rate of the targets. When tailored for HMIs, the proposed US A-mode system and processing pipeline can successfully regress hand kinematics both in offline and online settings with performances comparable or superior to previously published interfaces. Wearable US technology may provide a new generation of HMIs that use muscular deformation to estimate limb movements. The wearable US system allowed for robust proportional and simultaneous control over multiple DoFs in both offline and online settings.

EEG generation mechanism of lower limb active movement intention and its virtual reality induction enhancement: a preliminary study.

Active rehabilitation requires active neurological participation when users use rehabilitation equipment. A brain-computer interface (BCI) is a direct communication channel for detecting changes in the nervous system. Individuals with dyskinesia have unclear intentions to initiate movement due to physical or psychological factors, which is not conducive to detection. Virtual reality (VR) technology can be a potential tool to enhance the movement intention from pre-movement neural signals in clinical exercise therapy. However, its effect on electroencephalogram (EEG) signals is not yet known. Therefore, the objective of this paper is to construct a model of the EEG signal generation mechanism of lower limb active movement intention and then investigate whether VR induction could improve movement intention detection based on EEG. Firstly, a neural dynamic model of lower limb active movement intention generation was established from the perspective of signal transmission and information processing. Secondly, the movement-related EEG signal was calculated based on the model, and the effect of VR induction was simulated. Movement-related cortical potential (MRCP) and event-related desynchronization (ERD) features were extracted to analyze the enhancement of movement intention. Finally, we recorded EEG signals of 12 subjects in normal and VR environments to verify the effectiveness and feasibility of the above model and VR induction enhancement of lower limb active movement intention for individuals with dyskinesia. Simulation and experimental results show that VR induction can effectively enhance the EEG features of subjects and improve the detectability of movement intention. The proposed model can simulate the EEG signal of lower limb active movement intention, and VR induction can enhance the early and accurate detectability of lower limb active movement intention. It lays the foundation for further robot control based on the actual needs of users.

A systematic investigation of detectors for low signal-to-noise ratio EMG signals

Background Active participation of stroke survivors during robot-assisted movement therapy is essential for sensorimotor recovery. Robot-assisted therapy contingent on movement intention is an effective way to encourage patients’ active engagement. For severely impaired stroke patients with no residual movements, a surface electromyogram (EMG) has been shown to be a viable option for detecting movement intention. Although numerous algorithms for EMG detection exist, the detector with the highest accuracy and lowest latency for low signal-to-noise ratio (SNR) remains unknown. Methods This study, therefore, investigates the performance of 13 existing EMG detection algorithms on simulated low SNR (0dB and -3dB) EMG signals generated using three different EMG signal models: Gaussian, Laplacian, and biophysical model. The detector performance was quantified using the false positive rate (FPR), false negative rate (FNR), and detection latency. Any detector that consistently showed FPR and FNR of no more than 20%, and latency of no more than 50ms, was considered an appropriate detector for use in robot-assisted therapy. Results The results indicate that the Modified Hodges detector – a simplified version of the threshold-based Hodges detector introduced in the current study – was the most consistent detector across the different signal models and SNRs. It consistently performed for ~90% and ~40% of the tested trials for 0dB and -3dB SNR, respectively. The two statistical detectors (Gaussian and Laplacian Approximate Generalized Likelihood Ratio) and the Fuzzy Entropy detectors have a slightly lower performance than Modified Hodges. Conclusions Overall, the Modified Hodges, Gaussian and Laplacian Approximate Generalized Likelihood Ratio, and the Fuzzy Entropy detectors were identified as the potential candidates that warrant further investigation with real surface EMG data since they had consistent detection performance on low SNR EMG data.

Liquid Metal Composites-Enabled Real-Time Hand Gesture Recognizer with Superior Recognition Speed and Accuracy.

Prosthetic hands play a vital role in restoring forearm functionality for patients who have suffered hand loss or deformity. The hand gesture intention recognition system serves as a critical component within the prosthetic hand system. However, accurately and swiftly identifying hand gesture intentions remains a challenge in existing approaches. Here, a real-time motion intention recognition system utilizing liquid metal composite sensor bracelets is proposed. The sensor bracelet detects pressure signals generated by forearm muscle movements to recognize hand gesture intent. Leveraging the remarkable pressure sensitivity of liquid metal composites and the efficient classifier based on the optimized recognition algorithm, this system achieves an average offline and real-time recognition accuracy of 98.2% and 92.04%, respectively, with an average recognition speed of 0.364 s. Thus, this wearable system shows advantages in superior recognition speed and accuracy. Furthermore, this system finds applications in master-slave control of prosthetic hands in unmanned scenarios, such as electrically powered operations, space exploration, and telemedicine. The proposed system promises significant advances in next-generation intent-controlled prosthetic hands and robots.

Infants on the move: bibliometric analyses of observational vs. digital means of screening infant development.

Neurodevelopmental disorders are on the rise, yet their average diagnosis is after 4.5 years old. This delay is partly due to reliance on social-communication criteria, which require longer maturation than scaffolding elements of neuromotor control. Much earlier criteria could include reflexes, monitoring of the quality of spontaneous movements from central pattern generators and maturation of intentional movements and their overall sensation. General Movement Assessment (GMA) studies these features using observational means, but the last two decades have seen a surge in digital tools that enable non-invasive, continuous tracking of infants' spontaneous movements. Despite their importance, these tools are not yet broadly used. In this work, using CiteSpace, VOSViewer and SciMAT software, we investigate the evolution of the literature on GMA and the methods in use today, to estimate how digital techniques are being adopted. To that end, we created maps of key word co-occurrence networks, co-author networks, document co-citation analysis and strategic diagrams of 295 publications based on a search in the Web of Science, Dimensions and SCOPUS databases for: 'general movement assessment' OR 'general movements assessment'. The nodes on the maps were categorized by size, cluster groups and year of publication. We found that the state-of-the-art methodology to diagnose neurodevelopmental disorders still relies heavily on observation. Several groups in classical GMA research have branched out to incorporate new techniques, but few groups have adopted digital means. We report on additional analyses of methods and biosensors usage and propose that combining traditional clinical observation criteria with digital means may allow earlier diagnoses and interventional therapies for infants.

A Semi-Autonomous Hierarchical Control Framework for Prosthetic Hands Inspired by Dual Streams of Human.

The routine use of prosthetic hands significantly enhances amputees' daily lives, yet it often introduces cognitive load and reduces reaction speed. To address this issue, we introduce a wearable semi-autonomous hierarchical control framework tailored for amputees. Drawing inspiration from the visual processing stream in humans, a fully autonomous bionic controller is integrated into the prosthetic hand control system to offload cognitive burden, complemented by a Human-in-the-Loop (HIL) control method. In the ventral-stream phase, the controller integrates multi-modal information from the user's hand-eye coordination and biological instincts to analyze the user's movement intention and manipulate primitive switches in the variable domain of view. Transitioning to the dorsal-stream phase, precise force control is attained through the HIL control strategy, combining feedback from the prosthetic hand's sensors and the user's electromyographic (EMG) signals. The effectiveness of the proposed interface is demonstrated by the experimental results. Our approach presents a more effective method of interaction between a robotic control system and the human.

Servo navigators: Linear regression and feedback control for rigid-body motion correction.

Navigator-based correction of rigid-body motion reconciling high precision with minimal acquisition, minimal calibration and simple, fast processing. A short orbital navigator (2.3 ms) is inserted in a three-dimensional (3D) gradient echo sequence for human head imaging. Head rotation and translation are determined by linear regression based on a complex-valued model built either from three reference navigators or in a reference-less fashion, from the first actual navigator. Optionally, the model is expanded by global phase and field offset. Run-time scan correction on this basis establishes servo control that maintains validity of the linear picture by keeping its expansion point stable in the head frame of reference. The technique is assessed in a phantom and demonstrated by motion-corrected imaging in vivo. The proposed approach is found to establish stable motion control both with and without reference acquisition. In a phantom, it is shown to accurately detect motion mimicked by rotation of scan geometry as well as change in global B0 . It is demonstrated to converge to accurate motion estimates after perturbation well beyond the linear signal range. In vivo, servo navigation achieved motion detection with precision in the single-digit range of micrometers and millidegrees. Involuntary and intentional motion in the range of several millimeters were successfully corrected, achieving excellent image quality. The combination of linear regression and feedback control enables prospective motion correction for head imaging with high precision and accuracy, short navigator readouts, fast run-time computation, and minimal demand for reference data.

Decoding hand and wrist movement intention from chronic stroke survivors with hemiparesis using a user-friendly, wearable EMG-based neural interface

ObjectiveSeventy-five percent of stroke survivors, caregivers, and health care professionals (HCP) believe current therapy practices are insufficient, specifically calling out the upper extremity as an area where innovation is needed to develop highly usable prosthetics/orthotics for the stroke population. A promising method for controlling upper extremity technologies is to infer movement intention non-invasively from surface electromyography (EMG). However, existing technologies are often limited to research settings and struggle to meet user needs.ApproachTo address these limitations, we have developed the NeuroLife® EMG System, an investigational device which consists of a wearable forearm sleeve with 150 embedded electrodes and associated hardware and software to record and decode surface EMG. Here, we demonstrate accurate decoding of 12 functional hand, wrist, and forearm movements in chronic stroke survivors, including multiple types of grasps from participants with varying levels of impairment. We also collected usability data to assess how the system meets user needs to inform future design considerations.Main resultsOur decoding algorithm trained on historical- and within-session data produced an overall accuracy of 77.1 ± 5.6% across 12 movements and rest in stroke participants. For individuals with severe hand impairment, we demonstrate the ability to decode a subset of two fundamental movements and rest at 85.4 ± 6.4% accuracy. In online scenarios, two stroke survivors achieved 91.34 ± 1.53% across three movements and rest, highlighting the potential as a control mechanism for assistive technologies. Feedback from stroke survivors who tested the system indicates that the sleeve’s design meets various user needs, including being comfortable, portable, and lightweight. The sleeve is in a form factor such that it can be used at home without an expert technician and can be worn for multiple hours without discomfort.SignificanceThe NeuroLife EMG System represents a platform technology to record and decode high-resolution EMG for the real-time control of assistive devices in a form factor designed to meet user needs. The NeuroLife EMG System is currently limited by U.S. federal law to investigational use.

Research on the method of identifying upper and lower limb coordinated movement intentions based on surface EMG signals.

Rehabilitation robots have gained considerable focus in recent years, aiming to assist immobilized patients in regaining motor capabilities in their limbs. However, most current rehabilitation robots are designed specifically for either upper or lower limbs. This limits their ability to facilitate coordinated movement between upper and lower limbs and poses challenges in accurately identifying patients' intentions for multi-limbs coordinated movement. This research presents a multi-postures upper and lower limb cooperative rehabilitation robot (U-LLCRR) to address this gap. Additionally, the study proposes a method that can be adjusted to accommodate multi-channel surface electromyographic (sEMG) signals. This method aims to accurately identify upper and lower limb coordinated movement intentions during rehabilitation training. By using genetic algorithms and dissimilarity evaluation, various features are optimized. The Sine-BWOA-LSSVM (SBL) classification model is developed using the improved Black Widow Optimization Algorithm (BWOA) to enhance the performance of the Least Squares Support Vector Machine (LSSVM) classifier. Discrete movement recognition studies are conducted to validate the exceptional precision of the SBL classification model in limb movement recognition, achieving an average accuracy of 92.87%. Ultimately, the U-LLCRR undergoes online testing to evaluate continuous motion, specifically the movements of "Marching in place with arm swinging". The results show that the SBL classification model maintains high accuracy in recognizing continuous motion intentions, with an average identification rate of 89.25%. This indicates its potential usefulness in future rehabilitation robot-active training methods, which will be a promising tool for a wide range of applications in the fields of healthcare, sports, and beyond.

Early Prediction of Human Intention for Human–Robot Collaboration Using Transformer Network

Abstract Human intention prediction plays a critical role in human–robot collaboration, as it helps robots improve efficiency and safety by accurately anticipating human intentions and proactively assisting with tasks. While current applications often focus on predicting intent once human action is completed, recognizing human intent in advance has received less attention. This study aims to equip robots with the capability to forecast human intent before completing an action, i.e., early intent prediction. To achieve this objective, we first extract features from human motion trajectories by analyzing changes in human joint distances. These features are then utilized in a Hidden Markov Model (HMM) to determine the state transition times from uncertain intent to certain intent. Second, we propose two models including a Transformer and a Bi-LSTM for classifying motion intentions. Then, we design a human–robot collaboration experiment in which the operator reaches multiple targets while the robot moves continuously following a predetermined path. The data collected through the experiment were divided into two groups: full-length data and partial data before state transitions detected by the HMM. Finally, the effectiveness of the suggested framework for predicting intentions is assessed using two different datasets, particularly in a scenario when motion trajectories are similar but underlying intentions vary. The results indicate that using partial data prior to the motion completion yields better accuracy compared to using full-length data. Specifically, the transformer model exhibits a 2% improvement in accuracy, while the Bi-LSTM model demonstrates a 6% increase in accuracy.