Neuromuscular Signals Research Articles

IP3T: an interpretable multimodal time-series model for enhanced gait phase prediction in wearable exoskeletons.

Wearable exoskeletons assist individuals with mobility impairments, enhancing their gait and quality of life. This study presents the iP3T model, designed to optimize gait phase prediction through the fusion of multimodal time-series data. The iP3T model integrates data from stretch sensors, inertial measurement units (IMUs), and surface electromyography (sEMG) to capture comprehensive biomechanical and neuromuscular signals. The model's architecture leverages transformer-based attention mechanisms to prioritize crucial data points. A series of experiments were conducted on a treadmill with five participants to validate the model's performance. The iP3T model consistently outperformed traditional single-modality approaches. In the post-stance phase, the model achieved an RMSE of 1.073 and an R2 of 0.985. The integration of multimodal data enhanced prediction accuracy and reduced metabolic cost during assisted treadmill walking. The study highlights the critical role of each sensor type in providing a holistic understanding of the gait cycle. The attention mechanisms within the iP3T model contribute to its interpretability, allowing for effective optimization of sensor configurations and ultimately improving mobility and quality of life for individuals with gait impairments.

EMG Instrumentation Modeling and Feature Processing Based On Discrete Wavelet Transform

Electromyography (EMG) instrumentation is essential in generating electrical signals from skeletal muscles. EMG sensors are helpful in various cases requiring the detection of human muscle contractions, neuromuscular disorders, and rehabilitation. EMG instrumentation is divided into two parts, namely, the analogue part and the digital part. The EMG instrumentation design comprises a digital-to-analog converter (DAC), instrumentation amplifier, filter, and analog-to-digital converter (ADC). Meanwhile, in digital signal processing adopting the Discrete Wavelet Transform (DWT) method, frequency analysis using DWT has proven superior. It is used in various research and has exceptionally detailed coefficient features for classifying neuromuscular disease signals. Therefore, this research aims to design analogue and digital EMG instrumentation and identify features in the form of detailed coefficients. The data used are two Physionet signals from the anterior tibialis body with myopathy and neuropathy disorders. The results obtained for EMG analogue instrumentation provide the expected results until they reach the filter component stage. The resulting signal forms a block in the filter component, different from the initial EMG signal. Meanwhile, the DWT decomposition results are of the Daubechies4 wavelet type with the highest level 17, which has a high detail coefficient at low frequencies, high dilation and the result of a mixture of neuropathy and myopathy EMG signals, or in other words, at low energies, this result is by the DWT theorem. Determining the efficiency of the DWT detailed coefficient feature requires further study with the same signal subject. The DWT features obtained can then be developed for various needs in EMG signal recognition.

Driver Steering Behaviour Modelling Based on Neuromuscular Dynamics and Multi-Task Time-Series Transformer

Driver steering intention prediction provides an augmented solution to the design of an onboard collaboration mechanism between human driver and intelligent vehicle. In this study, a multi-task sequential learning framework is developed to predict future steering torques and steering postures based on upper limb neuromuscular electromyography signals. The joint representation learning for driving postures and steering intention provides an in-depth understanding and accurate modelling of driving steering behaviours. Regarding different testing scenarios, two driving modes, namely, both-hand and single-right-hand modes, are studied. For each driving mode, three different driving postures are further evaluated. Next, a multi-task time-series transformer network (MTS-Trans) is developed to predict the future steering torques and driving postures based on the multi-variate sequential input and the self-attention mechanism. To evaluate the multi-task learning performance and information-sharing characteristics within the network, four distinct two-branch network architectures are evaluated. Empirical validation is conducted through a driving simulator-based experiment, encompassing 21 participants. The proposed model achieves accurate prediction results on future steering torque prediction as well as driving posture recognition for both two-hand and single-hand driving modes. These findings hold significant promise for the advancement of driver steering assistance systems, fostering mutual comprehension and synergy between human drivers and intelligent vehicles.

Hydrodynamic Diversity of Jets Mediated by Giant and Non-Giant Axon Systems in Brief Squid.

Neural input is critical for establishing behavioral output, but understanding how neuromuscular signals give rise to behaviors remains a challenge. In squid, locomotion through jet propulsion underlies many key behaviors, and the jet is mediated by two parallel neural pathways, the giant and non-giant axon systems. Much work has been done on the impact of these two systems on jet kinematics, such as mantle muscle contraction and pressure-derived jet speed at the funnel aperture. However, little is known about any influence these neural pathways may have on the hydrodynamics of the jet after it leaves the squid and transfers momentum to the surrounding fluid for the animal to swim. To gain a more comprehensive view of squid jet propulsion, we made simultaneous measurements of neural activity, pressure inside the mantle cavity, and wake structure. By computing impulse and time-averaged forces from the wake structures of jets associated with giant or non-giant axon activity, we show that the influence of neural pathways on jet kinematics could extend to hydrodynamic impulse and force production. Specifically, the giant axon system produced jets with, on average, greater impulse magnitude than those of the non-giant system. However, non-giant impulse could exceed that of the giant system, evident by the graded range of its output in contrast to the stereotyped nature of the giant system. Our results suggest that the non-giant system offers flexibility in hydrodynamic output, while recruitment of giant axon activity can provide a reliable boost when necessary.

SEMG APPROACH FOR SPEECH RECOGNITION

Speech is the most familiar and habitual way of communication used by most of us. Due to speech disabilities, many people find it difficult to properly voice their views and thus are at a disadvantage. The research tackles the issue of lack of speech from a speech impaired user by recognizing it with the use of ML models such as Gaussian Mixture Model - GMM and Convolutional Neural Network - CNN. With properly recorded and cleaned muscle activity from the facial muscles it is possible to predict the words being uttered/whispered with a certain accuracy. The intended system will additionally also have a visual aid system which can provide better accuracy when used together with the facial muscle activity-based system. Neuromuscular signals from the speech articulating muscles are recorded using Surface Electro Myo Graphy (SEMG) sensors, which will be used to train the machine learning models. In this paper we have demonstrated various signals synthesized through the ElectroMyography system and how they can be classified using machine learning models such as Gaussian Mixture Model and Convolutional Neural Network for the visual-based lip-reading system.

Silent Speech Interface With Vocal Speaker Assistance Based on Convolution-Augmented Transformer

Silent Speech Interface (SSI) based on neuromuscular signals has become popular in Human-Computer Interaction. Articulatory neuromuscular activity can be captured with surface electromyography (sEMG). However, in the absence of acoustic sounds, it remains challenging to convert the articulatory neuromuscular signals of silent speakers into corresponding audio. This paper proposes a Mandarin-based SSI, a novel converter between articulatory neuromuscular from silent speakers and audio signals, by providing an auxiliary set of sEMG and audio pairs from vocal speakers. The proposed method is based on a hybrid framework combining convolutional neural networks and Transformer to utilize both global and local information. We experimentally validate the feasibility of the proposed method by obtaining an average objective character error rate (CER) of 10.69% using an automatic speech recognition (ASR) evaluation tool with four silent speakers. The results show that our Mandarin-based SSI with vocal speaker assistance facilitates the conversion from sEMG into audio in the cross-subject scenario.

Mathematical description of proprioception through muscle activation signal generation in core musculoskeletal system

ObjectiveCentral Pattern Generators (CPGs) produce the majority of muscle activation signals during gait whereas, reflexive signals from proprioception deal with perturbations. The reflexive neuromuscular signals are rather difficult to measure directly. The objective of this study is to mathematically estimate the reflexive neuromuscular signals of the core muscles during a movement similar to a normal gait. Methods1-Musculoskeletal model: a 2-D musculoskeletal model is designed for the upper body. 2-Neuromuscular model: mathematical models for neuromuscular reflexes are derived based on their physiological function descriptions. 3-Simulation: mathematical equations are extracted from the model and solved using numerical methods. Kinematics, muscle forces, and activation signals are obtained from the simulation. 4-Verification: verification of the model is conducted through a sample study. Five IMU sensors and four surface EMG electrodes are utilized to gather data from the experiment. ResultsUpper body kinematics, core muscle forces, and activation signals of the subject are estimated through model simulation for the intended movement. The verification process indicates similarity between simulation outcomes and test results. ConclusionThe proposed mathematical description of proprioception can successfully simulate and estimate reflexive activation signals of core muscles. The estimated activation signals can be employed in conjunction with a musculoskeletal model to predict body kinematics. SignificanceReflexive signal estimation of core muscles can further enhance studies concerning body stability control. Such predictive models have also the aptitude to be employed as an auxiliary clinical tool to help the therapist choose appropriate treatment protocols.

Complexity of modular neuromuscular control increases and variability decreases during human locomotor development

When does modular control of locomotion emerge during human development? One view is that modularity is not innate, being learnt over several months of experience. Alternatively, the basic motor modules are present at birth, but are subsequently reconfigured due to changing brain-body-environment interactions. One problem in identifying modular structures in stepping infants is the presence of noise. Here, using both simulated and experimental muscle activity data from stepping neonates, infants, preschoolers, and adults, we dissect the influence of noise, and identify modular structures in all individuals, including neonates. Complexity of modularity increases from the neonatal stage to adulthood at multiple levels of the motor infrastructure, from the intrinsic rhythmicity measured at the level of individual muscles activities, to the level of muscle synergies and of bilateral intermuscular network connectivity. Low complexity and high variability of neuromuscular signals attest neonatal immaturity, but they also involve potential benefits for learning locomotor tasks.

Fuzzy inference system (FIS) - long short-term memory (LSTM) network for electromyography (EMG) signal analysis

A wide range of application domains,s such as remote robotic control, rehabilitation, and remote surgery, require capturing neuromuscular activities. The reliability of the application is highly dependent on an ability to decode intentions accurately based on captured neuromuscular signals. Physiological signals such as Electromyography (EMG) and Electroencephalography (EEG) generated by neuromuscular activities contain intrinsic patterns for users’ particular actions. Such actions can generally be classified as motor states, such as Forward, Reverse, Hand-Grip, and Hand-Release. To classify these motor states truthfully, the signals must be captured and decoded correctly. This paper proposes a novel classification technique using a Fuzzy Inference System (FIS) and a Long Short-Term Memory (LSTM) network to classify the motor states based on EMG signals. Existing EMG signal classification techniques generally rely on features derived from data captured at a specific time instance. This typical approach does not consider the temporal correlation of the signal in the entire window. This paper proposes an LSTM with a Fuzzy Logic method to classify four major hand movements: forward, reverse, raise, and lower. Features associated with the pattern generated throughout the motor state movement were extracted by exploring published data within a given time window. The classification results can achieve a 91.3% accuracy for the 4-way action (Forward/Reverse/GripUp/RelDown) and 95.1% (Forward/Reverse Action) and 96.7% (GripUp/RelDown action) for 2-way actions. The proposed mechanism demonstrates high-level, human-interpretable results that can be employed in rehabilitation or medical-device industries.

Novel Three-Dimensional and Biocompatible Lift-Off Method for Selective Metallization of a Scleral Contact Lens Electrode for Biopotential Detection.

Presbyopia describes the eye's physiological loss of the ability to see close objects clearly. The adaptation to different viewing distances, termed accommodation, is achieved by a change in the curvature of the eye lens induced by the ciliary muscle. A possible approach to correct presbyopia could be to detect the ciliary muscle's neuromuscular signals during accommodation and transfer these signals electronically to a biomimetic, micro-optical system to provide the necessary refractive power. As a preliminary step toward such a described system, a novel three-dimensional and biocompatible lift-off method was developed. In addition, the influence of the distance between the electrically conducting surfaces of the lens on the accommodated signal amplitudes was investigated. Compared to the conventional masking methods, this process has the advantage that three-dimensional surfaces can be masked with biocompatible gelling sugar by utilizing a direct writing process with a dispensing robot. Since gelling sugar can be used at room temperature and is water-soluble, the process presented is suitable for materials that should not be exposed to organic solvents or excessively high temperatures. Apart from investigating the shrinkage behavior of the gelling sugar during the physical vapor deposition (PVD) coating process, this paper also describes the approaches used to partially coat a commercial scleral contact lens with an electrically conductive material. It was shown that gelling sugar withstands the conditions during the PVD processes and a successful lift-off was performed. To investigate the influence of the spacing between the electrically conductive regions of the contact lens on the measured signals, three simplified electrode configurations with different distances were fabricated using a 3D printer. By testing these in an experimental setup, it could be demonstrated that the distance between the conductive surfaces has a significant influence on the amplitude. Regarding the described lift-off process using gelling sugar, it was found that the dispensing flow rate has a direct influence on the line uniformity. Future work should address the influence of the viscosity of the gelling sugar as well as the diameter of the cannula. It is assumed that they are the prevailing limitations for the lateral resolution.

Design of upper limb prosthesis using real-time motion detection method based on EMG signal processing

Many people suffer from disability, caused by arm amputees and congenital disorder of the musculoskeletal system. An upper limb prosthesis can help disabled people to do their usual life activity. The aim of the paper is to develop upper limb prosthetic device controlled by myoelectric signals. At present the most complicated problem in upper limb prosthetics is connected with processing of neuromuscular signals for their use in control of the prosthesis. In modern prosthesis the intensity of muscle activity is extracted from the electromyographic signals. Prosthetic devices should ideally compensate for the loss of fine, coordinated movements of the hand, provide proprioceptive feedback, and have an esthetic exterior. From this point of view, identification of single finger movements seems very attractive. On the one hand it allows to restore functionality of amputated limb close to the healthy one. On the other hand existing results in the field of multifinger control require complicated computations. This leads to cost increase of the end-user device. To overcome this problem, we propose amplitude-based window recognition method with low computational complexity. The proposed solution can be implemented in real-time control using low cost Arduino nano microcontroller. A prototype of prosthetic hand with novel engineering solutions and proposed software is suggested. Experimental and statistical validation of identification scheme on a group of healthy people is considered. The proposed solution is tested in real time in autonomous mode. Experimental results for disabled man is provided. A discussion of the obtained results is given.

Gait and neuromuscular dynamics during level and uphill walking carrying military loads

ABSTRACT The neuromuscular system responds to perturbation and increasing locomotor task difficulty by altering the stability of neuromuscular output signals. The purpose of this study was to determine the effects of two different military load carriage systems on the dynamic stability of gait and muscle activation signals. 14 army office cadets (20 ± 1 years) performed 4-minute treadmill walking trials on level (0%) and uphill (10%) gradients while unloaded, and with 11 kg backpack and 11 kg webbing loads while the activity of 6 leg and trunk muscles and the motion of the centre of mass (COM) were recorded. Loaded and uphill walking decreased stability and increased magnitude of muscle activations compared to loaded and level gradient walking. Backpack loads increased the medio-lateral stability of COM and uphill walking decreased stability of vertical COM motion and increased stride time variability. However, there was no difference between the two load carriage systems for any variable. The reduced stability of muscle activations in loaded and uphill conditions indicates an impaired ability of the neuromuscular control systems to accommodate perturbations in these conditions which may have implications on the operational performance of military personnel. However, improved medio-lateral stability in backpack conditions may indicate that participants were able to compensate for the loads used in this study, despite the decreased vertical stability and increased stride time variability evident in uphill walking. This study did not find differences between load carriage systems however, specific load carriage system effects may be elicited by greater load carriage masses.

Design of an Intent Recognition System for Dynamic, Rapid Motions in Unstructured Environments

Abstract In this study, we developed an offline, hierarchical intent recognition system for inferring the timing and direction of motion intent of a human operator when operating in an unstructured environment. There has been an increasing demand for robot agents to assist in these dynamic, rapid motions that are constantly evolving and require quick, accurate estimation of a user’s direction of travel. An experiment was conducted in a motion capture space with six subjects performing threat evasion in eight directions, and their mechanical and neuromuscular signals were recorded for use in our intent recognition system (XGBoost). Investigated against current, analytical methods, our system demonstrated superior performance with quicker direction of travel estimation occurring 140 ms earlier in the movement and a 11.6 deg reduction of error. The results showed that we could also predict the start of the movement 100 ms prior to the actual, thus allowing any physical systems to start up. Our direction estimation had an optimal performance of 8.8 deg, or 2.4% of the 360 deg range of travel, using three-axis kinetic data. The performance of other sensors and their combinations indicate that there are additional possibilities to obtain low estimation error. These findings are promising as they can be used to inform the design of a wearable robot aimed at assisting users in dynamic motions, while in environments with oncoming threats.

Voiceless Speech Recognition System

Speech recognition is a very important part of the human-machine interaction system. The commonly used methods for speech recognition are invariantly based on the analysis of speech signals. These signals might not be always viable especially in noisy environments or in environments where discretion is of utmost importance. The “Thought Predictor system” allows the user to converse silently with a computing device. It thus enables the user to communicate with devices and AI assistants in a voiceless and discrete manner. The user’s intention to speak and internal speech is characterized by neuromuscular signals by the speech articulating muscles that are captured by this system to reconstruct this speech.

Contrast of multi‐resolution analysis approach to transhumeral phantom motion decoding

In signal processing, multiresolution decomposition techniques allow for the separation of an acquired signal into sub levels, where the optimal level within the signal minimises redundancy, uncertainties, and contains the information required for the characterisation of the sensed phenomena. In the area of physiological signal processing for prosthesis control, scenarios where a signal decomposition analysis are required: the wavelet decomposition (WD) has been seen to be the favoured time-frequency approach for the decomposition of non-stationary signals. From a research perspective, the WD in certain cases has allowed for a more accurate motion intent decoding process following feature extraction and classification. Despite this, there is yet to be a widespread adaptation of the WD in a practical setting due to perceived computational complexity. Here, for neuromuscular (electromyography) and brainwave (electroencephalography) signals acquired from a transhumeral amputee, a computationally efficient time domain signal decomposition method based on a series of heuristics was applied to process the acquired signals before feature extraction. The results showed an improvement in motion intent decoding prowess for the proposed time-domain-based signal decomposition across four different classifiers for both the neuromuscular and brain wave signals when compared to the WD and the raw signal.

Phantom motion intent decoding for transhumeral prosthesis control with fused neuromuscular and brain wave signals

In recent years, the electroencephalography (EEG) brain–computer interface (BCI) has been researched in the area of upper-limb prosthesis control due to the promise of being able to record neurological signals which follow activation patterns in the cortex directly from the brain with non-invasive electrodes. This is seen as a way of bypassing the limitation posed by acquiring neuromuscular signals predominantly with electromyography (EMG) directly from the stump, which possesses residual limb anatomy post-amputation. In this study, the sequential forward selection algorithm to form a 10-optimal-channel representation, alongside an extended signal feature vector was applied, to investigate the motion intent decoding performance of EMG-only, EEG-only, and a fused EMG–EEG sensing configuration for four transhumeral amputees with varying stump lengths. The results showed a considerable improvement for the EMG-only configuration with the advanced feature vector, but only a small increase for the EEG-only, and thus a marginal improvement when information from both signals was fused together. This is likely due to the EEG requiring a greater number of channels spread across the skull to provide a reliable intent decoding. Further work will now involve optimisation studies to find a greater representation of electrode representation and parsimony, to minimise the number of channels while boosting motion intent decoding accuracy.

Towards optimizing electrode configurations for silent speech recognition based on high-density surface electromyography

Objective. Silent speech recognition (SSR) based on surface electromyography (sEMG) is an attractive non-acoustic modality of human-machine interfaces that convert the neuromuscular electrophysiological signals into computer-readable textual messages. The speaking process involves complex neuromuscular activities spanning a large area over the facial and neck muscles, thus the locations of the sEMG electrodes considerably affected the performance of the SSR system. However, most of the previous studies used only a quite limited number of electrodes that were placed empirically without prior quantitative analysis, resulting in uncertainty and unreliability of the SSR outcomes. Approach. In this study, the technique of high-density sEMG was proposed to provide a full representation of the articulatory muscle activities so that the optimal electrode configuration for SSR could be systemically explored. A total of 120 closely spaced electrodes were placed on the facial and neck muscles to collect the high-density sEMG signals for classifying ten digits (0–9) silently spoken in both English and Chinese. The sequential forward selection algorithm was adopted to explore the optimal electrodes configurations. Main Results. The results showed that the classification accuracy increased rapidly and became saturated quickly when the number of selected electrodes increased from 1 to 120. Using only ten optimal electrodes could achieve a classification accuracy of 86% for English and 94% for Chinese, whereas as many as 40 non-optimized electrodes were required to obtain comparable accuracies. Also, the optimally selected electrodes seemed to be mostly distributed on the neck instead of the facial region, and more electrodes were required for English recognition to achieve the same accuracy. Significance. The findings of this study can provide useful guidelines about electrode placement for developing a clinically feasible SSR system and implementing a promising approach of human-machine interface, especially for patients with speaking difficulties.

Interlimb Neuromuscular Responses During Fatiguing, Bilateral, Leg Extension Exercise at a Moderate Versus High Load.

This study determined the load- and limb-dependent neuromuscular responses to fatiguing, bilateral, leg extension exercise performed at a moderate (50% one-repetition maximum [1RM]) and high load (80% 1RM). Twelve subjects completed 1RM testing for the bilateral leg extension, followed by repetitions to failure at 50% and 80% 1RM, on separate days. During all visits, the electromyographic (EMG) and mechanomyographic (MMG), amplitude (AMP) and mean power frequency (MPF) signals were recorded from the vastus lateralis of both limbs. There were no limb-dependent responses for any of the neuromuscular signals and no load-dependent responses for EMG AMP, MMG AMP, or MMG MPF (p = .301-.757), but there were main effects for time that indicated increases in EMG and MMG AMP and decreases in MMG MPF. There was a load-dependent decrease in EMG MPF over time (p = .032) that suggested variability in the mechanism responsible for metabolite accumulation at moderate versus high loads. These findings suggested that common drive from the central nervous system was used to modulate force during bilateral leg extension performed at moderate and high loads.

Evaluation of Non-Invasive Ankle Joint Effort Prediction Methods for Use in Neurorehabilitation Using Electromyography and Ultrasound Imaging.

Reliable measurement of voluntary human effort is essential for effective and safe interaction between the wearer and an assistive robot. Existing voluntary effort prediction methods that use surface electromyography (sEMG) are susceptible to prediction inaccuracies due to non-selectivity in measuring muscle responses. This technical challenge motivates an investigation into alternative non-invasive effort prediction methods that directly visualize the muscle response and improve effort prediction accuracy. The paper is a comparative study of ultrasound imaging (US)-derived neuromuscular signals and sEMG signals for their use in predicting isometric ankle dorsiflexion moment. Furthermore, the study evaluates the prediction accuracy of model-based and model-free voluntary effort prediction approaches that use these signals. The study evaluates sEMG signals and three US imaging-derived signals: pennation angle, muscle fascicle length, and echogenicity and three voluntary effort prediction methods: linear regression (LR), feedforward neural network (FFNN), and Hill-type neuromuscular model (HNM). In all the prediction methods, pennation angle and fascicle length significantly improve the prediction accuracy of dorsiflexion moment, when compared to echogenicity. Also, compared to LR, both FFNN and HNM improve dorsiflexion moment prediction accuracy. The findings indicate FFNN or HNM approach and using pennation angle or fascicle length predict human ankle movement intent with higher accuracy. The accurate ankle effort prediction will pave the path to safe and reliable robotic assistance in patients with drop foot.

Silent Speech Decoding Using Spectrogram Features Based on Neuromuscular Activities.

Silent speech decoding is a novel application of the Brain–Computer Interface (BCI) based on articulatory neuromuscular activities, reducing difficulties in data acquirement and processing. In this paper, spatial features and decoders that can be used to recognize the neuromuscular signals are investigated. Surface electromyography (sEMG) data are recorded from human subjects in mimed speech situations. Specifically, we propose to utilize transfer learning and deep learning methods by transforming the sEMG data into spectrograms that contain abundant information in time and frequency domains and are regarded as channel-interactive. For transfer learning, a pre-trained model of Xception on the large image dataset is used for feature generation. Three deep learning methods, Multi-Layer Perception, Convolutional Neural Network and bidirectional Long Short-Term Memory, are then trained using the extracted features and evaluated for recognizing the articulatory muscles’ movements in our word set. The proposed decoders successfully recognized the silent speech and bidirectional Long Short-Term Memory achieved the best accuracy of 90%, outperforming the other two algorithms. Experimental results demonstrate the validity of spectrogram features and deep learning algorithms.