Finger Joint Angles Research Articles

A multimodal multistream multilevel fusion network for finger joint angle estimation with hybrid sEMG and FMG sensing

Finger joint angle (FJA) estimation, as a dynamic and fine-grained decoding mode, can support intuitive and natural human–machine interactions. This study is pioneering work dedicated to achieving accurate and clinically friendly FJA estimation. In this context, a novel surface electromyography and force myography-based multimodal multistream multilevel fusion network (MMMFNet) is proposed. MMMFNet utilises gate-recurrent units (GRU) networks to capture spatial dependencies among features and adopts a multilevel fusion strategy to extract more comprehensive representations from information sources than a single-level fusion. A cross-stream interaction (CSI) block is added between the GRU networks to focus on important information in the prediction and optimise the learning process and representation capability by facilitating information interaction among branch networks. The experimental results suggest that the MMMFNet has higher estimation accuracy (RMSE: 6.225±0.276 vs. 6.953±0.282, R2: 0.879±0.019 vs 0.846±0.021), shorter training time (4.146±0.074 min vs. 16.536±0.430 min), and lower computational complexity (5.4×106 FLOPs vs 8.8×106 FLOPs) than state-of-the-art fusion methods. Moreover, MMMFNet, which quantifies feature importance using attention weights, is the first explainable multimodal model for hand movement intention decoding. In summary, compared with state-of-the-art methods, MMMFNet can not only estimate FJAs more accurately but is also more clinically friendly.

Design and Verification of Piano Playing Assisted Hand Exoskeleton Robot.

Finger technique is a crucial aspect of piano learning, and hand exoskeleton mechanisms effectively assist novice piano players in maintaining correct finger technique consistently. Addressing current issues with exoskeleton robots, such as the inability to provide continuous correction of finger technique and their considerable weight, a novel hand exoskeleton robot has been developed to enhance finger technique through continuous correction and reduced weight. Initial data are gathered using finger joint angle sensors to analyze movements during piano playing, focusing on the trajectory and angular velocity of key strikes. This analysis informs the design of a 6-bar double-closed-loop mechanism with an end equivalent sliding pair, using analytical methods to establish the relationship between motor extension and input rod rotation. Simulation studies assess the exoskeleton's motion space and dynamics, confirming its capability to meet structural and functional demands for accurate key striking. Prototype testing validates the exoskeleton's ability to maintain correct finger positioning and mimic natural strike speeds, thus improving playing technique while ensuring comfort and safety.

Bioinspired, highly sensitive interlocked flexible textile pressure sensor based on multilayer SWCNTs/PVP/rGO dendritic for gesture recognition

The development of wearable gesture recognition systems, capable of converting intricate human gestures into electric signals and graphical representations, poses significant challenges for future advancements in health monitoring and human-machine interfaces. In response to this demand, we present an approach centered on a cell-inspired, multiscale hierarchical structural piezoresistive pressure sensor designed for gesture recognition applications. The breathable, hierarchically interlocked textile pressure sensor (PFPA), featuring a SWCNTs/polyvinylpyrrolidone (PVP)/reduced graphene oxide (rGO) dendritic structure combined with a PANI/PVA cylindrical array, exhibits remarkable properties. The dendritic electrodes, composed of SWCNTs and PVP in a 1:2 ratio, maintain optimal regularity. Meanwhile, the cylindrical PANI/PVA array is meticulously engineered with a radius of 200 μm, a height of 200 μm, and an inter-cylindrical spacing of 150 μm. The PFPA demonstrates an unprecedented ultrahigh sensitivity of 80.8 kPa−1 across a pressure range of 0–5 kPa, coupled with a rapid response time of 150 ms and minimal hysteresis of 16.1 %. The PFPA also demonstrates outstanding repeatability, with no significant performance degradation observed after 4000 cycles. By correlating the bending angles of finger joints with resistance changes, we successfully integrated these sensors into a glove, enabling precise gesture recognition. The multilayer SWCNTs/PVP/rGO dendritic and PANI/PVA cylindrical array pressure sensor exhibits superior sensing performance and accurate posture capture. This demonstrates its significant potential for applications in wearable electronics and intelligent devices, marking a substantial advancement in the field.

Mechanical Interaction Between the Tendons of the Extrinsic Finger Flexors

ABSTRACTHumans possess an incredible capacity for dexterity, but independent finger control is limited. One factor constraining finger independence is the connections between the tendons of the extrinsic finger muscles. The aim of this study was to assess to what extent the linkages between the distal tendons of flexor digitorum profundus (FDP) and those of the flexor digitorum superficialis (FDS) constrain finger independence. Experiments on human fresh frozen cadaveric upper extremities (n = 6) were performed. First, one finger (target) was flexed whereas the other (nontarget) fingers were held in a nearly extended position. The change (Δ) in total flexion angle (∑Θ; i.e., the sum of angles of the different finger joints) of the target finger from the ∑Θ corresponding to the extended position at the start of the movement until the ∑Θ corresponding to the onset of force exertion at the nontarget fingers was assessed. Second, the distribution of force across the four fingers upon loading the tendon of the target finger was assessed for two finger positions (extended, 90° flexion of metacarpal phalangeal joint). For both muscles and for all fingers, the range of independent movement was small (<7°). Δ∑Θ at force onset was lowest for fingers immediately adjacent to the target finger and highest for more distant fingers. For both muscles and for all fingers, some of the target finger force (<14% for FDP, <2% for FDS) was distributed to the nontarget fingers, which increased (up to 58%) only for FDP in response to target finger flexion. We conclude that mechanical connections between the FDP and FDS tendons constrain finger independence. Such constraints become apparent when moving one finger relative to the other fingers.

Design, Engineering Analysis, and Fabrication of a Prototype Electromechanical Finger Fixator Control System

This paper investigates a new external fixator system for treating finger contractures. This innovative system is electric and user-friendly, providing precise readings of the angles of diseased finger joints. Accurate measurements of the angles of the diseased finger joints can assist therapists and patients during rehabilitation procedures. The fixator consists of ten parts assembled using polyether-ether-ketone (PEEK) for the plastic components and stainless steel for the metal components. The design model was engineered, drawn, and analyzed using SolidWorks Computer-Aided Design software. The verification process utilizes finite element analysis to demonstrate that the maximum stress was lower than the yield strength of the chosen materials. As a result, the new device design is robust and stable enough to withstand the anticipated loading conditions of human fingers. Subsequently, a prototype was fabricated using advanced additive manufacturing technology, specifically fused deposition modeling (FDM). The proposed fixator is simple to control, reliable, easy to use, and reproducible. It enables device users to exercise their finger joints throughout the day without requiring the assistance of specialists.

Evaluating the Reliability of Three-dimensional Models Constructed Photogrammetry Software 3DF Zephyr by Measuring Joint Angles of Fingers: A Comparison to a Conventional Goniometer

Evaluating the Reliability of Three-dimensional Models Constructed Photogrammetry Software 3DF Zephyr by Measuring Joint Angles of Fingers: A Comparison to a Conventional Goniometer

Is a smartphone application as accurate as a traditional goniometer for assessing finger joint angles in Dupuytren’s disease?

ObjectivesSmartphone applications are available using the smartphone accelerometer to measure angles. This study compared finger joint angle measurements using a traditional goniometer (TG) versus a smartphone application (SPA) in a group of patients with Dupuytren’s disease, to determine if they were equally accurate. This was a clinical measurement case series. A group of patients with Dupuytren’s disease were invited to participate. Material and methodsThe Apple iPhone 7 “Measure” application was used. Participants were asked to place their hand as flat as possible on the table in a clinic room. Each joint of the ring and little fingers was measured using the TG and the SPA. Power calculation by paired t-test determined a sample size of 25. Results28 hands were measured. The Pearson correlation coefficients for the various finger joints were: metacarpophalangeal joint (MCPJ), 0.989; proximal interphalangeal joint (PIPJ), 0.997; and distal interphalangeal joint (DIPJ), 0.977. The root mean square errors (RMSE) for the various joints were: MCPJ, 3.190; PIPJ, 2.019; and DIPJ, 1.897. Bland-Altman analysis showed the two methods to be in agreement. ConclusionThe SPA was consistent, reliable and in agreement with a TG for assessing ring and little finger joint angles. The SPA could be used by healthcare professionals instead of a TG.

Estimating finger joint angles by surface EMG signal using feature extraction and transformer-based deep learning model

Human-machine interfaces frequently use electromyography (EMG) signals. Based on previous work, feature extraction has a great deal of influence on the performance of EMG pattern recognition. Furthermore, the Deep Learning method is supposed to increase performance and not depend on feature engineering. However, directly processing raw signals will require a higher computation rate. This study proposed a new method that combines feature extraction and Deep Learning to address those issues while improving performance, reducing architecture size, and producing a more representative output. The proposed architecture employs the Transformer model as the backbone to get the correlation between elements and focus on the important information for estimating the flexion-extension of finger joint angles. This study uses experiment three of the NinaPro (Non-Invasive Adaptive Hand Prosthetics) DB5 dataset. Each experiment produces 16 Surface EMG data streams from two Myo Armbands devices representing 22 finger joint angles as output. This study compares the windowing process, feature extraction, execution time, and results with previous studies. The results show that the proposed model outperforms the previous study, from 0.957 in the previous study to become 0.970 in this study for the R-Square score. This result is obtained using 100 data points for the windowing process and Median Frequency for the best feature extraction method.

Lightweight Soft Robotic Glove with Whole-Hand Finger Motion Tracking for Hand Rehabilitation in Virtual Reality.

Soft robotic gloves have attracted significant interest in hand rehabilitation in the past decade. However, current solutions are still heavy and lack finger-state monitoring and versatile treatment options. To address this, we present a lightweight soft robotic glove actuated by twisted string actuators (TSA) that provides whole-hand finger motion tracking. We have developed a virtual reality environment for hand rehabilitation training, allowing users to interact with various virtual objects. Fifteen small inertial measurement units are placed on the glove to predict finger joint angles and track whole-hand finger motion. We performed TSA experiments to identify design and control rules, by understanding how their response varies with input load and voltages. Grasping experiments were conducted to determine the grasping force and range of motion. Finally, we showcase an application of the rehabilitation glove in a Unity-based VR interface, which can actuate the operator's fingers to grasp different virtual objects.

Master-Slave Control System for Virtual-Physical Interactions Using Hands.

Among the existing technologies for hand protection, master-slave control technology has been extensively researched and applied within the field of safety engineering to mitigate the occurrence of safety incidents. However, it has been identified through research that traditional master-slave control technologies no longer meet current production and lifestyle needs, and they have even begun to pose new safety risks. To resolve the safety risks exposed by traditional master-slave control, this research fuses master-slave control technology for hands with virtual reality technology, and the design of a master-slave control system for hands based on virtual reality technology is investigated. This study aims to realize the design of a master-slave control system for virtual-physical interactions using hands that captures the position, orientation, and finger joint angles of the user's hand in real time and synchronizes the motion of the slave interactive device with that of a virtual hand. With amplitude limiting, jitter elimination, and a complementary filtering algorithm, the original motion data collected by the designed glove are turned into a Kalman-filtering-algorithm-based driving database, which drives the synchronous interaction of the virtual hand and a mechanical hand. As for the experimental results, the output data for the roll, pitch, and yaw were in the stable ranges of -0.1° to 0.1°, -0.15° to 0.15°, and -0.15° to 0.15°, respectively, which met the accuracy requirements for the system's operation under different conditions. More importantly, these data prove that, in terms of accuracy and denoising, the data-processing algorithm was relatively compatible with the hardware platform of the system. Based on the algorithm for the virtual-physical interaction model, the authors introduced the concept of an auxiliary hand into the research, put forward an algorithmic process and a judgement condition for the stable grasp of the virtual hand's, and solved a model-penetrating problem while enhancing the immersive experience during virtual-physical interactions. In an interactive experiment, a dynamic accuracy test was run on the system. As shown by the experimental data and the interactive effect, the system was satisfactorily stable and interactive.

Action-Conditioned Generation of Bimanual Object Manipulation Sequences

The generation of bimanual object manipulation sequences given a semantic action label has broad applications in collaborative robots or augmented reality. This relatively new problem differs from existing works that generate whole-body motions without any object interaction as it now requires the model to additionally learn the spatio-temporal relationship that exists between the human joints and object motion given said label. To tackle this task, we leverage the varying degree each muscle or joint is involved during object manipulation. For instance, the wrists act as the prime movers for the objects while the finger joints are angled to provide a firm grip. The remaining body joints are the least involved in that they are positioned as naturally and comfortably as possible. We thus design an architecture that comprises 3 main components: (i) a graph recurrent network that generates the wrist and object motion, (ii) an attention-based recurrent network that estimates the required finger joint angles given the graph configuration, and (iii) a recurrent network that reconstructs the body pose given the locations of the wrist. We evaluate our approach on the KIT Motion Capture and KIT RGBD Bimanual Manipulation datasets and show improvements over a simplified approach that treats the entire body as a single entity, and existing whole-body-only methods.

Influence of Force-Length Relationship and Task-Specific Constraints on Finger Force-Generating Capacities.

Grip strength loss in extended and flexed wrist postures has been explained by reduced force-generating capacities of extrinsic finger flexor resulting from non-optimal length, owing to the force-length relationship. Recent works suggested that other muscles, especially wrist extensors, participate in this grip strength loss. The objective of this study was to clarify the role of the force-length relationship in finger force production. 18 participants performed maximal isometric finger force production during pinch grip (Pinch) and four-finger pressing (Press) tasks in four different wrist postures (extended, flexed, neutral, spontaneous). The maximum finger force (MFF), finger and wrist joint angles, as well as activation of four muscles were determined using dynamometry, motion capture, and electromyography. The force and length of the four muscles were estimated from joint angles and muscle activation using a musculoskeletal model. MFF decreased for flexed wrist during Pinch but remained stable across wrist postures during Press. The results suggested that the loss of pinch grip force in deviated wrist posture is partially related to force-length relationship of finger extensors. In opposition, MFF during Press was not influenced by the modulation of muscle capacities but was probably first limited by mechanical and neural factors related to finger interdependence.

Concurrent and Continuous Prediction of Finger Kinetics and Kinematics via Motoneuron Activities.

Robust neural decoding of intended motor output is crucial to enable intuitive control of assistive devices, such as robotic hands, to perform daily tasks. Few existing neural decoders can predict kinetic and kinematic variables simultaneously. The current study developed a continuous neural decoding approach that can concurrently predict fingertip forces and joint angles of multiple fingers. We obtained motoneuron firing activities by decomposing high-density electromyogram (HD EMG) signals of the extrinsic finger muscles. The identified motoneurons were first grouped and then refined specific to each finger (index or middle) and task (finger force and dynamic movement) combination. The refined motoneuron groups (separate matrix) were then applied directly to new EMG data in real-time involving both finger force and dynamic movement tasks produced by both fingers. EMG-amplitude-based prediction was also performed as a comparison. We found that the newly developed decoding approach outperformed the EMG-amplitude method for both finger force and joint angle estimations with a lower prediction error (Force: 3.47±0.43 vs 6.64±0.69% MVC, Joint Angle: 5.40±0.50° vs 12.8±0.65°) and a higher correlation (Force: 0.75±0.02 vs 0.66±0.05, Joint Angle: 0.94±0.01 vs 0.5±0.05) between the estimated and recorded motor output. The performance was also consistent for both fingers. The developed neural decoding algorithm allowed us to accurately and concurrently predict finger forces and joint angles of multiple fingers in real-time. Our approach can enable intuitive interactions with assistive robotic hands, and allow the performance of dexterous hand skills involving both force control tasks and dynamic movement control tasks.

Measuring Arm and Hand Joint Kinematics to Estimate Impairment During a Functional Reach and Grasp Task after Stroke

Background Current approaches to characterizing deficits in upper limb movements after stroke typically focus either on changes in a functional measure, for example, how well a patient can complete a task, or changes in impairment, for example, isolated measurements of joint range of motion. However, there can be notable dissociations between static measures of impairment versus those of function. Objective We develop a method to measure upper limb joint angles during performance of a functional task and use measurements to characterize joint impairment in the context of a functional task. Methods We developed a sensorized glove that can precisely measure select finger, hand, and arm joints while participants complete a functional reach-to-grasp task involving manipulation of a sensorized object. Results We first characterized the accuracy and precision of the glove’s joint angle measurements. We then measured joint angles in neurologically intact participants (n = 4 participants, 8 limbs) to define the expected distribution of joint angle variation during task execution. These distributions were used to normalize finger, hand, and arm joint angles in stroke participants (n = 6) as they performed the task. We present a participant-specific visualization of functional joint angle variance which illustrated that stroke participants with nearly identical clinical scores exhibited unique patterns of joint angle variation. Conclusions Overall, measuring individual joint angles in the context of a functional task may inform whether changes in functional scores over recovery or rehabilitation are driven by changes in impairment or the development of compensatory strategies, and provide a quantified path toward personalized rehabilitative therapy.

Continuous motion finger joint angle estimation utilizing hybrid sEMG-FMG modality driven transformer-based deep learning model

Continuous motion joint angle estimation plays an important role in human–machine interaction (HMI). However, it is still a challenge to estimate continuous motion finger joint angles (CMFJA) accurately. To improve CMFJA estimation accuracy, we used hybrid surface electromyography-force myography (sEMG-FMG) modality as the decoding scheme since combining two sensing modalities could potentially compensate and correct for single sensing modality, and proposed a biosignals driven convolution neural networks (CNN) and Transformer model (BioCNN-T) to estimate CMFJA motivated by the potential of the Transformer architecture, which extracts local features through CNN, and captures dependencies between global features through Transformer encoder. The metacarpophalangeal joint angles of 6 hand movements commonly used in daily life were selected as estimation objects. The experimental results show that CMFJA estimation by hybrid sEMG-FMG modality can achieve higher accuracy than single modality. And we found that the addition of convolution block before Transformer encoder plays a positive role in improving estimation accuracy and pixel-level fusion is the most suitable among the information fusion strategies appearing in the paper. What’s more, compare with the CNN and long short-term memory neural networks (CNN-LSTM), the BioCNN-T has higher accuracy and less computing cost. (Average normalized root mean square error, 0.0456 ± 0.0030 versus 0.0539 ± 0.0035; Inference time, 9.12 × 10-6 s versus 3.20 × 10-4 s). To our knowledge, our work is a pioneer in the use of hybrid sEMG-FMG modality and Transformer architecture to estimate CMFJA. The Implications of it have promising potential in flexible and fine-grained HMI.

A Transfer Learning based Cross-subject Generic Model for Continuous Estimation of Finger Joint Angles from a New User.

Continuous estimation of finger joints based on surface electromyography (sEMG) has attracted much attention in the field of human-machine interface (HMI). A couple of deep learning models were proposed to estimate the finger joint angles for specific subject. When applied onto a new subject, however, the performance of the subject-specific model would degrade significantly due to the inter-subject differences. Therefore, a novel cross-subject generic (CSG) model was proposed in this study to estimate continuous kinematics of finger joints for new users. Firstly, a multi-subject model based on the LSTA-Conv network was built by using sEMG and finger joint angles data from multiple subjects. Then, the subjects adversarial knowledge (SAK) transfer learning strategy was adopted to calibrate the multi-subject model with the training data from a new user. With the updated model parameters and the testing data from the new user, multiple finger joint angles could be estimated afterwards. The overall performance of the CSG model for new users was validated on three public datasets from Ninapro. The results showed that the newly proposed CSG model significantly outperformed five subject-specific models and two transfer learning models in terms of Pearson correlation coefficient, root mean square error, and coefficient of determination. Comparison analysis showed that both the long short-term feature aggregation (LSTA) module and the SAK transfer learning strategy contributed to the CSG model. Moreover, increasing number of subjects in training set improved the generalization capability of the CSG model. The novel CSG model would facilitate the application of robotic hand control and other HMI settings.

Synchronization of kinetic and kinematic hand tasks with electrocorticography and cortical stimulation during awake craniotomies.

Awake craniotomies provide unique and invaluable scientific opportunities for neurophysiological experimentation in consenting human subjects. While such experimentation carries a long history, rigorous reporting of methodologies focusing on synchronizing data across multiple platforms is not universally reported and often not translatable to across operating rooms, facilities, or behavioral tasks. Therefore, here we detail an intraoperative data synchronization methodology designed to work across multiple commercially available platforms to collect behavioral and surgical field videos, electrocorticography, brain stimulation timing, continuous finger joint angles, and continuous finger force production. Our technique was developed to be nonobstructive to operating room (OR) staff and generalizable to a variety of hand-based tasks. We hope that the detailed reporting of our methods will support the scientific rigor and reproducibility of future studies, as well as aid other groups interested in performing related experiments.

Soft-Sensor System for Grasp Type Recognition in Underactuated Hand Prostheses.

This paper presents the development of an intelligent soft-sensor system to add haptic perception to the underactuated hand prosthesis PrHand. Two sensors based on optical fiber were constructed, one for finger joint angles and the other for fingertips' contact force. Three sensor fabrications were tested for the angle sensor by axially rotating the sensors in four positions. The configuration with the most similar response in the four rotations was chosen. The chosen sensors presented a polynomial response with R2 higher than 92%. The tactile force sensors tracked the force made over the objects. Almost all sensors presented a polynomial response with R2 higher than 94%. The system monitored the prosthesis activity by recognizing grasp types. Six machine learning algorithms were tested: linear regression, k-nearest neighbor, support vector machine, decision tree, k-means clustering, and hierarchical clustering. To validate the algorithms, a k-fold test was used with a k = 10, and the accuracy result for k-nearest neighbor was 98.5%, while that for decision tree was 93.3%, enabling the classification of the eight grip types.

A Practical System for 3-D Hand Pose Tracking Using EMG Wearables With Applications to Prosthetics and User Interfaces

Ubiquitous finger motion tracking enables a number of exciting applications in augmented reality, sports analytics, rehabilitation-healthcare, haptics, etc. This article presents NeuroPose, a system that shows the feasibility of 3-D finger motion tracking using a platform of wearable electromyography (EMG) sensors. EMG sensors can sense electrical potential from muscles due to finger activation, thus offering rich information for fine-grained finger motion sensing. However, converting the sensor information to 3-D finger poses is non trivial since signals from multiple fingers superimpose at the sensor in complex patterns. Toward solving this problem, NeuroPose fuses information from anatomical constraints of finger motion with machine learning architectures on recurrent neural networks (RNNs), encoder–decoder networks, and ResNets to extract 3-D finger motion from noisy EMG data. The generated motion pattern is temporally smooth as well as anatomically consistent. Furthermore, a transfer learning algorithm is leveraged to adapt a pretrained model on one user to a new user with minimal training overhead. A systematic study with 12 users demonstrates a median error of 6.24° and a 90%-ile error of 18.33° in tracking 3-D finger joint angles. The accuracy is robust to natural variation in sensor mounting positions as well as changes in wrist positions of the user. In addition, this article validates the feasibility of mirrored bilateral training approach with applications in prosthetic devices. Finally, NeuroPose is comprehensively evaluated on both low-end and recent smartphones with a processing latency of 0.019 s and low energy overhead.

Relative motion between the flexor digitorum superficialis tendon and subsynovial connective tissue is time dependent.

The subsynovial connective tissue is an integral component of flexor tendon gliding in the carpal tunnel, which is strained during longitudinal tendon displacement. We tested the effects of repetition frequency and finger load on flexor tendon function throughout active finger movement. Eleven participants performed metacarpophalangeal joint flexion/extension of the long finger cyclically at three repetition frequencies (0.75, 1.00, 1.25 Hz) and two finger loads (3.5, 7 N). Relative displacement between the flexor digitorum superficialis tendon and subsynovial connective tissue was assessed as the shear-strain index with color ultrasound throughout the entire time history of finger flexion and extension. In addition, long finger joint angles were measured with electrogoniometry while flexor digitorum superficialis and extensor digitorum muscle activities were measured with fine-wire electromyography to characterize the finger movements. The shear-strain index increased with greater finger flexion (p = 0.001), representing higher relative displacement between tendon and subsynovial connective tissue; however, no changes were observed throughout finger extension. The shear-strain index also increased with higher repetition frequencies (p = 0.013) and finger loads (p = 0.029), further modulating time-dependent effects during finger flexion versus extension. Using ultrasound, we characterized the time-dependent response of the shear-strain index, in vivo, providing valuable data on flexor tendon function during active finger movement. Our results infer greater subsynovial connective tissue strain and shear during repetitive and forceful finger movements. Future research characterizing time-dependent effects in carpal tunnel syndrome patients may further elucidate the relations between subsynovial connective tissue function, damage, and carpal tunnel syndrome.