Human Arm Movements Research Articles

Comparison Between InterCriteria and Correlation Analyses over sEMG Data from Arm Movements in the Horizontal Plane

InterCriteria analysis (ICrA) and two kinds of correlation analyses, Pearson (PCA) and Spearman (SCA), were applied to surface electromyography (sEMG) signals obtained from human arm movements in the horizontal plane. Ten healthy participants performed ten movements, eight of which were cyclic. Each cyclic movement (CM) consisted of flexion and extension phases with equal duration (10 s, 6 s, 2 s, and 1 s) and two 5 s rest poses between them. The CMs were performed with and without an added load of 0.5 kg on the wrists of the participants. The sEMG signals from six different muscles or separate muscle heads (m. deltoideus pars clavicularis, m. deltoideus pars spinata, m. brachialis, m. anconeus, m. biceps brachii, and m. triceps brachii long head) were recorded and used to compare the results of the ICrA, PCA, and SCA. All three methods found identical consonance pairs for the flexion and extension CM phases. Additionally, PCA detected two more consonance pairs in the extension phases. In this investigation, ICrA, PCA, and SCA were proven to be reliable tools when applied separately or in combination for sEMG data. These three methods are appropriate for researching arm movements in the horizontal plane and experimental protocol revision.

Application of Artificial Neuromolecular System in Robotic Arm Control to Assist Progressive Rehabilitation for Upper Extremity Stroke Patients

Freedom of movement of the hands is the most desired hope of stroke patients. However, stroke recovery is a long, long road for many patients. If artificial intelligence can assist human arm movement, the possibility of stroke patients returning to normal hand movement might be significantly increased. This study uses the artificial neuromolecular system (ANM system) developed in our laboratory as the core of motion control, in an attempt to learn to control the mechanical arm to produce actions similar to human rehabilitation training and the transition between different activities. This research adopts two methods. The first is hypothetical exploration, the so-called “artificial world” simulation method. The detailed approach uses the V-REP (Virtual Robot Experimentation Platform) to conduct different experimental runs to capture relevant data. Our policy is to establish an action database systematically to a certain extent. From these data, we use the ANM system with self-organization and learning capabilities to develop the relationship between these actions and establish the possibility of conversion between different activities. The second method of this study is to use the data from a hospital in Toronto, Canada. Our experimental results show that the ANM system can continuously learn for problem-solving. In addition, our three experimental results of adaptive learning, transfer learning, and cross-task learning further confirm that the ANM system can use previously learned systems to complete the delivered tasks through autonomous learning (instead of learning from scratch).

Blind parameter identification of implicit differential equations using the collocation discretization and homotopy optimization methods

In this paper, our objectives are to estimate the moments of inertia and reconstruct the inputs of a two-link pendulum that models a human arm. A blind parameter identification routine to determine the inertia properties of human limbs without input data based on a combination of collocation discretization and homotopy optimization is suggested. Without the input data, inertia parameters are structurally unidentifiable. Complementary equations in terms of the ratio of inertia parameters in the cost function and the rate of change of the inputs in the constraints are introduced to make the problem structurally identifiable. Numerous simulations are performed to validate our approach. Experiments to record human upper arm and forearm oscillatory movements were also performed, and moment of inertia terms were evaluated. The significance of the proposed method is that the method can be used to evaluate the moments of inertia of human body segments only from the experimental kinematic data. The advantages of the method are: numerical integration of dynamic and sensitivity equations is avoided and the record of the inputs to the system is not needed.

Mapping Method of Human Arm Motion Based on Surface Electromyography Signals.

This paper investigates a method for precise mapping of human arm movements using sEMG signals. A multi-channel approach captures the sEMG signals, which, combined with the accurately calculated joint angles from an Inertial Measurement Unit, allows for action recognition and mapping through deep learning algorithms. Firstly, signal acquisition and processing were carried out, which involved acquiring data from various movements (hand gestures, single-degree-of-freedom joint movements, and continuous joint actions) and sensor placement. Then, interference signals were filtered out through filters, and the signals were preprocessed using normalization and moving averages to obtain sEMG signals with obvious features. Additionally, this paper constructs a hybrid network model, combining Convolutional Neural Networks and Artificial Neural Networks, and employs a multi-feature fusion algorithm to enhance the accuracy of gesture recognition. Furthermore, a nonlinear fitting between sEMG signals and joint angles was established based on a backpropagation neural network, incorporating momentum term and adaptive learning rate adjustments. Finally, based on the gesture recognition and joint angle prediction model, prosthetic arm control experiments were conducted, achieving highly accurate arm movement prediction and execution. This paper not only validates the potential application of sEMG signals in the precise control of robotic arms but also lays a solid foundation for the development of more intuitive and responsive prostheses and assistive devices.

“Deep Learning Based Classification of Single-Hand South Indian Sign Language Gestures”

Gesture recognition is a branch of computer science and language technology dedicated to utilizing mathematical algorithms for the analysis of human gestures. Within the realm of non-verbal communication, the pivotal role of human arm movements and gestures remains a focal point. This research introduces advanced multi-stream deep transfer learning models tailored for identifying signs from South Indian languages, specifically Kannada, Tamil, and Telugu. The primary aim is to offer support to individuals encountering speech disorders or disabilities. The key deep transfer learning models utilized include Inception-V3, VGG-16, and ResNet-50, which have been modified and improved to attain heightened classification efficacy. The dataset comprises 35,000 images capturing single-hand gestures. In the realm of models, Inception-V3 exhibits the utmost test accuracy, standing at 91.45%, alongside a validation accuracy of 93.45% when tasked with the classification of single-hand gesture images across thirty-five distinct categories. The importance of this study lies in its prospective utility for creating an automated system that can support and improve the functional capabilities of individuals facing speech disorders or disabilities.

Implementation and Comparison of Wearable Exoskeleton Arm Design with Fuzzy Logic and Machine Learning Control

In this study, a wearable exoskeleton arm was designed and controlled with different control methods to help people with muscle disorders in their arms and support treatment. The developed robot arm was transferred to Simulink software with the Simmechanics application. Two electromyography (EMG) muscle sensors and the ADXL335 position and acceleration sensors attach to the human arm’s biceps and triceps muscle areas. As the human moved the arm, data were obtained from the EMG muscle sensors and the ADXL335 position and acceleration sensor. The received data were first trained with the fuzzy logic algorithm. The same data were then trained with machine learning algorithms in Simulink software. It has been determined that the best result is the quadratic support vector machine (SVM) algorithm. The fuzzy logic algorithm trained with the PID controller block and the received sensor data have been added to the degrees of freedom regions that will enable rotation in the block diagram of the previously exported system. Later, the fuzzy logic block was removed and the machine learning algorithm, the quadratic SVM algorithm, was added. The designed system was operated with two different control systems, and the control algorithm closest to the human arm movement was determined. In addition, each part of the system, whose design was prepared, was removed and assembled separately with a 3D printer. ESP32 microcontroller development board was used to control the system, and it was run in real-time with EMG muscle sensors and position sensors.

“Deep Learning Based Classification of Single-Hand South Indian Sign Language Gestures”

Gesture recognition is a branch of computer science and language technology dedicated to utilizing mathematical algorithms for the analysis of human gestures. Within the realm of non-verbal communication, the pivotal role of human arm movements and gestures remains a focal point. This research introduces advanced multi-stream deep transfer learning models tailored for identifying signs from South Indian languages, specifically Kannada, Tamil, and Telugu. The primary aim is to offer support to individuals encountering speech disorders or disabilities. The key deep transfer learning models utilized include Inception-V3, VGG-16, and ResNet-50, which have been modified and improved to attain heightened classification efficacy. The dataset comprises 35,000 images capturing single-hand gestures. In the realm of models, Inception-V3 exhibits the utmost test accuracy, standing at 91.45%, alongside a validation accuracy of 93.45% when tasked with the classification of single-hand gesture images across thirty-five distinct categories. The importance of this study lies in its prospective utility for creating an automated system that can support and improve the functional capabilities of individuals facing speech disorders or disabilities.

A Dexterous Hand-Arm Teleoperation System Based on Hand Pose Estimation and Active Vision.

Markerless vision-based teleoperation that leverages innovations in computer vision offers the advantages of allowing natural and noninvasive finger motions for multifingered robot hands. However, current pose estimation methods still face inaccuracy issues due to the self-occlusion of the fingers. Herein, we develop a novel vision-based hand-arm teleoperation system that captures the human hands from the best viewpoint and at a suitable distance. This teleoperation system consists of an end-to-end hand pose regression network and a controlled active vision system. The end-to-end pose regression network (Transteleop), combined with an auxiliary reconstruction loss function, captures the human hand through a low-cost depth camera and predicts joint commands of the robot based on the image-to-image translation method. To obtain the optimal observation of the human hand, an active vision system is implemented by a robot arm at the local site that ensures the high accuracy of the proposed neural network. Human arm motions are simultaneously mapped to the slave robot arm under relative control. Quantitative network evaluation and a variety of complex manipulation tasks, for example, tower building, pouring, and multitable cup stacking, demonstrate the practicality and stability of the proposed teleoperation system.

Motion planning framework based on dual-agent DDPG method for dual-arm robots guided by human joint angle constraints.

Reinforcement learning has been widely used in robot motion planning. However, for multi-step complex tasks of dual-arm robots, the trajectory planning method based on reinforcement learning still has some problems, such as ample exploration space, long training time, and uncontrollable training process. Based on the dual-agent depth deterministic strategy gradient (DADDPG) algorithm, this study proposes a motion planning framework constrained by the human joint angle, simultaneously realizing the humanization of learning content and learning style. It quickly plans the coordinated trajectory of dual-arm for complex multi-step tasks. The proposed framework mainly includes two parts: one is the modeling of human joint angle constraints. The joint angle is calculated from the human arm motion data measured by the inertial measurement unit (IMU) by establishing a human-robot dual-arm kinematic mapping model. Then, the joint angle range constraints are extracted from multiple groups of demonstration data and expressed as inequalities. Second, the segmented reward function is designed. The human joint angle constraint guides the exploratory learning process of the reinforcement learning method in the form of step reward. Therefore, the exploration space is reduced, the training speed is accelerated, and the learning process is controllable to a certain extent. The effectiveness of the framework was verified in the gym simulation environment of the Baxter robot's reach-grasp-align task. The results show that in this framework, human experience knowledge has a significant impact on the guidance of learning, and this method can more quickly plan the coordinated trajectory of dual-arm for multi-step tasks.

Estimating Surface EMG Activity of Human Upper Arm Muscles Using InterCriteria Analysis

Electromyography (EMG) is a widely used method for estimating muscle activity and could help in understanding how muscles interact with each other and affect human movement control. To detect muscle interactions during elbow flexion and extension, a recently developed InterCriteria Analysis (ICrA) based on the mathematical formalisms of index matrices and intuitionistic fuzzy sets is applied. ICrA has had numerous implementations in different fields, including biomedicine and quality of life; however, this is the first time the approach has been used for establishing muscle interactions. Six human upper arm large surface muscles or parts of muscles responsible for flexion and extension in shoulder and elbow joints were selected. Surface EMG signals were recorded from four one-joint (pars clavicularis and pars spinata of m. deltoideus [DELcla and DELspi, respectively], m. brachialis [BRA], and m. anconeus [ANC]) and two two-joint (m. biceps brachii [BIC] and m. triceps brachii-caput longum [TRI]) muscles. The outcomes from ten healthy subjects performing flexion and extension movements in the sagittal plane at four speeds with and without additional load are implemented in this study. When ICrA was applied to examine the two different movements, the BIC–BRA muscle interaction was distinguished during flexion. On the other hand, when the ten subjects were observed, four interacting muscle pairs, namely DELcla-DELspi, BIC-TRI, BIC-BRA, and TRI-BRA, were detected. The results obtained after the ICrA application confirmed the expectations that the investigated muscles contribute differently to the human upper arm movements when the flexion and extension velocities are changed, or a load is added.

Efficient and Collision-Free Human–Robot Collaboration Based on Intention and Trajectory Prediction

Human-Robot collaboration (HRC) is an important topic for manufacturing and household robotics. It is very challenging to ensure both efficiency and safety in HRC. This paper presents an HRC pipeline that generates efficient and collision-free robot trajectories based on predictions of the human arm and hand (AH) motions. We train a recurrent neural network for AH trajectory prediction based on observed initial trajectory segments. To increase the accuracy of target estimation at an early stage, the observed and the predicted hand palm trajectory are combined to predict the current AH motion target using Gaussian Mixture Models (GMMs). An optimization-based trajectory generation algorithm is proposed to ensure the safety of the human while collaborating with the robot. The proposed system is validated in a shared-workspace scenario with human pick-and-place motions. The task can be safely and efficiently completed. The results demonstrate that our proposed pipeline can predict the human AH trajectory and estimate the motion target intended by the human accurately and early.

Underwater manipulator that imitates the movements of the human arm

In this paper, the design of a five degrees of freedom (5DoF) underwater manipulator system is presented and discussed. The forward and inverse kinematics problem are solved and shown. Additionally, a system that mimics the human arm movements using a manipulator is proposed. In order to imitate the movement of the human arm, the layout diagram of the IMU sensors and the method of first calibration were presented. The work also presents a device with a simple haptic feedback used to facilitate control of the gripper. The presented approach to the design of a robot control system is dedicated to an underwater robot with the main assumption of reducing the complexity of the system. Taking into account the authors’ knowledge, this approach distinguishes the presented system from solutions known from the literature.

Augmenting inertial motion capture with SLAM using EKF and SRUKF data fusion algorithms

This paper proposes quaternion-based extended and square-root unscented Kalman filters to estimate human body segment positions and orientations using low-cost IMUs in conjunction with a SLAM method. The Kalman filters use measurements based on SLAM output, multilink biomechanical constraints, and vertical referencing to correct errors. In addition to the sensor biases, the fusion algorithm is capable of estimating link geometries, allowing the imposition of biomechanical constraints without a priori knowledge of sensor positions. The proposed algorithms achieve up to 5.87 (cm) and 1.1 (deg) accuracy in position and attitude estimation in various scenarios of human arm movements. Compared to the EKF, the SRUKF algorithm presents a smoother and higher convergence rate but is 2.4 times more computationally demanding. After convergence, the SRUKF is up to 17% less and 36% more accurate than the EKF in position and attitude estimation, respectively.

Applying an Artificial Neuromolecular System to the Application of Robotic Arm Motion Control in Assisting the Rehabilitation of Stroke Patients-An Artificial World Approach.

Stroke patients cannot use their hands as freely as usual. However, recovery after a stroke is a long road for many patients. If artificial intelligence can assist human arm movement, it is believed that the possibility of stroke patients returning to normal hand movement can be significantly increased. In this study, the artificial neuromolecular system (ANM system) developed by our laboratory is used as the core motion control system to learn to control the mechanical arm, produce similar human rehabilitation actions, and assist patients in transiting between different activities. The strength of the ANM system lies in its ability to capture and process spatiotemporal information by exploiting the dynamic information processing inside neurons. Five experiments are conducted in this research: continuous learning, dimensionality reduction, moving problem domains, transfer learning, and fault tolerance. The results show that the ANM system can find out the arm movement trajectory when people perform different rehabilitation actions through the ability of continuous learning and reduce the activation of multiple muscle groups in stroke patients through the learning method of reducing dimensions. Finally, using the ANM system can reduce the learning time and performance required to switch between different actions through transfer learning.

Intended Human Arm Movement Direction Prediction using Eye Tracking

ABSTRACT Collaborative robots are becoming increasingly popular in industries, providing flexibility and increased productivity for complex tasks. However, the robots are still not interactive enough since they cannot yet interpret humans and adapt to their behaviour, mainly due to limited sensory input. Prediction of human movement intentions could be one way to improve these robots. This paper presents a system that uses a recurrent neural network to predict the intended human arm movement direction, solely based on eye gaze, utilizing the notion of uncertainty to determine whether to trust a prediction or not. The network was trained with eye tracking data gathered using a virtual reality environment. The presented deep learning solution makes predictions on continuously incoming data and reaches an accuracy of 70.7%, for predictions with high certainty, and correctly classifies 67.89% of the movements at least once. The movements are, in 99% of the cases, correctly predicted the first time, before the hand reaches the target and more than 24% ahead of time in 75% of the cases. This means that a robot could receive warnings regarding in which direction an operator is likely to move and adjust its behaviour accordingly.

The concepts of muscle activity generation driven by upper limb kinematics

BackgroundThe underlying motivation of this work is to demonstrate that artificial muscle activity of known and unknown motion can be generated based on motion parameters, such as angular position, acceleration, and velocity of each joint (or the end-effector instead), which are similarly represented in our brains. This model is motivated by the known motion planning process in the central nervous system. That process incorporates the current body state from sensory systems and previous experiences, which might be represented as pre-learned inverse dynamics that generate associated muscle activity.MethodsWe develop a novel approach utilizing recurrent neural networks that are able to predict muscle activity of the upper limbs associated with complex 3D human arm motions. Therefore, motion parameters such as joint angle, velocity, acceleration, hand position, and orientation, serve as input for the models. In addition, these models are trained on multiple subjects (n=5 including , 3 male in the age of 26±2 years) and thus can generalize across individuals. In particular, we distinguish between a general model that has been trained on several subjects, a subject-specific model, and a specific fine-tuned model using a transfer learning approach to adapt the model to a new subject. Estimators such as mean square error MSE, correlation coefficient r, and coefficient of determination R2 are used to evaluate the goodness of fit. We additionally assess performance by developing a new score called the zero-line score. The present approach was compared with multiple other architectures.ResultsThe presented approach predicts the muscle activity for previously through different subjects with remarkable high precision and generalizing nicely for new motions that have not been trained before. In an exhausting comparison, our recurrent network outperformed all other architectures. In addition, the high inter-subject variation of the recorded muscle activity was successfully handled using a transfer learning approach, resulting in a good fit for the muscle activity for a new subject.ConclusionsThe ability of this approach to efficiently predict muscle activity contributes to the fundamental understanding of motion control. Furthermore, this approach has great potential for use in rehabilitation contexts, both as a therapeutic approach and as an assistive device. The predicted muscle activity can be utilized to guide functional electrical stimulation, allowing specific muscles to be targeted and potentially improving overall rehabilitation outcomes.

Comparison of LSTM, Transformers, and MLP-mixer neural networks for gaze based human intention prediction.

Collaborative robots have gained popularity in industries, providing flexibility and increased productivity for complex tasks. However, their ability to interact with humans and adapt to their behavior is still limited. Prediction of human movement intentions is one way to improve the robots adaptation. This paper investigates the performance of using Transformers and MLP-Mixer based neural networks to predict the intended human arm movement direction, based on gaze data obtained in a virtual reality environment, and compares the results to using an LSTM network. The comparison will evaluate the networks based on accuracy on several metrics, time ahead of movement completion, and execution time. It is shown in the paper that there exists several network configurations and architectures that achieve comparable accuracy scores. The best performing Transformers encoder presented in this paper achieved an accuracy of 82.74%, for predictions with high certainty, on continuous data and correctly classifies 80.06% of the movements at least once. The movements are, in 99% of the cases, correctly predicted the first time, before the hand reaches the target and more than 19% ahead of movement completion in 75% of the cases. The results shows that there are multiple ways to utilize neural networks to perform gaze based arm movement intention prediction and it is a promising step toward enabling efficient human-robot collaboration.

Development of a portable assistive exoskeleton for human arm movements

AbstractOver the past two decades, the number of people with arm disabilities has dramatically increased. Many researchers have begun to concentrate on developing technologies to address this challenge. Exoskeleton is one of the most successful technologies out of these since it does not require medical staff to accompany the patients and can continually rehabilitate the patients until they regain the ability to move voluntarily. However, the majority of exoskeletons are anchored to the ground and individuals are unable to purchase one due to its high manufacturing costs. Furthermore, since this type of exoskeleton is anchored to the ground, patients must go to hospitals to get rehabilitation services. In this study, a portable assistive exoskeleton for human arm motions is designed to address this problem. This device can carry patients' arms to the desired position by adhering to a prescribed trajectory by medical specialists. The main parts of this exoskeleton are made of polylactic acid and produced using 3D printing technology. Therefore, the total manufacturing costs of the exoskeleton are not excessive and the weight of it is not high as well.

How the timing of visual feedback influences goal-directed arm movements: delays and presentation rates

Visual feedback normally helps guide movements to their goal. When moving one’s hand, such guidance has to deal with a sensorimotor delay of about 100 ms. When moving a cursor, it also has to deal with a delay of tens of milliseconds that arises between the hand moving the mouse and the cursor moving on the screen. Moreover, the cursor is presented at a certain rate, so only positions corresponding with the position of the mouse at certain moments are presented. How does the additional delay and the rate at which cursor positions are updated influence how well the cursor can be guided to the goal? We asked participants to move a cursor to consecutive targets as quickly as they could. They did so for various additional delays and presentation rates. It took longer for the mouse to reach the target when the additional delay was longer. It also took longer when a lower presentation rate was achieved by not presenting the cursor all the time. The fraction of the time during which the cursor was present was more important than the rate at which the cursor’s position was updated. We conclude that the way human arm movements are guided benefits from continuous access to recent visual feedback.

Biomimetic Approaches for Human Arm Motion Generation: Literature Review and Future Directions.

In recent years, numerous studies have been conducted to analyze how humans subconsciously optimize various performance criteria while performing a particular task, which has led to the development of robots that are capable of performing tasks with a similar level of efficiency as humans. The complexity of the human body has led researchers to create a framework for robot motion planning to recreate those motions in robotic systems using various redundancy resolution methods. This study conducts a thorough analysis of the relevant literature to provide a detailed exploration of the different redundancy resolution methodologies used in motion generation for mimicking human motion. The studies are investigated and categorized according to the study methodology and various redundancy resolution methods. An examination of the literature revealed a strong trend toward formulating intrinsic strategies that govern human movement through machine learning and artificial intelligence. Subsequently, the paper critically evaluates the existing approaches and highlights their limitations. It also identifies the potential research areas that hold promise for future investigations.