Human Hand Motion Research Articles

Advancing robots with greater dynamic dexterity: A large-scale multi-view and multi-modal dataset of human-human throw&catch of arbitrary objects

Learning and imitating behavioral intelligence from human demonstrations is a promising approach towards the intuitive programming of robots for enhanced dynamic dexterity. However, there has been no publicly available dataset in this domain. To address this gap, we introduce the first large-scale dataset and recording framework specifically designed for studying human collaborative dynamic dexterity in throw&catch tasks. The dataset, named H2TC, contains 15,000 multi-view and multi-modal synchronized recordings of diverse Human-Human Throw-and-Catch activities. It involves 34 human subjects with typical motor abilities and a variety of 52 objects frequently manipulated through throw&catch in domestic and/or industrial scenarios. The dataset is supplemented with a hierarchy of manually annotated semantic and dense labels, such as the ground truth human body, hand and object motions captured with specialized high-precision motion tracking systems. These rich annotations make the dataset well-suited for a wide range of robot studies, including both low-level motor skill learning and high-level cognitive planning and recognition. We envision that the proposed dataset and recording framework will facilitate learning pipelines to extract insights on how humans coordinate both intra- and interpersonally to throw and catch objects, ultimately leading to the development of more capable and collaborative robots. The dataset, along with a suite of utility tools, such as those for visualization and annotation, can be accessed from our project page at https://h2tc-roboticsx.github.io/ .

Incorporating shape dependent power law in motion planning for drawing robots

Incorporating human natural features into the algorithmic functions of robots can enhance performance efficiency. One of the most popular features of human movements is the power law that defines the connection between movement speed and path curvature. To contribute to this area, by discussing how the power law can provide a reasonable balance between velocity and efficiency, we propose a novel method to design motion profiles based on the power law. The novelty of this solution lies in the adjustment approach for the power law. In this work, inspired by features of human hand movements, the overall curvature of non-circular shapes is considered as the shape-dependent criterion for motion planning. Also, a framework to apply the proposed approach to any open non-circular curvy contours is presented. To investigate the efficiency of the approach, we considered a simple drawing robot. The simulation and experimental results verify the efficacy of the proposed motion planning method.

Ablation threshold, structural modification, and sensing of graphene oxide thin film induced by UV nanosecond laser

Reduction of graphene oxide by laser irradiation is crucial for developing conductive thin films. This study investigates the ablation threshold, reduction of graphene oxide, and high-precision patterning by UV nanosecond laser irradiation of graphene oxide films on flexible substrates. The ablation threshold of graphene oxide thin film was evaluated by examining the relationship between micropore diameters and laser pulse energy density. The single pulse ablation threshold was 2.29 J/cm², indicating that the UV nanosecond laser energy density is sufficient for the ablation of graphene oxide. Based on ablation threshold evaluation, graphene oxide was reduced by UV nanosecond laser irradiation, and laser-irradiated graphene oxide-based flexible thin film with four different line widths was fabricated. UV nanosecond laser irradiation graphene oxide-based grid structure patterns on the flexible substrates were fabricated by analyzing these different line widths. In addition, laser-irradiated graphene oxide-based grid structure flexible electrode were used to monitor human hand motion and finger press sensing. This study underscores the significance of laser irradiation of graphene oxide, a controllable strategy, and highlights the novelty and practical applications for fabricating flexible graphene oxide-based electronics.

Human hand gesture recognition using fast Fourier transform with coot optimization based on deep neural network

ABSTRACT Hand motion detection is particularly important for managing the movement of individuals who have limbs amputated. The existing algorithm is complex, time-consuming and difficult to achieve better accuracy. A DNN is suggested to recognize human hand movements in order to get over these problems.Initially, the raw input EMG signal is captured then the signal is pre-processed using high-pass Butterworth filter and low-pass filter which is utilized to eliminate the noise present in the signal. After that pre-processed EMG signal is segmented using sliding window which is used for solving the issue of overlapping. Then the features are extracted from the segmented signal using Fast Fourier Transform. Then selected the appropriate and optimal number of features from the feature subset using coot optimization algorithm. After that selected features are given as input for deep neural network classifier for recognizing the hand movements of human. The simulation analysis shows that the proposed method obtain 95% accuracy, 0.05% error, precision is 94%, and specificity is 92%.The simulation analysis shows that the developed approach attain better performance compared to other existing approaches. This prediction model helps in controlling the movement of amputee patients suffering from disable hand motion and improve their living standard.

High-Performance Multidirectional Flexible Strain Sensor for Human Motion and Health Monitoring.

Multidirectional strain sensors are pivotal for wearable electronic devices and human-computer interaction. In this investigation, we translocate carbon/graphene (CB/Gr) conductive nanocomposites onto an Ecoflex flexible substrate via a facile technique encompassing reverse molding and spraying, culminating in the fabrication of a 45° strain rosette-shaped multidirectional flexible strain sensor. The sensor distinguishes itself with extraordinary performance characteristics, including high sensitivity (boasting a gauge factor of 35), an extensive strain range from 0 to 100%, exceptional linearity, a rapid response time of merely 200 ms, remarkable stability, and outstanding durability, effortlessly withstanding over 5000 stretch-release cycles. The sensor exhibits its exceptional capability to discern intricate movements, particularly in detecting human hand and neck motions. The sensor's remarkable comprehensive performance and strain direction recognition ability underscore its significant potential for diverse applications, notably in human-computer interaction, human motion monitoring, and health monitoring.

Positional Analysis of Assisting Muscles for Handling-Assisted Exoskeletons.

In order to better design handling-assisted exoskeletons, it is necessary to analyze the biomechanics of human hand movements. In this study, Anybody Modeling System (AMS) simulation was used to analyze the movement state of muscles during human handling. Combined with surface electromyography (sEMG) experiments, specific analysis and verification were carried out to obtain the position of muscles that the human body needs to assist during handling. In this study, the simulation and experiment were carried out for the manual handling process. A treatment group and an experimental group were set up. This study found that the vastus medialis muscle, vastus lateralis muscle, latissimus dorsi muscle, trapezius muscle, deltoid muscle and triceps brachii muscle require more energy in the process of handling, and it is reasonable and effective to combine sEMG signals with the simulation of the musculoskeletal model to analyze the muscle condition of human movement.

Stretchable glove for accurate and robust hand pose reconstruction based on comprehensive motion data

We propose a compact wearable glove capable of estimating both the finger bone lengths and the joint angles of the wearer with a simple stretch-based sensing mechanism. The soft sensing glove is designed to easily stretch and to be one-size-fits-all, both measuring the size of the hand and estimating the finger joint motions of the thumb, index, and middle fingers. The system was calibrated and evaluated using comprehensive hand motion data that reflect the extensive range of natural human hand motions and various anatomical structures. The data were collected with a custom motion-capture setup and transformed into the joint angles through our post-processing method. The glove system is capable of reconstructing arbitrary and even unconventional hand poses with accuracy and robustness, confirmed by evaluations on the estimation of bone lengths (mean error: 2.1 mm), joint angles (mean error: 4.16°), and fingertip positions (mean 3D error: 4.02 mm), and on overall hand pose reconstructions in various applications. The proposed glove allows us to take advantage of the dexterity of the human hand with potential applications, including but not limited to teleoperation of anthropomorphic robot hands or surgical robots, virtual and augmented reality, and collection of human motion data.

Feature Extraction Based on Sparse Coding Approach for Hand Grasp Type Classification

The kinematics of the human hand exhibit complex and diverse characteristics unique to each individual. Various techniques such as vision-based, ultrasonic-based, and data-glove-based approaches have been employed to analyze human hand movements. However, a critical challenge remains in efficiently analyzing and classifying hand grasp types based on time-series kinematic data. In this paper, we propose a novel sparse coding feature extraction technique based on dictionary learning to address this challenge. Our method enhances model accuracy, reduces training time, and minimizes overfitting risk. We benchmarked our approach against principal component analysis (PCA) and sparse coding based on a Gaussian random dictionary. Our results demonstrate a significant improvement in classification accuracy: achieving 81.78% with our method compared to 31.43% for PCA and 77.27% for the Gaussian random dictionary. Furthermore, our technique outperforms in terms of macro-average F1-score and average area under the curve (AUC) while also significantly reducing the number of features required.

Robotic Rehabilitation for Paralyzed Arm

This paper involves the design and implementation of a wearable 3D printed robotic hand controlled by motion of human hand. In the recent trend in technology, the demand and number of studies in Robotics field has been increased. In the literature there are number of studies on Robotics systems in various fields to facilitate the human life.

Advancing Robotic Fruit Picking: Adaptive End-Effector Approach

In modern agriculture, the demand for efficient fruit picking methods is constantly increasing due to labor shortages and the need to improve productivity. Traditional fruit picking methods often rely on manual labor, which is labor-intensive, time-consuming, and prone to inconsistency. This study aims to develop a novel end effector for agricultural fruit picking robots, which improves the efficiency and accuracy of fruit picking through highly flexible and adaptive design. It can effectively handle fruits of diverse types, shapes, and hardness. By adopting a three-finger scheme and independent servo motor control, the simulated movements of human hands are realized. Shape memory alloy (SMA) and electroactive polymer (EAP) as intelligent driving materials, as well as flexible materials such as silicone, are explored to optimize the grip performance and fruit protection of the actuator. High-precision visual and tactile sensors are also integrated to support the accurate identification of fruits of different maturity. The performance of the actuator is evaluated through advanced data analysis techniques, including descriptive statistics, hypothesis testing and regression analysis. The findings suggest that the proposed design optimization measures based on data analysis results can further enhance the adaptability and efficiency of the robot in future applications, thereby addressing the challenges faced in modern agricultural practices and contributing to increased productivity and sustainability in the agriculture sector.

A Flexible Double-Sided Curvature Sensor Array for Use in Soft Robotics.

This paper describes the design, fabrication, integration, characterization, and demonstration of a novel flexible double-sided curvature sensor array for use in soft robotics. The paper explores the performance and potential applications of a piezoresistive sensor array consisting of four gold strain gauges on a flexible polyimide (PI) substrate arranged in a Wheatstone bridge configuration. Multiple sensor strips were arranged like the fingers of a hand. Integrating Shape Memory Alloy (SMA) foils alongside the fingers was explored to mimic a human hand-gripping motion controlled with temperature, while curvature sensor array strips measure the resulting finger shapes. Moreover, object sensing in a flexible granular material gripper was demonstrated. The sensors were embedded within Polydimethylsiloxane (PDMS) to enhance their tactile feel and adhesive properties. The findings of this study are promising for future applications, particularly in robotics and prosthetics, as the ability to accurately mimic human hand movements and reconstruct sensor surfaces paves the way for robotic hand functionality.

An Object Deformation-Agnostic Framework for Human–Robot Collaborative Transportation

In this study, an adaptive object deformability-agnostic human-robot collaborative transportation framework is presented. The proposed framework enables to combine the haptic information transferred through the object with the human kinematic information obtained from a motion capture system to generate reactive whole-body motions on a mobile collaborative robot. Furthermore, it allows rotating the objects in an intuitive and accurate way during co-transportation based on an algorithm that detects the human rotation intention using the torso and hand movements. First, we validate the framework with the two extremities of the object deformability range (i.e., purely rigid aluminum rod and highly deformable rope) by utilizing a mobile manipulator which consists of an Omni-directional mobile base and a collaborative robotic arm. Next, its performance is compared with an admittance controller during a co-carry task of a partially deformable object in a 12-subjects user study. Quantitative and qualitative results of this experiment show that the proposed framework can effectively handle the transportation of objects regardless of their deformability and provides intuitive assistance to human partners. Finally, we have demonstrated the potential of our framework in a different scenario, where the human and the robot co-transport a manikin using a deformable sheet. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Transportation of objects which requires the cooperation of multiple partners, is a common task in industrial settings such as factories and warehouses. The existing human-robot collaboration solutions for this task have focused only on purely rigid objects, although deformable objects need to be carried frequently in real-world applications. In this paper, we introduce a human-robot collaborative transportation framework that can handle objects with different deformability ranging from purely rigid to highly deformable. In particular, the proposed framework generates whole-body movements on a mobile collaborative robot by combining of the haptic information transmitted through the object and the human motion information obtained from a motion capture system. Moreover, the framework includes an intuitive way to rotate the object during the execution based on human hand and torso motion. The results of the experiments where objects with various deformability characteristics were transported in collaboration with a mobile manipulator demonstrated the high potential of the proposed approach in a laboratory setting. In the future, we plan to employ a less expensive vision-based human motion tracking system instead of the IMU-based system used in this study. With this change, we will be able to eliminate the need for wearable sensors from the framework presented, which would enhance its usability in real-world scenarios.

Gesture Detection System using OpenCV

In the rapidly evolving landscape of human-computer interaction, our project delves into the innovative realm of gesture detection technology. This study presents the development of a sophisticated Gesture Detection System, a responsive interface bridging the gap between human hand movements and digital actions. With the invaluable support of our project supervisor, Mrs. V. M. Khanapure, our project not only explores the intricate nuances of gesture recognition but also showcases the collaborative spirit of our team. Utilizing OpenCV and MediaPipe libraries, our system employs advanced computer vision algorithms for real-time hand tracking and precise gesture recognition. The project’s success is not only attributed to the technical acumen of our team members but also to the seamless cooperation and encourageme Among them. Furthermore, the availability of essential resources provided by Mr.V. Nitnaware Principal, PLGPL, significantly bolstered our project development process. This abstract encapsulates a journey of collaborative innovation, highlighting the symbiotic relationship between technology and teamwork. Through this Gesture Detection System, we aim to redefine user experiences, offering an intuitive and interactive interface that responds to human gestures with precision and fluidity

Human Hand Movement Classification based on EMG Signal using different Feature Extractor

Electromyography (EMG) based hand movement classification plays a significant role in various fields, namely in prosthetics, rehabilitation, biomechanics, etc. This paper presents the study of EMG-based hand movement classification of 3 human hand gestures (hand at rest, wrist flexion, and wrist extension). The dataset was officially collected from the University of California, Irvine (UCI) machine learning repository. The dataset contains 8 channels and 3 classes representing 3 human hand gestures, with 15000 rows of EMG data for each class. The dataset obtained was raw and unprocessed, to filter this dataset Notch and Butterworth filters were used. After filtering, the sliding window was performed. Various feature extraction techniques, namely frequency domain features (FD) and discrete wavelet transform (DWT) were applied separately on the window dataset and then accuracy was tested on different classifiers, namely random forest (RF), k- nearest neighbor (KNN), and decision tree (DT). As a novel approach, time domain (TD) and DWT extracted features were fused together and then given to the classifiers to test accuracy. Among all these feature extractors, the features extracted by FD provided the highest accuracy of 81.69 for the RF classifier.

Bionic Hand Motion Control Method Based on Imitation of Human Hand Movements and Reinforcement Learning

Bionic Hand Motion Control Method Based on Imitation of Human Hand Movements and Reinforcement Learning

Learning dexterity from human hand motion in internet videos

To build general robotic agents that can operate in many environments, it is often useful for robots to collect experience in the real world. However, unguided experience collection is often not feasible due to safety, time, and hardware restrictions. We thus propose leveraging the next best thing as real world experience: videos of humans using their hands. To utilize these videos, we develop a method that retargets any 1st person or 3rd person video of human hands and arms into the robot hand and arm trajectories. While retargeting is a difficult problem, our key insight is to rely on only internet human hand video to train it. We use this method to present results in two areas: First, we build a system that enables any human to control a robot hand and arm, simply by demonstrating motions with their own hand. The robot observes the human operator via a single RGB camera and imitates their actions in real-time. This enables the robot to collect real-world experience safely using supervision. See these results at https://robotic-telekinesis.github.io. Second, we retarget in-the-wild human internet video into task-conditioned pseudo-robot trajectories to use as artificial robot experience. This learning algorithm leverages action priors from human hand actions, visual features from the images, and physical priors from dynamical systems to pretrain typical human behavior for a particular robot task. We show that by leveraging internet human hand experience, we need fewer robot demonstrations compared to many other methods. See these results at https://video-dex.github.io

Biomimetic learning of hand gestures in a humanoid robot.

Hand gestures are a natural and intuitive form of communication, and integrating this communication method into robotic systems presents significant potential to improve human-robot collaboration. Recent advances in motor neuroscience have focused on replicating human hand movements from synergies also known as movement primitives. Synergies, fundamental building blocks of movement, serve as a potential strategy adapted by the central nervous system to generate and control movements. Identifying how synergies contribute to movement can help in dexterous control of robotics, exoskeletons, prosthetics and extend its applications to rehabilitation. In this paper, 33 static hand gestures were recorded through a single RGB camera and identified in real-time through the MediaPipe framework as participants made various postures with their dominant hand. Assuming an open palm as initial posture, uniform joint angular velocities were obtained from all these gestures. By applying a dimensionality reduction method, kinematic synergies were obtained from these joint angular velocities. Kinematic synergies that explain 98% of variance of movements were utilized to reconstruct new hand gestures using convex optimization. Reconstructed hand gestures and selected kinematic synergies were translated onto a humanoid robot, Mitra, in real-time, as the participants demonstrated various hand gestures. The results showed that by using only few kinematic synergies it is possible to generate various hand gestures, with 95.7% accuracy. Furthermore, utilizing low-dimensional synergies in control of high dimensional end effectors holds promise to enable near-natural human-robot collaboration.

HandFi: WiFi Sensing based Hand Gesture Recognition using Channel State Information

Gesture recognition using WiFi sensing is an emerging and innovative approach that leverages the variations in WiFi signals caused by human hand movements to enable non-contact and intuitive human-computer interaction. The identification of the hand used to perform a gesture significantly impacts the channel state information (CSI) characteristics. It plays a crucial role in data analysis and interpretation. This paper presents a model for classifying the gesture and the hand with which it is performed. We use an Intel 5300 network interface card as a receiver and a commodity WiFi router to collect the CSI values. The CSI values are collected for six left-hand and six right-hand gestures (push left, push right, increase, decrease, stop, and next). The collected CSI values are pre-processed to find the CSI ratio among the sub-carriers. Then four different methods including proposed method are used for classifying the hand and the gesture. The four methods are (1) use of the Hampel filter, (2) use of CSI ratio only, (3) use of correlation matrix along with CSI ratio, and (4) proposed CSI ratio and interquartile distance based method. Then various machine learning techniques such as KNN, Decision trees, and SVM are used to classify the hand (right or left) used for signaling the gestures and the type of gestures. It is found that using the top 30 inter-quartile distance of the CSI-ratios corresponding to the CSI values of the sub-carriers while preserving their positions as the feature vector, the KNN machine learning model can classify the hand and gesture with approximately 99% accuracy for test data. Similar performance is also observed for other performance evaluation parameters like precision and recall.

Enhancing Dexterous Grasping and Manipulation through Linkage-Driven Underactuated Five- Fingered Robotic Hand: A Computational Approach

This research aims to enhance dexterous grasping and manipulation capabilities through a computational approach involving a novel design of a linkage-driven, underactuated, five-fingered robotic hand. Leveraging underactuated mechanisms, the study addresses the complexity associated with individually actuated joints, offering a streamlined and efficient solution for replicating human-like hand movements. The proposed hand comprises 12 actuators and 21° of freedom, emphasizing the need for enhanced adaptability and reduced complexity in replicating human hand movements. Utilizing SolidWorks for mechanical design, Automated Dynamic Analysis of Mechanical Systems (ADAMS) is employed for dynamic simulations, enabling a comprehensive evaluation of the robotic hand’s performance in realistic scenarios. MATLAB is utilized for the required algorithm, and Analysis System (ANSYS) for structural analysis, ensuring the robustness and reliability of the robotic hand under different loading conditions. This approach integrates mechanical engineering principles with advanced simulation tools. The abstract concludes by presenting comprehensive research results, featuring general quantitative data from the entire research process, underscoring the efficacy and applicability of the computational approach in advancing robotic manipulation capabilities.

Gesture-Based Control of Multimedia Player Using Python and OpenCV

The increasing significance of computers in our daily lives, coupled with the rise of ubiquitous computing, has necessitated effective human-computer interaction. Hand gesture recognition systems have emerged as a real-time video-based solution for detecting and interpreting hand gestures, offering intelligent and natural human-computer interaction (HCI) methods. This project focuses on leveraging human hands as input devices for computer operation. Developed using Python and the OpenCV library, the program utilizes a computer webcam to capture and analyze hand shapes and patterns. The program provides real-time feedback by displaying recognized hand gestures on the live video stream. The ultimate outcome of this project is an application that enhances user experiences in contactless systems. The project recognizes and detects human hand motions using the Python computer language through a process flow that includes background subtraction, hand ROI segmentation, contour detection, and finger recognition. Techniques for processing images are used, including hand gesture detection, pattern recognition, thresholding, and contour detection. The processing of incoming photos and the creation of related keystrokes are made possible by OpenCV, a rich set of image processing tools