Hand Shape Research Articles

On the Use of Magnets to Robustify the Motion Control of Soft Hands

In this letter, we propose a physics-based framework to exploit magnets in robotic manipulation. More specifically, we suggest equipping soft and underactuated hands with magnetic elements, which can generate a magnetic actuation able to synergistically interact with tendon-driven and pneumatic actuations, engendering a complementarity that enriches the capabilities of the actuation system. Magnetic elements can act as additional Degrees of Actuation (DoAs), robustifying the motion control of the device and augmenting the hand manipulation capabilities. We investigate the interaction of a soft hand with itself for enriching possible hand shaping, and the interaction of the hand with the environment for enriching possible grasping capabilities. Physics laws and notions reported in the manuscript can be used as a guidance for DoAs augmentation and can provide tools for the design of novel soft hands.

REAL TIME FINGER WRITING RECOGNITION USING WEBCAMERA

This paper proposes a real time finger writing recognition system using a web camera to enable user to interact with computer systems by their own bare hands. The hand region is firstly extracted by background subtraction and blob labelling . Then, convex hull and convexity defect are used to count the fingers and detect the fingertip coordinates. The trajectory of the fingertip is smoothed and drawn to become a finger writing character. This paper describes a system which uses multiple visual process to detect like Hand Tracking, Volume Gesture, Finger counter which ultimately tracks Finger Recognition based on computer vision. In modern technologies video tracking and processing the feed has been very essential. This processed data can be used for many research purposes or to express a particular output on a particular system. There are various methods for processing and manipulation of data to get the required output. This application is created using OpenCV module and Mediapipe. OpenCV is a computer vision and machine learning software library that includes many common image analysis algorithms that will help us build custom intelligent computer vision applications. In this application frequent image feed results in video tracking of our particular finger. Video tracking is the process of locating a moving object (or multiple objects) over time using a camera. The ability to perceive the shape and motion of hands can be a vital component in improving the user experience across a variety of technological domains and platforms .One such technology is using MediaPipe .MediaPipe is a framework mainly used for building audio, video, or any time series data. With the help of the MediaPipe framework, we can build very impressive pipelines for different media processing functions.

Independent risk factors of carpal tunnel syndrome: Assessment of body mass index, hand, wrist and finger anthropometric measurements.

The most prevalent neuropathy in the upper extremity is carpal tunnel syndrome (CTS). A variety of related risk factors such as biomechanical exposures, body mass index (BMI), sex and hand shape are reported to be related to CTS. We aimed to identify the role of BMI, hand, wrist and finger anthropometric dimensions in the development of CTS, and to compare these measured variables between control and CTS participants. A cross-sectional, case control study (n = 240, CTS = 120, controls = 120) with participants recruited from a convenience sample diagnosed with CTS and referred for anthropometric measurements. The control participants were matched by age and sex. The body height, weight, hand width, hand length, wrist depth, wrist width, wrist circumference, and finger length were measured. Hand, wrist and finger indices, hand to height ratio, and BMI were calculated. Mean values of all dimensions were compared between cases and controls, and the role of independent risk factors were determined by logistic regression analysis. The mean BMI, age, weight, sex and height were not significant between the two groups. Among the measured dimensions and calculated indices the significantly different variables between two groups were the wrist width, wrist depth, wrist circumference, hand index, hand to height index, and wrist index. Regression analysis showed that the wrist index (β=-1.7, p = 0.0001), wrist depth (β=0.25, p = 0.0001) and wrist width (β=0.21, p = 0.0001) were the strongest factors in CTS development in the sample. Wrist parameters have a strong role in predicting the development of CTS, while BMI was not confirmed as an independent risk factor.

Self-designed hierarchical network based hand shape intelligent recognition and evaluation for Chinese zither performing

A proper hand shape is the foundation for professional musical instrument performance. In this paper, the image recognition technology based on artificial intelligence is introduced into the hand shape recognition of performing Chinese zither, which is referred to as Zheng in Chinese, for the first time to realize the function of hand shape intelligent evaluation and the self-designed hierarchical network is proposed to recognize and evaluate correct hand shape for Zheng performing. The intra-class difference is larger than the inter-class difference for Zheng performing hand shape image, which belongs to fine-grained image. Therefore, we use the first layer network to determine four classes of images acquired from different viewpoints. Meanwhile, the feature maps from different convolutional blocks of this layer are concatenated as the input of the second layer, which performs fine classification of Zheng performing hand shape images. Consequently, the learning ability of the network can be improved and the complexity of the network can be reduced at the same time. We design an experimental paradigm for instrumentalist hand shape performance evaluation, formulate a Zheng hand shape evaluation merit based on image recognition, and construct a Chinese zither hand shape Dataset (CZ-dataset V3) for the real scene. The experiments show that the method proposed in this paper can effectively improve the recognition accuracy of fine-grained hand shape images and the result is consistent with the evaluation of professional advisors, which realizes the perfect combination of the intelligent image recognition and the hand shape evaluation for Chinese traditional instrument performing.

Sign Language Detection System Using Adaptive Neuro Fuzzy Inference System (ANFIS) Method

Sign language is a language that prioritizes communication with hands, body language, and lip movements to communicate. The deaf are the main group who use this language, often combining hand shape, hand, arm and body orientation and movement, and facial expressions to express their thoughts. The sign language detection system is designed using the Adaptive Neuro Fuzzy Inference System (ANFIS). This study uses data from the kaggle.com dataset, which is a site that provides research data on artificial intelligence. This study was conducted to recognize empty hand signals. Where it will help users naturally without any additional help. The test is carried out using a data set as evidenced by 1 display. In this process, The characteristics of the hand were carried out using the Histogram Oriented Gradient (HOG) method. Meanwhile, to separate it from the background image, it is used with color segmentation. The results of the process are then taken for classification. The classification process uses the Adaptive Neuro Fuzzy Inference System method. The results of the tests carried out for accuracy are as much as

Improving Recognition of SIBI Gesture by Combining Skeleton and Hand Shape Features

SIBI (Sign System for Indonesian Language) is an official sign language system used in school for hearing impairment students in Indonesia. This work uses the skeleton and hand shape features to classify SIBI gestures. In order to improve the performance of the gesture classification system, we tried to fuse the features in several different ways. The accuracy results achieved by the feature fusion methods are, in descending order of accuracy: 88.016%, when using sequence-feature-vector concatenation, 85.448% when using Conneau feature vector concatenation, 83.723% when using feature-vector concatenation, and 49.618% when using simple feature concatenation. The sequence-feature-vector concatenation techniques yield noticeably better results than those achieved using single features (82.849% with skeleton feature only, 55.530% for the hand shape feature only). The experiment results show that the combined features of the whole gesture sequence can better distinguish one gesture from another in SIBI than the combined features of each gesture frame. In addition to finding the best feature combination technique, this study also found the most suitable Recurrent Neural Network (RNN) model for recognizing SIBI. The models tested are 1-layer, 2-layer LSTM, and GRU. The experimental results show that the 2-layer bidirectional LSTM has the best performance.

A Wearable Multi-Joint Wrist Contour Measuring Device for Hand Shape Recognition

Recently, various types of hand shape recognition systems have been developed for human–machine interfaces. However, most wearable recognition systems cannot robustly handle the variations in attachment positions of the devices. Thus, we propose a hand shape recognition system using a wearable multi-joint wrist contour measuring device to realize robust and effective recognition of hand shape, regardless of attachment position variations. In particular, this device can measure the wrist contour and band flexion data to recognize hand shape. The wrist contour data are measured using photo-reflectors mounted inside the device, and the band flexion data are measured using photo-interrupters installed at the device joints. Additionally, the attachment position information is extracted from the wrist contour or band flexion data using dimensional compression or attachment position recognition to achieve robustness against the position variations. Subsequently, the extracted information is incorporated into the hand shape recognizer. The results of the recognition experiment demonstrate that the attachment position information extracted from the band flexion data using dimensional compression could effectively realize robust hand shape recognition, considering variations in the device attachment position.

NIMBLE

Emerging Metaverse applications demand reliable, accurate, and photorealistic reproductions of human hands to perform sophisticated operations as if in the physical world. While real human hand represents one of the most intricate coordination between bones, muscle, tendon, and skin, state-of-the-art techniques unanimously focus on modeling only the skeleton of the hand. In this paper, we present NIMBLE, a novel parametric hand model that includes the missing key components, bringing 3D hand model to a new level of realism. We first annotate muscles, bones and skins on the recent Magnetic Resonance Imaging hand (MRI-Hand) dataset [Li et al. 2021] and then register a volumetric template hand onto individual poses and subjects within the dataset. NIMBLE consists of 20 bones as triangular meshes, 7 muscle groups as tetrahedral meshes, and a skin mesh. Via iterative shape registration and parameter learning, it further produces shape blend shapes, pose blend shapes, and a joint regressor. We demonstrate applying NIMBLE to modeling, rendering, and visual inference tasks. By enforcing the inner bones and muscles to match anatomic and kinematic rules, NIMBLE can animate 3D hands to new poses at unprecedented realism. To model the appearance of skin, we further construct a photometric HandStage to acquire high-quality textures and normal maps to model wrinkles and palm print. Finally, NIMBLE also benefits learning-based hand pose and shape estimation by either synthesizing rich data or acting directly as a differentiable layer in the inference network.

Gesture Vocalizer Using Flex Sensor and Software Visualization

Abstract: People have the voice ability for communicating among one another. Tragically, not every person has the ability to talk and hear. Gesture based communication utilized among the local area of individuals who are distinctive abled as the methods for correspondence. Communication through signing is a motion portrayal that includes concurrent joining hand shapes and development of the hands, arms or body, and looks to communicate. The people who can't talk utilizes the gesture based communications to speak with other individual vocally hindered individual and even with other ordinary individuals who knows the implications of gesture based communications or an interpreter is expected to decipher the implications of gesture based communications to others who can talk and do comprehend the gesture based communications. In any case, it's anything but consistently workable for a person to associate with constantly to decipher the communications via gestures and not every person can gain proficiency with the communications through signing. In this way, another option is that we can utilize a gadget Gesture Vocalizer as an arbiter. The Gesture Vocalizer can take the motions a contribution from the vocally impeded individual and interaction it's anything but a literary and sound type of yield. Hardware version of the project Gesture Vocalizer is a multi-microcontroller-based framework being intended to work with the correspondence among the imbecilic, hard of hearing and visually impaired networks and their correspondence with the ordinary individuals. This system can be flexibly changed to function as a "smart device.". In this project we showcases a gesture vocalizer based on a microcontroller and sensors. Gesture vocalizer designed is basically a hand glove and a microcontroller system

An efficient hand gesture recognition based on optimal deep embedded hybrid convolutional neural network‐long short term memory network model

AbstractHand gestures are the nonverbal communication done by individuals who cannot represent their thoughts in form of words. It is mainly used during human‐computer interaction (HCI), deaf and mute people interaction, and other robotic interface applications. Gesture recognition is a field of computer science mainly focused on improving the HCI via touch screens, cameras, and kinetic devices. The state‐of‐art systems mainly used computer vision‐based techniques that utilize both the motion sensor and camera to capture the hand gestures in real‐time and interprets them via the usage of the machine learning algorithms. Conventional machine learning algorithms often suffer from the different complexities present in the visible hand gesture images such as skin color, distance, light, hand direction, position, and background. In this article, an adaptive weighted multi‐scale resolution (AWMSR) network with a deep embedded hybrid convolutional neural network and long short term memory network (hybrid CNN‐LSTM) is proposed for identifying the different hand gesture signs with higher recognition accuracy. The proposed methodology is formulated using three steps: input preprocessing, feature extraction, and classification. To improve the complex visual effects present in the input images, a histogram equalization technique is used which improves the size of the gray level pixel in the image and also their occurrence probability. The multi‐block local binary pattern (MB‐LBP) algorithm is employed for feature extraction which extracts the crucial features present in the image such as hand shape structure feature, curvature feature, and invariant movements. The AWMSR with the deep embedded hybrid CNN–LSTM network is applied in the two‐benchmark datasets namely Jochen Triesch static hand posture and NUS hand posture dataset‐II to detect its stability in identifying different hand gestures. The weight function of the deep embedded CNN‐LSTM architecture is optimized using the puzzle optimization algorithm. The efficiency of the proposed methodology is verified in terms of different performance evaluation metrics such as accuracy, loss, confusion matrix, Intersection over the union, and execution time. The proposed methodology offers recognition accuracy of 97.86% and 98.32% for both datasets.

Correlation of Handedness with Hand Shape Index in Right Hander and Left Hander Medical Students of Bangladesh

This cross-sectional, analytical type of study was performed in Department of Anatomy, Dhaka Medical College, Bangladesh, from July 2016 to June 2017 on 52 right hander male and 52 right hander female, 40 left hander male and 34 left hander female Bangladeshi medical students. Convenient purposive sampling technique was adopted. History of any injury of upper limb was excluded to construct standard measurement. Handedness of each medical student was determined by the Edinburgh handedness inventory. Hand breadth and hand length was measured with the help of vernier caliper. Paired and unpaired student‟s „t‟ test and Pearson‟s correlation coefficient test were done for statistical analyses. Handedness score showed nonsignificant positive correlation with right hand shape index (r = +0.087, P>0.05) and left-hand shape index (r = +0.008, P>0.05) in right hander males. Handedness score showed non-significant negative correlation with right hand shape index (r = -0.017, P>0.05) and left-hand shape index (r = -0.003, P>0.05) in right hander females. Handedness score showed non-significant negative correlation with right hand shape index (r= - 0.210, P>0.05) and left-hand shape index (r = -0.247, P>0.05) in left hander males. Handedness score showed non-significant positive correlation with right hand shape index (r= +0.051, P>0.05) and left-hand shape index (r = +0.213, P>0.05) in left hander females. To summarize, handedness score showed nonsignificant positive correlation with right hand shape index and left-hand shape index in right hander males and left hander females. However, handedness score showed non-significant negative correlation with right hand shape index and left-hand shape index in right hander females and left hander males.
 CBMJ 2022 January: vol. 11 no. 01 P: 14-20

Comparison of Muscle Activities in Upper Extremity Muscles During the Knee-flexed and Standard Push-ups With Different Hand Shape

Comparison of Muscle Activities in Upper Extremity Muscles During the Knee-flexed and Standard Push-ups With Different Hand Shape

Two Types of Withdraw-to-Eat Movement Related to Food Size in Long-Tailed Macaques (Macaca fascicularis): Insights into the Evolution of the Visual Control of Hand Shaping in Anthropoid Primates

The details of the evolutionary steps in the transition from nonvisual guidance of hand movements for feeding, as displayed by many non-primate species, to visual guidance of hand movements in primates are sparse. Contemporary theory holds that a small-bodied stem primate evolved visual control of the reach to guide a hand to obtain small insects and fruit items from the terminal branches of trees. The subsequent evolution of the visual control of hand and finger shaping movements of the grasp of anthropoids is uncertain. The present study finds that Balinese long-tailed macaques (Macaca fascicularis), video recorded while spontaneously eating at the Sacred Monkey Forest Sanctuary in Ubud in Indonesia, displayed two types of hand movements associated with two types of withdraw movements to place food items in the mouth. Small food items were brought directly to the mouth with hand supination, often with no visual monitoring after grasping. Large food items that protruded from the hand were visually monitored to orient the food item on the initial part of the withdraw but visually disengaged with a head movement and often a blink before the item was placed in the mouth. The results are discussed in relation to the idea that visual information related to orientating food items of varying sizes to an appropriate position in the mouth contributed to the evolution of the visual control of hand shaping skills in anthropoid primates.

Gesticulating Biochemistry: Exploring the Impact of using Augmented Reality Models on Students’ Usage of Gestures when Talking about Biochemistry

There are multiple examples in the literature highlighting the connection between gestures and meaning‐making by students learning chemical concepts in chemistry; however, there is little known about whether this connection extends to student learning in biochemistry. Moreover, learning biochemistry often involves the use and coordination of multiple external representations (ranging from 2D illustrations and physical models to 3D virtual models) of macromolecules. How does use of these external representations impact student gesture production and meaning‐making when learning new biochemical content? In this study, we used a newly developed augmented reality (AR) model of the KscA channel to investigate the influence of external representations on student produced gestures during a discussion on KscA function. We focused on several research questions: what are the different biochemistry content knowledge that students express through gestures? What are the different types of gestures and patterns of gesture production used by students to express content knowledge? What are the differences, if any, in student produced gestures before and after the introduction of an AR model? What are the relationships between student produced gestures and the AR model? Semi‐structured clinical interviews were conducted and recorded through Zoom. Seven undergraduate students who enrolled in an upper‐division biochemistry course participated in the interview. During the interview, participants were presented with an AR model of KcsA channel after proposing a theory of how potassium ions move through the channel. Participants were then asked to modify their theory. In the first stage of analysis, interview participants’ utterances were transcribed and their gestures were described in terms of hand shape and movement. The identification of unique gestures as part of the coding process highlighted the importance of hand shape vs movement vs a combination of both as ways to delineate different gestures. In the second stage of analysis, qualitative and quantitative differences in student gesture production were identified through a comparative analysis of student gestures before and after their exposure to the AR model. Exposure to the AR model resulted in a shift from gestures that were more general (for example, holding two hands parallel to represent the channel) to more specific to the channel representation they say (for example, holding two hands to form a funnel shape or curling one hand into a tube shape). Our findings add to existing literature on the use of external representation in biochemistry education by highlighting the relationships between AR mediated external representations and gesture mediated external representations. Implications for both the development of new external representations and instructor’s choice of external representations are discussed.

Shape Transformers: Topology‐Independent 3D Shape Models Using Transformers

AbstractParametric 3D shape models are heavily utilized in computer graphics and vision applications to provide priors on the observed variability of an object's geometry (e.g., for faces). Original models were linear and operated on the entire shape at once. They were later enhanced to provide localized control on different shape parts separately. In deep shape models, nonlinearity was introduced via a sequence of fully‐connected layers and activation functions, and locality was introduced in recent models that use mesh convolution networks. As common limitations, these models often dictate, in one way or another, the allowed extent of spatial correlations and also require that a fixed mesh topology be specified ahead of time. To overcome these limitations, we present Shape Transformers, a new nonlinear parametric 3D shape model based on transformer architectures. A key benefit of this new model comes from using the transformer's self‐attention mechanism to automatically learn nonlinear spatial correlations for a class of 3D shapes. This is in contrast to global models that correlate everything and local models that dictate the correlation extent. Our transformer 3D shape autoencoder is a better alternative to mesh convolution models, which require specially‐crafted convolution, and down/up‐sampling operators that can be difficult to design. Our model is also topologically independent: it can be trained once and then evaluated on any mesh topology, unlike most previous methods. We demonstrate the application of our model to different datasets, including 3D faces, 3D hand shapes and full human bodies. Our experiments demonstrate the strong potential of our Shape Transformer model in several applications in computer graphics and vision.

3D hand pose and shape estimation from RGB images for keypoint-based hand gesture recognition

Estimating the 3D pose of a hand from a 2D image is a well-studied problem and a requirement for several real-life applications such as virtual reality, augmented reality, and hand gesture recognition. Currently, reasonable estimations can be computed from single RGB images, especially when a multi-task learning approach is used to force the system to consider the shape of the hand when its pose is determined. However, depending on the method used to represent the hand, the performance can drop considerably in real-life tasks, suggesting that stable descriptions are required to achieve satisfactory results. In this paper, we present a keypoint-based end-to-end framework for 3D hand and pose estimation and successfully apply it to the task of hand gesture recognition as a study case. Specifically, after a pre-processing step in which the images are normalized, the proposed pipeline uses a multi-task semantic feature extractor generating 2D heatmaps and hand silhouettes from RGB images, a viewpoint encoder to predict the hand and camera view parameters, a stable hand estimator to produce the 3D hand pose and shape, and a loss function to guide all of the components jointly during the learning phase. Tests were performed on a 3D pose and shape estimation benchmark dataset to assess the proposed framework, which obtained state-of-the-art performance. Our system was also evaluated on two hand-gesture recognition benchmark datasets and significantly outperformed other keypoint-based approaches, indicating that it is an effective solution that is able to generate stable 3D estimates for hand pose and shape.

A Survey on GAN-Based Data Augmentation for Hand Pose Estimation Problem

Deep learning solutions for hand pose estimation are now very reliant on comprehensive datasets covering diverse camera perspectives, lighting conditions, shapes, and pose variations. While acquiring such datasets is a challenging task, several studies circumvent this problem by exploiting synthetic data, but this does not guarantee that they will work well in real situations mainly due to the gap between the distribution of synthetic and real data. One recent popular solution to the domain shift problem is learning the mapping function between different domains through generative adversarial networks. In this study, we present a comprehensive study on effective hand pose estimation approaches, which are comprised of the leveraged generative adversarial network (GAN), providing a comprehensive training dataset with different modalities. Benefiting from GAN, these algorithms can augment data to a variety of hand shapes and poses where data manipulation is intuitively controlled and greatly realistic. Next, we present related hand pose datasets and performance comparison of some of these methods for the hand pose estimation problem. The quantitative and qualitative results indicate that the state-of-the-art hand pose estimators can be greatly improved with the aid of the training data generated by these GAN-based data augmentation methods. These methods are able to beat the baseline approaches with better visual quality and higher values in most of the metrics (PCK and ME) on both the STB and NYU datasets. Finally, in conclusion, the limitation of the current methods and future directions are discussed.

Variation in classifi er predications in Polish Sign Language: a pilot study

This paper describes the fi ndings of a pilot research on the application of object classifi ers and body parts in classifi er predications referring to one designatum, i.e. the dog in the picture story Frog, Where Are You?. The research showed that the frequency of the classifi ers in informants’ texts varied. One classifi er type prevailed when describing the majority of situations. Moreover, for each classifi er type used in a given situation, there was a clear predominance of one of the possible shapes of hand, while the remaining ones were similar to it. From among body part classifi ers, the least specifi c shapes of hand prevailed, which rendered the look of dog head to the smallest extent. These observations prove a high level of grammaticalisation of object classifi ers and poorly specifi c body part classifi ers

Design, Implementation, and Evaluation of a Variable Stiffness Transradial Hand Prosthesis.

We present the design, implementation, and experimental evaluation of a low-cost, customizable, easy-to-use transradial hand prosthesis capable of adapting its compliance. Variable stiffness actuation (VSA) of the prosthesis is based on antagonistically arranged tendons coupled to nonlinear springs driven through a Bowden cable based power transmission. Bowden cable based antagonistic VSA can, not only regulate the stiffness and the position of the prosthetic hand but also enables a light-weight and low-cost design, by the opportunistic placement of motors, batteries, and controllers on any convenient location on the human body, while nonlinear springs are conveniently integrated inside the forearm. The transradial hand prosthesis also features tendon driven underactuated compliant fingers that allow natural adaption of the hand shape to wrap around a wide variety of object geometries, while the modulation of the stiffness of their drive tendons enables the prosthesis to perform various tasks with high dexterity. The compliant fingers of the prosthesis add inherent robustness and flexibility, even under impacts. The control of the variable stiffness transradial hand prosthesis is achieved by an sEMG based natural human-machine interface.

Sign Language Detector using cloud

Abstract: Peoples who are deaf/dumb use Sign language to convey their message to normal people. Sign language detector using cloud provides us an innovative, user friendly way of interaction with the computer which is more familiar to human beings. Sign language detector has a wide area of application including human machine interaction, sign language, immersive game technology etc. By keeping in mind, the similarities of human hand shape with four fingers and one thumb, this paper aims to present a real time system for sign language detector on the basis of detection of some meaningful shape-based features like orientation, center of mass (centroid), status of fingers, thumb in terms of raised or folded fingers of hand and their respective location in image. The approach introduced in this paper is totally depending on the shape parameters of the hand gesture. It does not consider any other means of hand gesture recognition like skin color, texture because these image-based features are extremely variant to different light conditions and other influences. To implement this approach, we have utilized a simple web cam which is working on 20 fps with 7 mega pixel intensity. Keywords: Sign language detector, cloud