Hand Shape Research Articles

Classification Of Hand Images by Person, Age and Gender with The Median Robust Extended Local Binary Model

Biometric technologies try to automatically recognize individuals by considering the physiological and behavioral characteristics of individuals. Although the methods used here are very diverse, the personal qualities used also vary. Facial features, finger and vein prints, iris, retina, ear, hand, and finger recognition are only some of the physiological features. It may be preferred to use one or more of these personal features to reduce the margin of error that may arise depending on the security level in the applications used. Biometric recognition systems have varying requirements in security systems applications. Fingerprint and iris recognition work well in applications that require high security levels, while applications that require low security levels are not suitable due to privacy concerns. On the other hand, identification from hand images is more accepted based on the idea that it does not have a very high distinctiveness. But it is sufficient for medium security applications. Apart from these, palm images have many advantages such as reliability, stability, user-friendliness, non-intrusiveness, and flexible use. In this study, it is aimed to identify people, determine their ages, and determine their gender by using both upper surface and inner surface images of right-left hand data of hand shape. For this purpose, images of both the inner surface of the hand (10) and the outer surface of the hand (10) of 100 different people were collected. This was done separately for the right and left hands, and a total of 3955 images were obtained. The features of these images were extracted using the Median Robust Extended Local Binary Model (MRELBP). Images are classified for person, age and gender. The results were 91.4%, 85.9% and 92.6%, respectively.

Data Glove with Bending Sensor and Inertial Sensor Based on Weighted DTW Fusion for Sign Language Recognition

There are numerous communication barriers between people with and without hearing impairments. Writing and sign language are the most common modes of communication. However, written communication takes a long time. Furthermore, because sign language is difficult to learn, few people understand it. It is difficult to communicate between hearing-impaired people and hearing people because of these issues. In this research, we built the Sign-Glove system to recognize sign language, a device that combines a bend sensor and WonderSense (an inertial sensor node). The bending sensor was used to recognize the hand shape, and WonderSense was used to recognize the hand motion. The system collects a more comprehensive sign language feature. Following that, we built a weighted DTW fusion multi-sensor. This algorithm helps us to combine the shape and movement of the hand to recognize sign language. The weight assignment takes into account the feature contributions of the sensors to further improve the recognition rate. In addition, a set of interfaces was created to display the meaning of sign language words. The experiment chose twenty sign language words that are essential for hearing-impaired people in critical situations. The accuracy and recognition rate of the system were also assessed.

Sign Language Recognition using Deep Neural Network

Abstract—Sign language recognition is an attractive research field with a wide range of applications including telesurgery techniques. Another important application of hand gesture recognition is the translation of sign language, which is a complicated structured form of hand gestures. In sign language, the fingers’ configuration, the hand’s orientation, and the hand’s relative position to the body are the basis of structured expressions. The importance of hand gesture recognition has increased due to the prevalence of touchless applications and the rapid growth of the hearing-impaired population. However, developing an efficient recognition system needs to overcome the challenges of hand segmentation, local hand shape representation, global body configuration representation, and gesture sequence modeling. In this paper, a novel system is proposed for dynamic hand gesture recognition using multiple deep learning architectures for hand segmentation, local and global feature representations, and sequence feature globalization and recognition. The proposed system is evaluated on a very challenging dataset, which consists of dynamic hand gestures performed by subjects in an uncontrolled environment. The results show that the proposed system outperforms state-of- the-art approaches, demonstrating its effectiveness.

3D Hand Pose and Shape Estimation from Single RGB Image for Augmented Reality

In the realm of Human-Computer Interaction (HCI), the importance of hands cannot be overstated. Hands serve as a fundamental means of communication, expression, and interaction in the physical world. In recent years, Augmented Reality (AR) has emerged as a next-generation technology that seamlessly merges the digital and physical worlds, providing transformative experiences across various domains. In this context, accurate hand pose and shape estimation plays a crucial role in enabling natural and intuitive interactions within AR environments. Augmented Reality, with its ability to overlay digital information onto the real world, has the potential to revolutionize how we interact with technology. From gaming and education to healthcare and industrial training, AR has opened up new possibilities for enhancing user experiences. This study proposes an innovative approach for hand pose and shape estimation in AR applications. The methodology commences with the utilization of a pre-trained Single Shot Multi-Box (SSD) model for hand detection and cropping. The cropped hand image is then transformed into the HSV color model, followed by applying histogram equalization on the value band. To precisely isolate the hand, specific bounds are set for each band of the HSV color space, generating a mask. To refine the mask and diminish noise, contouring techniques are applied to the mask, and gap-filling methods are employed. The resultant refined mask is then combined with the original cropped image through logical AND operations to accurately delineate the hand boundaries. This meticulous approach ensures robust hand detection even in complex scenes. To extract pertinent features, the detected hand undergoes two concurrent processes. Firstly, the Scale-Invariant Feature Transform (SIFT) algorithm identifies distinctive keypoints on the hand's outer surface. Simultaneously, a pre-trained lightweight Convolutional Neural Network (CNN), namely MobileNet, is employed to extract 3D hand landmarks, the hand's center (middle finger metacarpophalangeal joint), and handedness information. These extracted features, encompassing hand keypoints, landmarks, center, and handedness, are aggregated and compiled into a CSV file for further analysis. A Gated Recurrent Unit (GRU) is then employed to process the features, capturing intricate dependencies between them. The GRU model successfully predicts the 3D hand pose, achieving high accuracy even in dynamic scenarios. The evaluation results for the proposed model are very promising that the Mean Per Joint Position Error in 3D (MPJPE) is 0.0596 between the predicted pose and the ground truth hand landmarks, while the Percentage of Correct Keypoints (PCK) is 95%. Upon predicting the hand pose, a mesh representation is employed to reconstruct the 3D shape of the hand. This mesh provides a tangible representation of the hand's structure and orientation, enhancing the realism and usability of the AR application. By integrating sophisticated detection, feature extraction, and predictive modeling techniques, this method contributes to creating more immersive and intuitive AR experiences, thereby fostering the seamless fusion of the digital and physical worlds.

A Hybrid Deep Learning Model for Real Time Hand Gestures Recognition

The performance of Hand Gesture Recognition (HGR) depends on the hand shape. Segmentation helps in the recognition of hand gestures for more accuracy and improves the overall performance compared to other existing deep neural networks. The crucial segmentation task is extremely complicated because of the background complexity, variation in illumination etc. The proposed modified UNET and ensemble model of Convolutional Neural Networks (CNN) undergoes a two stage process and results in proper hand gesture recognition. The first stage is segmenting the regions of the hand and the second stage is gesture identification. The modified UNET segmentation model is trained using resized images to generate a cost effective semantic segmentation model. The Central Processing Unit (CPU) utilization and training time taken by these models with respect to three public benchmark datasets are also analyzed. Recognition is carried out with the ensemble learning model consisting of EfficientNet B0, EfficientNet B4 and ResNet V2 152. Experimentation on NUS hand posture dataset-II, OUHANDS and HGRI benchmark datasets show that our architecture achieves a maximum recognition rate of 99.07% through semantic segmentation and the Ensemble learning model.

A Novel Machine Learning–Based Hand Gesture Recognition Using HCI on IoT Assisted Cloud Platform

Machine learning is a technique for analyzing data that aids the construction of mathematical models. Because of the growth of the Internet of Things (IoT) and wearable sensor devices, gesture interfaces are becoming a more natural and expedient human-machine interaction method. This type of artificial intelligence that requires minimal or no direct human intervention in decision-making is predicated on the ability of intelligent systems to self-train and detect patterns. The rise of touch-free applications and the number of deaf people have increased the significance of hand gesture recognition. Potential applications of hand gesture recognition research span from online gaming to surgical robotics. The location of the hands, the alignment of the fingers, and the hand-to-body posture are the fundamental components of hierarchical emotions in gestures. Linguistic gestures may be difficult to distinguish from nonsensical motions in the field of gesture recognition. Linguistic gestures may be difficult to distinguish from nonsensical motions in the field of gesture recognition. In this scenario, it may be difficult to overcome segmentation uncertainty caused by accidental hand motions or trembling. When a user performs the same dynamic gesture, the hand shapes and speeds of each user, as well as those often generated by the same user, vary. A machine-learning-based Gesture Recognition Framework (ML-GRF) for recognizing the beginning and end of a gesture sequence in a continuous stream of data is suggested to solve the problem of distinguishing between meaningful dynamic gestures and scattered generation. We have recommended using a similarity matching-based gesture classification approach to reduce the overall computing cost associated with identifying actions, and we have shown how an efficient feature extraction method can be used to reduce the thousands of single gesture information to four binary digit gesture codes. The findings from the simulation support the accuracy, precision, gesture recognition, sensitivity, and efficiency rates. The Machine Learning-based Gesture Recognition Framework (ML-GRF) had an accuracy rate of 98.97%, a precision rate of 97.65%, a gesture recognition rate of 98.04%, a sensitivity rate of 96.99%, and an efficiency rate of 95.12%.

Flexible soft Pneumatic Bionic Hand Based on Multi-Jointed Structure

Compared with rigid robotic hands, soft hands can provide better safety and adaptability. In the process of gradual development, the multi-jointed structure that mimics the shape of the human hand has shown significant progress in realizing its personification. In this article, we propose a multi-jointed pneumatic soft hand, which is composed of multi-jointed soft fingers, thumb, thenar and 3D printed palm. It can express letters through sign language gestures and can grasp objects with different sizes, shapes, weights and surface textures. We tested the bending ability of the actuators under different air pressures to characterize the performance of actuators made of silicone rubber. Based on the strain energy density function of silicone rubber, the Yeoh model is used to calculate the relationship between the air pressure required and the bending angle. In addition, a dedicated pneumatic control system is designed and manufactured to enable the soft hand to automatically perform tasks set by the specific program. This new multi-jointed pneumatic soft hand has flexible bionic fingers, and has the advantages of fast response speed, low cost, easy manufacturing, assembly and replacement.

Multiscaled Multi-Head Attention-Based Video Transformer Network for Hand Gesture Recognition

Dynamic gesture recognition is one of the challenging research areas due to variations in pose, size, and shape of the signer's hand. In this letter, Multiscaled Multi-Head Attention Video Transformer Network (MsMHA-VTN) for dynamic hand gesture recognition is proposed. A pyramidal hierarchy of multiscale features is extracted using the transformer multiscaled head attention model. The proposed model employs different attention dimensions for each head of the transformer which enables it to provide attention at the multiscale level. Further, in addition to single modality, recognition performance using multiple modalities is examined. Extensive experiments demonstrate the superior performance of the proposed MsMHA-VTN with an overall accuracy of 88.22% and 99.10% on NVGesture and Briareo datasets, respectively.

Enhanced Bald Eagle Search Optimizer with Transfer Learning-based Sign Language Recognition for Hearing-impaired Persons

Sign language recognition and classification for hearing-impaired people is a vital application of computer vision (CV) and machine learning (ML) approaches. It contains developing structures that take sign language gestures carried out by individuals and transform them into textual or auditory output for transmission aspects. It is critical to realize that establishing a robust and correct sign language recognition and classification method is a difficult task because of several challenges like differences in signing styles, occlusions, lighting conditions, and individual variances in hand movements and shapes. Thus, it needs a group of CV approaches, ML systems, and a varied and representative database for training and testing. In this study, we propose an Enhanced Bald Eagle Search Optimizer with Transfer Learning Sign Language Recognition (EBESO-TLSLR) technique for hearing-impaired persons. The presented EBESO-TLSLR technique aims to offer effective communication among hearing-impaired persons and normal persons using deep learning models. In the EBESO-TLSLR technique, the SqueezeNet model is used for feature map generation. For recognition of sign language classes, the long short-term memory (LSTM) method can be used. Finally, the EBESO approach is exploited for the optimal hyperparameter election of the LSTM method. The simulation results of the EBESO-TLSLR method are validated on the sign language dataset. The simulation outcomes illustrate the superior results of the EBESO-TLSLR technique in terms of different measures.

Data Glove with Self-Compensation Mechanism Based on High-Sensitive Elastic Fiber-Optic Sensor

With the development of virtual reality (VR) interaction technology, data glove has become one of the most popular devices for human-computer interaction. It's valuable to design high-sensitive and flexible sensor for data glove. Therefore, a low-cost data glove based on self-compensating elastic optical fiber sensor with self-calibration function is proposed. The tunable and stretchable elastic fiber was fabricated by a simple, economical and controllable method. The fiber has good flexibility and high stability under stretching, bending and indentation deformation. The optical fibers are installed in the sensor in a U shape with a bending radius of 5 mm. Compared with the straight fiber, the response sensitivity of the U-shaped fiber to deformation is increased by about 7 times at most. The reference optical fiber is connected to the sensor, which effectively improves the stability and accuracy of the sensor system. In addition, the sensors are easy to install so that the data gloves can be customized for different hand shapes. In the gesture capture test, it can respond quickly and guide the manipulator to track the gesture. This responsive and stable data glove has broad development potential in motion monitoring, telemedicine and human-computer interaction.

HandVoxNet++: 3D Hand Shape and Pose Estimation Using Voxel-Based Neural Networks.

3D hand shape and pose estimation from a single depth map is a new and challenging computer vision problem with many applications. Existing methods addressing it directly regress hand meshes via 2D convolutional neural networks, which leads to artifacts due to perspective distortions in the images. To address the limitations of the existing methods, we develop HandVoxNet++, i.e., a voxel-based deep network with 3D and graph convolutions trained in a fully supervised manner. The input to our network is a 3D voxelized-depth-map-based on the truncated signed distance function (TSDF). HandVoxNet++ relies on two hand shape representations. The first one is the 3D voxelized grid of hand shape, which does not preserve the mesh topology and which is the most accurate representation. The second representation is the hand surface that preserves the mesh topology. We combine the advantages of both representations by aligning the hand surface to the voxelized hand shape either with a new neural Graph-Convolutions-based Mesh Registration (GCN-MeshReg) or classical segment-wise Non-Rigid Gravitational Approach (NRGA++) which does not rely on training data. In extensive evaluations on three public benchmarks, i.e., SynHand5M, depth-based HANDS19 challenge and HO-3D, the proposed HandVoxNet++ achieves the state-of-the-art performance. In this journal extension of our previous approach presented at CVPR 2020, we gain 41.09% and 13.7% higher shape alignment accuracy on SynHand5M and HANDS19 datasets, respectively. Our method is ranked first on the HANDS19 challenge dataset (Task 1: Depth-Based 3D Hand Pose Estimation) at the moment of the submission of our results to the portal in August 2020.

Local Scale Adaptation to Hand Shape Model for Accurate and Robust Hand Tracking

AbstractThe accuracy of hand tracking algorithms depends on how closely the geometry of the mesh model resembles the user's hand shape. Most existing methods rely on a learned shape space model; however, this fails to generalize to unseen hand shapes with significant deviations from the training set. We introduce local scale adaptation to augment this data‐driven shape model and thus enable modeling hands of substantially different sizes. We also present a framework to calibrate our proposed hand shape model by registering it to depth data and achieve accurate and robust tracking. We demonstrate the capability of our proposed adaptive shape model over the most widely used existing hand model by registering it to subjects from different demographics. We also validate the accuracy and robustness of our tracking framework on challenging public hand datasets where we improve over state‐of‐the‐art methods. Our adaptive hand shape model and tracking framework offer a significant boost towards generalizing the accuracy of hand tracking.

“Gesture Identifier”

Abstract: In modern society, where information security is emphasized, the importance of biometrics, which identifies individuals based on their physical and behavioral characteristics, is increasing. Hand geometry based biometric systems are gaining acceptance in low-to-medium security applications. Hand geometry -based identification systems use geometric characteristics of the hand, such as finger length and width, palm diameter and circumference. The proposed system is an authentication system that uses these hand shape features for user authentication. This project presents a low-cost, high-performance, and easy-to-use hand-geometry based biometric personal authentication system. One of the novelties of this work involves the introduction of position-based, position-independent feature extraction and hand shape identification, useful for problems related to image processing and pattern recognition. Today, the presence of students (classes) has become an important part of every organization/institution. Traditional methods of recording attendance through paper attribution or signatures are very time consuming, insecure and inefficient. This paper describes manual student attendance management in a computerized system for convenience or data reliability. Therefore, the system was developed by integrating a ubiquitous computing system into the classroom to manage student attendance with palmprint scanners

Simulation of Hand Anatomy Using Medical Imaging

Precision modeling of the hand internal musculoskeletal anatomy has been largely limited to individual poses, and has not been connected into continuous volumetric motion of the hand anatomy actuating across the hand's entire range of motion. This is for a good reason, as hand anatomy and its motion are extremely complex and cannot be predicted merely from the anatomy in a single pose. We give a method to simulate the volumetric shape of hand's musculoskeletal organs to any pose in the hand's range of motion, producing external hand shapes and internal organ shapes that match ground truth optical scans and medical images (MRI) in multiple scanned poses. We achieve this by combining MRI images in multiple hand poses with FEM multibody nonlinear elastoplastic simulation. Our system models bones, muscles, tendons, joint ligaments and fat as separate volumetric organs that mechanically interact through contact and attachments, and whose shape matches medical images (MRI) in the MRI-scanned hand poses. The match to MRI is achieved by incorporating pose-space deformation and plastic strains into the simulation. We show how to do this in a non-intrusive manner that still retains all the simulation benefits, namely the ability to prescribe realistic material properties, generalize to arbitrary poses, preserve volume and obey contacts and attachments. We use our method to produce volumetric renders of the internal anatomy of the human hand in motion, and to compute and render highly realistic hand surface shapes. We evaluate our method by comparing it to optical scans, and demonstrate that we qualitatively and quantitatively substantially decrease the error compared to previous work. We test our method on five complex hand sequences, generated either using keyframe animation or performance animation using modern hand tracking techniques.

Interactive Dashboard with Visual Sensing and Fast Reactivity

These days, technology is growing rapidly, and the market has been introduced with lots of fascinating ways to interact with computers. The advancement of deep learning models and hardware technology also enables more applications with fancy features to be built. The importance of hand gesture recognition has increased due to the prevalence of touchless applications. However, developing an efficient recognition system needs to overcome the challenges of hand segmentation, local hand shape representation, global body configuration representation, and a gesture sequence model. This paper proposed an interactive dashboard that could react to hand gestures. This is also an initiative of the Tunku Abdul Rahman University College (TAR UC) Smart Campus project. Deep learning models were investigated in this research and the optimal model was selected for the dashboard. In addition, 20BN Jester Dataset was used for the dashboard development. To set up a more user-friendly dashboard, the data communication stream between the captured input stream and commands among the devices were studied. As to achieve higher responsiveness from the dashboard, evaluation on data communication protocols which were used to pass the input data were included in the study.

Deep Neural Network Regression of MANO from a Single Image

In order to improve the precision and efficiency of estimating the pose and shape of the 3D human hand from a monocular RGB image and accelerate 3D reconstruction, combining the efficiency of deep neural networks and accuracy of traditional optimization, DeepMANO is proposed. First, a DNN model is used to extract sparse features from image, then using these features to regress MANO model parameters. The regressed parameters are then used to initialize the iterative optimization routine which fits MANO to 3D joints. Finally, the parameters obtained by the routine are employed to supervise the network. Two large hand motion datasets, FreiHand and ObMan, are utilized to train the network. The approach assumes a weak perspective model. Experimental results show that the hand pose error is reduced by 52% and mesh error is reduced by 59% compared with state-of-the-art (SOTA) methods. The running time is about three times faster than SOTA methods.

Performance boosting of image matching-based iris recognition systems using deformable circular hollow kernels and uniform histogram fusion images

Identification of people using different biometric data is becoming more important in network society. Biometrics include voice, ears, palms, fingerprints, faces, iris, retina, and hand shapes. Among these features, iris detection gets more attention because each iris type is unique and does not change throughout life. In this study, an iris recognition framework is proposed using deformable circular hollow kernels and uniform histogram fusion images (UHFIs). This system introduces two different approaches for the iris recognition: one with image matching without machine learning and the other one with machine learning. In the first approach, image matching through Gabor features is employed. After getting the circularly cropped UHFI, Gabor features are extracted and employed in the image matching-based recognition system. In addition, a normalized cross-correlation coefficient is used as a similarity metric to compare the Gabor feature vectors. In the second approach, Gabor feature images are extracted and employed in deep learning (DL). According to the experimental results, the proposed system reaches around 89% accuracy on MMU1, 86% accuracy on MMU2, and about 50% accuracy on the CASIAV3 and CASIAV4 datasets when no machine learning is employed. Extensive benchmarking results also indicate that the proposed system boosts the performance of conventional systems by at least around 40% in terms of accuracy in the absence of machine learning. However, in the presence of machine learning, experimental results show that the proposed method with DL achieves an accuracy of up to 100% for MMU1, MMU2, CASIAV3, and CASIAV4 datasets.

Grassmannian learning mutual subspace method for image set recognition

This paper addresses the problem of object recognition given a set of images as input (e.g., multiple camera sources and video frames). Convolutional neural network (CNN)-based frameworks do not exploit these sets effectively, processing a pattern as observed, not capturing the underlying feature distribution as it does not consider the variance of images in the set. To address this issue, we propose the Grassmannian learning mutual subspace method (G-LMSM), a NN layer embedded on top of CNNs that can process image sets more effectively and can be trained in an end-to-end manner. The image set is first represented by a low-dimensional input subspace and then this input subspace is matched with dictionary subspaces by a similarity of their canonical angles, an interpretable and easy to compute metric. The key idea of G-LMSM is that the dictionary subspaces are learned as points on the Grassmann manifold, optimized with Riemannian stochastic gradient descent. This learning is stable, efficient and theoretically well-grounded. We demonstrate the effectiveness of our proposed method on hand shape recognition, face identification, and facial emotion recognition.

Parental Perspectives on Adolescent Health-Related Confidentiality: Trust, Responsibility, and Disease Etiology as Key Themes

PurposeAs children progress through adolescence, they become more independent and more responsible regarding their health. This shift in responsibility from the parents to the adolescent poses a challenge for healthcare professionals who must consider both parties. Pediatricians and other healthcare professionals may encounter problems regarding consent and confidentiality. This study aimed to investigate the opinions of Belgian parents of adolescents concerning cases about confidentiality in adolescent health problems. MethodsA qualitative methodology with semi-structured interviews and a case-based approach was chosen to answer our study aim. Belgian parents of adolescents were recruited voluntarily; 20 parents were interviewed. Parents’ opinions on four different cases regarding confidentiality were obtained. Interviews were audio- and video-recorded and transcribed verbatim. Independent coding of the transcripts was conducted. ResultsParents’ opinions differ considerably when asked if a physician has to maintain confidentiality toward the adolescent, depending on the content of the case. Opinions appear underpinned by three factors: trust, responsibility of the different parties, and the etiology of the problem. DiscussionThis study shows that the nature, severity, and frequency of the medical issue at hand shape the opinions of parents toward patient confidentiality, on top of the trust and responsibility factors also highlighted in previous work. This is in contrast to the Belgian legislation, which focuses on maturity regardless of context.

Detection of The Deaf Signal Language Using The Single Shot Detection (SSD) Method

Sign Language is a language that prioritizes manual communication, body language, and lip movements, instead of sound, to communicate. Deaf people are the main group who use this language, usually by combining hand shape, orientation and movement of the hands, arms, and body, and facial expressions to express their thoughts. Therefore, the researcher created an image recognition program in sign language using the Single Shot Detection (SSD) method, which is a convolution activity by combining several layers of preparation, by utilizing several components that move together and are motivated by a biological sensory system. The letters used in making sign language programs use the letters of the alphabet (az). This sign language detection programming that runs on the Google Collaboratory application