Hand Segmentation Research Articles

A Vision-Based Framework for Spotting and Segmentation of Gesture-Based Assamese Characters Written in the Air

The task of automatic gesture spotting and segmentation is challenging for determining the meaningful gesture patterns from continuous gesture-based character sequences. This paper proposes a vision-based automatic method that handles hand gesture spotting and segmentation of gestural characters embedded in a continuous character stream simultaneously, by employing a hybrid geometrical and statistical feature set. This framework shall form an important constituent of gesture-based character recognition (GBCR) systems, which has gained tremendous demand lately as assistive aids for overcoming the restraints faced by people with physical impairments. The performance of the proposed system is validated by taking into account the vowels and numerals of Assamese vocabulary. Another attribute to this proposed system is the implementation of an effective hand segmentation module, which enables it to tackle complex background settings.

Elderly Care Based on Hand Gestures Using Kinect Sensor

Technological advances have allowed hand gestures to become an important research field especially in applications such as health care and assisting applications for elderly people, providing a natural interaction with the assisting system through a camera by making specific gestures. In this study, we proposed three different scenarios using a Microsoft Kinect V2 depth sensor then evaluated the effectiveness of the outcomes. The first scenario used joint tracking combined with a depth threshold to enhance hand segmentation and efficiently recognise the number of fingers extended. The second scenario utilised the metadata parameters provided by the Kinect V2 depth sensor, which provided 11 parameters related to the tracked body and gave information about three gestures for each hand. The third scenario used a simple convolutional neural network with joint tracking by depth metadata to recognise and classify five hand gesture categories. In this study, deaf-mute elderly people performed five different hand gestures, each related to a specific request, such as needing water, meal, toilet, help and medicine. Next, the request was sent via the global system for mobile communication (GSM) as a text message to the care provider’s smartphone because the elderly subjects could not execute any activity independently.

골볼 터닝 드로우 동작의 운동역학적 분석

The purpose of this study is to analyze the kinematics of the goal ball turning throwing motion, and the conclusions are as follows. First, the total time for goal ball turning and throwing motion was 1.82 s, 76.95 % in P1 and 23.02 % in P2. Second, it was found that the COM (center of mass) left and right displacement of the goalball turning throwing motion moved leftward from P1 (E1~E2) and moved to the right from P2 (E2~E3). COM forward and backwark displacement Moved to the forward according to the progress of the event. COM vertical displacement moved downward according to the progress of the event. Third, the COM left and right velocities of the goal ball turning throwing motion showed the fastest velocity from E3 to the right, and the COM forward and backward velocities showed the fastest speed from E2 to the forward. COM vertical velocity showed the fastest downward velocity at E2. Fourth, when the goal ball turning throwing motion was released, the velocity of the ball was 12.00±0.88 ㎧, and the stride width was 1.09±0.12 m. Fifth, the body segment’s angular momentum at the time of release of the goal ball turning throwing motion was found to be the largest in the trunk. And the upper arm, forearm, and hand segment of the upper limb all showed greater angular momentum than the left. In the case of the lower extremities, the right thigh and foot segments showed greater angular momentum than the left, but the left shank segment showed greater angular momentum than the right. Sixth, when the goal ball turning throwing motion was released, the ground reaction force of the forefoot showed the force in the right direction, the rear direction, and the vertical upward direction, and at this time, the large force was in the order of Fz, Fy, and Fx.

Ensemble Learning with Multiclassifiers on Pediatric Hand Radiograph Segmentation for Bone Age Assessment.

In the study of pediatric automatic bone age assessment (BAA) in clinical practice, the extraction of the object area in hand radiographs is an important part, which directly affects the prediction accuracy of the BAA. But no perfect segmentation solution has been found yet. This work is to develop an automatic hand radiograph segmentation method with high precision and efficiency. We considered the hand segmentation task as a classification problem. The optimal segmentation threshold for each image was regarded as the prediction target. We utilized the normalized histogram, mean value, and variance of each image as input features to train the classification model, based on ensemble learning with multiple classifiers. 600 left-hand radiographs with the bone age ranging from 1 to 18 years old were included in the dataset. Compared with traditional segmentation methods and the state-of-the-art U-Net network, the proposed method performed better with a higher precision and less computational load, achieving an average PSNR of 52.43 dB, SSIM of 0.97, DSC of 0.97, and JSI of 0.91, which is more suitable in clinical application. Furthermore, the experimental results also verified that hand radiograph segmentation could bring an average improvement for BAA performance of at least 13%.

Automated Bone Age Assessment with Image Registration Using Hand X-ray Images

One of the methods for identifying growth disorder is by assessing the skeletal bone age. A child with a healthy growth rate will have approximately the same chronological and bone ages. It is important to detect any growth disorder as early as possible, so that mitigation treatment can be administered with less negative consequences. Recently, the most popular approach in assessing the discrepancy between bone and chronological ages is through the subjective protocol of Tanner–Whitehouse that assesses selected regions in the hand X-ray images. This approach relies heavily on the medical personnel experience, which produces a high intra-observer bias. Therefore, an automated bone age prediction system with image registration using hand X-ray images is proposed in order to complement the inexperienced doctors by providing the second opinion. The system relies on an optimized regression network using a novel residual separable convolution model. The regressor network requires an input image to be 299 × 299 pixels, which will be mapped to the predicted bone age through three modules of the Xception network. Moreover, the images will be pre-processed or registered first to a standardized and normalized pose using separable convolutional neural networks. Three steps image registration are performed by segmenting the hand regions, which will be rotated using angle calculated from four keypoints of interest, before positional alignment is applied to ensure the region of interest is located in the middle. The hand segmentation is based on DeepLab V3 plus architecture, while keypoints regressor for angle alignment is based on MobileNet V1 architecture, where both of them use separable convolution as the core operators. To avoid the pitfall of underfitting, synthetic data are generated while using various rotation angles, zooming factors, and shearing images in order to augment the training dataset. The experimental results show that the proposed method returns the lowest mean absolute error and mean squared error of 8.200 months and 121.902 months2, respectively. Hence, an error of less than one year is acceptable in predicting the bone age, which can serve as a good supplement tool for providing the second expert opinion. This work does not consider gender information, which is crucial in making a better prediction, as the male and female bone structures are naturally different.

A new framework for sign language alphabet hand posture recognition using geometrical features through artificial neural network (part 1)

Hand pose tracking is essential in sign languages. An automatic recognition of performed hand signs facilitates a number of applications, especially for people with speech impairment to communication with normal people. This framework which is called ASLNN proposes a new hand posture recognition technique for the American sign language alphabet based on the neural network which works on the geometrical feature extraction of hands. A user’s hand is captured by a three-dimensional depth-based sensor camera; consequently, the hand is segmented according to the depth analysis features. The proposed system is called depth-based geometrical sign language recognition as named DGSLR. The DGSLR adopted in easier hand segmentation approach, which is further used in segmentation applications. The proposed geometrical feature extraction framework improves the accuracy of recognition due to unchangeable features against hand orientation compared to discrete cosine transform and moment invariant. The findings of the iterations demonstrate the combination of the extracted features resulted to improved accuracy rates. Then, an artificial neural network is used to drive desired outcomes. ASLNN is proficient to hand posture recognition and provides accuracy up to 96.78% which will be discussed on the additional paper of this authors in this journal.

Shoulder muscle activation strategies differ when lifting or lowering a load.

Lowering a load could be associated with abnormal shoulder and scapular motion. We tested the hypothesis that lowering a load involves different shoulder muscle coordination strategies compared to lifting a load. EMG activity of 13 muscles was recorded in 30 healthy volunteers who lifted and lowered a 6, 12 or 18kg box between three shelves. Kinematics, EMG levels and muscle synergies, extracted using non-negative matrix factorization, were analyzed. We found greater muscle activity level during lowering in four muscles (+ 1-2% MVC in anterior deltoid, biceps brachii, serratus anterior and pectoralis major). The movements were performed faster during lifting (18.2 vs. 15.9cm/s) but with similar hand paths and segment kinematics. The number of synergies was the same in both tasks. Two synergies were identified in ~ 75% of subjects, and one synergy in the others. Synergy #1 mainly activated prime movers' muscles, while synergy #2 co-activated several antagonist muscles. Synergies' structure was similar between lifting and lowering (Pearson's r ≈ 0.9 for synergy #1 and 0.7-08 for synergy #2). Synergy #2 was more activated during lowering and explained the greater activity observed in anterior deltoid, serratus anterior and pectoralis. Lifting and lowering a load were associated with similar synergy structure. In 3/4 of subjects, lowering movements involved greater activation of a "multiple antagonists" synergy. The other subjects co-contracted all shoulder muscles as a unit in both conditions. These inter-individual differences should be investigated in the occurrence of shoulder musculoskeletal disorders.

DBSCAN for Hand Tracking and Gesture Recognition

Hand segmentation and tracking are important issues for hand-gesture recognition. Using depth data, it can speed up the segmentation process because we can delete unnecessary data like the background of the image easily. In this research, we modify DBSCAN clustering algorithm to make it faster and suitable for our system. This method is used in both hand tracking and hand gesture recognition. The results show that our method performs well in this system. The proposed method can outperform the original DBSCAN and the other clustering method in terms of computational time.

Hand Gesture Recognition Based on Computer Vision: A Review of Techniques.

Hand gestures are a form of nonverbal communication that can be used in several fields such as communication between deaf-mute people, robot control, human–computer interaction (HCI), home automation and medical applications. Research papers based on hand gestures have adopted many different techniques, including those based on instrumented sensor technology and computer vision. In other words, the hand sign can be classified under many headings, such as posture and gesture, as well as dynamic and static, or a hybrid of the two. This paper focuses on a review of the literature on hand gesture techniques and introduces their merits and limitations under different circumstances. In addition, it tabulates the performance of these methods, focusing on computer vision techniques that deal with the similarity and difference points, technique of hand segmentation used, classification algorithms and drawbacks, number and types of gestures, dataset used, detection range (distance) and type of camera used. This paper is a thorough general overview of hand gesture methods with a brief discussion of some possible applications.

Kinematic sequence and time lag for pitch and lob shots in female pro-golfers

ABSTRACT This study aimed to determine the differences in the kinematic sequence and time lag of body segments for shots (pitch and lob shots) at different distances (20 and 30 m) around the green. Ten skilled female golfers participated voluntarily. A repeated-measures one-way analysis of variance was performed to assess the effect of the peak angular velocity of the shots (20-m pitch, 30-m pitch, 20-m lob, and 30-m lob shots) on the segments and the effect of the peak angular velocity and time lag of the body segments (pelvis, thorax, upper arm, forearm, and hand) on the shots. All shots taken revealed a summation effect of velocity from proximal to distal body segments, regardless of the shot type (pitch or lob shot) or distance (20 or 30 m). The peak angular velocity of the body segments demonstrated that the velocities associated with the 30-m shots were significantly higher than those associated with the 20-m shots, and the velocities associated with lob shots were significantly higher than those associated with pitch shots, irrespective of the shot distance. This study provided additional insight into the mechanism for improving performances in the shots around the green.

Generalizability of Hand-Object Interaction Detection in Egocentric Video across Populations with Hand Impairment.

Stroke survivors often experience unilateral sensorimotor impairment. The restoration of upper limb function is an important determinant of quality of life after stroke. Wearable technologies that can measure hand function at home are needed to assess the impact of new interventions. Egocentric cameras combined with computer vision algorithms have been proposed as a means to capture hand use in unconstrained environments, and have shown promising results in this application for individuals with cervical spinal cord injury (cSCI). The objective of this study was to examine the generalizability of this approach to individuals who have experienced a stroke. An egocentric camera was used to capture the hand use (hand-object interactions) of 6 stroke survivors performing daily tasks in a home simulation laboratory. The interaction detection classifier previously trained on 9 individuals with cSCI was applied to detect hand use in the stroke survivors. The processing pipeline consisted of hand detection, hand segmentation, feature extraction, and interaction detection. The resulting average F1 scores for affected and unaffected hands were 0.66 ± 0.25 and 0.80 ± 0.15, respectively, indicating that the approach is feasible and has the potential to generalize to stroke survivors. Using stroke-specific training data may further increase the accuracy obtained for the affected hand.

Developing a hybrid soft mechanism for assisting individualized flexion and extension of finger joints.

Various robotic devices have been developed for home rehabilitation and support of therapists. Special attention has been focused on soft actuators due to their high viscoelasticity and flexibility, which can contribute to the safety and affinity with the users. However, most of them aimed at the assist of finger flexion, and few have been designed to support extension actively. Moreover, for most soft actuator based mechanisms, the individual-adaptability have not been considered nor appropriately evaluated. Consequently, the effect of individual difference on the assistance using soft robotic devices is unclear, and for the purpose of dealing with the individual difference, the whole mechanism should be designed and fabricated for each individual user, which is ineffective for the rehabilitation support. In this study, we proposed a hybrid soft mechanism with modularized fiber-reinforced elastomer actuators for joint-dependent flexion support and McKibben actuators for finger extension support. Without further changing the design of the elastomer actuator, the hybrid mechanism could be adapted to individual hand difference: proportions of hand segments, range of motion (ROM) and torque characteristics of the joints. A prototype of the mechanism was fabricated and evaluated. The results showed that the mechanism could meet the requirement of finger function assist. Moreover, the mechanism could be fine-tuned towards the individual hand by changing the fiber-reinforcement and adjusting the fittings of the actuators.

Optimisation of both classifier and fusion based feature set for static American sign language recognition

Sign language recognition becomes a popular research field in human–computer interaction. Attention on hand signal analysis helps to make easy communication among computer and human for information sharing. Major focus of the gesture recognition system is to identify and recognise various gestures, by a computer. This study introduces optimisation of both classifier and feature set for static American sign language recognition. Initially, the hand part is segmented from other parts of the image through effective edge and skin colour detection. Thereafter, robust features are obtained using discrete cosine transform, Zernike moment, scale-invariant feature transform, speeded-up robust features, histogram of oriented gradients and binary object features from the segmented hand image. From these extracted features, an optimal feature set is selected by social ski driver optimisation algorithm. Deep Elman recurrent neural network classifier is then introduced for recognition purpose. Optimisation is performed on feature sets, derived by fusion of features obtained from the above methods, based on precision, accuracy, F-measure and recall. Finally, optimised feature set and best classifier are used to recognise the hand gesture for classification purpose. The performance of this proposed method is evaluated and compared with existing literature.

A PointNet-Based Solution for 3D Hand Gesture Recognition.

Gesture recognition is an intensively researched area for several reasons. One of the most important reasons is because of this technology’s numerous application in various domains (e.g., robotics, games, medicine, automotive, etc.) Additionally, the introduction of three-dimensional (3D) image acquisition techniques (e.g., stereovision, projected-light, time-of-flight, etc.) overcomes the limitations of traditional two-dimensional (2D) approaches. Combined with the larger availability of 3D sensors (e.g., Microsoft Kinect, Intel RealSense, photonic mixer device (PMD), CamCube, etc.), recent interest in this domain has sparked. Moreover, in many computer vision tasks, the traditional statistic top approaches were outperformed by deep neural network-based solutions. In view of these considerations, we proposed a deep neural network solution by employing PointNet architecture for the problem of hand gesture recognition using depth data produced by a time of flight (ToF) sensor. We created a custom hand gesture dataset, then proposed a multistage hand segmentation by designing filtering, clustering, and finding the hand in the volume of interest and hand-forearm segmentation. For comparison purpose, two equivalent datasets were tested: a 3D point cloud dataset and a 2D image dataset, both obtained from the same stream. Besides the advantages of the 3D technology, the accuracy of the 3D method using PointNet is proven to outperform the 2D method in all circumstances, even the 2D method that employs a deep neural network.

Signer independent real-time hand gestures recognition using multi-features extraction and various classifiers

In this research paper, an effort has been placed to convert 24 American Sign Language (ASL) signer independent, real time hand gesture alphabets into human or machine recognizable English text. In the proposed work, the ASL hand gestures used for cognition and recognition process is completely invariant to scale, luminance, gender, and distance in the complex background of indoor location. The Viola-Jones algorithm, CIE Lab color model and canny approximation to the derivative are used for proper hand segmentation. In both the cognition and recognition process, the various features such as boundary, centroid, entropy, Hu moments, Zernike moments, Gabor filters, Histogram of Oriented Gradients (HOG) and Local Phase Quantization (LPQ) are extracted from the hand gestures. The K-Nearest Neighbor (KNN), Multiclass-Support Vector Machines (M-SVM) and Decision Tree (DT) classifiers are used for classifying the hand gestures. In recognition task, these classifiers are applied independently on the same set of hand gestures to check the optimality of recognition rate and recognition time. With the detailed experimentation, it is found that, the KNN classifier achieved an average recognition rate and average recognition time of 92.71% and 0.48 s per gestures. This recognition rate and time is better and optimal compared to M-SVM and DT classifiers. Also it is an inspiring result compared to state of art techniques in real time environment by considering various invariants.

Real-time hand area segmentation for hand gesture recognition in desktop environment

the process of separating the hand area from a complex background, knownas hand segmentation, is a prerequisite for any vision-based hand gesture recognitionsystem. In some applications, only a rough estimate of the hand area is needed, but in otherapplications an exact segmentation of the hand, if possible, is needed. In this paper, threemethods for extracting the hand from the background in real-time were tested. Thesemethods use off-line learning of skin color; these methods are Histogram Intersection,Color Histogram, and Skin Color Modeling and Adaptation. The three methods weretested using a video sequence of 100 frames using four different lighting conditions, with400 frames in total. The different lighting conditions are fluorescent light only, a mixture offluorescent and daylight, daylight without the sun light, and daylight with the sun light.These video streams were taken from the same person under the above different lightingconditions. A comparative study between the three methods was performed. The resultsshowed that hand segmentation using Skin Color Modeling and Adaptation withFluorescent light produced the best results among the three methods. The objective of ourresearch is to accurately recognize the hand gestures, and then use it in many differentapplications. The work in this paper is the first stage in our research. The contribution inthis paper is the comparative study between three different techniques that use skin-colormodeling under different lighting conditions.

DenseAttentionSeg: Segment hands from interacted objects using depth input

Hand segmentation is an important task in computer vision, which is usually the foundation of hand pose recognition, hand tracking, and reconstruction. For hand segmentation, it is more challenging when the hand is interacting with objects, but handling interacting motions is more important for applications like HCI and VR. In this paper, we propose a real-time DNN-based technique to segment hand and object in interacting motions from a single depth input. Our model is called DenseAttentionSeg, which contains a dense attention mechanism which effectively fuses information in different scales and improves the quality of result with skip-connections. Besides, we introduce a contour loss in model training, which helps to generate accurate hand and object boundaries. Finally, we propose our InterSegHands dataset, a fine-scale hand segmentation dataset containing about 52k depth maps of hand-object interactions, with the ground truth segmentation masks. Our experiments evaluate the effectiveness of our techniques and datasets, and indicate that our method outperforms the current state-of-the-art deep segmentation methods in handling hand-object interactions.

Analyzing the Response Behavior of Lumbriculus variegatus (Oligochaeta: Lumbriculidae) to Different Concentrations of Copper Sulfate Based on Line Body Shape Detection and a Recurrent Self-Organizing Map.

Point detection (e.g., the centroid of the body) of species has been conducted in numerous studies. However, line detection (i.e., the line body shape) of elongated species has rarely been investigated under stressful conditions. We analyzed the line movements of an Oligochaeta Lumbriculus variegatus in response to treatments with a toxic chemical, copper sulfate, at low concentrations (0.01 mg/L and 0.1 mg/L). The automatic line-tracking system was devised to identify the movement of body segments (body length) and the movements of segments (i.e., the speed and angles between segments) were recorded before and after treatment. Total body length was shortened from 31.22 (±5.18) mm to 20.91 (±4.65) mm after the 0.1 mg/L treatment. The Shannon entropy index decreased from 0.44 (±0.1) to 0.28 (±0.08) after treatment. On the other hand, the body and movement segments did not significantly change after the 0.01 mg/L treatment. Sequential movements of test organisms were further analyzed with a recurrent self-organizing map (RSOM) to determine the pattern of time-series line movements. The RSOM made it feasible to classify sequential behaviors of indicator organisms and identify various continuous body movements under stressful conditions.

A Study of Hand Gesture Segmentation Techniques for Sign Languages

This paper presents about an overview on several methods of segmentation techniques for hand gesture recognition. Hand gesture recognition has evolved tremendously in the recent years because of its ability to interact with machine. Mankind tries to incorporate human gestures into modern technologies like touching movement on screen, virtual reality gaming and sign language prediction. This research aims towards employed on hand gesture recognition for sign language interpretation as a human computer interaction application. Sign Language which uses transmits the sign patterns to convey meaning by hand shapes, orientation and movements to fluently express their thoughts with other person and is normally used by the physically challenged people who cannot speak or hear. Automatic Sign Language which requires robust and accurate techniques for identifying hand signs or a sequence of produced gesture to help interpret their correct meaning. Hand segmentation algorithm where segmentation using different hand detection schemes with required morphological processing. There are many methods which can be used to acquire the respective results depending on its advantage.

The influence of tracking marker locations on three-dimensional wrist kinematics

ObjectivesTo determine the influence of tracking marker locations on wrist kinematics during free movements and the golf swing, with the intention of recommending a solution that generates meaningful three-dimensional wrist kinematics. DesignRepeated measures. MethodsSix participants performed free movements of flexion/extension, radial/ulnar deviation and forearm supination/pronation, with a further sixteen participants performing golf drives. A passive motion capture system tracked four different marker sets located on participants’ hand and forearm segments. Variables of peak angle and range of motion were used to compare marker sets during free movements and angles at the top of the backswing and impact were compared during the golf swing. ResultsWrist marker set had a large (η2≥0.557) and often significant (p≤0.051) effect on the variables measured during free movements, and a mixed (η2≥0.108, p≤0.198) effect on wrist angles during the golf swing. Wrist axial rotation range of motion during free forearm supination/pronation revealed the greatest difference between marker sets (∼42°). The large values generated by two of the marker sets for this rotation appeared to influence the values of flexion/extension and radial/ulnar deviation during the golf swing. ConclusionsThe location of markers used to measure wrist kinematics can have a large effect on the angles generated. A solution of two markers located at the distal end of the forearm and one at the proximal, appears to minimise values of wrist axial rotation during free forearm supination/pronation and, consequently, produce more meaningful three-dimensional wrist kinematics.