Hand Segmentation Research Articles

Development of a 3D scan posture-correction procedure to facilitate the direct-digital splinting approach

ABSTRACTThe direct-digital upper-limb orthopaedic splinting process employs portable 3D scanning systems to capture anatomical surface geometry. However, the resultant quality of a complete upper-limb scan geometry may be detrimentally affected by involuntary movements of an unsupported limb, and the inability to maintain the correct posture due to patient disability. The aim of this research study is to develop an upper-limb 3D scanning posture-correction procedure that addresses these limitations. The methodology covers four stages: (a) 3D scanning; (b) rigging; (c) posture correction; and (d) geometry inspection. Manual and semi-automated rigging procedures were developed to define segments and joints of the wrist on the upper-limb scans. A semi-automated definition was developed to precisely and repeatedly articulate the defined hand segment. To assess feasibility and accuracy of the developed posture-correction procedure, the hand segment on scans in neutral hand posture were articulated with the developed posture-correction procedure as closely as possible to the corresponding hand positions on the scans in extended hand postures. The reported virtual posture-correction procedure provides a basis for research into anatomy articulation and the optimisation of complex splints and orthoses.

MobiGesture: Mobility-aware hand gesture recognition for healthcare

Abstract Accurate recognition of hand gestures while moving is still a significant challenge, which prevents the wide use of existing gesture recognition technology. In this paper, we propose a novel mobility-aware hand gesture segmentation algorithm to detect and segment hand gestures. We also propose a Convolutional Neural Network (CNN) to classify hand gestures with mobility noises. Based on the segmentation and classification algorithms, we develop MobiGesture, a mobility-aware hand gesture recognition system for healthcare. For the leave-one-subject-out cross-validation test, experiments with human subjects show that the proposed segmentation algorithm achieves 94.0% precision, and 91.2% recall when the user is moving. The proposed hand gesture classification algorithm is 16.1%, 15.3%, and 14.4% more accurate than state-of-the-art work when the user is standing, walking and jogging, respectively.

A musculoskeletal model of the hand and wrist: model definition and evaluation

To improve our understanding on the neuromechanics of finger movements, a comprehensive musculoskeletal model is needed. The aim of this study was to build a musculoskeletal model of the hand and wrist, based on one consistent data set of the relevant anatomical parameters. We built and tested a model including the hand and wrist segments, as well as the muscles of the forearm and hand in OpenSim. In total, the model comprises 19 segments (with the carpal bones modeled as one segment) with 23 degrees of freedom and 43 muscles. All required anatomical input data, including bone masses and inertias, joint axis positions and orientations as well as muscle morphological parameters (i.e. PCSA, mass, optimal fiber length and tendon length) were obtained from one cadaver of which the data set was recently published. Model validity was investigated by first comparing computed muscle moment arms at the index finger metacarpophalangeal (MCP) joint and wrist joint to published reference values. Secondly, the muscle forces during pinching were computed using static optimization and compared to previously measured intraoperative reference values. Computed and measured moment arms of muscles at both index MCP and wrist showed high correlation coefficients (r = 0.88 averaged across all muscles) and modest root mean square deviation (RMSD = 23% averaged across all muscles). Computed extrinsic flexor forces of the index finger during index pinch task were within one standard deviation of previously measured in-vivo tendon forces. These results provide an indication of model validity for use in estimating muscle forces during static tasks.

Multi‐modality‐based Arabic sign language recognition

With the increase in the number of deaf‐mute people in the Arab world and the lack of Arabic sign language (ArSL) recognition benchmark data sets, there is a pressing need for publishing a large‐volume and realistic ArSL data set. This study presents such a data set, which consists of 150 isolated ArSL signs. The data set is challenging due to the great similarity among hand shapes and motions in the collected signs. Along with the data set, a sign language recognition algorithm is presented. The authors’ proposed method consists of three major stages: hand segmentation, hand shape sequence and body motion description, and sign classification. The hand shape segmentation is based on the depth and position of the hand joints. Histograms of oriented gradients and principal component analysis are applied on the segmented hand shapes to obtain the hand shape sequence descriptor. The covariance of the three‐dimensional joints of the upper half of the skeleton in addition to the hand states and face properties are adopted for motion sequence description. The canonical correlation analysis and random forest classifiers are used for classification. The achieved accuracy is 55.57% over 150 ArSL signs, which is considered promising.

Coarse-to-fine online learning for hand segmentation in egocentric video

Hand segmentation is one of the most fundamental and crucial steps for egocentric human-computer interaction. The special egocentric view brings new challenges to hand segmentation tasks, such as the unpredictable environmental conditions. The performance of traditional hand segmentation methods depend on abundant manually labeled training data. However, these approaches do not appropriately capture the whole properties of egocentric human-computer interaction for neglecting the user-specific context. It is only necessary to build a personalized hand model of the active user. Based on this observation, we propose an online-learning hand segmentation approach without using manually labeled data for training. Our approach consists of top-down classifications and bottom-up optimizations. More specifically, we divide the segmentation task into three parts, a frame-level hand detection which detects the presence of the interactive hand using motion saliency and initializes hand masks for online learning, a superpixel-level hand classification which coarsely segments hand regions from which stable samples are selected for next level, and a pixel-level hand classification which produces a fine-grained hand segmentation. Based on the pixel-level classification result, we update the hand appearance model and optimize the upper layer classifier and detector. This online-learning strategy makes our approach robust to varying illumination conditions and hand appearances. Experimental results demonstrate the robustness of our approach.

A technique of hand digit reconstruction using scarred and deformed donor segments.

Hand digit reconstruction with transposition of hand segments in cases of hand injury is not used frequently by surgeons, at least in part because of potential ischemic complications after cicatricial changes in tissues. The aim of this study was to investigate the opportunities for expanding the set of indications for hand digit reconstruction using scarred and deformed donor segments. We developed a technique of ischemic preconditioning and preforming the segment before transfer to improve macro-circulation and micro-circulation in scarred donor segments with a view to preventing ischemic complications in reconstructed digits. It was validated using clinical, radiological, morphological, and biophysical (IR imaging, laser Doppler flowmetry, rheovasography) methods. A total of 184 patients were included. Primary digit reconstruction was performed in 19 patients while secondary reconstruction-in 165 patients. Single-stage transfer was performed in 122 hands, delayed transfer-in 67 hands. The total number of reconstructed thumbs and fingers was 203. Digit viability, stability, mobility and range of motion were very good with the use of new technique. When transferring hand segments with severe and highly severe deformations, tissue scarring and vascular damage, it is beneficial to pre-train the donor segment against hypoxia by both exsanguination and surgical methods.

A navel approach of gradient and non-gradient search algorithms for segmentation of hand written data

A navel approach of gradient and non-gradient search algorithms for segmentation of hand written data

Dynamic ROI extraction method for hand vein images

The region of interest (ROI) extraction is important in hand vein recognition system. The main challenges for accurate extraction of the vein region are to overcome variability in hand size, lighting conditions, orientation, appearance, noisy background, and non‐uniform grey levels in foreground region. Here, we propose a new dynamic hand vein ROI extraction, preserving the whole vein area. A hand segmentation process robust to the mentioned challenges, contributing to an accurate definition of hand edge delimitations is proposed. Our approach is validated on both dorsal vein Bosphorus database and palm vein Vera database. Our proposed method accuracy is ∼98% for Bosphorus database and 90% for Vera database. To illustrate the efficiency of the proposed ROI extraction, we insert it as a first block in a hand vein recognition system. Then, a comparison study at system level with recent approaches is carried on, showing an improvement of the whole system area under the curve by a rate of 12% and 2% for Bosphorus and Vera databases, respectively. The speed performances demonstrate a mean run time of 0.73 s for Bosphorus database and 1.2 s for Vera database, proving that the proposed method can be conveniently used on a real‐time application.

American Sign Language Alphabet Translator Android Application: Hand Shapes into Text

American Sign Language is used by the deaf-mute community in expressing their thoughts. Many applications have been developed to assist the deaf-mute person, but most of the application integrates on computer or in desktop. These leads to create a mobile application that will recognize hand pose into text. American Sign Language Alphabet Translator Android Application: Hand Shapes into Text is a mobile application that translates American Sign Language Alphabet into text. These applications translate the ASL alphabet to a person who is not capable in understanding the language of a deaf-mute person. The application used the process of hand segmentation and feature extraction to get information needed to extract the hand and data set that used to match the hand pose to display the equivalent letters. The study is successful in recognizing most of the alphabet hand shape as long as it is well detected.

HAND POSTURE RECOGNITION USING MODIFIED ENSEMBLE OF SHAPE FUNCTIONS AND GLOBAL RADIUS-BASED SURFACE DESCRIPTOR

This paper presents an approach to recognition of static hand gestures based on data acquired from 3D cameras and point cloud descriptors: Ensemble of Shape Functions and Global Radius-based Surface Descriptor. We describe the recognition algorithm consisting of: hand segmentation, noise removal and downsampling of point cloud, dividing point cloud bounding box to cells, feature extraction and normalization, gesture classification. Modifications of the descriptors are proposed in order to increase hand posture recognition rates and decrease quantity of used features as well as computational cost of the algorithm. The experiments performed on four challenging datasets using cross-validation tests prove the usefulness of our approach.

Research on Gesture Recognition Method Based on Computer Vision

Gesture recognition is an important way of human-computer interaction. With time going on, people are no longer satisfied with gesture recognition based on wearable devices, but hope to perform gesture recognition in a more natural way. Computer vision-based gesture recognition can transfer human feelings and instructions to computers conveniently and efficiently, and improve the efficiency of human-computer interaction significantly. The gesture recognition based on computer vision is mainly based on hidden Markov, dynamic time rounding algorithm and neural network algorithm. The process is roughly divided into three steps: image collection, hand segmentation, gesture recognition and classification. This paper reviews the computer vision-based gesture recognition methods in the past 20 years, analyses the research status at home and abroad, summarizes its current development, the advantages and disadvantages of different gesture recognition methods, and looks forward to the development trend of gesture recognition technology in the next stage.

Towards contactless palm region extraction in complex environment

Palm region of interest (ROI) extraction is an indispensable procedure in palmprint recognition. Prior works generally perform well on palm ROI extraction because of dedicated devices and well-controlled environment. To make hand placement less-constrained and improve usability, mobile palmprint recognition has attracted a wide attention in recent years. For mobile phone images captured in complex natural environment, palm ROI extraction is a challenging work due to varying illumination, complex background and contactless acquisition mode. In this paper, a mobile palmprint dataset (SPIC) is at first established with five smartphones, comprising 4000 images collected from 128 persons in two separate sessions. Furthermore, a novel pre-processing approach is proposed to achieve ROI extraction in mobile scenarios, which include colour component selection, learning-based fast hand segmentation and geometry-driven valley point location. Experimental results demonstrate that the proposed method can achieve high extraction accuracy and computational efficiency on PolyU1.0, HA-BJTU and SPIC palmprint databases.

Sign language recognition in complex background scene based on adaptive skin colour modelling and support vector machine

With the advances of wearable cameras, the user can record the first-person view videos for gesture recognition or even sign language recognition to help the deaf or hard of hearing people communicate with others. In this paper, we propose a purely vision-based sign language recognition system which can be used in complex background scene. We design an adaptive skin colour modelling method for hand segmentation so that the hand contour can be derived more accurately even when different users use our system in various light conditions. Four kinds of feature descriptors are integrated to describe the contours and the salient points of hand gestures, and support vector machine (SVM) is applied to classify hand gestures. Our recognition method is evaluated by two datasets: 1) the CSL dataset collected by ourselves in which images were captured in three different environments including complex background; 2) the public ASL dataset, in which images of the same gesture were captured in different lighting conditions. The proposed recognition method achieves acceptable accuracy rates of 100.0% and 94.0% for the CSL and ASL datasets, respectively.

Sign language recognition in complex background scene based on adaptive skin colour modelling and support vector machine

With the advances of wearable cameras, the user can record the first-person view videos for gesture recognition or even sign language recognition to help the deaf or hard of hearing people communicate with others. In this paper, we propose a purely vision-based sign language recognition system which can be used in complex background scene. We design an adaptive skin colour modelling method for hand segmentation so that the hand contour can be derived more accurately even when different users use our system in various light conditions. Four kinds of feature descriptors are integrated to describe the contours and the salient points of hand gestures, and support vector machine (SVM) is applied to classify hand gestures. Our recognition method is evaluated by two datasets: 1) the CSL dataset collected by ourselves in which images were captured in three different environments including complex background; 2) the public ASL dataset, in which images of the same gesture were captured in different lighting conditions. The proposed recognition method achieves acceptable accuracy rates of 100.0% and 94.0% for the CSL and ASL datasets, respectively.

Cost-management in correspondent banking relationships

Cost-management is essential and highly specific sphere, which requires applying adequate decision- making approach as a part of the bank’s internal value creation process. Correspondent banking is a dynamically growing area of management and controlling methods applied in banks on the one hand and high risk financial segment of the regulators’ measures and expectations around the world on the other. The purpose of this research is to outline the main challenges for cost-management development in correspondent banking relationships (CBRs) around the world with recommendations for Ukraine. The paper explains the key reasons and essential components of cost-management system for managing risks and costs in CBRs based on the analytical results of transactions volume, comparing drivers of restriction of CBRs, rapidly growing number of different types of compliance and operational costs. As a result, the study highlights the cost-cutting measures based on digital assets solutions and blockchain technologies that can help banks to eliminate and lower costs of customer on-boarding, due diligence and money laundering prevention, foreign exchange and currency hedging, treasury and payment operations, liquidity and capital raising.

A Novel String Grammar Unsupervised Possibilistic C-Medians Algorithm for Sign Language Translation Systems

Sign language is a basic method for solving communication problems between deaf and hearing people. In order to communicate, deaf and hearing people normally use hand gestures, which include a combination of hand positioning, hand shapes, and hand movements. Thai Sign Language is the communication method for Thai hearing-impaired people. Our objective is to improve the dynamic Thai Sign Language translation method with a video captioning technique that does not require prior hand region detection and segmentation through using the Scale Invariant Feature Transform (SIFT) method and the String Grammar Unsupervised Possibilistic C-Medians (sgUPCMed) algorithm. This work is the first to propose the sgUPCMed algorithm to cope with the unsupervised generation of multiple prototypes in the possibilistic sense for string data. In our experiments, the Thai Sign Language data set (10 isolated sign language words) was collected from 25 subjects. The best average result within the constrained environment of the blind test data sets of signer-dependent cases was 89–91%, and the successful rate of signer semi-independent cases was 81–85%, on average. For the blind test data sets of signer-independent cases, the best average classification rate was 77–80%. The average result of the system without a constrained environment was around 62–80% for the signer-independent experiments. To show that the proposed algorithm can be implemented in other sign languages, the American sign language (RWTH-BOSTON-50) data set, which consists of 31 isolated American Sign Language words, is also used in the experiment. The system provides 88.56% and 91.35% results on the validation set alone, and for both the training and validation sets, respectively.

Hand Wrist Bone Identification Using Quadrant Ballon Snake

In this research, hand wrist bone Identification for human forensic is discussed. Hand wrist bone is one of the effective methods used in the forensic science for age identification. There are four techniques used in this research: cropping image, preprocessing, Quadrant Ballon Snake and identification. The first step is to crop an image on metaphysis and epiphysis bone. The second step is preprocessing using morphology and edge detection. The third step is to apply Quadrant Ballon Snake to segment hand wrist bone. The last step is to use ratio metaphysis and epiphysis to identify a person. The performance segmentation for assessment hand wrist bone showed an average age identification 91%, bone age metaphysis 95% and bone age epiphysis 95%. The experiments resulted in the fact that Quadrant Ballon Snake method is able to find and improve the segmentation of hand wrist bone images accurately. This indicates that this method is effective for segmenting hand wrist bone.

Finger reconstruction in children by the transfer of hand segments

Aim. We aimed to evaluate the possibility of reconstruction of amputated fingers using various methods of hand segment transfer in children.
 Materials and methods. A retrospective analysis of the reconstruction of the first and triphalangeal fingers of an injured hand using a segment transfer method in 31 children was performed. Eleven patients had a mechanical injury; 12, a gunshot injury; 7, a burn injury; and 1, a freezing injury. The reconstruction of 32 fingers was performed by the transfer of an intact triphalangeal finger (3), defective finger (3), finger stump (14), and a metacarpal (12). The surgery was performed using traditional (16) and original (16) methods. In 26 cases, skin grafting using donor resources from remote areas was required for segment transfer. Newly developed approaches enable the transfer of the defective finger and any stump of the main finger phalanges and metacarpals, irrespective of their location, amputation level, hand defect character, level of cicatrical changes of the soft tissues, and impaired circulation, while adequately preventing ischemic complications.
 Results. Survival of all the transferred segments, including those with total cicatrical tissue changes and vascular disruptions, was achieved. The analysis of results showed that handgrip was restored in 31 hands. The best results were achieved in the transfer of intact fingers. Transfer of the utile segments enabled the reconstruction of the double-sided grip with minimum donor retrieval. Two-point discrimination was 2 mm for finger transfer, 4.5 mm for finger stump, and 6.5–7.4 mm for the metacarpal stump.
 Conclusion. The transfer of injured hand segments in children for finger reconstruction provides acceptable functional and anatomical results. This method can be used in combination with other finger reconstruction methods. Advanced technical and tactical approaches have widened the criteria of segment suitability and indications for the use of this method.

An Automatic Arabic Sign Language Recognition System (ArSLRS)

Sign language recognition system (SLRS) is one of the application areas of human computer interaction (HCI) where signs of hearing impaired people are converted to text or voice of the oral language. This paper presents an automatic visual SLRS that translates isolated Arabic words signs into text. The proposed system has four main stages: hand segmentation, tracking, feature extraction and classification. A dynamic skin detector based on the face color tone is used for hand segmentation. Then, a proposed skin-blob tracking technique is used to identify and track the hands. A dataset of 30 isolated words that used in the daily school life of the hearing impaired children was developed for evaluating the proposed system, taking into consideration that 83% of the words have different occlusion states. Experimental results indicate that the proposed system has a recognition rate of 97% in signer-independent mode. In addition to, the proposed occlusion resolving technique can outperform other methods by accurately specify the position of the hands and the head with an improvement of 2.57% at τ = 5 that aid in differentiating between similar gestures.

Spatial and Temporal Patterns in Sequential Precision Reach Movements.

Introduction: Sequential reach tasks are a common component of manual assembly jobs. These tasks typically involve manipulating a work object or material and reaching to successive target locations...