Fingertip Position Research Articles

A multi-objective optimization model and its evolution-based solutions for the fingertip localization problem

Exact fingertip positions are of particular importance to the fingertip-based human–computer interaction. We build a multi-objective optimization model for the problem of fingertip localization, and present a method to solve the above model based on evolutionary algorithms. When building the model, we take the positions of a series of pixels as the decision variable, the shape of the hand-edge curve corresponding to each of the pixels as one objective function, and the distance between each of the pixels and the gravity center of the palm as the other objective function. In addition, based on the correlation among the positions of pixels of the fingertip regions, we present a multi-objective estimation of distribution algorithm to solve the model so as to obtain the best pixel set, thus gaining the fingertip positions. The experimental results demonstrate the effectiveness of the proposed model and algorithm.

Role of digit placement control in sensorimotor transformations for dexterous manipulation.

Dexterous manipulation relies on the ability to modulate grasp forces to variable digit position. However, the sensorimotor mechanisms underlying such critical ability are not well understood. The present study addressed whether digit force-to-position modulation relies entirely on feedback of digit placement and force, or on the integration of such feedback with motor commands responsible for digit positioning. In two experiments, we asked 25 subjects to estimate the index fingertip position relative to the thumb (perception test) or to grasp and lift an object with an asymmetrical mass distribution while preventing object roll (action test). Both tests were performed after subjects' digits were placed actively or passively at different distances (active and passive condition, respectively) and without visual feedback. Because motor commands for digit positioning would be integrated with position and force feedback in the active condition, we hypothesized this condition to be characterized by greater accuracy of digit position estimation and digit force-to-position modulation. Surprisingly, discrimination of digit position and force-to-position modulation was statistically indistinguishable in the active and passive conditions. We conclude that voluntary commands for digit positioning are not essential for accurate estimation of finger position or modulation of digit forces to variable digit position. Thus digit force-to-position modulation can be implemented by integrating sensory feedback of digit position and voluntary commands of digit force production following contact.NEW & NOTEWORTHY This study was designed to understand the sensorimotor mechanisms underlying digit force-to-position modulation required for manipulation. Surprisingly, estimation of relative digit position and force-to-position modulation was accurate regardless of whether the digits were passively or actively positioned. Therefore, accurate estimation of digit position does not require an efference copy of active digit positioning, and the hypothesized advantage of active over passive movement on estimation of end-point position appears to be task and effector dependent.

Does practicing a wide range of joint angle configurations lead to higher flexibility in a manual obstacle-avoidance target-pointing task?

Flexibility in motor actions can be defined as variability in the use of degrees of freedom (e.g., joint angles in the arm) over repetitions while keeping performance (e.g., fingertip position) stabilized. We examined whether flexibility can be increased through enlarging the joint angle range during practice in a manual obstacle-avoidance target-pointing task. To establish differences in flexibility we partitioned the variability in joint angles over repetitions in variability within (GEV) and variability outside the solution space (NGEV). More GEV than NGEV reflects flexibility; when the ratio of the GEV and NGEV is higher, flexibility is higher. The pretest and posttest consisted of 30 repetitions of manual pointing to a target while moving over a 10 cm high obstacle. To enlarge the joint angle range during practice participants performed 600 target-pointing movements while moving over obstacles of different heights (5–9 cm, 11–15 cm). The results indicated that practicing movements over obstacles of different heights led participants to use enlarged range of joint angles compared to the range of joint angles used in movements over the 10 cm obstacle in the pretest. However, for each individual obstacle neither joint angle variance nor flexibility were higher during practice. We also did not find more flexibility after practice. In the posttest, joint angle variance was in fact smaller than before practice, primarily in GEV. The potential influences of learning effects and the task used that could underlie the results obtained are discussed. We conclude that with this specific type of practice in this specific task, enlarging the range of joint angles does not lead to more flexibility.

Bilateral Tele-operated Hand Robot with Communicational Time Delay

Various tasks require a humanoid robot instead of human. This paper proposes a bilateral tele-operation system between a human operator and a humanoid hand robot. The proposed system is able to communicate the master’s fingertip force and the slave’s fingertip position with a time delay. The experimental results show the effectiveness of the proposed system using a multi-fingered haptic interface robot and humanoid hand robot with three-dimensional force feedback.

Fingertip-Based Feature Analysis for the Push and Stroke Manipulation of Elastic Objects.

In this study, to quantitatively understand finger operations used to manipulate elastic objects, we explore robust fingertip-based feature descriptors that are invariant to operator, finger position, and target object. To measure the tactile information generated when an object is directly touched by a fingertip, we used a wearable system that enables the simultaneous measurement of fingertip position and strain without inhibiting the operator's sense of touch. This paper focuses on the quantitative classification of the push and stroke operations of a single finger, and conducted user experiments to obtain time-series fingertip position and strain from 10 subjects touching nine types of elastic objects. The recognition rate was investigated by binary classification using a support vector machine and cross validation. The results show that the two-dimensional features obtained from fingertip position and strain within a 0.9-s time frame can stably recognize push and stroke operations on elastic bodies of different shapes, stiffnesses, and thicknesses at a higher recognition rate.

Texture design for light touch perception

Abstract This study focused on active light touch with predefined textures specially-designed for tactile perception. The counter-body material is stainless steel sheet. Three geometric structures (grid, crater and groove) were fabricated by pulsed laser surface texturing. A total number of twenty volunteers participated in the research which contains two parts: perception tests and skin friction measurements. The perception tests focused mainly on the participants׳ perceptual attributes: perceived roughness and perceived stickiness. For the skin friction measurements, a multi-axis force/torque transducer was used to measure the normal force and friction force between skin and counter-surface along with the fingertip position. The results of the predefined textures showed the ability to reduce skin friction due to the reduction of contact area. Moreover, the participants׳ perceptual attributes were greatly influenced by the predefined micro-structures in the light touch regime.

Recognize the tilt fingertips by partial scan algorithm

To meet the needs of finger interaction system and the recognition of tilt fingertips, a fast and accurate method is described to detect the position of fingertips and real-time recognize tilt fingertip gesture. This method uses YCbCr color-space-segmentation-algorithm to devise complexion roughly. After that, preprocessing regions are segmented carefully by the concept of proportion of perimeter and acreage to eliminate interference region other than the hand skin color. In addition, obtain the finger profile by binomial method of least squares fitting algorithm. Besides, the improved convex algorithms are employed to complete the fingertips detection and correct tilt angle. Finally, fingertips are recognized by means of partial scan method. Experimental results demonstrate that the proposed method can realize the recognition refers to easy tilt fingertips of zero to night, and reach the high recognition rate of 95.7%. Apart from this, stable performance are achieved.

The impact of efferent oculomotor signals on size and distance perception

PurposeTo demonstrate the influence of efferent oculomotor signals on perceived size and distance of visual objects.MethodsBy fingertip, a pencil target has to be localized. The target is seen at 25 cm of distance on a 45° mirror mounted in front of the subject. With the finger behind the mirror, the subject is deprived from visual feedback. Observation is done either by naked eye or by looking through binocular prisms of 6 pdpt b.i. The fingertip estimate is recorded by an ElGuide® ultrasound position control system (Zebris Medical, Isny, Germany).ResultsIn first results, the prisms produced a mean 25% overestimation of target distance as recorded by fingertip position. The target appeared visibly larger. This prompted us to do trials with a minus −3 dpt lens instead of prisms, forcing the subject to overaccommodation. Hereby, the target appeared visibly smaller and more distant, but now, fingertip pointing ends up at reduced distance.ConclusionsWith reduced convergence the object appears larger and nearer. The distance of the object is over‐estimated by fingertip localization. Hyperaccommodation results in a smaller and more distant appearance of the object, but its fingertip distance is underestimated. Sense and purpose of these contradictory effects are not yet well understood. Visitors of the poster are encouraged to self experience.

Unsteady steady-states: central causes of unintentional force drift.

We applied the theory of synergies to analyze the processes that lead to unintentional decline in isometric fingertip force when visual feedback of the produced force is removed. We tracked the changes in hypothetical control variables involved in single fingertip force production based on the equilibrium-point hypothesis, namely the fingertip referent coordinate (R FT) and its apparent stiffness (C FT). The system's state is defined by a point in the {R FT; C FT} space. We tested the hypothesis that, after visual feedback removal, this point (1) moves along directions leading to drop in the output fingertip force, and (2) has even greater motion along directions that leaves the force unchanged. Subjects produced a prescribed fingertip force using visual feedback and attempted to maintain this force for 15s after the feedback was removed. We used the "inverse piano" apparatus to apply small and smooth positional perturbations to fingers at various times after visual feedback removal. The time courses of R FT and C FT showed that force drop was mostly due to a drift in R FT toward the actual fingertip position. Three analysis techniques, namely hyperbolic regression, surrogate data analysis, and computation of motor-equivalent and non-motor-equivalent motions, suggested strong covariation in R FT and C FT stabilizing the force magnitude. Finally, the changes in the two hypothetical control variables {R FT; C FT} relative to their average trends also displayed covariation. On the whole, the findings suggest that unintentional force drop is associated with (a) a slow drift of the referent coordinate that pulls the system toward a low-energy state and (b) a faster synergic motion of R FT and C FT that tends to stabilize the output fingertip force about the slowly drifting equilibrium point.

EP 32. Automated high resolution fMRI mapping of the cortical sensory fingertip somatotopy in group examinations

Objective The increasing spatial precision in functional brain mapping is a methodological key issue driving the progress in brain research. Mapping of previously invisible fine scale details permits the discovery of unknown functional brain properties. Currently, the available resolution of fMRI examinations is increasing rapidly. However, data analysis techniques lag behind this development, and achieve an effective resolution which is smaller as the maximum possible spatial precision. We present a data analysis technique with maximum spatial precision in group analyses of high resolution functional magnetic resonance imaging (fMRI) data. Methods The contralateral fingertip representations of thumb, index and middle finger were investigated in a group of 18 participants, for right and left hand. A block design was applied (absolute duration of 200 s) including long counting pulses as a temporal identification task for attention monitoring ( Fig. 1 ). Imaging was carried out with a 3 Tesla MRI-scanner (Verio, Siemens, Germany). FreeSurfer recommendations for the anatomy scans were followed (standard MPRAGE sequence). A standard gradient-echo EPI sequence was used for fMRI (resolution 1.5 × 1.5 × 2 mm 3 , 17 slices, field of view = 191 × 179 mm 2 , TE/TR = 3.6 ms/21 ms). Data analysis was performed with an automated protocol ( Pfannmoller et al., 2015 ), including vessel artefact removal. Cortical positions of the fingertips were computed on individual and MNI brains, using volume-based linear or surface-based non-linear normalization. An assessment of cortical distance measures, for mapping of functional properties, was carried out including straight-line distances, using the 3-D Euclidean measure, and shortest connections within the cortical surface, either with the Dijkstra or LOS-Floyd algorithms. Results In BA3b a distinct somatotopy was found for both hands. The maximum spatial fingertip spread in BA3b after linear normalization was at least two times larger than for the non-linear normalization. All properties of the BA3b fingertip somatotopy were left-right symmetric. Since the representation in BA1 did not exhibit any of those features it was neglected. The LOS-Floyd algorithm achieved an error free result for the distances within measurement accuracy. In contrast, Euclidean distances did not account for the relevant cortical properties. Dijkstra distances were sufficiently precise in the individual brains, but excessively overestimated the left-hand distances after non-linear normalization. Discussion & conclusion Since the representations in BA1 and BA3b are entirely different, the cytoarchitectonic divisions need to be accounted for. The convergence of the BA3b fingertip positions after non-linear normalization implies a removal of the anatomical variability, leaving exclusively functional variability. In the distance mapping, application of the LOS-Floyd algorithm was mandatory in the group template. The Dijkstra algorithm represents a fast alternative for distance mapping in individual brains, while the precision of Euclidean distances did not suffice for our purposes. In conclusion, our methodology improves the spatial precision of somatotopic mapping at 3 T. At larger field strengths improvements to even higher resolutions are anticipated.

Impact of Long Flexor Versus Intrinsic Dominance in the Generation of Arc of Finger Flexion.

Background: Finger flexion is a composite movement involving both long flexors and intrinsic hand muscles. Previous studies have characterized this but have not investigated differences within the normal population. Were discrete finger motion patterns identified, this could guide rehabilitation programs following flexor tendon surgery. Methods: Twelve volunteers repeatedly flexed and extended at a comfortable speed, resting their hand on a horizontal surface. Video was recorded perpendicular to the little finger flexion plane, and the little finger tip position was identified frame by frame to create a composite curve. Its highest point was noted, and the horizontal distance was measured from this point to the palmar digital crease (ΔXH). Results: In addition, 2 investigators independently reviewed frame-by-frame images and allocated subjects into groups based on the motion pattern. ΔXH demonstrated 2 clusters within our study population, and there was a statistically significant (P < .036) difference between groups. Conclusions: This study demonstrated that qualitative and quantitative differences exist in flexion curves between individuals.

Spatio-Temporal Hough Forest for efficient detection–localisation–recognition of fingerwriting in egocentric camera

Recognising fingerwriting in mid-air is a useful input tool for wearable egocentric camera. In this paper we propose a novel framework to this purpose. Specifically, our method first detects a writing hand posture and locates the position of index fingertip in each frame. From the trajectory of the fingertip, the written character is localised and recognised simultaneously. To achieve this challenging task, we first present a contour-based view independent hand posture descriptor extracted with a novel signature function. The proposed descriptor serves both posture recognition and fingertip detection. As to recognising characters from trajectories, we propose Spatio-Temporal Hough Forest that takes sequential data as input and perform regression on both spatial and temporal domain. Therefore our method can perform character recognition and localisation simultaneously. To establish our contributions, a new handwriting-in-mid-air dataset with labels for postures, fingertips and character locations is proposed. We design and conduct experiments of posture estimation, fingertip detection, character recognition and localisation. In all experiments our method demonstrates superior accuracy and robustness compared to prior arts.

3D Air-Touch User Interface With High Touch Accuracy on Stereoscopic Displays

To interact with stereoscopic images through 3D air-touch technology, the depth information for a touched 3D virtual object and the fingertip position should be aligned accurately. However, the fused 3D position is affected when the virtual object is blocked by the fingertip. Additionally, the 3D image with a large disparity decreases 3D touch accuracy. In order to improve the touch accuracy, virtual references and 3D image feedback were introduced respectively, and the touch accuracy was more than 90% based on a human interaction experiment.

視覚障害者のための指差し文字検出システムの開発

This paper proposes a system to assist the visually handicapped by extracting text characters on the surface of an object. The visually handicapped user cannot know exactly what the object is and cannot read the text written on the surface. The user touches a point on the surface with his or her index finger; the system extracts characters around the fingertip. To implement these functions, we propose fingertip detection and character extraction methodologies. For fingertip detection, the user wears a ring, in which three acrylic infrared-reflective beads are embedded, on the index finger. The system detects the beads and locates their positions from an infrared image, then, calculates the fingertip position based on bead arrangement. For character extraction, a region is generated according to the position of the fingertip. The characters are extracted from the region based on color histogram and shape. Then, the extracted characters are combined as text. Finally, the text is recognized by commercially available OCR software and is read aloud to the user by reading software. We have experimentally verified that these methodologies achieved character extraction accuracy of at least 70.0% expect the case of narrow character extraction. The experiments demonstrated the basic validity of our proposed method.

Postural optimization during functional reach while kneeling and standing.

[Purpose] The purpose of the present study was to examine the validity of functional reach models by comparing actual values with estimated values. [Subjects and Methods] Twenty-eight volunteers were included in this study (male: 14, female: 14, age: 21 ± 1 years, height: 166.8 ± 9.0 cm, and body mass: 60.1 ± 8.5 kg). The maximum forward fingertip position and joint angles were measured using the original equipment. In addition, the maximum forward fingertip position, shoulder joint angle, and knee or ankle joint angle were estimated using the functional reach model. [Results] The correlation coefficients between actual data and estimated data for the maximum forward fingertip position, shoulder joint angle, and ankle joint angle while standing were 0.93, 0.83, and 0.73, respectively. The correlation coefficients between actual data and estimated data for the maximum forward fingertip position, shoulder joint angle, and knee joint angle while kneeling were 0.86, 0.81, and 0.72, respectively. [Conclusion] The validity of both functional reach models in estimating optimal posture was confirmed. Therefore, the functional reach model is useful for evaluation of postural control and optimal postural control exercises.

A fine motor skill training system using multi-fingered haptic interface robot

This paper focuses on a haptic training system for fine motor skills using multiple fingers. To transfer skills involving such fine motor tasks, instruction on how to move the fingers and use fingertip forces is essential. In this paper, we propose an optimal method for transmitting knowledge regarding fingertip forces from trainer to trainee; we then construct a haptic training system based on the proposed method. The resulting system conveys knowledge of fingertip forces and positions in three-dimensional space by combining an image display system with a five-fingered haptic interface robot capable of alternating force presentation. Finally, we describe several experiments carried out to optimize and evaluate the performance of the proposed method.

Deducing the reachable space from fingertip positions.

The reachable space of the hand has received significant interests in the past from relevant medical researchers and health professionals. The reachable space was often computed from the joint angles acquired from a motion capture system such as gloves or markers attached to each bone of the finger. However, the contact between the hand and device can cause difficulties particularly for hand with injuries, burns or experiencing certain dermatological conditions. This paper introduces an approach to find the reachable space of the hand in a non-contact measurement form utilizing the Leap Motion Controller. The approach is based on the analysis of each position in the motion path of the fingertip acquired by the Leap Motion Controller. For each position of the fingertip, the inverse kinematic problem was solved under the physiological multiple constraints of the human hand to find a set of all possible configurations of three finger joints. Subsequently, all the sets are unified to form a set of all possible configurations specific for that motion. Finally, a reachable space is computed from the configuration corresponding to the complete extension and the complete flexion of the finger joint angles in this set.

3D Finger CAPE: Clicking Action and Position Estimation under Self-Occlusions in Egocentric Viewpoint.

In this paper we present a novel framework for simultaneous detection of click action and estimation of occluded fingertip positions from egocentric viewed single-depth image sequences. For the detection and estimation, a novel probabilistic inference based on knowledge priors of clicking motion and clicked position is presented. Based on the detection and estimation results, we were able to achieve a fine resolution level of a bare hand-based interaction with virtual objects in egocentric viewpoint. Our contributions include: (i) a rotation and translation invariant finger clicking action and position estimation using the combination of 2D image-based fingertip detection with 3D hand posture estimation in egocentric viewpoint. (ii) a novel spatio-temporal random forest, which performs the detection and estimation efficiently in a single framework. We also present (iii) a selection process utilizing the proposed clicking action detection and position estimation in an arm reachable AR/VR space, which does not require any additional device. Experimental results show that the proposed method delivers promising performance under frequent self-occlusions in the process of selecting objects in AR/VR space whilst wearing an egocentric-depth camera-attached HMD.

Workspace Modeling and Analysis for Dexterous Hands

The workspace is a fundamental feature for a dexterous hand to grasping plan, motion control, and mechanical design. Although the graphic, numerical, or analytical methods are valid to generate the workspace of dexterous hands, but there exist several defects for these methods to describe the workspace characteristics, such as forms, boundaries, volumes, and intersection workspaces. We propose a combined modeling approach to visualize and analyze the workspace of YWZ dexterous hand, which has five fingers, 20 degrees of freedom (Dofs). The proposed approach is also fit for similar dexterous hands to generate their precise workspaces. After a brief mechanism introduction of YWZ dexterous hand, a displacement equation of its index finger is deduced to calculate the fingertip positions in three-dimension (3D) Euclidean space. Then, a two-dimension (2D) boundary figure enclosing the flexion–extension motion field of the finger is drawn by the displacement equation. Taking this boundary figure as a sketch in a CAD modeling environment, we further model a vivid 3D finger workspace, which is relative with the abduction–adduction motion of the finger. Besides, we generate the whole workspace of YWZ dexterous hand and analyze its volumes and intersection workspaces. The kinematic simulations and physical prototype experimentations are carried out to validate this approach can construct perfect workspace of dexterous hands. Compared with the Monte Carlo method, the form and boundary of the 3D finger workspace generated by our proposed approach have more accurate, integrated, vivid, and intuitional.

自動運転のインターフェース構築を目的とした視点と指先位置の測定による指示方向推定手法の構築

In recent years, development of the automated driving system is aggressively promoted for the purpose of reducing burden of drivers and for the enhanced safety. Transferring the intention of the driver to the automobile control, such as to stop on the way or to change destinations, is essential even if a direct control from the driver is no longer necessary. Therefore, an in-car interface to conduct intention of the driver promptly and accurately to the automobile control is needed. In this paper, we proposed an interface which recognizes the intended target of the driver using gestures and speech recognition. The speech recognition technology was previously developed. Hence, we developed an interface to recognize the direction in which the driver is pointing toward from the gesture. We hypothesize that we can estimate the direction in which the driver is pointing toward from the fingertip and eye position. Verification using three-dimensional motion analysis system showed that it is possible to recognize the direction of the object with a maximum error of 5.5° and an average error of 1.2°. After verifying the hypothesis, a position measurement device that integrates Leap Motion® and KINECT®, the non-contact sensor, was developed to recognize gestures of the driver in the automobiles. The maximum error of the angle recognition in accordance with the present device was 16.1°. However, the value of the error exists only in a certain interval. For this reason, by taking into account the movement of the eye position, and provide adequate tuning for each driver, proposed interface is sufficiently usable.