Finger Gestures Research Articles

Neural activation during imitation with or without performance feedback: An fMRI study

In our daily lives, we often receive performance feedback (PF) during imitative learning, and we adjust our behaviors accordingly to improve performance. However, little is known regarding the neural mechanisms underlying this learning process. We hypothesized that appropriate PF would enhance neural activation or recruit additional brain areas during subsequent action imitation. Pictures of 20 different finger gestures without any social meaning were shown to participants from the first-person perspective. Imitation with or without PF was investigated by functional magnetic resonance imaging in 30 healthy subjects. The PF was given by a real person or by a computer. PF from a real person induced hyperactivation of the parietal lobe (precuneus and cuneus), cingulate cortex (posterior and anterior), temporal lobe (superior and transverse temporal gyri), and cerebellum (posterior and anterior lobes) during subsequent imitation. The positive PF and negative PF from a real person, induced the activation of more brain areas during the following imitation. The hyperactivation of the cerebellum, posterior cingulate cortex, precuneus, and cuneus suggests that the subjects exhibited enhanced motor control and visual attention during imitation after PF. Additionally, random PF from a computer had a small effect on the next imitation. We suggest that positive and accurate PF may be helpful for imitation learning.

Recognizing arbitrarily connected and superimposed handwritten numerals in intangible writing interfaces

An intangible writing interface is a new input mechanism that allows users to write in the air with hands or fingers. Since all written characters are connected in one writing trajectory, each writing trajectory is referred to as a one-stroke finger gesture. Because of no visual aid to indicate the effective writing area in the air, users often write conservatively the texts in a restrictive area to avoid invalid motion sensing. Consequently, each handwritten character may not only arbitrarily connect to others, but also highly superimpose over others. Aiming at addressing the challenging issue to segment and recognize the arbitrarily connected and superimposed characters in one-stroke finger gestures, this paper proposes a solution comprising (1) a simple but effective method to spot any numeral of interest, called a key numeral, from arbitrary one-stroke finger gestures and (2) a novel method to simultaneously accomplish the segmentation and recognition of all individual numerals in one-stroke finger gestures by a graph-based path finding algorithm. The evaluated performance shows that the proposed solution achieves satisfactory recognition accuracy even though no sophisticated matching features and context-dependent information are employed.

Index finger and thumb kinematics and performance measurements for common touchscreen gestures

This study aimed to quantify differences in 7 touchscreen gestures. Eighteen participants performed index finger tapping, sliding in 4 orthogonal directions, and index finger and thumb pinch and stretch gestures on a touchscreen tablet computer. We hypothesized that two finger gestures would require longer task completion time and greater finger joint excursions than sliding gestures using only the index finger. We measured task completion times and finger joint kinematics. Tapping showed the fastest average (±SD) task completion time, 567(190) ms, of all gestures (p < 0.001). Pinch had faster task completion time, 765(277) ms, than all single-finger sliding gestures (p < 0.001). Stretch was faster to complete at 843(317) ms (p < 0.001) than all sliding gestures except slide right. Stretch demonstrated greater mean index finger metacarpophalangeal flexion/extension joint excursions, 63(16)°, compared to sliding gestures, 34(10)°, and tapping, 27(13)° (p < 0.01). Overall, two-finger gestures were faster to complete and showed greater joint excursions than single-finger sliding gestures.

Multi-Finger 3D Landmark Detection using Bi-Directional Hierarchical Regression

Over the past years, many application technologies related to Human Computer Interaction (HCI) have been developed for intelligent devices and wearable sensors. Moreover, they also have been received attention from both the research commu-nity and the industry. Almost of them make use of hand infor-mation such as hand gesture and posture. Consequently, the technical performance is highly depends on the information of humans’ hand and also, the number of products equipped with these technologies rapidly increases. Therefore, the accurate estimation of hand pose becomes a crucial issue to improve the performance of HCI applications. To detect landmarks, many algorithms have been develop-ing in this research field. Most of them are based on the pixel-based classification (1) and deformable part models (DPM) (2). However, they does not fit well in finger pose estimation since there are large pose variations due to camera viewpoints and various finger gestures. It is challenging to precisely detect the principal landmarks of fingers, as they flexibly deform with a large degree of freedom.In this paper, the regression based approach is used for esti-mating 3D multi-finger landmarks. Cascaded regression ap-proaches (3, 4) already used for hand pose estimation. Our ap-proach, however, concentrates on hierarchical structure of the detection strategy and initialization for better accuracy and stability. In our method, the bi-directional search structure is used for estimate landmarks with random forest regression. To

Creating a Simulated Pharmacy.

The acquisition and use of pharmaceuticals in simulation is a common challenge for many institutions and simulation centers. There are 2 major avenues of obtaining medications, namely, via expired drugs donated by manufacturers or via purchasing simulated medication vials filled with inactive ingredients. Neither option is ideal to keep pace with busy simulation centers or to meet the specialized requirements of many educational programs. We describe an alternative solution through internal production of simulated drugs using readily available materials from laboratory suppliers. Serum vials, stoppers, and flip off vial caps can be purchased in 2-, 5-, 10-, and 20-mL volumes. Vials can be filled with substances such as 0.9% NaCl or table salt to mimic a variety of available injectable drugs. Labels can be produced using the free online Web application, Simulated Online Pharmaceutical Image Editor (SOPHIE) and printed on glossy adhesive photo paper for application onto completed vials. A wide range of simulated drugs, customized to the needs of each center, can be produced in an affordable and reliable manner. The appearance of the vials can be tailored for each program to meet educational objectives and enhance fidelity in simulation. Low-cost production of simulated drug vials increases learning opportunities for participants to practice skills related to drug delivery and preparation. Further expansion can include nonintravenous drugs, code syringes, or reconstitution devices used for blood factors. Simulation centers should consider safety, availability, affordability, and fidelity concerns before integrating an in-house simulated pharmacy.

Effect of eigenvalue solution on the characteristics of double diffusive salt fingers

Double diffusive finger convection is driven by the varying diffusivity of the two components, contributing to the density variation. When the faster diffusing component heat stabilizes the system, and slow diffusing component salinity destabilizes the system with the overall density stratification remaining stable, the convection takes the form of rising and sinking fingers, which are known as salt fingers. We considered previous theories on the salt fingers’ growth rate, and show how different approaches to solve the same Eigen value equation lead to a different prediction of finger characteristics. We precisely solved the Eigen value growth rate equations, and show that the assumptions taken in the previous theories lead to unphysical system behavior in the limit of density ratio approaching unity. We show that the finger system becomes transient in this limit, and steady state assumption is not a valid approximation. Furthermore, other theories have also been proposed, and wide fingers are generally believed to transport larger fluxes when compared with thin fingers. In this paper, we show through numerical simulation that the assumption of wide finger-large flux ratio is not physically consistent in light of recent findings.

Design for Sharing Emotional Touches during Phone Calls

Background As the importance of mobile phones as an emotional communication medium is growing, non-verbal behaviors such as facial expressions, touching behaviors and gestures ought to be considered to enhance phone conversations. Among various non-verbal signals, we focus on sharing specific touches that could be used for exchanging contextual and emotional cues. Along with this, to make the new phone conversation involving touching more natural, we consider maintaining natural audio conversation with phones by keeping the receiver on the ear and the transmitter on the mouth while holding the phone to the cheek. Methods In this paper, we focus on investigating ways to deliver four touches: pat, slap, tickle and kiss with sound and tactile feedback while holding the phone to the cheek as in typical phone calls. The interaction technique is called CheekTouch, and it is based on enabling users to share touches by representing finger gestures on one phone screen to the other party’s cheek using a vibrotactile display and sound stimulations. We asked the 30 students who participated to put the device on their cheek and wear earphones, and we provided them with 24 different stimulations. Results An evaluation was conducted to propose the most appropriate type of stimulation to deliver a pat, slap, tickle and kiss using CheekTouch. We showed that the best way to deliver a pat was to use a vibrotactile display combined with sound; however, a tickle was best delivered with only the vibrotactile display. A kiss and slap, on the other hand, were best delivered when there was only sound. Conclusions Considering the trends in sharing delicate emotions during phone-mediated communications, it is significant to investigate ways to convey touches during phone conversations beyond sharing visual emoticons. Here, we focused on findings ways to pat, slap, tickle and kiss by using existing phone technologies (vibrotactile motors and sound). Through the quantitative evaluation of those four touches, we discovered which stimulation type is best for delivering each of the touches. The results do not show whether those touches can deliver emotions, however, we believe CheekTouch and the findings from the evaluation can be used for sharing different types of non-verbal signals during audio-based phone conversations and enable further studies in the field of remote tactile interaction.

Classification of Phantom Finger, Hand, Wrist, and Elbow Voluntary Gestures in Transhumeral Amputees With sEMG.

Decoding finger and hand movements from sEMG electrodes placed on the forearm of transradial amputees has been commonly studied by many research groups. A few recent studies have shown an interesting phenomenon: simple correlations between distal phantom finger, hand and wrist voluntary movements and muscle activity in the residual upper arm in transhumeral amputees, i.e., of muscle groups that, prior to amputation, had no physical effect on the concerned hand and wrist joints. In this study, we are going further into the exploration of this phenomenon by setting up an evaluation study of phantom finger, hand, wrist and elbow (if present) movement classification based on the analysis of surface electromyographic (sEMG) signals measured by multiple electrodes placed on the residual upper arm of five transhumeral amputees with a controllable phantom limb who did not undergo any reinnervation surgery. We showed that with a state-of-the-art classification architecture, it is possible to correctly classify phantom limb activity (up to 14 movements) with a rather important average success (over 80% if considering basic sets of six hand, wrist and elbow movements) and to use this pattern recognition output to give online control of a device (here a graphical interface) to these transhumeral amputees. Beyond changing the way the phantom limb condition is apprehended by both patients and clinicians, such results could pave the road towards a new control approach for transhumeral amputated patients with a voluntary controllable phantom limb. This could ease and extend their control abilities of functional upper limb prosthetics with multiple active joints without undergoing muscular reinnervation surgery.

The interlocking finger test in patients with Parkinson’s disease and healthy subjects

The interlocking finger test (ILFT) is a bedside screening test in which the subject must imitate four bimanual finger gestures without symbolic meaning. We assessed the utility of the test in the cognitive evaluation of patients with Parkinson’s disease (PD). We evaluated 88 healthy subjects and 101 patients with PD using a simplified motor score of the Unified Parkinson’s Disease Rating Scale (UPDRS), Hoehn and Yahr and Schwab and England scales, Geriatric Depression Scale, Pfeffer Functional Activities Questionnaire, Clinical Dementia Rating, Mini-Mental State Examination, clock drawing test, digit span, word list battery of the Consortium to Establish a Registry for Alzheimer’s Disease assessment, Frontal Assessment Battery, semantic verbal fluency test, and the ILFT. Diagnoses of mild cognitive impairment and dementia were made using the Movement Disorder Society diagnostic criteria. ILFT scores in healthy subjects correlated significantly with age (p=0.001) and only one healthy subject scored 2 in the test. ILFT scores were significantly lower in patients with PD and dementia (p=0.001) and significantly correlated with cognitive and functional tests, but not with depressive symptoms (p=0.607), Hoehn and Yahr scores (p=0.907), or Schwab and England scores (p=0.701). Twenty-five patients with dementia, three patients with mild cognitive impairment, and six patients with apparently normal cognition scored less than 3 in the ILFT. The area under the receiver operating characteristic curve for the ILFT to discriminate patients with dementia from those without it was 0.76 (cut-off score of 3/2: sensitivity of 61%, specificity of 0.85). In conclusion, the ILFT seems to be a useful bedside test to assess cognitive impairment in patients with PD.

Augmented reality as a digital teaching environment to develop spatial thinking

ABSTRACTDigital three-dimensional (3D) cartography environments provide new ways of landscape visualization and topographic interpretation: digital terrain modeling (DTM) is added to the traditional cartographic technique, where relief interpretation skill is needed to interpret 3D topographic information from 2D representations. The emergence of augmented reality (AR) technology for DTM representation offers a new way to develop map-reading skills. It is in the representation of the landforms (hills, dunes, depths, valleys, mountains, shapes, slopes, and elevations) where the AR allows a new way to interact with the landscape representation. Once AR technology is combined with tablet computers, the manipulation of DTM can be done by finger gestures. This paper analyzes the potential of AR as an innovative teaching tool to improve relief interpretation skill based on the results of a workshop with 73 engineering students from La Laguna University, Spain. Participants performed exercises with 2D traditional techniques of relief representation and with AR. To compare the effect with and without AR, another group of 22 students conducted the workshop with only 2D relief representations. The gain was significantly better with the AR group; students who did not use AR had also a significantly better result, but their scores were still lower.

Clinical Neuropsychology of Dementia with Lewy Bodies

Dementia with Lewy bodies (DLB) shows lesser memory impairment and more severe visuospatial disability than Alzheimer disease (AD). Although deficits in both consolidation and retrieval underlie the memory impairment, retrieval deficit is predominant in DLB. Visuospatial dysfunctions in DLB are related to the impairments in both ventral and dorsal streams of higher visual information processing, and lower visual processing in V1/V2 may also be impaired. Attention and executive functions are more widely disturbed in DLB than in AD. Imitation of finger gestures is impaired more frequently in DLB than in other mild dementia, and provides additional information for diagnosis of mild dementia, especially for DLB. Pareidolia, which lies between hallucination and visual misperception, is found frequently in DLB, but its mechanism is still under investigation.

Helping older adults conquer digital tablets

Purpose Three questions were asked about voluntary peer group meetings attended by older adults who wanted to improve their competence with digital tablets: What did people need to learn? Could interface changes reduce cognitive demands? Which elements of the meetings enhanced their success? Method From May 2013 adults aged 50+ belonging to a UK branch of the University of the Third Age met once a month. 76 people attended at least one meeting during a 27 month period. They were helped to use their tablets and interface design implications were noted. The elements within meetings (for instance, discussion, demonstration, practice, etc.) were modified as problems arose. Results and Discussion Average monthly attendance grew from 11 to 15 but 63% of the attendees dropped out after one or more meetings. Necessary learning included finger gestures, distributing attention, and multiple routes to similar goals. Complete beginners needed individual help before benefitting from the group. The important elements of meetings were consistent with cognitive learning theories: reducing distraction by increasing group homogeneity (separating iPad and Android users); including a projected demonstration of new procedures as an advanced organiser; aiding memory through a printed handout. Group cohesion was enhanced through discussion and by putting handouts on the group’s web page so absentees could stay informed. Conclusions Digital tablets offer viable internet access for older adults but pose learning challenges that would be fewer if interface design better supported cognitive processes such as memory and attention. Within group meetings of older adults a balance between social and instructional activities enhances satisfaction.

一种手指姿态识别系统设计

手指康复运动具有高移动精度与高控制分辨率的需求，加上因人而异的数据多样性，以及在运动过程中的模糊信号识别等问题，实时准确的手指姿态识别能够大大提高手指的康复效果。针对这一问题，本文提出了一种简易、便携的手指姿态识别系统设计。使用多分类支持向量机对手部运动进行分析和识别，大量训练者所采集到的数据分为离线训练集和在线测试集，经过大量的离线训练与在线测试，结果表明，多分类支持向量机在分类和识别过程中具有高效性和实用性，并且极大可能有助于手指的康复过程。 With high precision and high resolution of the mobile control demand, plus the data, it differs from man to man. Diversity, and in the process of movement of fuzzy signal recognition, real-time and accurate finger gesture recognition can greatly improve the effect of rehabilitation of finger. To solve this problem, this paper presents a simple, portable finger gesture recognition system design. The use of multi class support vector machine hand motion analysis and recognition, a large number of training data collected is divided into offline and online training set test set; after the test, a large number of online and offline training results show that the multi class support vector machine is efficient and practical in classification and recognition in the process of the rehabilitation process and which may contribute to the finger.

タッチパネル用ジェスチャ型日本語入力インターフェイスの開発

Currently, the text entry method of the portable information devices including smart phones and tablet, is a soft keyboard that appears mainly in the viewing screen. It considered the limited screen of portable information device is inefficient. This paper proposes a gesture-based text entry method which does not require predetermined area. It can be available anywhere on the screen, thus touch type entry is possible. This method is based on the property of Japanese Hiragana-50-phonemic-table and two step sequence of finger gesture is utilized. The first step gesture corresponds to the row entry such as “a ka sa ta na”. The second one corresponds to the column entry of it, such as “a i u e o”. From the experiments, the proposed entry method is easy to learn and entry speed is acceptable. The experimental results show that the proposed method is practically applicable as a new text entry method.

동심원 추적 알고리즘을 사용한 손가락 동작 인식

In this paper, we propose a novel algorithm, Concentric-Circle Tracing algorithm, which recognizes finger's shape and counts the number of fingers of hand using low-cost web-camera. We improve algorithm’s usability by using low-price web-camera and also enhance user's comfortability by not using a additional marker or sensor. As well as counting the number of fingers, it is possible to extract finger's shape information whether finger is straight or folded, efficiently. The experimental result shows that the finger gesture can be recognized with an average accuracy of 95.48%. It is confirmed that the hand-gesture is an useful method for HCI input and remote control command.

Cognitive and Emotional Structure of a Robotic Game Player in Turn-based Interaction

This paper focuses on how cognitive and emotional structures affect humans during long-term interaction. We design an interaction with a turn-based game, the Chopstick Game, in which two agents play with numbers using their fingers. While a human and a robot agent alternate turn, the human user applies herself to play the game and to learn new winning skills from the robot agent. Conventional valence and arousal space is applied to design emotional interaction. For the robotic system, we implement finger gesture recognition and emotional behaviors that are designed for three-dimensional virtual robot. In the experimental tests, the properness of the proposed schemes is verified and the effect of the emotional interaction is discussed.

Interaction on-the-go: a fine-grained exploration on wearable PROCAM interfaces and gestures in mobile situations

Wearable projector and camera (PROCAM) interfaces, which provide a natural, intuitive and spatial experience, have been studied for many years. However, existing hand input research into such systems revolved around investigations into stable settings such as sitting or standing, not fully satisfying interaction requirements in sophisticated real life, especially when people are moving. Besides, increasingly more mobile phone users use their phones while walking. As a mobile computing device, the wearable PROCAM system should allow for the fact that mobility could influence usability and user experience. This paper proposes a wearable PROCAM system, with which the user can interact by inputting with finger gestures like the hover gesture and the pinch gesture on projected surfaces. A lab-based evaluation was organized, which mainly compared two gestures (the pinch gesture and the hover gesture) in three situations (sitting, standing and walking) to find out: (1) How and to what degree does mobility influence different gesture inputs? Are there any significant differences between gesture inputs in different settings? (2) What reasons cause these differences? (3) What do people think about the configuration in such systems and to what extent does the manual focus impact such interactions? From qualitative and quantitative points of view, the main findings imply that mobility impacts gesture interactions in varying degrees. The pinch gesture undergoes less influence than the hover gesture in mobile settings. Both gestures were impacted more in walking state than in sitting and standing states by all four negative factors (lack of coordination, jittering hand effect, tired forearms and extra attention paid). Manual focus influenced mobile projection interaction. Based on the findings, implications are discussed for the design of a mobile projection interface with gestures.

Hand Mouse Manipulation by Supporting Moving Sensitivity Modification and Click Event

Abstract In this paper, we propose new mouse manipulation system by using “OK” finger gesture which is human familiar and intuitive. The proposed method can control the sensitivity of cursor movement compared with previous hand mouse method. At the captured hand image, the closed “O” region generated by thumb and index fingers is detected through binarization and component labelling. The cursor position is

라즈베리 파이와 립 모션을 이용한 지화 번역 시스템 설계 및 구현

Deaf are it is difficult to communicate to represent the voice heard, so theay use mostly using the speech, sign language, writing, etc. to communicate. It is the best way to use sign language, in order to communicate deaf and normal people each other. But they must understand to use sign language. In this paper, we designed and implementated finger language translation system to support communicate between deaf and normal people. We used leap motion as input device that can track finger and hand gesture. We used raspberry pi that is low power sing board computer to process input data and translate finger language. We implemented application used Node.js and MongoDB. The client application complied with HTML5 so that can be support any smart device with web browser.

Differences and Similarities between Finger and Pen Stroke Gestures on Stationary and Mobile devices

This study investigated differences and similarities between finger and pen gestures on stationary devices (sitting posture) and mobile devices (sitting and walking postures). The recorded gestures were analyzed according to multiple gesture features. We found (1) pen and index finger gestures were different in features like size ratio but similar in features like angle difference ; (2) implement (pen vs. index finger vs. thumb) interacted with gesture complexity and size in features like articulation time ; (3) features like time and shape distance , were different between the pen and index finger on mobile devices (walking) but similar on stationary devices; (4) one-handed thumb gestures had worse performances than index finger gestures by time and accuracy in sitting but similar performances in walking; and (5) for the three implements, gesture drawing time and accuracy on mobile devices reduced from sitting to walking condition. We discuss these findings with implications for future gesture design and research.