Gesture Training Research Articles

Sign Language Recognition System Using Cnn

In this study, we have built an automatic sign language translation system for deaf and dumb persons to communicate with ordinary people. According to the Statistics Department of the Taiwan Ministry of Health and Welfare, there are 119,682 hearing impaired persons, and 14,831 voice function or language dysfunctions. The Deaf and dumb persons’ account for 11.7% of the population with physical and mental disabilities. However, there are only 488 qualified people with the sign language translation skill certificate, which shows the importance of automatic sign language translation systems. This system collects 11 signals including five fingers’ curvature, 3-axis gyroscope and 3-axis accelerometer from left and right hand separately. In addition, a total of 22 signals are collected by the two sensors, Flex sensor and GY-521 six-axis with single-board computer Arduino MEGA 2560; and then uploaded to server via ESP-01S Wi-Fi module. While server receives the 22 signals, it converts to a RGB picture using PHP program. As a result, we can compare the picture with the model trained by TensorFlow and the compared result is stored in the database. Meanwhile, the comparison stored in database which can be accessed by APP programs would be displayed on the screen of the mobile device and be read aloud. The TensorFlow training model collects 25 sign language gestures, each based on 100 training gesture pictures, and a sign language recognition training model is Convolutional Neural Network (CNN). In this study, the results of the sign language recognition training model are further confirmed by 10 people other than those in training database. So far, the indeed recognition rate of sign language is about 84.4%, and the system response time is about 2.243 seconds.

3D Vector Reconstruction of the Atlas from Anatomical Sections of Korean Visible Human at the Laboratory of Clinical and Digital Anatomy of Paris Descartes University

Aim: Carry out a 3D vector reconstruction of the atlas from the anatomical sections of the Visible Human for educational purposes. Material and methods: The anatomical subject was a 33-year-old Korean man who died of leukemia. It measured 164 cm and weighed 55 kg. This man donated his body to science. Her body was frozen and cut into several anatomical sections after an MRI and CT scan. These anatomical sections were made using a special saw called a 0.2 mm thick cryomacrotome. Thus 8,100 cuts were obtained. Only the sections numbered 780 to 860 were used for our study. A segmentation by manual contouring of the different parts of the atlas was done using the software Winsurf version 3.5 on a laptop PC running Windows 7 equipped with a Ram of 8 gigas. Results: Our 3D vector model of the atlas is easy to manipulate using the Acrobat 3DPDF interface. The atlas accessible in a menu can be displayed, hidden or made transparent, and 3D labels are available as well as educational menus for learning anatomy. Conclusion: This original work constitutes a remarkable educational tool for the anatomical study of the atlas and can also be used as a 3D atlas for simulation purposes for training in therapeutic gestures.

Charlie Chaplin and gesture training in severe aphasia: A controlled double-blind single-case experimental design.

Aphasia following a stroke is a frequent and disabling condition that decreases quality of life. The use of gesture has been proposed as a way to enhance aphasia recovery. We aimed to explore whether 2 types of gesture interventions could improve communication in individuals with severe aphasia. This was a pilot study performed at home in routine care by an outreach team. The study had a controlled double-blind single-case experimental design (SCED): a controlled multiple baseline design across 3 participants and 2 behaviors (gesture and naming). Three male patients with stroke-induced severe chronic aphasia, non-functional perseverative speech and severe associated impairments underwent a passive gesture intervention, in which participants watched movies selected for their intensive use of gesture, and an active gesture intervention, in which they actively practiced gestures by using visual action therapy. The main outcome measures were naming score, gesture score and nonverbal subscale score of the Lillois Test of Communication, with 3-month follow-up. In all 3 participants, gesture interventions improved the ability to gesture a list of words (Tau-U=0.38-0.67 for combined gesture intervention effect) and increased nonverbal communication activity. Benefits were maintained at 3-month follow-up. Mute films that use intensive nonverbal communication may be a useful add-on to speech therapy for individuals with aphasia. Improving naming in severe and chronic aphasia may not be feasible, and more effort could be devoted to improving gesture-based and nonverbal communication.

3D Vector Reconstruction of the Typical Cervical Vertebra from Anatomical Sections of Korean Visible Human at the Laboratory of Clinical and Digital Anatomy of Paris Descartes University

Aim: To carry out a 3D vector reconstruction of the typical cervical vertebra from anatomical sections of the “Korean Visible Human” for educational purposes. Material and Methods: The anatomical subject was a 33-year-old Korean man who died of leukemia. He was 164 cm tall and weighed 55 kg. This man donated his body to science. Her body was frozen and cut into several anatomical sections after an MRI and CT scan. These anatomical sections were made using a special saw called a 0.2 mm thick cryomacrotome. Thus 8100 cuts were obtained. Only the sections numbered 940 to 1200 were used for our study. A segmentation by manual contouring of the different parts of the typical cervical vertebra was made using the software Winsurf version 3.5 on a laptop PC running Windows 7 equipped with a Ram of 8 gigas. Results: Our 3D vector model of the typical cervical vertebra is easily manipulated using the Acrobat 3DPDF interface. Each part of the vertebra accessible in a menu can be displayed, hidden or made transparent, and 3D labels are available as well as educational menus for learning anatomy. Conclusion: This original work constitutes a remarkable educational tool for the anatomical study of the typical cervical vertebra and can also be used as a 3D atlas for simulation purposes for training in therapeutic gestures.

In Search of Training Gestures: What Regularities can be observed when Teaching how to Teach Written Production?

A partir de uma pesquisa empírica sobre as práticas de formação do professorado na Suíça que reúne 28 sequências de formação sobre a produção escrita, este artigo propõe uma visão geral dos gestos profissionais que desenvolvem os formadores ao preparar os futuros docentes. Seis grandes gestos profissionais organizam a dinâmica da formação observada e o agir do formador. Dois gestos se realizam fora das aulas: planificação e avaliação do trabalho de treinamento. Quatro gestos têm lugar no curso da implementação das sequências em presença dos docentes: conceptualização da escrita e do seu ensino e aprendizagem, mobilização das práticas e experiências da produção escrita na aula, equipar com recursos e ferramentas para o ensino a escrita, regulação das aprendizagens e das práticas dos professores. O artigo ilustra com exemplos o agir do formador no domínio da produção escrita. Os resultados da pesquisa mostram a importância do agir do formador para o desenvolvimento de competências profissionais para o ensino da produção escrita.
 Palavras-chave: Práticas de formação de professores. Sequências de treinamento. Produção escrita. Gestos profissionais.

Learning speech-internal cues to pronoun interpretation from co-speech gesture: a training study.

This study explores whether children can learn a structural processing bias relevant to pronoun interpretation from brief training. Over three days, 42 five-year-olds were exposed to narratives exhibiting a first-mentioned tendency. Two characters were introduced, and the first-mentioned was later described engaging in a solo activity. In our primary condition of interest, the Gesture Training condition, the solo-activity sentence contained an ambiguous pronoun, but co-speech gesture clarified the referent. There were two comparison conditions. In the Gender Training condition the characters were different genders, thereby avoiding ambiguity. In the Name Training condition, the first-mentioned name was simply repeated. Ambiguous pronoun interpretation was tested pre- and post-training. Children in the Gesture condition were significantly more likely to interpret ambiguous pronouns as the first-mentioned character after training. Results from the comparison conditions were ambiguous: there was a small but non-significant effect of training, but also no significant differences between conditions.

User Evaluation of the Smartphone Screen Reader VoiceOver with Visually Disabled Participants

Touchscreen assistive technology is designed to support speech interaction between visually disabled people and mobile devices, allowing hand gestures to interact with a touch user interface. In a global perspective, the World Health Organization estimates that around 285 million people are visually disabled with 2/3 of them over 50 years old. This paper presents the user evaluation of VoiceOver, a built-in screen reader in Apple Inc. products, with a detailed analysis of the gesture interaction, familiarity and training by visually disabled users, and the system response. Six participants with prescribed visual disability took part in the tests in a usability laboratory under controlled conditions. Data were collected and analysed using a mixed methods approach, with quantitative and qualitative measures. The results showed that the participants found most of the hand gestures easy to perform, although they reported inconsistent responses and lack of information associated with several functionalities. User training on each gesture was reported as key to allow the participants to perform certain difficult or unknown gestures. This paper also reports on how to perform mobile device user evaluations in a laboratory environment and provides recommendations on technical and physical infrastructure.

Robot-based intervention may reduce delay in the production of intransitive gestures in Chinese-speaking preschoolers with autism spectrum disorder

BackgroundPast studies have shown that robot-based intervention was effective in improving gestural use in children with autism spectrum disorders (ASD). The present study examined whether children with ASD could catch up to the level of gestural production found in age-matched children with typical development and whether they showed an increase in verbal imitation after the completion of robot-based training. We also explored the cognitive and motor skills associated with gestural learning.MethodsChildren with ASD were randomly assigned to two groups. Four- to 6-year-old children with ASD in the intervention group (N = 15) received four 30-min robot-based gestural training sessions. In each session, a social robot, NAO, narrated five stories and gestured (e.g., both hands clapping for an awesome expression). Children with ASD were told to imitate the gestures during training. Age-matched children with ASD in the wait-list control group (N = 15) and age-matched children with typical development (N = 15) received the gestural training after the completion of research. Standardized pretests and posttests (both immediate and delayed) were administered to assess the accuracy and appropriateness of gestural production in both training and novel stories. Children’s language and communication abilities, gestural recognition skills, fine motor proficiencies, and attention skills were also examined.ResultsChildren with ASD in the intervention condition were more likely to produce accurate or appropriate intransitive gestures in training and novel stories than those in the wait-list control. The positive learning outcomes were maintained in the delayed posttests. The level of gestural production accuracy in children with ASD in the delayed posttest of novel stories was comparable to that in children with typical development, suggesting that children with ASD could catch up to the level of gestural production found in children with typical development. Children with ASD in the intervention condition were also more likely to produce verbal markers while gesturing than those in the wait-list control. Gestural recognition skills were found to significantly predict the learning of gestural production accuracy in the children with ASD, with such relation partially mediated via spontaneous imitation.ConclusionsRobot-based intervention may reduce the gestural delay in children with ASD in their early childhood.

Coronary Heart Disease Preoperative Gesture Interactive Diagnostic System Based on Augmented Reality.

Coronary heart disease preoperative diagnosis plays an important role in the treatment of vascular interventional surgery. Actually, most doctors are used to diagnosing the position of the vascular stenosis and then empirically estimating vascular stenosis by selective coronary angiography images instead of using mouse, keyboard and computer during preoperative diagnosis. The invasive diagnostic modality is short of intuitive and natural interaction and the results are not accurate enough. Aiming at above problems, the coronary heart disease preoperative gesture interactive diagnostic system based on Augmented Reality is proposed. The system uses Leap Motion Controller to capture hand gesture video sequences and extract the features which that are the position and orientation vector of the gesture motion trajectory and the change of the hand shape. The training planet is determined by K-means algorithm and then the effect of gesture training is improved by multi-features and multi-observation sequences for gesture training. The reusability of gesture is improved by establishing the state transition model. The algorithm efficiency is improved by gesture prejudgment which is used by threshold discriminating before recognition. The integrity of the trajectory is preserved and the gesture motion space is extended by employing space rotation transformation of gesture manipulation plane. Ultimately, the gesture recognition based on SRT-HMM is realized. The diagnosis and measurement of the vascular stenosis are intuitively and naturally realized by operating and measuring the coronary artery model with augmented reality and gesture interaction techniques. All of the gesture recognition experiments show the distinguish ability and generalization ability of the algorithm and gesture interaction experiments prove the availability and reliability of the system.

O efeito do treino propriocetivo na prevenção de lesões da tibiotársica

ABSTRACT: Objectives – To evaluate and compare the efficacy of three different types of proprioceptive training in reducing the incidence of ankle sprains in male youth soccer players. Method – Muscle strength in tibiotarsal joint and proprioceptivity were evaluated and compared between three groups, with supervised proprioceptive training (GE), another unsupervised (GC1) and another with technical gesture training (GC2) over a nine week period, two to three times per week. Results – Both GE and GC1 showed significant increases (p<0.01) of proprioceptivity compared with GC2. Conclusion – The results suggest that after a short period of proprioceptive training, athletes improve their proprioceptivity but not necessarily tibiotarsal joint muscle strength. According to the study, supervision does not have an impact over training efficacy.

Physical Training Gesture Recognition Using Wristwatch Wearable Devices

Lately, many companies have launched smart watch products with their own strong points, drawing consumers’ attention. A smart watch has changed our life and made it more convenient, offering some different experiences compared to existing smart phones. More importantly, health care using this device has increasingly been the subject of people’s attention and studies. Most of the existing studies have explored mobile devices and multiple sensors recognizing activities related to the routine such as walking, running, and going up and down the stairs. This study has focused on the use in physical training. We made a wearable device with some features of a smart watch, and studied how to make it recognize the activity the user is performing with the built-in accelometer and gyroscope. It is expected to be helpful for the individuals to manage their own exercise systematically and be practical in the health care industry.

Training with rhythmic beat gestures benefits L2 pronunciation in discourse-demanding situations

Recent research has shown that beat gestures (hand gestures that co-occur with speech in spontaneous discourse) are temporally integrated with prosodic prominence and that they help word memorization and discourse comprehension. However, little is known about the potential beneficial effects of beat gestures in second language (L2) pronunciation learning. This study investigates the impact of beat gesture observation on the acquisition of native-like speech patterns in English by examining the effect of a brief training with or without beat gestures on participants’ ratings of accentedness. In a within-participants, pre-/post-test design, participants (undergraduate students learning English as a foreign language) watched a training video in which an L2 instructor gave spontaneous responses to discourse prompts. The prompts belonged to one of two categories (easy and difficult), and were presented by the instructor either with or without accompanying beat gestures. Participants’ own answers to the prompts were recorded before and after training and evaluated by five native speaker judges. The results of the comparison between the participants’ pre-training and post-training speech samples demonstrated that beat gesture training significantly improved the participants’ accentedness ratings on the set of difficult (more discourse-demanding) items. The results of the study support the role of beat gestures as highlighters of rhythmic information and have implications for pronunciation instruction practices.

Crossover learning of gestures in two ideomotor apraxia patients: A single case experimental design study

ABSTRACTCrossover learning may aid rehabilitation in patients with neurological disorders. Ideomotor apraxia (IMA) is a common sequela of left-brain damage that comprises a deficit in the ability to perform gestures to verbal commands or by imitation. This study elucidated whether crossover learning occurred in two post-stroke IMA patients without motor paralysis after gesture training approximately 2 months after stroke onset. We quantitatively analysed the therapeutic intervention history and investigated whether revised action occurred during gesture production. Treatment intervention was to examine how to influence improvement and generalisation of the ability to produce the gesture. This study used an alternating treatments single-subject design, and the intervention method was errorless learning. Results indicated crossover learning in both patients. Qualitative analysis indicated that revised action occurred during the gesture-production process in one patient and that there were two types of post-revised action gestures: correct and incorrect gestures. We also discovered that even when a comparably short time had elapsed since stroke onset, generalisation was difficult. Information transfer between the left and right hemispheres of the brain via commissural fibres is important in crossover learning. In conclusion, improvements in gesture-production skill should be made with reference to the left cerebral hemisphere disconnection hypothesis.

The effects of gesture and movement training on the intonation of children’s singing in vocal warm-up sessions

The main purpose of the current study was to examine the effects of gesture and movement training for beginning children’s choirs with regard to improving intonation. It was a between-subjects design with one independent variable Training Technique (TT). One dependent variable was measured: intonation in the singing of vocal pattern warm-up exercises. Fifty-three fifth grade students (10–11 years old) participated in the empirical investigation. They were randomly assigned into three training groups: (1) Group 1— students who received no gesture and movement training; (2) Group 2— students who received gesture training; and (3) Group 3— students who received gesture and movement training. The instructional unit, consisting of two 40-minute sessions per week for 24 sessions, focused on vocal development. Each individual was pretested and posttested on singing 5 vocal patterns, selected from some 20 learned in the sessions. Three experts were asked to judge the children’s individual singing of these patterns, assessing intonation. Results showed that the children who received gesture and/or movement training sang significantly more in tune than those who did not. It was concluded that gesture training has a positive effect on improving children’s intonation. Furthermore, the combination of gesture and movement training could be a powerful teaching strategy in choral rehearsals.

The Development of Conversation Training Therapy: A Concept Paper

To introduce the conceptual, theoretical, and practical foundations of a novel approach to voice therapy, called conversation training therapy (CTT), which focuses exclusively on voice awareness and efficient voice production in patient-driven conversational narrative, without the use of a traditional therapeutic hierarchy. CTT is grounded in motor learning theory, focused on training target voice goals in spontaneous, conversational speech in the first session and throughout. CTT was developed by a consensus panel of expert clinical voice-specialized speech-language pathologists (SLPs) and patients with voice problems. This is a prospective, clinical consensus design. A preliminary CTT approach to voice therapy was developed by the first and last authors (J.G-S. and A.I.G.) and incorporated six interchangeable tenets: clear speech, auditory/kinesthetic awareness, rapport building, negative practice, basic training gestures, and prosody. Five expert voice-specialized clinical SLPs (consensus group) were then presented CTT and a discussion ensued. Later, an informal interview by a neutral third party person occurred for further recommendations for CTT. The CTT approach was modified to reflect all the consensus groups' recommendations, which included the need for more detail and rationale in the program, troubleshooting suggestions, and the concern for potential challenges for novice clinicians. CTT is a new therapy approach based on motor learning theory, which exclusively uses patient-driven conversational narrative as the sole therapeutic stimuli. CTT is conceptually innovative because it represents an approach to voice therapy developed without the use of a traditional therapeutic hierarchy. It is also developed using input from patients with voice disorders and expert clinical providers.

Enhancing gesture quality in young singers

ABSTRACTStudies have shown positive results for the use of gesture as a successful technique in aiding children's singing. The main purpose of this study was to examine the effects of movement training for children with regard to enhancing gesture quality. Thirty-six fifth-grade students participated in the empirical investigation. They were randomly assigned into control and experimental groups: (1) control group – children who sang with gesture but without movement training and (2) experimental group – children who sang with gesture and received Dalcrozian movement training. The gestures were designed for developing singing voice, whereas the movement activities were designed for developing effective gestures for singing. The instructional unit, consisting of two 40-minute sessions per week for 24 sessions, focused on gesture use in singing. Each individual child was tested a Chinese art song. Three experts were asked to judge the children's individual gesture and singing performances by using a nine-point scale. Results showed that the children who received movement training gained a significantly higher score on gesture quality. It was concluded that the combination of gesture and movement training could be a powerful teaching strategy in children's singing.

The role of pictures and gestures as a support mechanism for novel word learning: A training study with 2-year-old children

A training study examined novel word learning in 2-year-old children and assessed two nonverbal mechanisms, pictures and gestures, which are commonly used as communication support. The aim was to (1) compare these two support mechanisms and measure their effects on expressive word learning and (2) to investigate these effects on word production over an extended time period. At baseline, the children’s performance was assessed on vocabulary and grammatical skills, and the groups were matched on these key variables. Eighteen participants were taught novel words either accompanied by a gesture or by a picture. The training consisted of four 20-minute sessions per week over a period of four weeks. Following training, the children were assessed on their ability to produce novel words at three time points: immediately after training, at a 2-week follow-up and at a 6-week follow-up. Gesture training supported word learning significantly better than picture training across all three testing points. Children in both groups showed the best production immediately after training, with a small but nonsignificant decline at 2-week follow-up. There was a significant decline in time 3 compared to time 1, but the children were still able to produce 6.8 out of 10 novel words, suggesting long-term learning. Our findings suggest that gesture may be used to support word learning and could benefit children with late emerging language. The importance of these findings for language development and language intervention are discussed.

A gesture-controlled projection display for CT-guided interventions.

The interaction with interventional imaging systems within a sterile environment is a challenging task for physicians. Direct physician-machine interaction during an intervention is rather limited because of sterility and workspace restrictions. We present a gesture-controlled projection display that enables a direct and natural physician-machine interaction during computed tomography (CT)-based interventions. Therefore, a graphical user interface is projected on a radiation shield located in front of the physician. Hand gestures in front of this display are captured and classified using a leap motion controller. We propose a gesture set to control basic functions of intervention software such as gestures for 2D image exploration, 3D object manipulation and selection. Our methods were evaluated in a clinically oriented user study with 12 participants. The results of the performed user study confirm that the display and the underlying interaction concept are accepted by clinical users. The recognition of the gestures is robust, although there is potential for improvements. The gesture training times are less than 10 min, but vary heavily between the participants of the study. The developed gestures are connected logically to the intervention software and intuitive to use. The proposed gesture-controlled projection display counters current thinking, namely it gives the radiologist complete control of the intervention software. It opens new possibilities for direct physician-machine interaction during CT-based interventions and is well suited to become an integral part of future interventional suites.

Systematic review of apraxia treatments to improve occupational performance outcomes.

The objective was to assess effectiveness of apraxia treatments using a systematic review. In contrast to previous reviews, each study was rated as to its applicability to occupational therapy practice and its focus on occupational performance using the FAME rating system (defined by four categories: Feasibility, Appropriateness, Meaningfulness, Effectiveness). This systematic review included eight studies: four randomized controlled trials (level 1 evidence) and four pre-post designs (level 3 evidence). Three treatment approaches were reported: errorless learning with training of details; gesture training; and strategy training. FAME scores ranged from A to C. All studies reported significant treatment effects, but only one demonstrated an impact on observed occupational performance that transferred from clinic to home.

Training experience in gestures affects the display of social gaze in baboons' communication with a human.

Gaze behaviour, notably the alternation of gaze between distal objects and social partners that accompanies primates' gestural communication is considered a standard indicator of intentionality. However, the developmental precursors of gaze behaviour in primates' communication are not well understood. Here, we capitalized on the training in gestures dispensed to olive baboons (Papio anubis) as a way of manipulating individual communicative experience with humans. We aimed to delineate the effects of such a training experience on gaze behaviour displayed by the monkeys in relation with gestural requests. Using a food-requesting paradigm, we compared subjects trained in requesting gestures (i.e. trained subjects) to naïve subjects (i.e. control subjects) for their occurrences of (1) gaze behaviour, (2) requesting gestures and (3) temporal combination of gaze alternation with gestures. We found that training did not affect the frequencies of looking at the human's face, looking at food or alternating gaze. Hence, social gaze behaviour occurs independently from the amount of communicative experience with humans. However, trained baboons-gesturing more than control subjects-exhibited most gaze alternation combined with gestures, whereas control baboons did not. By reinforcing the display of gaze alternation along with gestures, we suggest that training may have served to enhance the communicative function of hand gestures. Finally, this study brings the first quantitative report of monkeys producing requesting gestures without explicit training by humans (controls). These results may open a window on the developmental mechanisms (i.e. incidental learning vs. training) underpinning gestural intentional communication in primates.