Meaningful Gestures Research Articles

Anatomical or mirror mode imitation? A behavioral approach.

Imitation can occur in at least two forms: one, which can be defined as anatomical, is based primarily on the mental construct of the body schema and allows recognition of correspondences between own body anatomy and that of other individuals. The other form, defined as specular or mirror mode, is most probably based on the allocation of some form of attention to the same region of the environmental space both by model and imitator, and to the objects it contains. This study investigated the behavioral strategy of imitation in normal subjects, to assess whether they carried out task instructions using an anatomical or a mirror perspective. Twenty seven adults were asked to imitate intransitive meaningful and meaningless gestures shown by a model in video clips. Instructions about how to perform them were provided before each trial. Trials were free (intended to produce spontaneous imitation) or driven (intended to produce anatomical imitation); further driven trials were administered to verify participants' knowledge of bodily laterality and were used as control. Performances were interpreted as anatomical or mirror imitation, according to the observation of anatomical or spatial reference frames between stimulus and imitator. The results revealed that in spontaneous imitation the mirror mode was more frequent (61% of responses), in line with previous studies. The novel finding was the prevalence (93% of responses) of anatomical imitation in tasks involving detailed driven instructions.

Apraxia and Motor Dysfunction in Corticobasal Syndrome

BackgroundCorticobasal syndrome (CBS) is characterized by multifaceted motor system dysfunction and cognitive disturbance; distinctive clinical features include limb apraxia and visuospatial dysfunction. Transcranial magnetic stimulation (TMS) has been used to study motor system dysfunction in CBS, but the relationship of TMS parameters to clinical features has not been studied. The present study explored several hypotheses; firstly, that limb apraxia may be partly due to visuospatial impairment in CBS. Secondly, that motor system dysfunction can be demonstrated in CBS, using threshold-tracking TMS, and is linked to limb apraxia. Finally, that atrophy of the primary motor cortex, studied using voxel-based morphometry analysis (VBM), is associated with motor system dysfunction and limb apraxia in CBS.MethodsImitation of meaningful and meaningless hand gestures was graded to assess limb apraxia, while cognitive performance was assessed using the Addenbrooke's Cognitive Examination – Revised (ACE-R), with particular emphasis placed on the visuospatial subtask. Patients underwent TMS, to assess cortical function, and VBM.ResultsIn total, 17 patients with CBS (7 male, 10 female; mean age 64.4+/− 6.6 years) were studied and compared to 17 matched control subjects. Of the CBS patients, 23.5% had a relatively inexcitable motor cortex, with evidence of cortical dysfunction in the remaining 76.5% patients. Reduced resting motor threshold, and visuospatial performance, correlated with limb apraxia. Patients with a resting motor threshold <50% performed significantly worse on the visuospatial sub-task of the ACE-R than other CBS patients. Cortical function correlated with atrophy of the primary and pre-motor cortices, and the thalamus, while apraxia correlated with atrophy of the pre-motor and parietal cortices.ConclusionsCortical dysfunction appears to underlie the core clinical features of CBS, and is associated with atrophy of the primary motor and pre-motor cortices, as well as the thalamus, while apraxia correlates with pre-motor and parietal atrophy.

Functional Correlates of Increasing Gestural Articulatory Fluency Using a Miniature Second-Language Approach

Objectives: Gesture-based second languages have become an important tool in the rehabilitation of language-impaired subpopulations. Acquiring the ability to use manual gestures as a means to construct meaningful utterances places unique demands on the brain. This study identified changes in the blood oxygen level-dependent (BOLD) signal associated with the development of gestural fluency using a miniature second-language-based approach. Participants and Methods: Twelve healthy right-handed adults (19-31 years) were trained to produce sequences of meaningful gestures over a period of 2 weeks. Functional magnetic resonance imaging was used to identify brain regions involved in actual and imagined production of meaningful sentences both before (nonfluent production) and after (fluent production) practice. Results: Brain areas showing learning-dependent increases in activity associated with the development of fluency included sites associated with language articulation, while learning-related decreases in the BOLD signal were observed in cortical networks associated with motor imagery, and native language processing. Conclusion: These findings provide novel insights regarding the neural basis of fluency that could inform the design of interventions for treating speech disorders characterized by the loss of fluency.

Hand gesture recognition using low-budget data glove and cluster-trained probabilistic neural network

Purpose – Main purpose is to present methodology which allows efficient hand gesture recognition using low-budget, 5-sensor data glove. To allow widespread use of low-budget data gloves in engineering virtual reality (VR) applications, gesture dictionaries must be enhanced with more ergonomic and symbolically meaningful hand gestures, while providing high gesture recognition rates when used by different seen and unseen users. Design/methodology/approach – The simple boundary-value gesture recognition methodology was replaced by a probabilistic neural network (PNN)-based gesture recognition system able to process simple and complex static gestures. In order to overcome problems inherent to PNN – primarily, slow execution with large training data sets – the proposed gesture recognition system uses clustering ensemble to reduce the training data set without significant deterioration of the quality of training. The reduction of training data set is efficiently performed using three types of clustering algorithms, yielding small number of input vectors that represent the original population very well. Findings – The proposed methodology is capable of providing efficient recognition of simple and complex static gestures and was also successfully tested with gestures of an unseen user, i.e. person who took no part in the training phase. Practical implications – The hand gesture recognition system based on the proposed methodology enables the use of affordable data gloves with a small number of sensors in VR engineering applications which require complex static gestures, including assembly and maintenance simulations. Originality/value – According to literature, there are no similar solutions that allow efficient recognition of simple and complex static hand gestures, based on a 5-sensor data glove.

The cognitive models of limb apraxia and the specific properties of meaningful gestures

The cognitive models of limb apraxia and the specific properties of meaningful gestures

Young children's understanding of denial.

Although a fair amount is known about young children's production of negation, little is known about their comprehension. Here, we focus on arguably the most complex basic form, denial, and how young children understand denial, when it is expressed in response to a question with gesture, single word, or sentence. One hundred twenty-six children in 3 age groups (Ms = 1 year 9 months, 2 years 0 months, and 2 years 4 months) witnessed an adult look into 1 of 2 buckets and then, in response to a question about whether the toy was in there, communicate either something positive (positive head nod, "yes," "it is in this bucket") or negative (negative head shake, "No," "It's not in this bucket"). The youngest children did not search differently in response to any of the communicative cues (nor in response to an additional cue using both gesture and single word). Children at 2 years 0 months searched at above-chance levels only in response to the negative word and negative sentence. Children at 2 years 4 months were successful with all 3 types of cues in both positive and negative modalities, with the exception of the positive sentence. Young children thus seem to understand the denial of a statement before they understand its affirmation, and they understand linguistic means of expressing denial before they understand gestural means.

Does comprehension of symbolic gestures and corresponding-in-meaning words make use of motor simulation?

The present study aimed at determining whether or not the comprehension of symbolic gestures, and corresponding-in-meaning words, makes use of cortical circuits involved in movement execution control. Participants were presented with videos of an actress producing meaningful or meaningless gestures, pronouncing corresponding-in-meaning words or pseudo-words; they were required to judge whether the signal was meaningful or meaningless. Single pulse TMS was applied to forearm primary motor cortex area 150–200ms after the point when the stimulus meaning could be understood. MEPs were significantly greater when processing meaningless signals as compared to a baseline condition presenting a still-and-silent actress. In contrast, this was not the case for meaningful signals whose motor activation did not differ from that for the baseline stimulus. MEPs were significantly greater for meaningless than meaningful signals and no significant difference was found between gesture and speech. On the basis of these results, we hypothesized that the observation-of/listening-to meaningless signals recruits motor areas. In contrast, this did not occur when the signals were meaningful. Overall, the data suggest that the processes related to comprehension of symbolic gestures and communicative words do not involve primary motor area and probably use brain areas involved in semantics.

Dogs’ Eavesdropping from People’s Reactions in Third Party Interactions

Eavesdropping involves the acquisition of information from third-party interactions, and can serve to indirectly attribute reputation to individuals. There is evidence on eavesdropping in dogs, indicating that they can develop a preference for people based on their cooperativeness towards others. In this study, we tested dogs’ eavesdropping abilities one step further. In a first experiment, dogs could choose between cooperative demonstrators (the donors) who always gave food to an approaching third person (the beggar); here, the only difference between donors was whether they received positive or negative reactions from the beggar (through verbal and gestural means). Results showed that dogs preferentially approached the donor who had received positive reactions from the beggar. By contrast, two different conditions showed that neither the beggar’s body gestures nor the verbal component of the interaction on their own were sufficient to affect the dogs’ preferences. We also ran two further experiments to test for the possibility of dogs’ choices being driven by local enhancement. When the donors switched places before the choice, dogs chose at random. Similarly, in a nonsocial condition in which donors were replaced by platforms, subjects chose at chance levels. We conclude that dogs’ nonrandom choices in the present protocol relied on the simultaneous presence of multiple cues, such as the place where donors stood and several features of the beggar’s behavior (gestural and verbal reactions, and eating behavior). Nonetheless, we did not find conclusive evidence that dogs discriminated the donors by their physical features, which is a prerequisite of reputation attribution.

Experimentally Induced Increases in Early Gesture Lead to Increases in Spoken Vocabulary

Differences in vocabulary that children bring with them to school can be traced back to the gestures they produced at the age of 1;2, which, in turn, can be traced back to the gestures their parents produced at the same age (Rowe & Goldin-Meadow, 2009a). We ask here whether child gesture can be experimentally increased and, if so, whether the increases lead to increases in spoken vocabulary. Fifteen children aged 1;5 participated in an 8-week at-home intervention study (6 weekly training sessions plus follow-up 2 weeks later) in which all were exposed to object words, but only some were told to point at the named objects. Before each training session and at follow-up, children interacted naturally with caregivers to establish a baseline against which changes in communication were measured. Children who were told to gesture increased the number of gesture meanings they conveyed, not only during training but also during interactions with caregivers. These experimentally induced increases in gesture led to larger spoken repertoires at follow-up.

Nonverbal Cues Could Boost Kids' Vocabulary

Meaningful gestures and glances may help children learn more words, independent of how much parents talk to them

27. Modulation of motor cortex excitability and brain activation in professional pianists during music listening: A combined EEG and TMS study

Neuroimaging studies have reported mirroring motor activation to music listening in musicians. Actual corticospinal modulation remains to be clarified. We tested the motor cortex modulation in pianists while listening to well-known, predictable melodic sequences, using Transcranial Magnetic Stimulation (TMS) and electroencephalography (EEG). Two groups of subjects were studied: pianists (n = 10) and controls (n = 10). Participants listened to a melodic sequence containing the alternating repetition of a scale in two different octaves. Using a circular coil on the vertex, Motor Evoked Potentials (MEP) were recorded bilaterally on both hands at different time points, with 32 channels EEG monitoring. EEG event-related desynchronization (ERD) of the mu rhythm was assessed by comparing its spectral power during listening vs rest. Pianists, not controls, showed modulation of MEPs amplitude of right hand muscles according to the timing of their activation during actual performance. Left sensori-motor frontal mu ERD to the melodic sequence played with the right hand was present in 9 pianists vs 4 control. The dynamic auditory induced motor resonance to music listening observed in pianists suggests the presence of an audio-motor mirror system that can be related to the ability of musician to confer a gestural meaning to well-known melodies.

Using hand gestures to control mobile spoken dialogue systems

Speech and hand gestures offer the most natural modalities for everyday human-to-human interaction. The availability of diverse spoken dialogue applications and the proliferation of accelerometers on consumer electronics allow the introduction of new interaction paradigms based on speech and gestures. Little attention has been paid, however, to the manipulation of spoken dialogue systems (SDS) through gestures. Situation-induced disabilities or real disabilities are determinant factors that motivate this type of interaction. In this paper, six concise and intuitively meaningful gestures are proposed that can be used to trigger the commands in any SDS. Using different machine learning techniques, a classification error for the gesture patterns of less than 5 % is achieved, and the proposed set of gestures is compared to ones proposed by users. Examining the social acceptability of the specific interaction scheme, high levels of acceptance for public use are encountered. An experiment was conducted comparing a button-enabled and a gesture-enabled interface, which showed that the latter imposes little additional mental and physical effort. Finally, results are provided after recruiting a male subject with spastic cerebral palsy, a blind female user, and an elderly female person.

Left posterior parietal theta burst stimulation affects gestural imitation regardless of semantic content

ObjectiveNeuro-imaging studies have suggested that the ability to imitate meaningless and meaningful gestures may differentially depend on superior (SPL) and inferior (IPL) parietal lobule. Therefore, we hypothesized that imaging-guided neuro-navigated continuous theta burst stimulation (cTBS) over left SPL mainly affects meaningless and over left IPL predominantly meaningful gestures. MethodsTwelve healthy subjects participated in this study. High resolution structural MRI was used for imaging guided neuro-navigation cTBS. Participants were targeted with one train of cTBS in three experimental sessions: sham stimulation over vertex and real cTBS over left SPL and IPL, respectively. An imitation task, including 24 meaningless and 24 meaningful gestures, was performed ‘offline’. ResultscTBS over both left IPL and SPL significantly interfered with gestural imitation. There was no differential effect of SPL and IPL cTBS on gesture type (meaningless versus meaningful). ConclusionsOur findings confirm that left posterior parietal cortex plays a predominant role in gestural imitation. However, the hypothesis based on the dual route model suggesting a differential role of SPL and IPL in the processing of meaningless and meaningful gestures could not be confirmed. SignificanceLeft SPL and IPL play a common role within the posterior–parietal network in gestural imitation regardless of semantic content.

Selective imitation impairments differentially interact with language processing

Whether motor and linguistic representations of actions share common neural structures has recently been the focus of an animated debate in cognitive neuroscience. Group studies with brain-damaged patients reported association patterns of praxic and linguistic deficits whereas single case studies documented double dissociations between the correct execution of gestures and their comprehension in verbal contexts. When the relationship between language and imitation was investigated, each ability was analysed as a unique process without distinguishing between possible subprocesses. However, recent cognitive models can be successfully used to account for these inconsistencies in the extant literature. In the present study, in 57 patients with left brain damage, we tested whether a deficit at imitating either meaningful or meaningless gestures differentially impinges on three distinct linguistic abilities (comprehension, naming and repetition). Based on the dual-pathway models, we predicted that praxic and linguistic performance would be associated when meaningful gestures are processed, and would dissociate for meaningless gestures. We used partial correlations to assess the association between patients' scores while accounting for potential confounding effects of aspecific factors such age, education and lesion size. We found that imitation of meaningful gestures significantly correlated with patients' performance on naming and repetition (but not on comprehension). This was not the case for the imitation of meaningless gestures. Moreover, voxel-based lesion-symptom mapping analysis revealed that damage to the angular gyrus specifically affected imitation of meaningless gestures, independent of patients' performance on linguistic tests. Instead, damage to the supramarginal gyrus affected not only imitation of meaningful gestures, but also patients' performance on naming and repetition. Our findings clarify the apparent conflict between associations and dissociations patterns previously observed in neuropsychological studies, and suggest that motor experience and language can interact when the two domains conceptually overlap.

SVM을 이용한 동적 동작인식: 체감형 동화에 적용

본 연구에서는 다차원의 데이터 인식에 유리한 SVM을 이용한 동적 동작인식 알고리즘을 제안한다. 우선, Kinect 비디오 프레임에서 동작의 시작과 끝을 찾아 의미있는 동작 프레임을 분할하고, 프레임 수를 동일하게 정규화시킨다. 정규화된 프레임에서 인체 모델에 기반한 인체 부위의 위치와 부위 사이의 관계를 이용한 동작 특징을 추출하여 동작인식을 수행한다. 동작인식기인 C-SVM는 각 동작에 대해 positive 데이터와 negative 데이터로 구성된 학습 데이터로 학습된다. 최종 동작 선정은 각 C-SVM의 결과값 중 가장 큰 값을 갖는 동작으로 한다. 제안하는 동작인식 알고리즘은 플래시 구연동화에서 더 나아가 유아가 능동적으로 구연동화에 참여할 수 있도록 고안된 체감형 동화 콘텐츠에 동작 인터페이스로 적용되었다. This paper proposes a dynamic gesture recognition algorithm using SVM(Support Vector Machine) which is suitable for multi-dimension classification. First of all, the proposed algorithm locates the beginning and end of the gestures on the video frames at the Kinect camera, spots meaningful gesture frames, and normalizes the number of frames. Then, for gesture recognition, the algorithm extracts gesture features using body parts' positions and relations among the parts based on the human model from the normalized frames. C-SVM for each dynamic gesture is trained using training data which consists of positive data and negative data. The final gesture is chosen with the largest value of C-SVM values. The proposed gesture recognition algorithm can be applied to the interactive storybook as gesture interface.

How early do children understand gesture–speech combinations with iconic gestures?

Children understand gesture+speech combinations in which a deictic gesture adds new information to the accompanying speech by age 1;6 (Morford & Goldin-Meadow, 1992; 'push'+point at ball). This study explores how early children understand gesture+speech combinations in which an iconic gesture conveys additional information not found in the accompanying speech (e.g., 'read'+BOOK gesture). Our analysis of two- to four-year-old children's responses in a gesture+speech comprehension task showed that children grasp the meaning of iconic co-speech gestures by age three and continue to improve their understanding with age. Overall, our study highlights the important role gesture plays in language comprehension as children learn to unpack increasingly complex communications addressed to them at the early ages.

New insights into the medium hand: Discovering recurrent structures in gestures

This paper shows that an analysis of gestures based on the four parameters hand shape, orientation, movement, and position in gesture space, introduced for the notation of sign language (Battison 1974; Stokoe 1960) can uncover recurrent patterns and structures in gestures. By presenting two studies we argue for a discovery procedure that rests upon a systematic description of gestures applying the four parameters and upon a clear discrimination between form and meaning in the analytical process. Using this methodological framework it is possible to uncover clusters of recurrent forms and a systematic variation of form and meaning in a recurrent gesture.

Limb apraxia in acute ischemic stroke: A neglected clinical challenge?

Symptoms of limb apraxia and executive dysfunctions are currently not explicitly considered by the National Institutes of Health Stroke Scale and, thus, not routinely tested by clinicians in the acute care of patients with suspected stroke. Neuropsychological testing, clinical examination, MRI, and functional magnetic resonance imaging (fMRI) were performed in a right-handed patient with acute onset of left-sided sensorimotor hemiparesis due to a right hemisphere ischemic stroke. Deficits in the execution of meaningless and meaningful gestures were not detected properly on initial clinical examination but were revealed later on through neuropsychological testing. Instead, the patient’s inability to respond to specific instructions in the acute care setting was mistaken to reflect severe deficits in auditory comprehension. fMRI revealed right-hemispheric localization of language in the right-handed patient. We suggest including a bedside test for limb apraxia symptoms in acute clinical care of stroke patients. The distinction between deficits in limb praxis and impairments of language can be complicated owing to the common hemispheric co-localization of the two functions.

Conditional random field-based gesture recognition with depth information

Gesture recognition is useful for human-computer interaction. The difficulty of gesture recognition is that instances of gestures vary both in motion and shape in three-dimensional (3-D) space. We use depth information generated using Microsoft’s Kinect in order to detect 3-D human body components and apply a threshold model with a conditional random field in order to recognize meaningful gestures using continuous motion information. Body gesture recognition is achieved through a framework consisting of two steps. First, a human subject is described by a set of features, encoding the angular relationship between body components in 3-D space. Second, a feature vector is recognized using a threshold model with a conditional random field. In order to show the performance of the proposed method, we use a public data set, the Microsoft Research Cambridge-12 Kinect gesture database. The experimental results demonstrate that the proposed method can efficiently and effectively recognize body gestures automatically.

Type of gesture, valence, and gaze modulate the influence of gestures on observer's behaviors

The present kinematic study aimed at determining whether the observation of arm/hand gestures performed by conspecifics affected an action apparently unrelated to the gesture (i.e., reaching-grasping). In 3 experiments we examined the influence of different gestures on action kinematics. We also analyzed the effects of words corresponding in meaning to the gestures, on the same action. In Experiment 1, the type of gesture, valence and actor's gaze were the investigated variables Participants executed the action of reaching-grasping after discriminating whether the gestures produced by a conspecific were meaningful or not. The meaningful gestures were request or symbolic and their valence was positive or negative. They were presented by the conspecific either blindfolded or not. In control Experiment 2 we searched for effects of the sole gaze, and, in Experiment 3, the effects of the same characteristics of words corresponding in meaning to the gestures and visually presented by the conspecific. Type of gesture, valence, and gaze influenced the actual action kinematics; these effects were similar, but not the same as those induced by words. We proposed that the signal activated a response which made the actual action faster for negative valence of gesture, whereas for request signals and available gaze, the response interfered with the actual action more than symbolic signals and not available gaze. Finally, we proposed the existence of a common circuit involved in the comprehension of gestures and words and in the activation of consequent responses to them.