Non-linguistic Actions Research Articles

Angry lions and scared neighbors: Complex demonstrations in sign language role shift at the sign-gesture interface

Abstract Sign languages make use of the full expressive power of the visual-gestural modality to report the utterances and/or actions of another person. A signer can shift into the perspective of one or more persons and reproduce the utterances or actions from the perspective of these persons. This modality-specific device of utterance and action report is called role shift or constructed action. Especially in sign language narration, role shift is a productive and expressive means that can be used to demonstrate linguistic and non-linguistic actions. Recent developments in sign language linguistics put forth new formal semantic analyses of role shift at the interface between sign language and gesture, integrating insights from classical cognitive and formal analyses of quotation, demonstration and perspective/context shift in spoken and sign languages. In this article, I build on recent accounts of role shift as a modality-specific device of demonstration and show that a modified version of this theory even accounts for cases of complex demonstrations including hybrid demonstrations, multiple demonstrations and demonstrations involving a complex interaction of gestural and linguistic components.

Z problematyki formy czynności konwencjonalnych w prawie

The aim of the article is first of all an attempt to organize the ways of understanding the “form (of conventional actions in law)” and the contexts of the use of this term in detailed legal sciences. The author also considers the issue of performing conventional actions in law by non-linguistic actions and omissions, as well as the issue of significance of form ecognition in the context of identifying conventional actions in law. The results of the research presented in the paper (apart from their cognitive value) are to serve the development of the theoretical concept of conventional actions in law and, as far as possible, to put in order the dogmatic-legal discussion concerning the title issue.

A speech planning network for interactive language use.

During conversation, people take turns speaking by rapidly responding to their partners while simultaneously avoiding interruption1,2. Such interactions display a remarkable degree of coordination, as gaps between turns are typically about 200 milliseconds3-approximately the duration of an eyeblink4. These latencies are considerably shorter than those observed in simple word-production tasks, which indicates that speakers often plan their responses while listening to their partners2. Although a distributed network of brain regions has been implicated in speech planning5-9, the neural dynamics underlying the specific preparatory processes that enable rapid turn-taking are poorly understood. Here we use intracranial electrocorticography to precisely measure neural activity as participants perform interactive tasks, and we observe a functionally and anatomically distinct class of planning-related cortical dynamics. We localize these responses to a frontotemporal circuit centred on the language-critical caudal inferior frontal cortex10 (Broca's region) and the caudal middle frontal gyrus-a region not normally implicated in speech planning11-13. Using a series of motor tasks, we then show that this planning network is more active when preparing speech as opposed to non-linguistic actions. Finally, we delineate planning-related circuitry during natural conversation that is nearly identical to the network mapped with our interactive tasks, and we find this circuit to be most active before participant speech during unconstrained turn-taking. Therefore, we have identified a speech planning network that is central to natural language generation during social interaction.

Semantyka grzebowisk dla zwierząt na tle miejsc pochówków ludzi. Analiza porównawcza makroznaków

The contemporary practice of creating official animal burial sites exhibits certain analogies with cemeteries, which function not only as burial sites for humans but also as complex symbols of remembering the dead. Both are macro-signs, manifested through verbal and non-verbal codes. Human cemeteries are characterised by greater symbolic complexity and diversity, deriving from centuries-old cultural and religious traditions. The linguistic and non-linguistic actions performed with respect to cemeteries, in the broadly understood situational context, are closely related to the sphere of sacrum, represented by the symbol of the cross. Although pet cemeteries are identical to human cemeteries in terms of certain code elements (including headstone inscriptions, photographs, and figural sculptures), such cemeteries are less complex and relate chiefly to the sphere of profanum, represented by symbols of the widespread myth of the Rainbow Bridge. However, the common denominator of both macro-signs are the linguistic and non-linguistic actions that are inspired by the feelings of sadness, love, and gratitude.

Communication Scenarios of Misunderstanding

Introduction. The proposed paper discusses communication situations of mutual misunderstanding up to mutual rejection of each other by the parties. The research assumption is that misunderstanding in human communication is not necessarily accompanied by its overcoming. “Miscommunication communication” forms a communicative space that reveals the diversity of practices of personal self-realization, intergroup and intercultural interaction, while retaining the perception of the other side as incomprehensible.Methodology and sources. The methodological framework for analysis is a social constructionalist approach to the study of social reality, offering a conceptualization of the practical and observable actions of individuals or, in other words, “what people do when they act”. According to pragmatically oriented methodology, we cannot make an exhaustive conclusion about the internal reasons that motivate people to act in one way or another, but we can consider linguistic and non-linguistic actions that are perceived and interpreted by them as having a certain meaning and, therefore, trigger a certain response. The meanings that communicators give to a message are not pre-defined, but are created, produced, and constructed in a communicative interaction through contextspecific discursive procedures and practices, while also triggering specific socially recognizable types of contexts.Results and discussion. The variant of classification of various types of cognitive and communicative experience acquired by people in situations of misunderstanding is offered. Empirical data are the records of interviews, conversations, and comments that are at our possession. The basis for distinguishing between communicative scenarios of misunderstanding is the values in the range of “expanding one's own experience” ↔ “isolation from others' experience”. The structure of description of the selected situations includes: communicative status of the participant; verbal formulas that determine the choice of vector by the participants of the communication; characteristics of cognitive and communicative experience generated by a situation of misunderstanding; examples and illustrations containing replicas, comments, description of life situations of collision with misunderstanding, corresponding to a specific communicative scenario.Conclusion. Situations of misunderstanding are developed in scenarios that provide their participants with the resources to cope with the threat of risk to their personal or group (cultural) identity. The range of cognitive and communicative practices ranges from recognizing the value of cultural (social) diversity for social and personal development to discriminating against others, including violence and the exclusion of the incomprehensible from interaction. Discussion of the issue of “understanding misunderstanding” makes it possible to fit misunderstanding into the social fabric of human behavior practices as a vital resource for any social community.

Inclusion of Signing Deaf or Hard-of-Hearing Students: Factors That Facilitate Versus Challenge Access and Participation

PurposePast studies have consistently indicated a need for improvements in inclusive educational practices. Many have focused on a single actor (e.g., the student), not taking into account the interactive nature of communication.MethodA qualitative study, involving multiple participants in two subject area general education high school classrooms, was completed to examine factors that promote versus challenge deaf or hard-of-hearing (DHH) students' access to communication and participation in classroom activities. Participants included four classroom teachers, two sign language interpreters, and three DHH students who used sign language as their primary mode of communication. Data consisted of eight 60- to 75-min in-class observations, two interviews, and three questionnaires. Data were analyzed using a participation framework of interaction as coordinated linguistic and nonlinguistic actions between multiple actors.ResultsDespite the availability of technological tools and sign language interpretation, DHH students had inconsistent access to classroom communication and lower participation than their hearing peers.ConclusionsThe results correspond to those of past studies, which suggest a research-to-practice gap that needs to be addressed if we are to see improvements in inclusive practices, particularly for students who use signed communication. Several recommendations are proposed in consideration of teacher, interpreter, and student as integral participants. Speech-language pathologists can participate in assessing whether inclusive education is meeting desired results and contribute to the development and implementation of necessary adaptations. Future studies are needed to systematically examine the efficacy of specific education and training programs.

Is Language Production Planning Emergent From Action Planning? A Preliminary Investigation.

The nature of syntactic planning for language production may reflect language-specific processes, but an alternative is that syntactic planning is an example of more domain-general action planning processes. If so, language and non-linguistic action planning should have identifiable commonalities, consistent with an underlying shared system. Action and language research have had little contact, however, and such comparisons are therefore lacking. Here, we address this gap by taking advantage of a striking similarity between two phenomena in language and action production. One is known as syntactic priming—the tendency to re-use a recently produced sentence structure—and the second is hysteresis—the tendency to re-use a previously executed abstract action plan, such as a limb movement. We examined syntactic priming/hysteresis in parallel language and action tasks intermixed in a single experimental session. Our goals were to establish the feasibility of investigating language and action planning within the same participants and to inform debates on the language-specific vs. domain-general nature of planning systems. In both action and language tasks, target trials afforded two alternative orders of subcomponents in the participant’s response: in the language task, a picture could be described with two different word orders, and in the action task, locations on a touch screen could be touched in two different orders. Prime trials preceding the target trial promoted one of two plans in the respective domain. Manipulations yielded higher rates of primed behavior in both tasks. In an exploratory cross-domain analysis, there was some evidence for stronger priming effects in some combinations of action and language priming conditions than others. These results establish a method for investigating the degree to which language planning is part of a domain-general action planning system.

Shared neural and cognitive mechanisms in action and language: The multiscale information transfer framework.

This review compares how humans process action and language sequences produced by other humans. On the one hand, we identify commonalities between action and language processing in terms of cognitive mechanisms (e.g., perceptual segmentation, predictive processing, integration across multiple temporal scales), neural resources (e.g., the left inferior frontal cortex), and processing algorithms (e.g., comprehension based on changes in signal entropy). On the other hand, drawing on sign language with its particularly strong motor component, we also highlight what differentiates (both oral and signed) linguistic communication from nonlinguistic action sequences. We propose the multiscale information transfer framework (MSIT) as a way of integrating these insights and highlight directions into which future empirical research inspired by the MSIT framework might fruitfully evolve. This article is categorized under: Psychology > Language Linguistics > Language in Mind and Brain Psychology > Motor Skill and Performance Psychology > Prediction.

Conceptualizing politeness in Japanese and Greek

AbstractThe main aim of this paper is to investigate how Japanese and Greek female students conceptualize politeness and then compare the findings in order to tease out any cross-cultural similarities and differences. The data is drawn from a questionnaire filled in by two hundred female undergraduates (one hundred from each group). The results show that there are significant similarities as well as some differences. Although research on im/politeness has concentrated almost exclusively on linguistic performance, a significant similarity between the two groups is that politeness is conceptualized as primarily non-linguistic action. Another major similarity is that both groups conceptualize politeness mainly as “consideration to others” and “appropriate behavior,” the former expressed mostly non-linguistically and the latter involving both linguistic and non-linguistic manifestations. Most participants view politeness as conveyed through attentiveness/helping others, respect and empathy. Differences were located mostly in the numbers of participants who mentioned the various subcategories. For example, more Greek participants related a broad sense of “respect” to politeness, whereas more Japanese participants related it to “empathy” and only Japanese participants mentioned “honorifics.” Our participants’ understandings of politeness appear to be in contrast to earlier politeness theories which view politeness as strategic concern for conflict avoidance and closer to current approaches which view it as relational, expressing concern for the needs and feelings of others.

Developing communicative postures

AbstractThis article analyses the development of a composite communicative posture, the shrug (which can combine palm-up flips, lifted shoulders and a head tilt), in a video corpus of spontaneous interactions between a typically developing British girl, Ellie, and her mother, filmed at home one hour each month from Ellie’s tenth month to her fourth birthday. The systematic coding of every shrug yields a total of 124 tokens (Ellie: 98; her mother: 26), providing results in terms of forms, functions and input. Ellie’s first shrug components emerge from non-linguistic actions and she acquires them one at a time starting with the hands: these features recall the development of complex signs among deaf children of the same age (Reilly & Anderson, 2002for ASL). The functions of Ellie’s shrugs gradually diversify from the expression of absence at 1;04 to other epistemic and non-epistemic meanings (affective and dynamic). Adult intervention plays a crucial role as adults recurrently equate Ellie’s physical movements with speech, thereby contributing to the emergence of their communicative functions as gestural emblems (Ekman & Friesen, 1969).

Meaning in Phonology and Other Departures from Modularity in the Living Language

Abstract I review evidence of three kinds relating to leakages in modularity within language domains and between linguistic and nonlinguistic action. One kind of evidence shows that the form-meaning “rift” in language that enables the important principle of duality of patterning and the particulate principle of self-diversifying systems is bridged in many ways. Segmental language forms have iconic meanings, and form-meaning correlations of other kinds emerge cross linguistically. A second kind of evidence occurs in parallel transmission of linguistic prosodic information with iconic and emotional information conveyed suprasegmentally. The final kind of evidence shows the integrality of linguistic and nonlinguistic action (deictic points, speech-accompanying gestures, head motions, facial expressions, etc) in conveying communicative information in public language use. I suggest that these violations of modularity within language and between linguistic and nonlinguistic action reflect the dynamic effects of sets of competing and cooperating constraints including, among others, parity and learnability of language forms that shape communicative actions in social activity.

Areas Recruited during Action Understanding Are Not Modulated by Auditory or Sign Language Experience.

The observation of other people’s actions recruits a network of areas including the inferior frontal gyrus (IFG), the inferior parietal lobule (IPL), and posterior middle temporal gyrus (pMTG). These regions have been shown to be activated through both visual and auditory inputs. Intriguingly, previous studies found no engagement of IFG and IPL for deaf participants during non-linguistic action observation, leading to the proposal that auditory experience or sign language usage might shape the functionality of these areas. To understand which variables induce plastic changes in areas recruited during the processing of other people’s actions, we examined the effects of tasks (action understanding and passive viewing) and effectors (arm actions vs. leg actions), as well as sign language experience in a group of 12 congenitally deaf signers and 13 hearing participants. In Experiment 1, we found a stronger activation during an action recognition task in comparison to a low-level visual control task in IFG, IPL and pMTG in both deaf signers and hearing individuals, but no effect of auditory or sign language experience. In Experiment 2, we replicated the results of the first experiment using a passive viewing task. Together, our results provide robust evidence demonstrating that the response obtained in IFG, IPL, and pMTG during action recognition and passive viewing is not affected by auditory or sign language experience, adding further support for the supra-modal nature of these regions.

Editorial: What can we make of theories of embodiment and the role of the human mirror neuron system?

Editorial: What can we make of theories of embodiment and the role of the human mirror neuron system?

Pointing to Nothing? Empty Places Prime Infants' Attention to Absent Objects

People routinely point to empty space when referring to absent entities. These points to “nothing” are meaningful because they direct attention to places that stand in for specific entities. Typically, the meaning of places in terms of absent referents is established through preceding discourse and accompanying language. However, it is unknown whether nonlinguistic actions can establish locations as meaningful places, and whether infants have the capacity to represent a place as standing in for an object. In a novel eye-tracking paradigm, 18-month-olds watched objects being placed in specific locations. Then, the objects disappeared and a point directed infants' attention to an emptied place. The point to the empty place primed infants in a subsequent scene (in which the objects appeared at novel locations) to look more to the object belonging to the indicated place than to a distracter referent. The place–object expectations were strong enough to interfere when reversing the place–object associations. Findings show that infants comprehend nonlinguistic reference to absent entities, which reveals an ontogenetic early, nonverbal understanding of places as representations of absent objects.

Neuroanatomical substrates of action perception and understanding: an anatomic likelihood estimation meta-analysis of lesion-symptom mapping studies in brain injured patients.

Several neurophysiologic and neuroimaging studies suggested that motor and perceptual systems are tightly linked along a continuum rather than providing segregated mechanisms supporting different functions. Using correlational approaches, these studies demonstrated that action observation activates not only visual but also motor brain regions. On the other hand, brain stimulation and brain lesion evidence allows tackling the critical question of whether our action representations are necessary to perceive and understand others’ actions. In particular, recent neuropsychological studies have shown that patients with temporal, parietal, and frontal lesions exhibit a number of possible deficits in the visual perception and the understanding of others’ actions. The specific anatomical substrates of such neuropsychological deficits however, are still a matter of debate. Here we review the existing literature on this issue and perform an anatomic likelihood estimation meta-analysis of studies using lesion-symptom mapping methods on the causal relation between brain lesions and non-linguistic action perception and understanding deficits. The meta-analysis encompassed data from 361 patients tested in 11 studies and identified regions in the inferior frontal cortex, the inferior parietal cortex and the middle/superior temporal cortex, whose damage is consistently associated with poor performance in action perception and understanding tasks across studies. Interestingly, these areas correspond to the three nodes of the action observation network that are strongly activated in response to visual action perception in neuroimaging research and that have been targeted in previous brain stimulation studies. Thus, brain lesion mapping research provides converging causal evidence that premotor, parietal and temporal regions play a crucial role in action recognition and understanding.

Response-Locked Brain Dynamics of Word Production

The cortical regions involved in the different stages of speech production are relatively well-established, but their spatio-temporal dynamics remain poorly understood. In particular, the available studies have characterized neural events with respect to the onset of the stimulus triggering a verbal response. The core aspect of language production, however, is not perception but action. In this context, the most relevant question may not be how long after a stimulus brain events happen, but rather how long before the production act do they occur. We investigated speech production-related brain activity time-locked to vocal onset, in addition to the common stimulus-locked approach. We report the detailed temporal interplay between medial and left frontal activities occurring shortly before vocal onset. We interpret those as reflections of, respectively, word selection and word production processes. This medial-lateral organization is in line with that described in non-linguistic action control, suggesting that similar processes are at play in word production and non-linguistic action production. This novel view of the brain dynamics underlying word production provides a useful background for future investigations of the spatio-temporal brain dynamics that lead to the production of verbal responses.

Role of the lateral prefrontal cortex in speech monitoring

The role of lateral prefrontal cortex (LPFC) in speech monitoring has not been delineated. Recent work suggests that medial frontal cortex (MFC) is involved in overt speech monitoring initiated before auditory feedback. This mechanism is reflected in an event-related potential (ERP), the error negativity (Ne), peaking within 100 ms after vocal-onset. Critically, in healthy individuals the Ne is sensitive to the accuracy of the response; it is larger for error than correct trials. By contrast, patients with LPFC damage are impaired in non-verbal monitoring tasks showing no amplitude difference between the Ne measured in correct vs. error trials. Interactions between the LPFC and the MFC are assumed to play a necessary role for normal action monitoring. We investigated whether the LPFC was involved in speech monitoring to the same extent as in non-linguistic actions by comparing performance and EEG activity in patients with LPFC damage and in aged-matched controls performing linguistic (Picture Naming) and non-linguistic (Simon) tasks. Controls did not produce enough errors to allow the comparison of the Ne or other ERP in error vs. correct trials. PFC patients had worse performance than controls in both tasks, but their Ne was larger for error than correct trials only in Naming. This task-dependent pattern can be explained by LPFC-dependent working-memory requirements present in non-linguistic tasks used to study action monitoring but absent in picture naming. This suggests that LPFC may not be necessary for speech monitoring as assessed by simple picture naming. In addition, bilateral temporal cortex activity starting before and peaking around vocal-onset was observed in LPFC and control groups in both tasks but was larger for error than correct trials only in Naming, suggesting the temporal cortex is associated with on-line monitoring of speech specifically when access to lexical representations is necessary.

Exploring the movement dynamics of manual and oral articulation: Evidence from coarticulation

AbstractThis project explores three classes of human action through an investigation of long-distance coarticulation, defined here as the articulatory influence of one phonetic element (e.g., consonant or vowel) on another across more than one intervening element. Our first experiment investigated anticipatory vowel-to-vowel (VV) coarticulation in English. The second experiment was patterned after the first but deals instead with anticipatory location-to-location (LL) effects in American Sign Language (ASL). The sign experiment also incorporated a non-linguistic manual action, permitting a comparison of effects not only between spoken and signed language, but also between linguistic and non-linguistic manual actions.For the spoken-language study, sentences were created in which multiple consecutive schwas (target vowels) were followed by various context vowels. Eighteen English speakers were recorded as they repeated each sentence six times, and statistical tests were performed to determine the extent to which target vowel formant frequencies were influenced differently by different context vowels. For some speakers, significant effects of one vowel on another were found across as many as five intervening segments. In the sign study, motion-capture technology was used to investigate LL coarticulation in the signing of five ASL users. Some evidence was found of significant LL coarticulatory influence of one sign on another across as many as three intervening signs. However, LL effects overall were weaker and less pervasive than the VV effects found in the spoken-language study. The outcomes in the sign study's linguistic and non-linguistic conditions did not substantially differ. Collectively, these results support and complement previous research which has found that in comparisons of linguistic and non-linguistic manual and oral actions, movement patterns associated with oral and manual actions differ appreciably, while the linguistic vs. non-linguistic distinction appears to show little or no influence.

The Embodied Language: Why Language Should Not Be Conceived of in Abstraction from the Brain and Body, and the Consequences for Robotics

Until very recently, most language research has regarded linguistic phenomena as separated from the brain and body. This could lead one into believing that in order to emulate linguistic behavior, it suffices to develop ‘software’ operating on abstract representations that will work on any computational machine. This picture is inaccurate for several reasons, which are elucidated in this paper and extend beyond sensorimotor and semantic resonance. Without taking into account both the architecture of the human brain, and embodiment, it is unrealistic to replicate accurately the processes which take place during language acquisition, comprehension, or production, or during non-linguistic actions. While robots are far from isomorphic with humans, they could benefit from strengthened associative connections in the optimization of their processes, reactivity and sensitivity to environmental stimuli, and situated human-machine interaction. The concept of multisensory integration should be extended and cover linguistic input and the complementary information that the brain combines from temporally coincident sensory impressions.

Exploring perceptual processing of ASL and human actions: Effects of inversion and repetition priming

In this paper, we compare responses of deaf signers and hearing non-signers engaged in a categorization task of signs and non-linguistic human actions. We examine the time it takes to make such categorizations under conditions of 180° stimulus inversion and as a function of repetition priming, in an effort to understand whether the processing of sign language forms draws upon special processing mechanisms or makes use of mechanisms used in recognition of non-linguistic human actions. Our data show that deaf signers were much faster in the categorization of both linguistic and non-linguistic actions, and relative to hearing non-signers, show evidence that they were more sensitive to the configural properties of signs. Our study suggests that sign expertise may lead to modifications of a general-purpose human action recognition system rather than evoking a qualitatively different mode of processing, and supports the contention that signed languages make use of perceptual systems through which humans understand or parse human actions and gestures more generally.