Hearing Signers Research Articles

Between bodily action and conventionalized structure: The neural mechanisms of constructed action in sign language comprehension

Sign languages (SLs) are expressed through different bodily actions, ranging from re-enactment of physical events (constructed action, CA) to sequences of lexical signs with internal structure (plain telling, PT). Despite the prevalence of CA in signed interactions and its significance for SL comprehension, its neural dynamics remain unexplored. We examined the processing of different types of CA (subtle, reduced, and overt) and PT in 35 adult deaf or hearing native signers. The electroencephalographic-based processing of signed sentences with incongruent targets was recorded. Attenuated N300 and early N400 were observed for CA in deaf but not in hearing signers. No differences were found between sentences with CA types in all signers, suggesting a continuum from PT to overt CA. Deaf signers focused more on body movements; hearing signers on faces. We conclude that CA is processed less effortlessly than PT, arguably because of its strong focus on bodily actions.

Motor adaptation in deaf and hearing native signers.

Previous studies have suggested that deafness could lead to deficits in motor skills and other body-related abilities. However, the literature regarding motor skills in deaf adults is scarce and existing studies often included participants with heterogeneous language backgrounds and deafness etiologies, thus making it difficult to delineate the effects of deafness. In this study, we investigated motor learning in deaf native signers and hearing nonsigners. To isolate the effects of deafness and those of acquiring a signed language, we additionally tested a group of hearing native signers. Two well-established paradigms of motor learning were employed, in which participants had to adapt their hand movements to a rotation of the visual feedback (Experiment 1) or to the introduction of a force field (Experiment 2). Proprioceptive estimates were assessed before and after adaptation. Like hearing nonsigners, deaf and hearing signers showed robust adaptation in both motor adaptation paradigms. No significant differences in motor adaptation and memory were observed between deaf signers and hearing nonsigners, as well as between hearing signers and hearing nonsigners. Moreover, no discernible group differences in proprioceptive accuracy were observed. These findings challenge the prevalent notion that deafness leads to deficits in motor skills and other body-related abilities.

Characterizing language production across modalities

ABSTRACT This study investigates factors influencing lexical access in language production across modalities (signed and oral). Data from deaf and hearing signers were reanalyzed (Baus and Costa, 2015, On the temporal dynamics of sign production: An ERP study in Catalan Sign Language (LSC). Brain Research, 1609(1), 40-53. https://doi.org/10.1016/j.brainres.2015.03.013; Gimeno-Martínez and Baus, 2022, Iconicity in sign language production: Task matters. Neuropsychologia, 167, 108166. https://doi.org/10.1016/j.neuropsychologia.2022.108166) testing the influence of psycholinguistic variables and ERP mean amplitudes on signing and naming latencies. Deaf signers' signing latencies were influenced by sign iconicity in the picture signing task, and by spoken psycholinguistic variables in the word-to-sign translation task. Additionally, ERP amplitudes before response influenced signing but not translation latencies. Hearing signers' latencies, both signing and naming, were influenced by sign iconicity and word frequency, with early ERP amplitudes predicting only naming latencies. These findings highlight general and modality-specific determinants of lexical access in language production.

Introduction to the Special Issue: Learning Sign Languages as Additional Languages: Considering Language‐ and Modality‐Specific Factors

AbstractAdditional language (L2/Ln) research largely focuses on learners whose first languages are spoken and who are learning additional spoken languages. In the past few decades, sign languages have become increasingly popular for hearing students in schools. These students must not only learn the vocabulary and grammar of sign languages but also manage a different modality (that is, the channels of production and reception of language) than their first language. This raises questions about the role of both language and modality in the L2/Ln learning of sign languages for non‐signers. In other cases, deaf and hearing signers of a sign language learn a different sign language, raising questions about L2/Ln learning in the signed modality. This Special Issue consists of empirical contributions and a conceptual review article that examine how language and modality shape the learning of sign languages as additional languages. Theoretical issues concerning learning a sign language as another language are discussed.

Language-related motor facilitation in Italian Sign Language signers.

Linguistic tasks facilitate corticospinal excitability as revealed by increased motor evoked potential (MEP) induced by transcranial magnetic stimulation (TMS) in the dominant hand. This modulation of the primary motor cortex (M1) excitability may reflect the relationship between speech and gestures. It is conceivable that in healthy individuals who use a sign language this cortical excitability modulation could be rearranged. The aim of this study was to evaluate the effect of spoken language tasks on M1 excitability in a group of hearing signers. Ten hearing Italian Sign Language (LIS) signers and 16 non-signer healthy controls participated. Single-pulse TMS was applied to either M1 hand area at the baseline and during different tasks: (i) reading aloud, (ii) silent reading, (iii) oral movements, (iv) syllabic phonation and (v) looking at meaningless non-letter strings. Overall, M1 excitability during the linguistic and non-linguistic tasks was higher in LIS group compared to the control group. In LIS group, MEPs were significantly larger during reading aloud, silent reading and non-verbal oral movements, regardless the hemisphere. These results suggest that in hearing signers there is a different modulation of the functional connectivity between the speech-related brain network and the motor system.

Deafness and early language deprivation influence arithmetic performances.

It has been consistently reported that deaf individuals experience mathematical difficulties compared to their hearing peers. However, the idea that deafness and early language deprivation might differently affect verbal (i.e., multiplication) vs. visuospatial (i.e., subtraction) arithmetic performances is still under debate. In the present paper, three groups of 21 adults (i.e., deaf signers, hearing signers, and hearing controls) were therefore asked to perform, as fast and as accurately as possible, subtraction and multiplication operations. No significant group effect was found for accuracy performances. However, reaction time results demonstrated that the deaf group performed both arithmetic operations slower than the hearing groups. This group difference was even more pronounced for multiplication problems than for subtraction problems. Weaker language-based phonological representations for retrieving multiplication facts, and sensitivity to interference are two hypotheses discussed to explain the observed dissociation.

Sign language recognition by means of common spatial patterns: An analysis.

Currently there are around 466 million hard of hearing people and this amount is expected to grow in the coming years. Despite the efforts that have been made, there is a communication barrier between deaf and hard of hearing signers and non-signers in environments without an interpreter. Different approaches have been developed lately to try to deal with this issue. In this work, we present an Argentinian Sign Language (LSA) recognition system which uses hand landmarks extracted from videos of the LSA64 dataset in order to distinguish between different signs. Different features are extracted from the signals created with the hand landmarks values, which are first transformed by the Common Spatial Patterns (CSP) algorithm. CSP is a dimensionality reduction algorithm and it has been widely used for EEG systems. The features extracted from the transformed signals have been then used to feed different classifiers, such as Random Forest (RF), K-Nearest Neighbors (KNN) or Multilayer Perceptron (MLP). Several experiments have been performed from which promising results have been obtained, achieving accuracy values between 0.90 and 0.95 on a set of 42 signs.

The Roles of Manual and non-manual Cues in Recognizing Irony in Italian Sign Language

ABSTRACT In a previous study, our research group investigated the expression of irony in Italian Sign Language (LIS) and suggested that specific manual and non-manual markers signaled the signer’s meaning and attitude. The present research aimed at expanding those findings by analyzing whether these markers are used in irony recognition and whether they are language-specific. We designed an experiment in which we compared recognition of ironic remarks out of context considering three groups of Italians: Deaf signers, hearing signers, and hearing non-signers. Our aim was to verify whether the manual and non-manual markers we associated with the expression of irony in LIS constitute a reliable cue to detect irony and whether these cues were accessible to signers more than to non-signers sharing the same cultural background. Although the ironic intent in LIS was accessible also to hearing non-signers, we found, among hearing participants, that knowledge of LIS does improve accuracy in recognizing ironic remarks in LIS. This suggests that signers’ facial expressions and bodily movements do not solve a purely affective function, but are grammaticalized, at least to some degree.

Developmental experiences alter the temporal processing characteristics of the visual cortex: Evidence from deaf and hearing native signers.

To date, the extent to which early experience shapes the functional characteristics of neural circuits is still a matter of debate. In the present study, we tested whether congenital deafness and/or the acquisition of a sign language alter the temporal processing characteristics of the visual system. Moreover, we investigated whether, assuming cross-modal plasticity in deaf individuals, the temporal processing characteristics of possibly reorganised auditory areas resemble those of the visual cortex. Steady-state visual evoked potentials (SSVEPs) were recorded in congenitally deaf native signers, hearing native signers, and hearing nonsigners. The luminance of the visual stimuli was periodically modulated at 12, 21, and 40 Hz. For hearing nonsigners, the optimal driving rate was 12 Hz. By contrast, for the group of hearing signers, the optimal driving rate was 12 and 21 Hz, whereas for the group of deaf signers, the optimal driving rate was 21 Hz. We did not observe evidence for cross-modal recruitment of auditory cortex in the group of deaf signers. These results suggest a higher preferred neural processing rate as a consequence of the acquisition of a sign language.

Sign networks: Nucleated network sign languages and rural homesign in Papua New Guinea

AbstractThe sociodemographic typology of sign languages classifies them based on the characteristics and configurations of their users. When considering homesign and sign languages in rural areas, this typology needs further refinement. Here, I present new concepts to enable this. The study is based on fieldwork with twelve deaf people in Western Highlands, Papua New Guinea, and review of studies worldwide. Sign language communities can be mapped assign networks. Using this mapping, I propose a new typological category for languages with one central deaf user and many fluent hearing signers:nucleated network sign language. I usesign base analysisto determine lexical consistency between unconnected deaf signers in Western Highlands. The high level of consistency among largely unconnected deaf people is explained by aregional sign networkconnecting deaf and hearing signers. This research emphasises the role of both deaf and hearing signers in sign language emergence and maintenance. (Sign languages, social networks, sign networks, typology, homesign, rural sign languages, Papua New Guinea)*

Seeing Signs: Language Experience and Handshape Perception

Thousands of times every day you see signing around you. But what does it mean to "see" a sign? In this article we will discuss the difference between "seeing" and "perceiving". You see with your eyes, but you perceive with your mind. In this special issue on "Science and the Senses" we address what happens in our mind when we "see" signs, and how our experiences growing up around sign language shape the way our minds make sense of visual experience. We report the results of an experiment investigating whether deaf and hearing signers who learned to sign at different ages perceive handshapes differently using synthetic signs produced via animation. All signers tested categorized the handshapes in a similar manner, but signers differed in discrimination performance. Early learners of ASL tend to ignore meaningless phonetic variation in handshape tokens more than individuals who learned to sign later in life, potentially allowing faster and more efficient recognition of handshapes.

Emergence and Evolutions: Introducing Sign Language Sociolinguistics

Emergence and Evolutions:Introducing Sign Language Sociolinguistics Annelies Kusters (bio) and Ceil Lucas (bio) sign languages, rural sign language, village sign language, sociolinguistics, variation, multimodal The very nature of sign languages in society is such that all of the areas of sociolinguistics are relevant to their study (Schembri and Lucas 2015; Roy and Metzger 2014). Sign languages range from national ones such as American Sign Language (ASL; Supalla and Clark 2014); or rural sign languages such Adamorobe Sign Language used in Ghana and in many other communities, where members of a community, both hearing and deaf, use a sign language (Nyst 2007; Zeshan and de Vos 2012); to alternate sign languages (Kendon 1989) such as Plains Indian Sign Language used by the hearing speakers of mutually unintelligible Native American languages and sometimes used also by deaf members of these communities (Davis 2015). Sign languages that have emerged in differing contexts have been used on different scales too: rural sign languages are used locally, in wider language ecologies that feature the ample use of gesture (Branson et al. 1999); national sign languages, which mostly emerged in deaf schools and deaf families are widely used throughout countries and beyond national borders (e.g., in contexts of migration); and some sign languages are used around the world in international contexts, most notably American Sign Language (ASL) and International [End Page 320] Sign (Parks 2014; Parks and Kusters forthcoming). The geographical spread of sign languages has developed largely independently from that of spoken languages, and researchers have identified sign language families (Padden 2011). Sign sociolinguists have documented the varied social contexts of these sign languages, which emerged respectively in mostly deaf contexts (e.g., Palfreyman 2019), in mixed deaf/hearing contexts (e.g., Marsaja 2008), and in mostly hearing contexts (e.g., Kendon 1989), and have proposed accounts on how these contexts of use shape the structures of sign languages (e.g., Schembri et al. 2018; Ellis et al. 2019). Naturally, the sensory asymmetries between deaf and hearing people, and related sensory access to languages impacts on sign languages, their use, and conceptualizations of them (De Meulder et al. 2019). This is one crucial difference between sign language sociolinguistics and spoken language sociolinguistics, which we expand on below. In a Dialogue section of the Journal of Sociolinguistics (vol. 26, no. 1), author pairs introduce a number of themes and debates in sign language sociolinguistics, explore why these are debates; how the debates are situated within sociolinguistics as a whole; and how spoken language sociolinguistics does or does not have similar debates. In order to give authors room to think beyond established subject areas (such as "language contact" or "variationism"), we chose five titles that foreground and represent debates rather than subject areas: Geographies and Circulations; Lumping and Splitting; Hierarchies and Constellations; Classifications and Typologies; Natural and Elicited. These main themes and debates in sign language sociolinguistics, as well as main parallels and differences with spoken language sociolinguistics, are introduced below. Themes in Sign Language Sociolinguistics Sign languages do not exist in sociolinguistic vacuums—sign languages are used in contexts where spoken/written languages, and increasingly also other sign languages are in use. An increasing number of signers are fluent in multiple spoken/written and/or signed languages. In short, multilingualism as concerns deaf and hearing sign language users is an extremely rich topic for investigation (Van Herreweghe and Vermeerbergen 2004; [End Page 321] Zeshan and Webster 2020). Multilingualism of course is closely linked to language contact, with all of the phenomena attested in spoken language contact and bilingualism also seen in sign language situations, along with some unique ones related to the multimodal nature of sign languages and the sensory asymmetries between deaf and hearing signers. Examples of signed and spoken/written language contact include fingerspelling (i.e., the manual representation of a writing system which has been assimilated into the phonology of many sign languages), the (partial) mouthing of spoken words, codeblending (also called sign-speaking or code-mixing), and chaining (e.g., signing a concept and then fingerspelling it); (e.g., Bagga-Gupta 2000; Holmström and Schönström 2018; Lucas and Valli 1992; Tapio 2019). Adam and Braithwaite (2022...

Effects of deafness and sign language experience on the human brain: voxel-based and surface-based morphometry

ABSTRACT We investigated how deafness and sign language experience affect the human brain by comparing neuroanatomical structures across congenitally deaf signers (n = 30), hearing native signers (n = 30), and hearing sign-naïve controls (n = 30). Both voxel-based and surface-based morphometry results revealed deafness-related structural changes in visual cortices (grey matter), right frontal lobe (gyrification), and left Heschl's gyrus (white matter). The comparisons also revealed changes associated with lifelong signing experience: expansions in the surface area within left anterior temporal and left occipital lobes, and a reduction in cortical thickness in the right occipital lobe for deaf and hearing signers. Structural changes within these brain regions may be related to adaptations in the neural networks involved in processing signed language (e.g. visual perception of face and body movements). Hearing native signers also had unique neuroanatomical changes (e.g. reduced gyrification in premotor areas), perhaps due to lifelong experience with both a spoken and a signed language.

The sign superiority effect: Lexical status facilitates peripheral handshape identification for deaf signers.

Deaf signers exhibit an enhanced ability to process information in their peripheral visual field, particularly the motion of dots or orientation of lines. Does their experience processing sign language, which involves identifying meaningful visual forms across the visual field, contribute to this enhancement? We tested whether deaf signers recruit language knowledge to facilitate peripheral identification through a sign superiority effect (i.e., better handshape discrimination in a sign than a pseudosign) and whether such a superiority effect might be responsible for perceptual enhancements relative to hearing individuals (i.e., a decrease in the effect of eccentricity on perceptual identification). Deaf signers and hearing signers or nonsigners identified the handshape presented within a static ASL fingerspelling letter (Experiment 1), fingerspelled sequence (Experiment 2), or sign or pseudosign (Experiment 3) presented in the near or far periphery. Accuracy on all tasks was higher for deaf signers than hearing nonsigning participants and was higher in the near than the far periphery. Across experiments, there were different patterns of interactions between hearing status and eccentricity depending on the type of stimulus; deaf signers showed an effect of eccentricity for static fingerspelled letters, fingerspelled sequences, and pseudosigns but not for ASL signs. In contrast, hearing nonsigners showed an effect of eccentricity for all stimuli. Thus, deaf signers recruit lexical knowledge to facilitate peripheral perceptual identification, and this perceptual enhancement may derive from their extensive experience processing visual linguistic information in the periphery during sign comprehension. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Pre-output Language Monitoring in Sign Production.

A domain-general monitoring mechanism is proposed to be involved in overt speech monitoring. This mechanism is reflected in a medial frontal component, the error negativity (Ne), present in both errors and correct trials (Ne-like wave) but larger in errors than correct trials. In overt speech production, this negativity starts to rise before speech onset and is therefore associated with inner speech monitoring. Here, we investigate whether the same monitoring mechanism is involved in sign language production. Twenty deaf signers (American Sign Language [ASL] dominant) and 16 hearing signers (English dominant) participated in a picture-word interference paradigm in ASL. As in previous studies, ASL naming latencies were measured using the keyboard release time. EEG results revealed a medial frontal negativity peaking within 15 msec after keyboard release in the deaf signers. This negativity was larger in errors than correct trials, as previously observed in spoken language production. No clear negativity was present in the hearing signers. In addition, the slope of the Ne was correlated with ASL proficiency (measured by the ASL Sentence Repetition Task) across signers. Our results indicate that a similar medial frontal mechanism is engaged in preoutput language monitoring in sign and spoken language production. These results suggest that the monitoring mechanism reflected by the Ne/Ne-like wave is independent of output modality (i.e., spoken or signed) and likely monitors prearticulatory representations of language. Differences between groups may be linked to several factors including differences in language proficiency or more variable lexical access to motor programming latencies for hearing than deaf signers.

The effect of cultural transmission on shared sign language persistence

In this paper, we revisit a mathematical model of sign language persistence by Aoki and Feldman (Theor Popul Biol 39(3):358–372, 1991), which investigates the evolution of genes causing deafness, affected by an assortative mating parameter, and the cultural transmission of sign language. To assess their model, we reimplement it as an agent-based simulation to be able to easily represent structured relationships in a finite population. We study the persistence of shared sign languages, a categorization of sign languages, which are typically shared by deaf and hearing members of a small community with a high incidence of hereditary deafness (Nyst, 2012. Shared sign languages. Sign language: An international handbook, pp. 552–574). We observe how shared sign language persistence is affected by hearing signers, marriage patterns, and various modes of sign language transmission: vertical, horizontal, oblique, and grandparental transmission. In contrast to Aoki and Feldman’s (Theor Popul Biol 9(3):358–372, 1991) finding that modes of transmission other than vertical are negligible, in the agent-based model we find that adding modes of transmission helps to ensure shared sign language persistence. A better understanding of sign language persistence has relevance for processes of cultural evolution, (sign language) linguistics, and language endangerment.

The impact of early language exposure on the neural system supporting language in deaf and hearing adults

Deaf late signers provide a unique perspective on the impact of impoverished early language exposure on the neurobiology of language: insights that cannot be gained from research with hearing people alone. Here we contrast the effect of age of sign language acquisition in hearing and congenitally deaf adults to examine the potential impact of impoverished early language exposure on the neural systems supporting a language learnt later in life. We collected fMRI data from deaf and hearing proficient users (N ​= ​52) of British Sign Language (BSL), who learnt BSL either early (native) or late (after the age of 15 years) whilst they watched BSL sentences or strings of meaningless nonsense signs.There was a main effect of age of sign language acquisition (late ​> ​early) across deaf and hearing signers in the occipital segment of the left intraparietal sulcus. This finding suggests that late learners of sign language may rely on visual processing more than early learners, when processing both linguistic and nonsense sign input – regardless of hearing status. Region-of-interest analyses in the posterior superior temporal cortices (STC) showed an effect of age of sign language acquisition that was specific to deaf signers. In the left posterior STC, activation in response to signed sentences was greater in deaf early signers than deaf late signers. Importantly, responses in the left posterior STC in hearing early and late signers did not differ, and were similar to those observed in deaf early signers. These data lend further support to the argument that robust early language experience, whether signed or spoken, is necessary for left posterior STC to show a ‘native-like’ response to a later learnt language.

The DRM paradigm in sign language: An investigation of associative memory errors in deaf and hearing signers

There is a lack of research investigating associative memory errors in sign language. Thus, the Deese/Roediger-McDermott (DRM) paradigm was used to test recall and recognition errors in deaf signers following four types of encoding: written words, signs, written words presented along with signs, or written words combined with self-produced signs. The results revealed poor memory for studied material along with a relatively low level of false memory for critical items. However, dual coding appeared to enhance veridical memory. Interestingly, a high rate of noncritical intrusions was noticed across all four encoding conditions. Similar results were obtained for recognition, however, there were no differences between encoding conditions in the false alarms rate toward critical lures. The results point to difficulties in accessing the meaning of words in deaf participants. To test this hypothesis a follow-up study was conducted where false recall and recognition of words or signs were tested in hearing participants fluent in sign language. The second experiment revealed semantic memory errors comparable to previous studies with hearing participants. The results from both experiments are discussed in light of previous studies on vocabulary knowledge in deaf individuals.

A Spanish Sign Language (LSE) Adaptation of the Communicative Development Inventories.

This article presents the adaptation of the MacArthur Communicative Development Inventory (CDI; Fenson et al., 1993, Guide and technical manual for the MacArthur Communicative Development Inventories. San Diego, CA: Singular Press; Fenson et al. 1994, Variability in early communicative development. Monographs of the Society for Research in Child Development, 59, 1-173) to Spanish Sign Language (LSE). Data were collected from 55 participants (32 boys and 23 girls; 17 deaf signers, 38 hearing signers) who, evaluated by their caregivers every 4months, presented a total of 170 records. The parents reported the signs that the children could understand or produce between 8 and 36months. Results suggested that the CDI adapted to LSE is a valid and reliable instrument. Signing children could understand more signs than they produced at this early developmental stage. There were no significant differences between boys and girls, or between deaf and hearing children. The development of LSE is similar to other sign languages, although with a lower production of signs in the early stages, perhaps due to the bilingualism of most of the children of our study.

Sign language experience redistributes attentional resources to the inferior visual field

While a substantial body of work has suggested that deafness brings about an increased allocation of visual attention to the periphery there has been much less work on how using a signed language may also influence this attentional allocation. Signed languages are visual-gestural and produced using the body and perceived via the human visual system. Signers fixate upon the face of interlocutors and do not directly look at the hands moving in the inferior visual field. It is therefore reasonable to predict that signed languages require a redistribution of covert visual attention to the inferior visual field. Here we report a prospective and statistically powered assessment of the spatial distribution of attention to inferior and superior visual fields in signers – both deaf and hearing – in a visual search task. Using a Bayesian Hierarchical Drift Diffusion Model, we estimated decision making parameters for the superior and inferior visual field in deaf signers, hearing signers and hearing non-signers. Results indicated a greater attentional redistribution toward the inferior visual field in adult signers (both deaf and hearing) than in hearing sign-naïve adults. The effect was smaller for hearing signers than for deaf signers, suggestive of either a role for extent of exposure or greater plasticity of the visual system in the deaf. The data provide support for a process by which the demands of linguistic processing can influence the human attentional system.