American Sign Language Signs Research Articles

Thoughts About a Possible Bridge From ASL to English Literacy

Thoughts About a Possible Bridge From ASL to English Literacy Peter V. Paul This issue contains a special section, composed of five articles that address what I call a possible bridge from American Sign Language (ASL) to the development of English literacy. Grushkin (this issue) provides his perspectives on the development of a written-language component for ASL (i.e., written ASL), which can be used to facilitate the acquisition of English literacy (reading and writing) skills. I invited four reactants with the hope of securing a panorama of views on this possible bridge. From one standpoint, it can be argued that this special section is a follow-up to a previously published special issue, edited by Wang and Andrews (2014, 2015)—Reading and Deaf Individuals: Perspectives on the Qualitative Similarity Hypothesis. In the conclusion of the second part of that special issue (Paul, 2015), I mentioned the work of Cripps, cited in Andrews, Byrne, and Clark (2015). Among other assertions, Cripps proposes that ASL sign writing might function as an intermediate writing system or an intermediary system. This intermediary system might serve as the bridge (e.g., cross-linguistic transfer) on which ASL-using individuals could travel along to the acquisition of English literacy skills. Does this bridge have any theoretical or research support? Should we explore this pathway regardless and ascertain its feasibility or practicality? To whom would this bridge be applicable? What exactly is the nature of this bridge? Is this a bridge to nowhere, or can it be a possible accessible connection to the development of English literacy skills? Per my editorial approach to any controversial topic in the Annals, I shall cherry-pick a few assertions and passages from Grushkin and the reactants; however, I strongly encourage you to read all of the articles carefully, given the complexity of the arguments. Of course, I provide my own one-and-one-half cents, which might be worth a little less now because you have to wonder if I am planning to offer anything new and exciting in light of the information already provided in my previous editorials (Paul, 2014, 2015). Only poets and philosophers can travel along the same route and offer something innovative—however, my repeating of a few timeworn salient points is always instructive and beneficial. Moores’s Response In reacting to Grushkin’s proposal, Moores (this issue) offers a historical perspective, buttressed by extensive amounts of professional and scholarly experience. This scholar believes that the construct of a written ASL is a topic that is timely and worthy of attention. In fact, Moores remarks, “I approached the article with a predisposition to accept its advocacy of a written ASL” (p. 537). In short, it is clear that Moores has argued that a written system should be developed (i.e., that it has inherent merit), especially if this system can become a beneficial educational tool for d/Deaf and hard of hearing (d/Dhh) children and adolescents. One of the most interesting assertions by Moores concerns the composition or nature of the written ASL system. He questions whether a written ASL system should be influenced (or, perhaps, constrained) by the structure of written English (or the written system of any other spoken language). This seems to be in response to Grushkin’s summary statements, two of which are the following: “The script should be alphabetic, for maximal congruence with English-learning strategies” and “The script should have a phoneme/grapheme relationship that is as clear as possible” (p. 524). In my view, Grushkin is attempting to construct a bridge to English literacy, and this approach provides the context for his summary points. In any case, Moores strongly encourages research and development related to Grushkin’s main ideas, but cautions that such research should not be constrained by the structure of a spoken language. It is possible that Moores believes that the written ASL system would have a range of differential benefits [End Page 505] for d/Dhh students. This written system may be a possible bridge to the development of English literacy; however, it can serve other critical educational functions. Supalla, Cripps, and Byrne’s Response It is safe to conclude that Supalla, Cripps...

A Wearable System for Recognizing American Sign Language in Real-Time Using IMU and Surface EMG Sensors.

A sign language recognition system translates signs performed by deaf individuals into text/speech in real time. Inertial measurement unit and surface electromyography (sEMG) are both useful modalities to detect hand/arm gestures. They are able to capture signs and the fusion of these two complementary sensor modalities will enhance system performance. In this paper, a wearable system for recognizing American Sign Language (ASL) in real time is proposed, fusing information from an inertial sensor and sEMG sensors. An information gain-based feature selection scheme is used to select the best subset of features from a broad range of well-established features. Four popular classification algorithms are evaluated for 80 commonly used ASL signs on four subjects. The experimental results show 96.16% and 85.24% average accuracies for intra-subject and intra-subject cross session evaluation, respectively, with the selected feature subset and a support vector machine classifier. The significance of adding sEMG for ASL recognition is explored and the best channel of sEMG is highlighted.

The Phonetics of Head and Body Movement in the Realization of American Sign Language Signs

Background/Aims: Because the primary articulators for sign languages are the hands, sign phonology and phonetics have focused mainly on them and treated other articulators as passive targets. However, there is abundant research on the role of nonmanual articulators in sign language grammar and prosody. The current study examines how hand and head/body movements are coordinated to realize phonetic targets. Methods: Kinematic data were collected from 5 deaf American Sign Language (ASL) signers to allow the analysis of movements of the hands, head and body during signing. In particular, we examine how the chin, forehead and torso move during the production of ASL signs at those three phonological locations. Results: Our findings suggest that for signs with a lexical movement toward the head, the forehead and chin move to facilitate convergence with the hand. By comparison, the torso does not move to facilitate convergence with the hand for signs located at the torso. Conclusion: These results imply that the nonmanual articulators serve a phonetic as well as a grammatical or prosodic role in sign languages. Future models of sign phonetics and phonology should take into consideration the movements of the nonmanual articulators in the realization of signs.

Response bias reveals enhanced attention to inferior visual field in signers of American Sign Language.

Deafness results in cross-modal plasticity, whereby visual functions are altered as a consequence of a lack of hearing. Here, we present a reanalysis of data originally reported by Dye et al. (PLoS One 4(5):e5640, 2009) with the aim of testing additional hypotheses concerning the spatial redistribution of visual attention due to deafness and the use of a visuogestural language (American Sign Language). By looking at the spatial distribution of errors made by deaf and hearing participants performing a visuospatial selective attention task, we sought to determine whether there was evidence for (1) a shift in the hemispheric lateralization of visual selective function as a result of deafness, and (2) a shift toward attending to the inferior visual field in users of a signed language. While no evidence was found for or against a shift in lateralization of visual selective attention as a result of deafness, a shift in the allocation of attention from the superior toward the inferior visual field was inferred in native signers of American Sign Language, possibly reflecting an adaptation to the perceptual demands imposed by a visuogestural language.

Semantic Integration and Age of Acquisition Effects in Code-Blend Comprehension.

Semantic and lexical decision tasks were used to investigate the mechanisms underlying code-blend facilitation: the finding that hearing bimodal bilinguals comprehend signs in American Sign Language (ASL) and spoken English words more quickly when they are presented together simultaneously than when each is presented alone. More robust facilitation effects were observed for semantic decision than for lexical decision, suggesting that lexical integration of signs and words within a code-blend occurs primarily at the semantic level, rather than at the level of form. Early bilinguals exhibited greater facilitation effects than late bilinguals for English (the dominant language) in the semantic decision task, possibly because early bilinguals are better able to process early visual cues from ASL signs and use these to constrain English word recognition. Comprehension facilitation via semantic integration of words and signs is consistent with co-speech gesture research demonstrating facilitative effects of gesture integration on language comprehension.

The acquisition of prosody in American Sign Language

This is the first comparative analysis of prosody in deaf, native-signing children (ages 5;0–8;5) and adults whose first language is American Sign Language (ASL). The goals of this study are to describe the distribution of prosodic cues during acquisition, to determine which cues across age groups are most predictive in determining clausal and prosodic boundaries, and to ascertain how much isomorphy there is in ASL between syntactic and prosodic units. The results show that all cues are acquired compositionally, and that the prosodic patterns in child and adult ASL signers exhibit important differences regarding specific cues; however, in all groups the manual cues are more predictive of prosodic boundaries than nonmanual markers. This is evidence for a division of labor between the cues that are produced to mark constituents and those that contribute to semantic and pragmatic meaning. There is also more isomorphy in adults than in children, so these results add to the debates about isomorphy, suggesting that while there is clear autonomy between prosody and syntax, productions exhibiting nonisomorphy are relatively rare overall.

The contribution of phonological knowledge, memory, and language background to reading comprehension in deaf populations.

While reading is challenging for many deaf individuals, some become proficient readers. Little is known about the component processes that support reading comprehension in these individuals. Speech-based phonological knowledge is one of the strongest predictors of reading comprehension in hearing individuals, yet its role in deaf readers is controversial. This could reflect the highly varied language backgrounds among deaf readers as well as the difficulty of disentangling the relative contribution of phonological versus orthographic knowledge of spoken language, in our case ‘English,’ in this population. Here we assessed the impact of language experience on reading comprehension in deaf readers by recruiting oral deaf individuals, who use spoken English as their primary mode of communication, and deaf native signers of American Sign Language. First, to address the contribution of spoken English phonological knowledge in deaf readers, we present novel tasks that evaluate phonological versus orthographic knowledge. Second, the impact of this knowledge, as well as memory measures that rely differentially on phonological (serial recall) and semantic (free recall) processing, on reading comprehension was evaluated. The best predictor of reading comprehension differed as a function of language experience, with free recall being a better predictor in deaf native signers than in oral deaf. In contrast, the measures of English phonological knowledge, independent of orthographic knowledge, best predicted reading comprehension in oral deaf individuals. These results suggest successful reading strategies differ across deaf readers as a function of their language experience, and highlight a possible alternative route to literacy in deaf native signers.Highlights:1. Deaf individuals vary in their orthographic and phonological knowledge of English as a function of their language experience.2. Reading comprehension was best predicted by different factors in oral deaf and deaf native signers.3. Free recall memory (primacy effect) better predicted reading comprehension in deaf native signers as compared to oral deaf or hearing individuals.4. Language experience should be taken into account when considering cognitive processes that mediate reading in deaf individuals.

Signs are symbols: evidence from the Stroop task

Most languages use spoken arbitrary symbols to access the conceptual system. Moreover, the link from spoken words to meaning is demonstrably automatic. Sign languages, by contrast, employ many iconic manual gestures. While some signs are arbitrary it is unclear whether such arbitrary signs automatically activate the conceptual system. To address this question, we examine the propensity of arbitrary colour signs in American Sign Language (ASL) to induce Stroop interference. Three experiments elicited colour naming of coloured videos depicting colour ASL signs – either congruent or incongruent with video colour – and an unrelated condition. Results showed that colour identification is modulated by its congruency with the ASL sign, and this finding replicated irrespective of response mode – signing vs. button-press – and the presence of congruent trials. These findings indicate that arbitrary signs automatically activate their meanings. We conclude that the capacity to link arbitrary phonological forms and meanings is an amodal design feature of language.

Real-time processing of ASL signs: Delayed first language acquisition affects organization of the mental lexicon.

Sign language comprehension requires visual attention to the linguistic signal and visual attention to referents in the surrounding world, whereas these processes are divided between the auditory and visual modalities for spoken language comprehension. Additionally, the age-onset of first language acquisition and the quality and quantity of linguistic input for deaf individuals is highly heterogeneous, which is rarely the case for hearing learners of spoken languages. Little is known about how these modality and developmental factors affect real-time lexical processing. In this study, we ask how these factors impact real-time recognition of American Sign Language (ASL) signs using a novel adaptation of the visual world paradigm in deaf adults who learned sign from birth (Experiment 1), and in deaf adults who were late-learners of ASL (Experiment 2). Results revealed that although both groups of signers demonstrated rapid, incremental processing of ASL signs, only native signers demonstrated early and robust activation of sublexical features of signs during real-time recognition. Our findings suggest that the organization of the mental lexicon into units of both form and meaning is a product of infant language learning and not the sensory and motor modality through which the linguistic signal is sent and received.

Language co-activation and lexical selection in bimodal bilinguals: Evidence from picture-word interference.

We used picture-word interference (PWI) to discover a) whether cross-language activation at the lexical level can yield phonological priming effects when languages do not share phonological representations, and b) whether semantic interference effects occur without articulatory competition. Bimodal bilinguals fluent in American Sign Language (ASL) and English named pictures in ASL while listening to distractor words that were 1) translation equivalents, 2) phonologically related to the target sign through translation, 3) semantically related, or 4) unrelated. Monolingual speakers named pictures in English. Production of ASL signs was facilitated by words that were phonologically related through translation and by translation equivalents, indicating that cross-language activation spreads from lexical to phonological levels for production. Semantic interference effects were not observed for bimodal bilinguals, providing some support for a post-lexical locus of semantic interference, but which we suggest may instead reflect time course differences in spoken and signed production in the PWI task.

Adaptation of a vocabulary test from British Sign Language to American Sign Language

This study describes the adaptation process of a vocabulary knowledge test for British Sign Language (BSL) into American Sign Language (ASL) and presents results from the first round of pilot testing with 20 deaf native ASL signers. The web-based test assesses the strength of deaf children’s vocabulary knowledge by means of different mappings of phonological form and meaning of signs. The adaptation from BSL to ASL involved nine stages, which included forming a panel of deaf/hearing experts, developing a set of new items and revising/replacing items considered ineffective, and piloting the new version. Results provide new evidence in support of the use of this methodology for assessing signed languages, making a useful contribution toward the availability of tests to assess deaf children’s signed language skills.

Lending a hand to signed language acquisition: Enactment and iconicity enhance sign recall in hearing adult American Sign Language learners

For hearing adults, signed language processing increases the salience of iconicity and motor system involvement relative to spoken language processing. Nevertheless, it is unclear how embodied action, mental imagery and iconicity influence their acquisition of signed language. The current study examines the impact of these factors on sign acquisition by manipulating how signs are learned, as well as their semantic and phonological relatedness. The results of Experiment 1 demonstrated that American Sign Language (ASL) signs are learned more effectively via enactment than via referent visualisation and meaningless hand motion, and that iconic signs are learned more effectively than other types of signs. The results of Experiment 2 demonstrate that, when learned via enactment, semantically related ASL signs are recalled more accurately than phonologically related ASL signs. These results indicate that hearing adults' sign language acquisition can be enhanced via a learning method that combines mental imagery and meaningful embodied action (i.e., enactment), strengthening connections between the forms of signs and their referents.

Watching language grow in the manual modality: Nominals, predicates, and handshapes

All languages, both spoken and signed, make a formal distinction between two types of terms in a proposition – terms that identify what is to be talked about (nominals) and terms that say something about this topic (predicates). Here we explore conditions that could lead to this property by charting its development in a newly emerging language – Nicaraguan Sign Language (NSL). We examine how handshape is used in nominals vs. predicates in three Nicaraguan groups: (1) homesigners who are not part of the Deaf community and use their own gestures, called homesigns, to communicate; (2) NSL cohort 1 signers who fashioned the first stage of NSL; (3) NSL cohort 2 signers who learned NSL from cohort 1. We compare these three groups to a fourth: (4) native signers of American Sign Language (ASL), an established sign language. We focus on handshape in predicates that are part of a productive classifier system in ASL; handshape in these predicates varies systematically across agent vs. no-agent contexts, unlike handshape in the nominals we study, which does not vary across these contexts. We found that all four groups, including homesigners, used handshape differently in nominals vs. predicates – they displayed variability in handshape form across agent vs. no-agent contexts in predicates, but not in nominals. Variability thus differed in predicates and nominals: (1) In predicates, the variability across grammatical contexts (agent vs. no-agent) was systematic in all four groups, suggesting that handshape functioned as a productive morphological marker on predicate signs, even in homesign. This grammatical use of handshape can thus appear in the earliest stages of an emerging language. (2) In nominals, there was no variability across grammatical contexts (agent vs. no-agent), but there was variability within- and across-individuals in the handshape used in the nominal for a particular object. This variability was striking in homesigners (an individual homesigner did not necessarily use the same handshape in every nominal he produced for a particular object), but decreased in the first cohort of NSL and remained relatively constant in the second cohort. Stability in the lexical use of handshape in nominals thus does not seem to emerge unless there is pressure from a peer linguistic community. Taken together, our findings argue that a community of users is essential to arrive at a stable nominal lexicon, but not to establish a productive morphological marker in predicates. Examining the steps a manual communication system takes as it moves toward becoming a fully-fledged language offers a unique window onto factors that have made human language what it is.

Cognitive, cultural, and linguistic sources of a handshape distinction expressing agentivity.

In this paper the cognitive, cultural, and linguistic bases for a pattern of conventionalization of two types of iconic handshapes are described. Work on sign languages has shown that handling handshapes (H-HSs: those that represent how objects are handled or manipulated) and object handshapes (O-HSs: those that represent the class, size, or shape of objects) express an agentive/non-agentive semantic distinction in many sign languages. H-HSs are used in agentive event descriptions and O-HSs are used in non-agentive event descriptions. In this work, American Sign Language (ASL) and Italian Sign Language (LIS) productions are compared (adults and children) as well as the corresponding groups of gesturers in each country using "silent gesture." While the gesture groups, in general, did not employ an H-HS/O-HS distinction, all participants (signers and gesturers) used iconic handshapes (H-HSs and O-HSs together) more often in agentive than in no-agent event descriptions; moreover, none of the subjects produced an opposite pattern than the expected one (i.e., H-HSs associated with no-agent descriptions and O-HSs associated with agentive ones). These effects are argued to be grounded in cognition. In addition, some individual gesturers were observed to produce the H-HS/O-HS opposition for agentive and non-agentive event descriptions-that is, more Italian than American adult gesturers. This effect is argued to be grounded in culture. Finally, the agentive/non-agentive handshape opposition is confirmed for signers of ASL and LIS, but previously unreported cross-linguistic differences were also found across both adult and child sign groups. It is, therefore, concluded that cognitive, cultural, and linguistic factors contribute to the conventionalization of this distinction of handshape type.

Tools for language: patterned iconicity in sign language nouns and verbs.

When naming certain hand-held, man-made tools, American Sign Language (ASL) signers exhibit either of two iconic strategies: a handling strategy, where the hands show holding or grasping an imagined object in action, or an instrument strategy, where the hands represent the shape or a dimension of the object in a typical action. The same strategies are also observed in the gestures of hearing nonsigners identifying pictures of the same set of tools. In this paper, we compare spontaneously created gestures from hearing nonsigning participants to commonly used lexical signs in ASL. Signers and gesturers were asked to respond to pictures of tools and to video vignettes of actions involving the same tools. Nonsigning gesturers overwhelmingly prefer the handling strategy for both the Picture and Video conditions. Nevertheless, they use more instrument forms when identifying tools in pictures, and more handling forms when identifying actions with tools. We found that ASL signers generally favor the instrument strategy when naming tools, but when describing tools being used by an actor, they are significantly more likely to use more handling forms. The finding that both gesturers and signers are more likely to alternate strategies when the stimuli are pictures or video suggests a common cognitive basis for differentiating objects from actions. Furthermore, the presence of a systematic handling/instrument iconic pattern in a sign language demonstrates that a conventionalized sign language exploits the distinction for grammatical purpose, to distinguish nouns and verbs related to tool use.

Temporal entrainment of visual attention in children: effects of age and deafness.

The major documented effect of auditory deprivation on visual processing is enhanced spatial attention, in particular to the visual periphery and to moving stimuli. However, there is a parallel literature that has reported deficits in temporal aspects of visual processing in individuals with profound hearing losses. This study builds upon previous work showing possible deficits in processing of rapid serial visual presentation streams in deaf children [Restorative Neurology and Neuroscience (2010), 28, 181-192]. Deaf native signers of American Sign Language and hearing children and adults were asked to perform a 2-AFC identification task with a visual target embedded in a stream of visual stimuli presented at 6 Hz. Both children and adults displayed attentional awakening, whereby target identification accuracy improved as the number of stimuli preceding the target increased. For deaf children, however, this awakening effect was less pronounced than that observed in hearing children, interpreted as difficulty sustaining entrainment to the stimulus stream. The data provide the first account of attentional awakening in children, showing that it improves across the 6-13 year age range. They also provide additional support to the possibility of domain-general alterations in the processing of temporal information in the absence of auditory input.

A Sign Language Recognition Approach for Human-Robot Symbiosis

This paper introduces a new concept for the establishment of human-robot symbiotic relationship. The system is based on the implementation of knowledge-based image processing methodologies for model based vision and intelligent task scheduling for an autonomous social robot. This paper aims to develop an automatic translation of static gestures of alphabets and signs in American Sign Language (ASL), using neural network with backpropagation algorithm. System deals with images of bare hands to achieve the recognition task. For each individual sign 10 sample images have been considered, which means in total300 samples have been processed. In order to compare between the training set of signs and the considered sample images, are converted into feature vectors. Experimental results reveal that this can recognize selected ASL signs (accuracy of 92.00%). Finally, the system has been implemented issuing hand gesture commands for ASL to a robot car, named “Moto-robo”.

Words in the bilingual brain: an fNIRS brain imaging investigation of lexical processing in sign-speech bimodal bilinguals.

Early bilingual exposure, especially exposure to two languages in different modalities such as speech and sign, can profoundly affect an individual's language, culture, and cognition. Here we explore the hypothesis that bimodal dual language exposure can also affect the brain's organization for language. These changes occur across brain regions universally important for language and parietal regions especially critical for sign language (Newman et al., 2002). We investigated three groups of participants (N = 29) that completed a word repetition task in American Sign Language (ASL) during fNIRS brain imaging. Those groups were (1) hearing ASL-English bimodal bilinguals (n = 5), (2) deaf ASL signers (n = 7), and (3) English monolinguals naïve to sign language (n = 17). The key finding of the present study is that bimodal bilinguals showed reduced activation in left parietal regions relative to deaf ASL signers when asked to use only ASL. In contrast, this group of bimodal signers showed greater activation in left temporo-parietal regions relative to English monolinguals when asked to switch between their two languages (Kovelman et al., 2009). Converging evidence now suggest that bimodal bilingual experience changes the brain bases of language, including the left temporo-parietal regions known to be critical for sign language processing (Emmorey et al., 2007). The results provide insight into the resilience and constraints of neural plasticity for language and bilingualism.

Scalar implicatures in a signed language

This paper tests the calculation of scalar implicatures in American Sign Language (ASL) in one of the first experimental pragmatic studies in the manual/visual modality. Both native signers of ASL and native speakers of English participated in an automated Felicity Judgment Task to compare implicatures based on two traditional scales as well as “ad hoc” scales in their respective languages. Results show that native signers of ASL calculate scalar implicatures based on a prototypical scale <all, some> in ASL in the same pattern as native speakers of English, within the same experimental paradigm. There are similarly high rates of exact interpretations of numbers <three, two> in ASL as in English, despite the iconicity of the numerals in ASL. Finally, an ad hoc scale was tested showing fewer implicatures in English than on the conventionalized scales. In ASL, there was a trend toward increased implicatures on the ad hoc scale which made use of the unique ability of ASL to convey spatial information using the classifier system. Taken together, these results show that conventionalized scales in ASL have the same semantic/pragmatic scalar properties as in spoken languages, although in non-conventionalized scales the inclusion of additional information such as spatial location may affect pragmatic interpretation.

On the linguistic effects of articulatory ease, with a focus on sign languages

Abstract: Spoken language has a well-known drive for ease of articulation, which Kirchner (1998, 2004) analyzes as reduction of the total magnitude of all biomechanical forces involved. We extend Kirchner’s insights from vocal articulation to manual articulation, with a focus on joint usage, and we discuss ways that articulatory ease might be realized in sign languages. In particular, moving more joints and/or joints more proximal to the torso results in greater mass being moved, and thus more articulatory force being expended, than moving fewer joints or moving more distal joints. We predict that in casual conversation, where articulatory ease is prized, moving fewer joints should be favored over moving more, and moving distal joints should be favored over moving proximal joints. We report on the results of our study of the casual signing of fluent signers of American Sign Language, which confirm our predictions: in comparison to citation forms of signs, the casual variants produced by the signers in our experiment exhibit an overall decrease in average joint usage, as well as a general preference for more distal articulation than is used in citation form. We conclude that all language, regardless of modality, is shaped by a fundamental drive for ease of articulation. Our work advances a cross-modality approach for considering ease of articulation, develops a potentially important vocabulary for describing variations in signs, and demonstrates that American Sign Language exhibits variation that can be accounted for in terms of ease of articulation. We further suggest that the linguistic drive for ease of articulation is part of a broader tendency for the human body to reduce biomechanical effort in all physical activities.