Finger-spelled Words Research Articles

Sign Language Recognition Using ResNet50 Deep Neural Network Architecture

Communication is a barrier between the deaf-mute community and the rest of the society. Sign Language is used for communication among such people who cannot speak and listen. The automation of sign language recognition has gained researchers attention in the last few years. Many complex and costly hardware systems have already been developed to assist the purpose. However, we propose to use deep learning approach for automated sign language recognition. We devised a novel 2-level ResNet50 based Deep Neural Network Architecture to classify fingerspelled words. The dataset used is the standard American Sign Language Hand gesture dataset by [1]. The dataset was first augmented using various augmentation techniques. In our 2-level ResNet50 based approach the Level 1 model classifies the input image into one of the 4 sets. After an image is classified into one of the sets it is provided as an input to the corresponding second level model for predicting the actual class of the image. Our approach yields an accuracy of 99.03% on 12,048 test images.

Fingerspelled and Printed Words Are Recoded into a Speech-based Code in Short-term Memory.

We conducted three immediate serial recall experiments that manipulated type of stimulus presentation (printed or fingerspelled words) and word similarity (speech-based or manual). Matched deaf American Sign Languagesigners and hearing non-signers participated (mean reading age=14-15 years). Speech-based similarity effects were found for both stimulus types indicating that deaf signers recoded both printed and fingerspelled words into a speech-based phonological code. A manual similarity effect was not observed for printed words indicating that print was not recoded into fingerspelling (FS). A manual similarity effect was observed for fingerspelled words when similarity was based on joint angles rather than on handshape compactness. However, a follow-up experiment suggested that the manual similarity effect was due to perceptual confusion at encoding. Overall, these findings suggest that FS is strongly linked to English phonology for deaf adult signers who are relatively skilled readers. This link between fingerspelled words and English phonology allows for the use of a more efficient speech-based code for retaining fingerspelled words in short-term memory and may strengthen the representation of English vocabulary.

Processing Orthographic Structure: Associations Between Print and Fingerspelling

Two lexical decision experiments are reported that investigate whether the same segmentation strategies are used for reading printed English words and fingerspelled words (in American Sign Language). Experiment 1 revealed that both deaf and hearing readers performed better when written words were segmented with respect to an orthographically defined syllable (the Basic Orthographic Syllable Structure [BOSS]) than with a phonologically defined syllable. Correlation analyses revealed that better deaf readers were more sensitive to orthographic syllable representations, whereas segmentation strategy did not differentiate the better hearing readers. In contrast to Experiment 1, Experiment 2 revealed better performance by deaf participants when fingerspelled words were segmented at the phonological syllable boundary. We suggest that English mouthings that often accompany fingerspelled words promote a phonological parsing preference for fingerspelled words. In addition, fingerspelling ability was significantly correlated with reading comprehension and vocabulary skills. This pattern of results indicates that the association between fingerspelling and print for adult deaf readers is not based on shared segmentation strategies. Rather, we suggest that both good readers and good fingerspellers have established strong representations of English and that fingerspelling may aid in the development and maintenance of English vocabulary.