American Sign Research Articles

Subcortical language localization using sign language and awake craniotomy for dominant posterior temporal glioma resection in a hearing-impaired patient

Intraoperative direct cortical stimulation (DCS) is the gold standard technique to maximize the extent of resection of tumors located in eloquent areas. To date, there are three cases reported of awake mapping for language centers in deaf patients who could communicate only with sign language. We present the case of DCS in a deaf patient who could communicate vocally, native to American Sign Language and English, that underwent intraoperative awake mapping. DCS showed similar disruption of expressive phonology to both pictorial and gestural stimuli, confirming that sign language follows the same pattern as oral language.

Assistive Data Glove for Isolated Static Postures Recognition in American Sign Language Using Neural Network

Sign language recognition is one of the most challenging tasks of today’s era. Most of the researchers working in this domain have focused on different types of implementations for sign recognition. These implementations require the development of smart prototypes for capturing and classifying sign gestures. Keeping in mind the aspects of prototype design, sensor-based, vision-based, and hybrid approach-based prototypes have been designed. The authors in this paper have designed sensor-based assistive gloves to capture signs for the alphabet and digits. These signs are a small but important fraction of the ASL dictionary since they play an essential role in fingerspelling, which is a universal signed linguistic strategy for expressing personal names, technical terms, gaps in the lexicon, and emphasis. A scaled conjugate gradient-based back propagation algorithm is used to train a fully-connected neural network on a self-collected dataset of isolated static postures of digits, alphabetic, and alphanumeric characters. The authors also analyzed the impact of activation functions on the performance of neural networks. Successful implementation of the recognition network produced promising results for this small dataset of static gestures of digits, alphabetic, and alphanumeric characters.

An Approach to Convert Sign to Text for Impaired People

Human Computer Interface is the study of how humans and computers interact. Hand gestures are a great way to communicate with people when they don’t understand exactly what we are saying. Understanding hand gestures is essential to make sure the listener understands what we are saying. The main idea of our project is to try different approaches to hand gesture recognition. This proposed work first with radar data and then with camera sensor to achieve hand gesture recognition. First, we tried to build hand gesture recognition using radar data, and since most people don’t know sign language and few interpreters, we developed an approach to real-time approach for American Sign Language based on neural networks finger spelling followed by another model with Media Pipe. We propose a complex neural network method to detect hand gestures of human behaviour from camera recorded images. The hand gesture first goes through the filter and after applying the filter the gesture goes through a classifier that predicts which type of hand gesture it is. In an existing system radar unable to detect static gestures in our approach, a deep learning-based image captioning algorithm captures both static and dynamic gestures through Media Pipe.

Parent American Sign Language skills correlate with child–but not toddler–ASL vocabulary size

ABSTRACT Most deaf children have hearing parents who do not know a sign language at birth and are at risk of limited language input during early childhood. Studying these children as they learn a sign language has revealed that timing of first-language exposure critically shapes language outcomes. But the input deaf children receive in their first language is not only delayed, it is much more variable than most first language learners, as many learn their first language from parents who are themselves new sign language learners. Much of the research on deaf children learning a sign language has considered the role of parent input using broad strokes, categorizing hearing parents as non-native, poor signers, and deaf parents as native, strong signers. In this study, we deconstruct these categories and examine how variation in sign language skills among hearing parents might affect children’s vocabulary acquisition. This study included 44 deaf children between 8- and 60-months-old who were learning American Sign Language (ASL) and had hearing parents who were also learning ASL. We observed an interactive effect of parent ASL proficiency and age, such that parent ASL proficiency was a significant predictor of child ASL vocabulary size, but not among the infants and toddlers. The proficiency of language models can affect acquisition above and beyond age of acquisition, particularly as children grow. At the same time, the most skilled parents in this sample were not as fluent as “native” deaf signers, and yet their children reliably had age-expected ASL vocabularies. Data and reproducible analyses are available at https://osf.io/9ya6h/.

A Survey on Sign Language Recognition with Video Chat

The use of sign language is a crucial tool for improving communi- cation between hearing-impaired people and the general public. SLR (Sign Language Recognition) systems in the past have been sophisticated and challenging to train. However, in this research, we provide a novel method that makes use of SSD MobileNet V2 FPNLite 320x320 pre-trained models and object recognition based on TensorFlow's object detection. By enabling the identification and detection of a set of images, this method streamlines the train- ing process. The suggested system will be trained and evaluated using 10 to 15 different American Sign Language symbols. A fun- damental social skill used to exchange information is communica- tion. It is frequently used to express oneself and to fulfill funda- mental human needs including the desire for protection, safety, and connection. Several stages, diverse methods, and distinct conse- quences are used in this procedure. It typically refers to a two-way exchange of information in the local vicinity of touch. Information flows far more easily when people are speaking the same language than when they are speaking languages that are from distinct lan- guage families. In order to facilitate video chat communication be- tween signers and non-signers, our proposed sign language recog- nition system is specifically created. Each peer can see and hear the other during a video conversation thanks to their individual cameras and microphones. Nevertheless, using our method, a spe- cific peer can also view the indicators that the other peer on the other end of the video chat exhibits or copies. Our system employs object detection to recognize and track the signer's hand motions in real-time in order to accomplish this. The technology then over- lays a graphic of the detected sign onto the non-video signer's feed. This overlay is positioned so that it does not obstruct the view of the signer by the non-signer and is not obtrusive. By using contem- porary technology, our suggested solution is made to enable distant and two-way communication between signers and non-signers. With the help of our system, video chat communication is im- proved, allowing signers and non-signers to communicate and in- teract more effectively. Index Terms— SLR(Sign Language Recognition), Video Chat, Object Detection, SSD MobileNet.

Healthcare communication access among deaf and hard-of-hearing people during pregnancy

ObjectiveDeaf and hard-of-hearing (DHH) people report ineffective healthcare communication. Limited research has been conducted on healthcare communication during pregnancy. This study’s aim was to assess communication accommodations and experiences during pregnancy for DHH people. MethodsAn accessible web-based survey was administered to a non-probability sample of DHH people through national organizations, social media, and snowball sampling. Eligibility criteria included being 21 years or older; given birth in the U.S. within the past 10 years, report hearing loss prior to the most recent birth. Questions focused on healthcare experiences and information access during their last pregnancy. The sample included 583 respondents for the present analysis. We describe the communication accommodations requested and received during pregnancy, segmented by preferred language. ResultsMost DHH participants reported communication with prenatal clinicians as “good” or “very good”. On-site interpreter services were most commonly requested by American Sign Language (ASL) only and bilingual DHH people. Interpersonal communication modification requests (e.g., speaking louder) were rarely obliged. ConclusionThis study is the first national examination of requested and received communication accommodations for DHH patients during the perinatal period. Practice implicationsHealthcare providers should work closely with patients to ensure effective communication access is provided.

Sign Language Based Video Calling App

Deaf and hard of hearing people communicate with others and within their own communities by using sign language. Computer recognition of sign language begins with the learning of sign gestures and continues through the production of text and speech. There are two types of sign gestures: static and dynamic. Both gesture recognition systems are crucial to the human community, even if static gesture recognition is easier than dynamic gesture recognition. We have conducted research on the steps required to convert static American sign language (ASL) to readable text and selected the best available methods to do so. Examined general steps are the data collection, pre-processing, transformation, feature extraction, and classification. There are also some recommendations for further study in this field.

Whole-food plant-based Jumpstart for a Deaf and Hard of Hearing cohort.

Deaf and Hard of Hearing (DHH) patients are at high risk of developing chronic illness, and when they do, are at higher risk of poor outcomes than in a hearing community. Rochester Lifestyle Medicine Institute adapted its online, Zoom-based, medically-facilitated 15 Day Whole-Food Plant-Based (WFPB) Jumpstart program, to give DHH participants knowledge, skills, and support to make dietary changes to improve their health. Adaptations included having a medical provider present who is fluent in American Sign Language (ASL), is board-certified in Lifestyle Medicine, and has a Master of Science in Deaf Education; spotlighting participants when asking a question during the Q&A session; using ASL interpreters; utilizing closed captioning/automatic transcription during all Zoom meetings; and employing a Success Specialist to provide outreach via text and email throughout the program. Participants had significant positive changes in their eating pattern. They reported improvements in biometric measures as well as in how they were feeling. They all reported that they planned to continue to eat a more WFPB diet than they did prior to Jumpstart. All either agreed or strongly agreed that they learned important information, were confident that they knew the best eating pattern for health, and gained the skills they needed to make changes. Although this was a small pilot program, it suggests that this model can be used to provide education and support for behavior change that will lead to improved health in a DHH community.

Considerations in the Management of Functional Neurological Disorders in Patients with Hearing Loss.

Adam is a 14-year-old adolescent boy with hearing loss who presented to the pediatric neurology clinic accompanied by his father for evaluation of new-onset left hand tremor for a duration of 1 month. An American Sign Language interpreter was present and used throughout the visit.Adam has bilateral sensorineural hearing loss related to premature birth at 28 weeks' gestation. He uses sign language and attends a school for the hearing impaired. He has been diagnosed with attention-deficit/hyperactivity disorder (ADHD) and a nonspecific learning disorder. His ADHD symptoms are well controlled with a stimulant medication. He is independent in activities of daily living, and there is no concern for intellectual disability. His father is concerned that Adam may have anxiety, but this has not been evaluated.After careful history, it is found that the tremor was first noted the day after burglars broke into his home and stole precious belongings. Current stressors include difficulties with schoolwork and a strained relationship with an extended family member. There is no family history of tremor.The tremor was intermittent initially, with episodes lasting around 30 minutes. Over time, the tremor became more persistent. Adam is left-handed, and the tremor is now interfering with handwriting, eating, and other fine motor skills. The tremor worsens when Adam is tired or stressed and improves with relaxation. No tremor has been noted in other body parts. Adam denies any other neurological symptoms, including headache, vision changes, or gait abnormalities.On examination, Adam seemed anxious but showed no significant distress and had normal vital signs. His general examination was unremarkable. His neurological examination showed intact cranial nerves, apart from the hearing impairment. He had normal muscle tone, intact strength and coordination, and a normal casual gait. Rhythmic shaking of the left upper extremity was present with action, while maintaining posture. Using specific examination techniques, the examiner was able to alter the rhythm of the tremor, and the tremor was noted to subside when the patient was engaged with the examiner. These findings in addition to signs of suggestibility and variable frequency/direction were consistent with a functional etiology.The diagnosis of a functional neurological disorder manifesting in the form of functional tremor was discussed with the patient and his father with assistance from the sign language interpreter. Counseling regarding management consisting mainly of cognitive behavioral therapy and evaluation of possible coexisting conditions, such as anxiety, was discussed.What factors would you consider in diagnosis and management of functional neurological disorder in a hearing-impaired child/adolescent?

Developing Coding Instruction Videos for K12 Hearing Impaired Students Using American Sign Language

Disability Inclusion and Accessibility in technology has today emerged as a necessity of national importance. NSF funded projects like AccessCSforALL, Bootstrap, and CSforAll are making efforts to make computer science (CS) inclusive to the 7.4 million K-12 students with disabilities. Bringing people with disabilities into the tech workforce would not only create equal opportunities for all but would also better equip the society to serve its diverse population. Our proposed project focuses on making CS concepts accessible to K-12 students who are deaf or hard-of-hearing (D/HH).

Development of a PC-based sign language translator

While a hearing-impaired individual depends on sign language and gestures, non-hearing-impaired person uses verbal language. Thus, there is need for means of arbitration to forestall situation when a non-hearing-impaired individual who does not understand the sign language wants to communicate with a hearing-impaired person. This paper is concerned with the development of a PC-based sign language translator to facilitate effective communication between hearing-impaired and non-hearing-impaired persons. Database of hand gestures in American sign language (ASL) is created using Python scripts. TensorFlow (TF) is used in the creation of a pipeline configuration model for machine learning of annotated images of gestures in the database with the real time gestures. The implementation is done in Python software environment and it runs on a PC equipped with a web camera to capture real time gestures for comparison and interpretations. The developed sign language translator is able to translate ASL/gestures to written texts along with corresponding audio renderings at an average duration of about one second. In addition, the translator is able to match real time gestures with the equivalent gesture images stored in the database even at 44% similarity.

Sign Language Recognition Based on Hands Symbol's Classification

One of the most natural and ancient types of conversational language is sign language. The technology that converts sign language into writing for people who have difficulty communicating, such as those who have speech issues, hearing disabilities, or are deaf, is the subject of this study. This paper is based on a real-time method based on finger writing and neural networks for American sign language. An interesting field of vision study is the automatic recognition of human gestures from video images. The Research recommend employing a convolution neural network (CNN) method to recognize human hand motions from a photograph. The objective is to identify hand movements used in human work activities from a camera image. Hand placement and orientation are employed to collect the training and assessment data for CNN. The hand is first put through a filter, and once that has been done, it is put through a classification, which determines what class the hand movements belong to. Then, CNN is trained using the measured pictures.

Real-time recognition of American sign language using long-short term memory neural network and hand detection

Sign language recognition is very important for deaf and mute people because it has many facilities for them, it converts hand gestures into text or speech. It also helps deaf and mute people to communicate and express mutual feelings. This paper's goal is to estimate sign language using action detection by predicting what action is being demonstrated at any given time without forcing the user to wear any external devices. We captured user signs with a webcam. For example; if we signed “thank you”, it will take the entire set of frames for that action to determine what sign is being demonstrated. The long short-term memory (LSTM) model is used to produce a real-time sign language detection and prediction flow. We also applied dropout layers for both training and testing dataset to handle overfitting in deep learning models which made a good improvement for the final result accuracy. We achieved a 99.35% accuracy after training and implementing the model which allows the deaf and mute communicate more easily with society.

The Boundaries of Human Nature: The Philosophical Animal from Plato to Haraway

The Boundaries of Human Nature: The Philosophical Animal from Plato to Haraway

Learning a Sign Language Does Not Hinder Acquisition of a Spoken Language.

The purpose of this study is to determine whether and how learning American Sign Language (ASL) is associated with spoken English skills in a sample of ASL-English bilingual deaf and hard of hearing (DHH) children. This cross-sectional study of vocabulary size included 56 DHH children between 8 and 60 months of age who were learning both ASL and spoken English and had hearing parents. English and ASL vocabulary were independently assessed via parent report checklists. ASL vocabulary size positively correlated with spoken English vocabulary size. Spoken English vocabulary sizes in the ASL-English bilingual DHH children in the present sample were comparable to those in previous reports of monolingual DHH children who were learning only English. ASL-English bilingual DHH children had total vocabularies (combining ASL and English) that were equivalent to same-age hearing monolingual children. Children with large ASL vocabularies were more likely to have spoken English vocabularies in the average range based on norms for hearing monolingual children. Contrary to predictions often cited in the literature, acquisition of sign language does not harm spoken vocabulary acquisition. This retrospective, correlational study cannot determine whether there is a causal relationship between sign language and spoken language vocabulary acquisition, but if a causal relationship exists, the evidence here suggests that the effect would be positive. Bilingual DHH children have age-expected vocabularies when considering the entirety of their language skills. We found no evidence to support recommendations that families with DHH children avoid learning sign language. Rather, our findings show that children with early ASL exposure can develop age-appropriate vocabulary skills in both ASL and spoken English.

RF-CM

Radio-Frequency (RF) based human activity recognition (HAR) enables many attractive applications such as smart home, health monitoring, and virtual reality (VR). Among multiple RF sensors, mmWave radar is emerging as a new trend due to its fine-grained sensing capability. However, laborious data collection and labeling processes are required when employing a radar-based sensing system in a new environment. To this end, we propose RF-CM, a general cross-modal human activity recognition framework. The key enabler is to leverage the knowledge learned from a massive WiFi dataset to build a radar-based HAR system with limited radar samples. It can significantly reduce the overhead of training data collection. In addition, RF-CM can work well regardless of the deployment setups of WiFi and mmWave radar, such as performing environments, users' characteristics, and device deployment. RF-CM achieves this by first capturing the activity-related variation patterns through data processing schemes. It then employs a convolution neural network-based feature extraction module to extract the high-dimensional features to be fed into the activity recognition module. Finally, RF-CM takes the generalization knowledge from WiFi networks as guide labels to supervise the training of the radar model, thus enabling a few-shot radar-based HAR system. We evaluate RF-CM by applying it to two HAR applications, fine-grained American sign language recognition (WiFi-cross-radar) and coarse-grained gesture recognition (WiFi-cross-RFID). The accuracy improvement of over 10% in both applications demonstrates the effectiveness of RF-CM. This cross-modal ability allows RF-CM to support more cross-modal applications.

How Well are Young Deaf Children in Early Intervention Doing on their Language Acquisition? An Assessment View

Little is known about the earliest language acquisition of deaf children who have a variety of different experiences. Cochlear implants tend to be implanted between six months to 12 months and often become active around 18 months. In addition, Speech Language Pathologists (SLPs) tend to evaluate articulation rather than language acquisition. In terms of evaluating sign language, there are few normed assessments developed explicitly for this modality. Therefore, a joint research project was established with an early intervention program and Lamar University's Deaf Studies and Deaf Education department to evaluate the earliest language milestones acquired by deaf children between birth and five years of age. The project used language assessments developed for parents and teachers in both American Sign Language (ASL) and English, with norms derived from native users of each language. All languages reported as used by the family were assessed. Results found that most families selected spoken language and those results varied across participants. It was found that younger children using spoken language were more likely to demonstrate age-typical language acquisition when compared to older children. As more linguistic complexity developed, older children using spoken language tended to show delays. Future research suggests investigating bidirectional interactions between children, their families, and the broader systems with which the deaf child interacts. Doing so would create a further understanding of the child’s language acquisition and help families understand how to determine if their child’s language acquisition is progressing in a typical manner.

The landscape of American Sign Language education

AbstractAlthough American Sign Language (ASL) education for second language (L2) or additional language (Ln) learners in postsecondary education has increased rapidly over the past four decades, not much is understood about the field. The scholarship of ASL education is relatively new, and little is known of the perspectives of ASL professionals in ASL education. This qualitative study included focus group discussions with 13 ASL professionals who have worked in the field of ASL education to build a better understanding of ASL education as a discipline in regard to its strengths, challenges, and directions. The results from the four focus group discussions show three emergent themes: ASL professionals, the structure of postsecondary education, and the purpose of ASL education. The challenges in developing and sustaining ASL education as a discipline are discussed, and recommendations on how ASL education as a discipline can be elevated and sustained are provided.

Intelligent hybrid hand gesture recognition system using deep recurrent neural network with chaos game optimisation

ABSTRACT Hand gesture recognition is considered an essential task in various human-computer interaction (HCI) applications. Therefore, developing a robust system for hand gesture recognition is a challenge. This work proposed a new hybrid approach named hybrid deep recurrent neural network (RNN) incorporated with a chaos game optimisation (CGO) algorithm for efficiently recognising hand gestures. The main objective of a hybrid recurrent neural network with chaos game optimisation (RNN-CGO) is to achieve the recognition of alphabet signs from 2D gesture images. The different stages included in this technique are pre-processing, feature extraction, feature selection, and classification. The proposed approach is executed using American Sign Language (ASL) dataset with the help of the PYTHON platform. The precision, accuracy, f1-score, and recall obtained for the proposed approach are 99.28%, 99.96%, 99.25%, and 99.28%, respectively. The inference time required for the proposed approach is 0.121 s. The proposed technique has less computational complexity. The results showed that the proposed approach is better than the existing hand gesture recognition model approaches.

Toward the Development of Evidence-Based Practices in American Sign Language Instructional Laboratories

Despite the upward trajectory of formal American Sign Language (ASL) instruction, evidence-based practices remain in a rudimentary stage of development. Previously, no known studies had distinctly investigated supplemental ASL laboratories (ASL labs), despite over half of ASL instructors utilizing them alongside their classes. This qualitative study explored the perspectives of eight postsecondary ASL students regarding their experiences and perspectives of ASL labs. Data were triangulated through digital questionnaires, semi-structured interviews, and follow-up member checks and was analyzed using directed content analysis. Results are presented as nine overarching themes, and the discussion culminates with formal recommendations for stakeholders, including the continuation and extension of this important line of research.