Deaf Research Articles

SignifAI: French sign language translation based on deep learning and large language models

In this project, we propose SignifAI which is an AI-powered software that translates French Sign Language into spoken French. The goal of SignifAI is to remove the language barrier that separates the deaf and hearing communities. The translation process proves to be a major challenge for existing methods, as most of them rely on traditional Natural Language Processing (NLP) methods which solve the task in a similar way as spoken languages (e.g., paying more attention to the logics and reasoning). The drastic syntactical difference between sign and spoken languages, however, makes such approaches not as effective and efficient. In SignifAI, we propose to leverage a combination of deep learning models for sign gesture recognition and large language models (LLM) for sentence generation from recognized words. The former allows for precise detection of individual words, while the latter takes the recognized words as inputs and returns a fluently reformulated translated sentence. With extremely limited data available for French sign language, we created our own enhanced dataset through synthetically augmenting existing videos. Utilizing the SlowFast model for this task, we have achieved a maximum test accuracy of 98.9%, and a fluid translation process after implementing ChatGPTs API. The potential applications of SignifAI range from aiding communication between the deaf and the hearing, to helping the deaf place phone calls in numerical services. Ultimately, SignifAI strives to improve all aspects of a deaf persons life, and compensates for their speaking disabilities.

Indexical deprivation: The dominant link between cochlear implants and global English among Taiwanese deaf individuals

AbstractThis paper develops the concept of “indexical deprivation” from the experiences of English learning in relation to cochlear implant use among Taiwanese deaf adults. Based on the framework of language ideological assemblage, this paper traces how institutional discourses and practices at different levels contribute to the indexicalization between cochlear implants and elevated proficiency in English as a global language. By examining the top‐down discourses and bottom‐up narratives of two Taiwanese deaf women, the study demonstrates how enhanced English proficiency has been linked to cochlear implants and how individuals are deprived of the capacity to recognize alternative links. This paper highlights how global English has promoted the status of cochlear implants in a sociolinguistic context where English is spoken as a foreign language and increasingly gains prominence at multiple societal levels.

LÍNGUA BRASILEIRA DE SINAIS: BREVE HISTÓRICO DOS ASPECTOS EDUCACIONAIS DOS SURDOS

This article aims to present a compendium of the evolutionary history of Brazilian Sign Language (Libras) and its reflections on the deaf community, particularly concerning education for all. Schooling must be ensured for everyone, and therefore, addressing any disparities caused by specific needs is crucial. This dynamism reflects the shared responsibility of public and private agencies within legislative and educational contexts to provide specific training related to the language and prepare the environment for promoting teaching and learning. The method used involved exploratory bibliographic research in scientific articles and books, with filters for the central theme of this work. The results reveal a concerning state of involvement and interaction of public policies and effective actions for the integration and inclusion of deaf individuals in social and educational contexts, highlighting the challenge for more in-depth research on the topic to promote solutions and address the social disparities faced by these individuals.

Malaysian Sign Language Detection with Convolutional Neural Network for Disabilities

Sign language is the main form of communication used by deaf people. Most of their activities like speaking and learning involved sign language. In Malaysia, instead of a deaf person, only a few people know sign language because it's not part of the co-curriculum. Hence, normal people need to know sign language so that they can communicate with deaf people effectively. However, learning sign language requires a lot of effort and takes years of study. Therefore, this research tends to initiate the sign language image detection system that utilizes the deep learning model. The initial effort involves the development of an image detection system that can identify the alphabet of sign language from A to Z. There will be 26 alphabets that contribute to 26 classes of image detection using the Convolutional Neural Network. 85 images will be collected for each alphabet and the total data will be distributed into 80% and 20% for training and testing the CNN model, respectively. The result showed that the proposed system produced more than 95% detection accuracy of the alphabet based on finger sign gestures. For future works, the proposed system will be evaluated with a more complex dataset in terms of words.

Needs of Deaf People Using Elevators: Identification of Accessibility and Safety Requirements

As urbanization and population growth continue, cities concentrate an increasing amount of people, energy, and economy. Multi-story buildings enable densification, requiring vertical transport for access to upper floors. This is crucial for people with disabilities, who may face barriers in the built environment. Elevators are essential for accessibility, allowing everyone, including people with disabilities, to comfortably access multi-story buildings. However, barriers to inclusivity remain, often subtle and hard to define. This paper highlights one such example, focusing on elevator use by individuals with varying degrees of hearing loss. Currently, they cannot establish one-to-one communication with the outside world if trapped in an elevator. Under EN standards, this issue stems from alarm system requirements that lack effective alternatives to voice communication. Based on this evidence, the research was carried out in two steps, with the aim of understanding the needs of deaf individuals when using elevators by directly involving them in the study. First, a questionnaire conducted in Italy collected information regarding the safety and usability of elevators. Second, a test campaign involving both deaf and normal-hearing participants was carried out to quantify the severity of the issue and evaluate potential solutions to address the identified challenges. The conclusions indicate that current alarm systems in elevators are inadequate for individuals with hearing impairments, and effective alternatives must be implemented.

Embodiment of sexuality by Israeli Sign Language signers and hearing Hebrew speakers

AbstractLanguage is a key resource for speakers and signers to index different aspects of their social identities, such as their ethnicities and sexualities. Yet, for users of sign language – who exploit movements of the hands, face, head and torso for linguistic purposes – it is often assumed that any communicative movement of the body is part of a sign language rather than a general feature of the body's potential to communicate social meaning, shared by hearing and deaf individuals. In this study, we test this claim by comparing the movement features produced by gay and straight Israeli Sign Language signers to the gestural movements produced by gay and straight Hebrew speakers. The findings reveal that deaf gay signers and hearing gay gesturers exploit similar movements of the body. By incorporating the notion of embodiment into sign language sociolinguistics, we can better conceptualize the relationship between sign language and social identities.

Techniques for Detecting the Start and End Points of Sign Language Utterances to Enhance Recognition Performance in Mobile Environments

Recent AI-based technologies in mobile environments have enabled sign language recognition, allowing deaf individuals to communicate effectively with hearing individuals. However, varying computational performance across different mobile devices can result in differences in the number of image frames extracted in real time during sign language utterances. The number of extracted frames is a critical factor influencing the accuracy of sign language recognition models. If the number of extracted frames is too small, the performance of the sign language recognition model may decline. Additionally, detecting the start and end points of sign language utterances is crucial for improving recognition accuracy, as the period before the start point and after the end point often involves no action being performed. These parts do not capture the unique characteristics of each sign language. Therefore, this paper proposes a technique to dynamically adjust the sampling rate based on the number of frames extracted in real time during sign language utterances in mobile environments, with the aim of accurately detecting the start and end points of the sign language. Experiments were conducted to compare the proposed technique with the fixed sampling rate method and with the no-sampling method as a baseline. Our findings show that the proposed dynamic sampling rate adjustment method improves performance by up to 83.64% in top-5 accuracy and by up to 66.54% in top-1 accuracy compared to the fixed sampling rate method. The performance evaluation results underscore the effectiveness of our dynamic sampling rate adjustment approach in enhancing the accuracy and robustness of sign language recognition systems across different operational conditions.

Enhancing sign language recognition and accessibility for the deaf community in China

This paper explores the critical role of sign language in enabling communication and social inclusion for the deaf community in China, where approximately 27.8 million individuals experience hearing impairments. Despite the recognition of linguistic human rights, Chinese law lacks formal acknowledgment of sign language, leading to fragmented regulations and insufficient support. The study highlights deficiencies in existing legislation and proposes recommendations for enhancing legal frameworks to protect and promote the rights of deaf individuals. Additionally, the paper examines the disparities between local and national sign languages and underscores the potential of technological advancements, such as sign language translation gloves, in bridging communication gaps and fostering social integration.

Kurdish Sign Language Recognition Using Convolutional Neural Network (CNN)

The present study examines the obstacles encountered by the deaf population, with a specific emphasis on the increasing significance of sign language in facilitating effective communication. The main mode of communication for deaf persons is Sign Language (SL), which is a visual and expressive method of communicating meaning via facial expressions, hand motions, and body gestures. The objective of this project is to automate the identification of sign language in order to improve accessibility and decrease reliance on interpreters. Specifically, the work focused on constructing an alphabet recognition system for Kurdish Sign Language (KSL). Due to its many intricacies and resemblances to the Arabic script, KSL requires a strong recognition model. The proposed method utilizes Convolutional Neural Networks (CNN) with an actual dataset to accurately recognize both numerical values and alphabetic characters in the Korean Sign Language (KSL). Real-time operation allows for rapid recognition of hand gestures and subsequent provision of textual output. The collection comprises 132,000 hand pictures, including 33 alphabetica and (0-9) numeral signs. Significantly, the use of MediaPipe, a method for processing 3D images, greatly improves the efficiency of detection. Multiple methodologies were investigated, and the integration of Convolutional Neural Networks (CNN), TensorFlow, and MediaPipe resulted in a remarkable accuracy of 99.87% with negligible dropout rates. This study establishes a foundation for enhanced communication and independence for the deaf population, representing a notable advancement in the automation of sign language recognition.

Insights from the Silent: Exploring Deaf Individuals' Perspectives on Using Learning Management Systems for Online Learning

The COVID-19 pandemic has led to a surge in the popularity of online learning, offering students greater flexibility and accessibility. However, students with hearing impairments encounter significant challenges in accessing learning materials and communicating with teachers and peers. This study aims to investigate the experiences of students with hearing impairments using a Learning Management System (LMS) for online learning focusing on their challenges they face and the strategies they employ in their learning. The research was conducted using a descriptive qualitative design at private universities in Yogyakarta. Semi-structured interviews were conducted to gather the data involving three students with hearing impairments who have used LMS in online learning as the participants. The findings revealed the challenges that included the use of multiple applications, lack of understanding on the LMS, lack of LMS notifications, material mismatch, unstable internet connection, tool constraints, lack of inadequate facilities, and there is no training on the use of the LMS. Additionally, the strategies used by the students included self-study, assignment submission via email, asking others for help, using additional apps, using a typist facility, and joining WhatsApp group discussions.

Virtual reality as a solution: Meeting the needs of deaf students with digital sign language interpreters

Sign language is the primary means of communication for people who are deaf or hard of hearing. Sign language interpreters help support deaf individuals in understanding lecture material in class. This study aimed to analyze the need for digital sign language interpreters based on Virtual Reality for deaf students' lectures. This online survey-based study via Google form examined the perceptions and needs of digital sign language interpreters based on virtual reality for deaf student lectures. Participants in this study consisted of 51.5% (male) and 48.5% (female) deaf students from various public and private universities in Indonesia. The results of a survey conducted on 200 deaf students about their interest in using VR showed that 64% (128 students) were in the agreed category. The primary need in lectures for deaf students is a sign language interpreter. The results of a study conducted via the Google form showed that 80% or as many as 160 out of 200 students agreed that they needed sign language interpreters to make it easier for deaf students in the lecture process. The study’s results on the perception of the needs of VR and sign language interpreters provide a basis for developing assistive technologies by combining these elements. This will make it easier for deaf students, who are greatly helped by visual media. VR, which can be integrated to display sign language for deaf students, will make explaining the lecturer’s lecture material in class easier.

Auditory areas are recruited for naturalistic visual meaning in early deaf people

Congenital deafness enhances responses of auditory cortices to non-auditory tasks, yet the nature of the reorganization is not well understood. Here, naturalistic stimuli are used to induce neural synchrony across early deaf and hearing individuals. Participants watch a silent animated film in an intact version and three versions with gradually distorted meaning. Differences between groups are observed in higher-order auditory cortices in all stimuli, with no statistically significant effects in the primary auditory cortex. Comparison between levels of scrambling revealed a heterogeneity of function in secondary auditory areas. Both hemispheres show greater synchrony in the deaf than in the hearing participants for the intact movie and high-level variants. However, only the right hemisphere shows an increased inter-subject synchrony in the deaf people for the low-level movie variants. An event segmentation validates these results: the dynamics of the right secondary auditory cortex in the deaf people consist of shorter-length events with more transitions than the left. Our results reveal how deaf individuals use their auditory cortex to process visual meaning.

Cognitive processing speed improvement after cochlear implantation.

Untreated hearing loss has an effect on cognition. It is hypothesized that the additional processing required to compensate for the sensory loss affects the cognitive resources available for other tasks and that this could be mitigated by a hearing device. The impact on cognition of cochlear implants (CIs) was tested in 100 subjects, ≥60 years old, with bilateral moderately-severe to profound post linguistic deafness using hearing aids. Data was compared pre and 12 and 18 months after cochlear implantation for the speech spatial qualities questionnaire, Mini Mental State Examination (MMSE), Trail making test B (TMTB) and digit symbol coding (DSC) from the Wechsler Adult Intelligence Scale version IV and finally the timed up and go test (TUG). Subjects were divided into young old (60-64), middle old (65-75) and old old (75+) groups. Cognitive test scores and times were standardized according to available normative data. Hearing significantly improved pre- to post-operatively across all age groups. There was no change post-implant in outcomes for TMTB, TUG or MMSE tests. Age-corrected values were within normal expectations for all age groups for the TUG and MMSE. However, DSC scores and TMTB times were worse than normal. There was a significant increase in DSC scores between baseline and 12-months for 60- to 64-year-olds (t[153] = 2.608, p = 0.027), which remained at 18 months (t[153] = 2.663, p = 0.023). The improved attention and processing speed in the youngest age group may be a consequence of reallocation of cognitive resources away from auditory processing due to greatly improved hearing. The oldest age group of participants had cognition scores closest to normal values, suggesting that only the most able older seniors tend to come forward for a CI. Severe to profoundly deaf individuals with hearing aids or cochlear implants were still poorer than age-equivalent normally hearing individuals with respect to cognitive flexibility, attention, working memory, processing speed and visuoperceptual functions. Due to a lack of data for the TUG, TMTB and DSC in the literature for hearing impaired individuals, the results reported here provide an important set of reference data for use in future research.

Effects of Number of Filters and Frequency Cutoff in Continuous Interleaved Sampling and Frequency Amplitude Modulation Encoding Schemes in Cochlear Implant

Cochlear implants are devices designed to transform sound into electrical signals perceived by the brain, making them vital prostheses for deaf individuals. This study examines two schemes used in cochlear implants, namely Continuous Interleaved Sampling (CIS) and Frequency Amplitude Modulation Encoding (FAME), to compare their performance while varying the number of bandpass filters and cutoff frequencies used. Both schemes were simulated using 8 and 5 bandpass filters, and cutoff frequencies of 2000 Hz and 200 Hz. Results show that the CIS scheme can maintain signal intelligibility despite the loss of some frequency components when the number of bandpass filters is lowered. Conversely, FAME retains more frequency details but presents perceptible delays. With a cut off frequency of 200 Hz, signals processed with CIS loses intelligibility significantly, whereas FAME-processed signals remain intelligible both at 200 Hz and 2000 Hz cut off frequencies. It is therefore concluded that FAME can provide better cochlear implant performance despite the lower number of bandpass filters and lower frequency cutoff.

Linguistic and cultural adaptation of the National Quality of Life Survey in Colombia for the deaf population

Abstract This article reports on the linguistic and cultural adaptation of Items of the National Quality of Life Survey (NQLS 2018), which are included in a National Survey that explores the availability of reasonable accommodation for Deaf users in a health care setting as well as health care access. We document the cultural and linguistic adaptation step-by-step, guided by a literature review, participatory group discussions, and qualitative testing. This research addresses the dearth of accessible research instruments for D‍/‍deaf persons in low and middle-income countries. This lack of accessible tools leads to double layers of exclusion, and data gaps often lead to erroneous generalizations that do not accurately represent D‍/‍deaf populations. Evidence gaps may result in poor service planning and inadequate policymaking. This study contributes to good practices in language interpreting in research settings, disability research, and global health.

القفاز الناطق الذكي بيد واحدة للصم و البكم

The sign language is an effective means of communication between the deaf and dumb communities and the general public, but it requires human translators. This paper describes the development of a smart speaking glove with one-hand deaf and Dumb individuals using Arabic sign language. The glove bridges the communication gap between those with hearing and speech problems and those who don't understand sign language. It accurately converts sign language motions into audio and text, enhancing dignity, independence, and social inclusion. The glove, made of flexible sensors and a six-dimensional motion tracking device, allows deaf individuals to communicate with regular people via a smartphone, reducing the gap between them. It has focused on six major scenarios based on sentences and words of foodstuff and welcome signs. In cooperation with the Zawia Center for the Deaf and Dumb, every scenario was evaluated. The glove performed well when tested for side bending, resistance, voltage, and angle.

Bengali-Sign: A Machine Learning-Based Bengali Sign Language Interpretation for Deaf and Non-Verbal People.

Sign language is undoubtedly a common way of communication among deaf and non-verbal people. But it is not common among hearing people to use sign language to express feelings or share information in everyday life. Therefore, a significant communication gap exists between deaf and hearing individuals, despite both groups experiencing similar emotions and sentiments. In this paper, we developed a convolutional neural network-squeeze excitation network to predict the sign language signs and developed a smartphone application to provide access to the ML model to use it. The SE block provides attention to the channel of the image, thus improving the performance of the model. On the other hand, the smartphone application brings the ML model close to people so that everyone can benefit from it. In addition, we used the Shapley additive explanation to interpret the black box nature of the ML model and understand the models working from within. Using our ML model, we achieved an accuracy of 99.86% on the KU-BdSL dataset. The SHAP analysis shows that the model primarily relies on hand-related visual cues to predict sign language signs, aligning with human communication patterns.

FNIRS Changes in the Middle Temporal and Occipital Cortices After a Cochlear Implant.

The relationship between the middle temporal (MTG) and occipital cortices in post-lingually deaf (PLD) individuals is unclear. This study aimed to investigate changes in the MTG and occipital cortices excitability and their effects on the occipital cortex in individuals with PLD after receiving a cochlear implant (CI). Twenty-six individuals with severe-to-profound binaural sensorineural PLD were assessed clinically. Nine individuals had received a unilateral cochlear implant over 6 months, while 17 had not. Brodmann area 19 (BA19, extra-striate occipital cortex) and MTG (auditory-related area of cortex) were selected as regions of interest. The excitability of the ROI was observed and compared in the surgery and no-surgery groups by functional near-infrared spectroscopy (fNIRS) in the resting state, and correlations between connectivity of the MTG and occipital cortex, and as well as the duration of time that had elapsed following CI surgery, were investigated. fNIRS revealed enhanced global cortical connectivity in the BA19 and MTG on the operative side (p < 0.05) and the connectivity between BA19 and the MTG also increased (p < 0.05). The connectivity between the MTG and BA19 was positively correlated with the duration of cochlear implantation, as was the case for BA18. There was evidence for remodeling of the cerebral cortex: increased excitability was observed in the MTG and BA19, and their connectivity was enhanced, indicating a synergistic effect. Moreover, the MTG may further stimulate the visual cortex by strengthening their connectivity after CI. 3 Laryngoscope, 2024.

Sign Language Detection Using Machine Learning

This paper investigates the application of machine learning for sign language detection. The objective is to develop a model that translates sign language into spoken language, bridging the communication gap between deaf and hearing individuals. You Only Look Once (YOLO), a deep learning object detection algorithm, is employed to train a model on a dataset of labeled sign language images derived from video data. The system achieves real-time sign detection in videos. However, challenges include the scarcity of large, labeled datasets and the inherent ambiguity of certain signs, which can lead to reduced detection accuracy. This research contributes to the field of Assistive Technologies (AT) by promoting accessibility and social inclusion for the deaf community.

The characteristic of empathy in deaf individuals: Evidence from Hybrid Eye Tracking-EEG

The characteristic of empathy in deaf individuals: Evidence from Hybrid Eye Tracking-EEG