Interpretation Of Signs Research Articles

At risk but not adequately included: People with disabilities’ experience of COVID-19 in Zambia

Background: COVID-19 had an impact on all sections of society, including people with disabilities.Objectives: The authors aimed to explore the needs and experiences of people with disabilities in Zambia during the COVID-19 pandemic.Method: In this hermeneutic phenomenological study, we used a semi-structured interview guide to collect data from a purposive and snowball sample of 40 people with disabilities and their caregivers. The participants were from 11 districts in 6 provinces in Zambia. The in-depth interviews were done between July 2022 and November 2022. Data were managed in NVivo and analysed using reflexive thematic analysis.Results: The three themes included: (1) awareness and experience of public health measures on COVID-19 among people with disabilities; (2) experience of othering and stigmatisation as people with disability during the COVID-19 pandemic and (3) experience of COVID-19 symptoms and having COVID-19 among people with disabilities.Conclusion: Interventions were largely unresponsive to the needs of people with disabilities, exacerbating the risk of exposure to infection. In future, adaptations like emergency risk communication in braille, audio and sign language interpretation in adapted communication formats should be made. Further studies are needed to quantify the gaps in access to health, explore policies and strategies to improve health outcomes for people with disabilities in LMICs like Zambia.Contribution: The findings may contribute to the development and enhancement of policies and interventions responsive to the needs of people with disabilities in future pandemics in the Zambian context.

UTILIZING CONVOLUTIONAL NEURAL NETWORKS TO COMPREHEND SIGN LANGUAGE AND RECOGNIZE EMOTIONS

The inability to communicate verbally is widely acknowledged as a significant disability. The primary objective of this research is to create a practical system aimed at individuals with hearing impairments, particularly those who depend on sign language as their primary means of communication. This system aims to enable deaf individuals to express themselves, communicate effectively and facilitate understanding of their language which would otherwise be challenging since most people are unfamiliar with sign language. By employing human gesture interpretation and motion capture, this technology can facilitate the translation of sign language into spoken language and vice versa. Despite the existence of various methods to convert sign language into voice, none of them currently provide an entirely intuitive user interface. Our objective is to create a system that not only translates sign language but also integrates a natural user interface, thus enhancing accessibility for individuals who are blind or have visual impairments. This system will achieve this by recognizing facial expressions and effectively conveying emotions behind words assisting visually impaired individuals in expressing themselves more effectively. Hence, in this project, we tentatively aim to build a system that can ease the lives of blind, deaf and dumb people to some extent. Normal communication like normal people do might never be possible for differently abled people but through our project, we try to provide them with a tool that can help them experience normalcy while communicating with normal people.

Investigating Media Accessibility for the Deaf and Hard of Hearing in Iraq

Media accessibility, (MA) for short, is still largely uncharted research territory in the Translation Studies scene in Iraq. To examine the status quo of media accessibility for the (DHH) community, this study has three aims to address: first, to subject the Iraqi constitution to a thematic analysis to highlight the legal status of media accessibility; second, to examine the real actions taken by the government and non-governmental organizations to ensure that the d/Deaf and hard of hearing (DHH) can access this service; third, to probe the DHH community’s reception and their level of expectations towards the available (MA) service. The findings suggest that the Constitution of the Republic of Iraq contains Articles that refer directly to rights of people with special needs to access the public service, but no reference to the right of media accessibility (MA) is identified. The study also shows that sign language interpreting (SLI), though scarcely employed in very few TV channels, has been the only audiovisual modality available for DHH viewers to access TV news whereas subtitling for the d/Deaf and hard of hearing (SDH) modality has never been implemented.

Reverse Sign Language Recognition System Using Machine Learning

Deaf and hard-of-hearing people communicate with each other and their society by using sign language. As this is a very natural method for pupils to stay in touch with computers, many academics are working on it to make it less complex and more convenient for use. So the main objective of gesture recognition research is to make systems which can interact and communicate while understanding human gestures and use them. In simple words to communicate information. Fast and extremely accurate hand detection and realtime hand gestures. Identification should be possible with vision-based hand gesture circumstances and interfaces. Learning and knowing sign movements and gestures is the kick start in making words and sentences for computer assisted sign language interpretation. Both dynamic and static sign actions are open and available. Both ways of gesture recognition are crucial to human culture, even if static gesture recognition is much easier than dynamic gesture recognition. When a human enters the value of alphabets and numerical value as input, the system immediately displays or outputs the appropriate recognised character shows the gesture on the monitor screen. In the following, research projects that have led to a proper system that uses convolutional neural networks to identify handwriting on the basis of the depth pictures and Hand Languages (Brain lipi) the collects. Keywords- Convolutional neural network,Text recognition ,Convert to image,StackGAN,simultaneous Tracking text and converting to image, gesture recognition,display output as sign gesture,training machine to(A-z to 1-0).

A New Sign Language Translation System Based on Expert Model

Abstract. The linguistic diversity of sign languages across regions presents significant challenges for the development of accurate and culturally sensitive recognition systems. To address this, This paper propose a modular sign language recognition system that detects the regional origin of a sign language and uses specialized sub-models for translation, bypassing the limitations of generalized models. This approach allows for more precise translations by catering to regional variations, ensuring higher accuracy and preserving cultural nuances.This paper also explore the creation of a standardized, diverse dataset, integrating multiple sign languages, which serves as the foundation for the systems region detection process. The dataset was meticulously annotated, normalized, and augmented to support robust model training. By adopting sub-models and utilizing region-specific data, the system improves efficiency and scalability, offering a practical solution for real-world sign language translation. This work emphasizes the importance of preserving sign language diversity while enhancing the usability and adaptability of recognition systems.

Dynamic gesture recognition using hand pose-based neural networks for sign language interpretation

Dynamic gesture recognition using hand pose-based neural networks for sign language interpretation

Comparative analysis of GPT-4, Gemini, and Ernie as gloss sign language translators in special education

While several comparative studies have analyzed the effectiveness of various large language models (LLMs), most of them were technical (i.e., comparing execution time, among others). Additionally, these comparative studies did not discuss special education. Consequently, scant information exists about how effective LLMs are in special education. To address this research gap, this study conducted a comparative study of three LLMs, namely GPT-4o, Gemini, and Ernie, as gloss sign language translators for learners with hearing impairments. Specifically, a mixed method was adopted, where the translated outputs of the three LLMs were compared (quantitatively and qualitatively) to two sign language outputs from a sign language expert. The obtained results highlighted that Gemini outperformed both GPT-4o and Ernie as an accurate gloss sign language translator. Additionally, GPT-4o had a high accurate rate, while Ernie had a very low translation performance. The findings of this study can help to raise awareness about the use of LLMs in special education as well as the best ones to use especially with hearing impairment learners.

DFCNet +: Cross-modal dynamic feature contrast net for continuous sign language recognition

In sign language communication, the combination of hand signs and facial expressions is used to convey messages in a fluid manner. Accurate interpretation relies heavily on understanding the context of these signs. Current methods, however, often focus on static images, missing the continuous flow and the story that unfolds through successive movements in sign language. To address this constraint, our research introduces the Dynamic Feature Contrast Net Plus (DFCNet+), a novel model that incorporates both dynamic feature extraction and cross-modal learning. The dynamic feature extraction module of DFCNet+ uses dynamic trajectory capture to monitor and record motion across frames and apply key features as an enhancement tool that highlights pixels that are critical for recognizing important sign language movements, allowing the model to follow the temporal variation of the signs. In the cross-modal learning module, we depart from the conventional approach of aligning video frames with textual descriptions. Instead, we adopt a gloss-level alignment, which provides a more detailed match between the visual signals and their corresponding text glosses, capturing the intricate relationship between what is seen and the associated text. The enhanced proficiency of DFCNet+ in discerning inter-frame details translates to heightened precision on benchmarks such as PHOENIX14, PHOENIX14-T and CSL-Daily. Such performance underscores its advantage in dynamic feature capture and inter-modal learning compared to conventional approaches to sign language interpretation. Our code is available at https://github.com/fyzjut/DFCNet_Plus.

Innovative Hand Sign to Text-and-Speech Conversion System

In present scenario, it is difficult for people who are unable to speak due to various reasons. It may not be a permanent condition, as muteness and deafness can be caused or manifest due to several different phenomena, such as physiological injury, illness, medical side effects, psychological trauma, developmental disorders, or neurological disorders. People who are suffering from this typically use the Sign Language to communicate with other people. However, it is not possible for everyone to be acquainted with communication using sign language. This project focuses on developing a solution which can be used by regular people to understand the people having mutism. Hand Sign Recognition Systems have evolved significantly with advancements in machine learning and deep learning techniques. These systems are designed to recognize gestures or signs, making them useful in various applications such as sign language interpretation, human-computer interaction, and virtual reality. This paper gives an overview of the current state of hand sign recognition systems, highlighting the techniques used, challenges, and possible future directions. The focus of the paper is on deep learning-based methods and their impact on the accuracy and usability of hand sign recognition in real-time applications [1]

IoT Based Sign to Speech Converter System

The idea introduces a system for facilitating communication between the deaf and the non-signing community through a wearable device called IoT based Sign-to-Speech Converter System. The System consist of glove & it utilizes Internet of Things (IoT) technology to bridge the communication gap by translating sign language (gestures) into spoken language in real-time. The core functionality of the system is augmented by seamless integration with an Android application, enhancing user experience and accessibility. The proposed system leverages a combination of sensors embedded within the glove to capture intricate hand movements and gestures commonly used in sign language. These data are transmitted wirelessly to a central processing unit, to interpret the gestures and generates corresponding spoken language output. The Android application serves as a user interface, allowing for customization of settings and additional features such as text-based communication. Effectiveness and practicality of the proposed system have been demonstrated, opening-up new avenues for inclusive communication and empowerment of the deaf community.

The Development of AI-Powered Automatic Video Sign Language Translation Systems

People who are deaf or hard of hearing often face challenges in fully understanding written subtitles in videos due to the differences between sign language and spoken language, each having unique grammar and structures. While many social media platforms provide automatic subtitles, they are often insufficient for accessibility. Sign language interpretation should be included to make video content fully accessible. Traditionally, adding sign language to videos involves a time-consuming process of recording and embedding separate translation videos, which must be redone with any changes to the original video. The plugin simplifies this process by dynamically translating updates directly from subtitle files, significantly reducing the time, effort, and cost involved. It allows seamless sign language support for any video without modifying the original content. The plugin integrates with video players, providing a user-controlled, customizable sign language window that can be activated, moved, resized, or turned off as desired.

Application of AI in Turkish Sign Language Translation: A Case Study of its Use and Purpose

In Turkey, around 3 million people have hearing impairments, and the shift to digital platforms during the pandemic has worsened accessibility challenges as websites and apps often ignore their needs. Research shows that 50% of hearing-impaired individuals struggle to understand written text due to Turkish Sign Language being their first language, with Turkish serving as a second language. Differences in grammar between Turkish and Turkish Sign Language, along with a limited sign language vocabulary, further hinder comprehension. To solve this, we developed AI-powered sign language translation systems that allow people who are deaf or hard of hearing to access digital content in Turkish Sign Language. SignForDeaf's system translates text into sign language videos using Natural Language Processing (NLP) and generates seamless videos with smooth word transitions. Currently, this system supports Turkish Sign Language, with future plans to include other languages like Arabic, American, British, and Finnish Sign Language. The system was designed in collaboration with sign language experts to ensure accuracy and an inclusive development of a digital environment.

Workshops: Enhancing communication for nursing students with patients with hearing disabilities

Abstract Background Nursing students need guidance on effective communication strategies for interacting with patients with hearing disabilities, emphasizing the importance of addressing this gap in their education as future healthcare professionals. Methods A cross-sectional survey was conducted among nursing students at seven institutions before and after workshops led by guides with hearing disabilities from a non-profit organization Ma’agalei Shema and funded by the National Insurance Institute. Data was collected from 2023 to March 2024 and analyzed using SPSS software. Results 279 participants completed the survey before the workshops. and 136 responding afterward. Almost 80% were female. Before the workshops, 68.46% of students expected to gain communication tools for patients with hearing disabilities. About one-third believed the healthcare system offers accessible treatment to these patients, while only 22.22% foresaw no barriers. One-third reported no clinical contact with such patients. Among those who did, lip-reading (25.53%), writing (17.73%), assistance from family members (15.6%), and sign language (9.22%) were commonly used. Only 2.84% used professional sign language interpreters, and 20.57% reported never meeting a patient with hearing disabilities (8% reported “other”). Post-workshop surveys showed that around 82.48% of participants felt the workshops would significantly enhance their communication skills with such patients, with only 5.11% reporting a lack of practical tools received. Conclusions Practical workshops can significantly enhance communication skills and improve patient care. This study underscores the potential impact of minor adjustments in communication strategies on patient care, such as increasing the use of professional interpreters. Main messages: Providing training to nursing students in effective communication with these patients can streamline interactions, diminish reliance on family intermediaries, and enhance medical care. Key messages • Nursing students need guidance on communicating with patients with hearing disabilities to improve future healthcare interactions. • Practical workshops enhance communication skills, stressing the importance of minor adjustments for better patient care.

SignifAI: French sign language translation based on deep learning and large language models

In this project, we propose SignifAI which is an AI-powered software that translates French Sign Language into spoken French. The goal of SignifAI is to remove the language barrier that separates the deaf and hearing communities. The translation process proves to be a major challenge for existing methods, as most of them rely on traditional Natural Language Processing (NLP) methods which solve the task in a similar way as spoken languages (e.g., paying more attention to the logics and reasoning). The drastic syntactical difference between sign and spoken languages, however, makes such approaches not as effective and efficient. In SignifAI, we propose to leverage a combination of deep learning models for sign gesture recognition and large language models (LLM) for sentence generation from recognized words. The former allows for precise detection of individual words, while the latter takes the recognized words as inputs and returns a fluently reformulated translated sentence. With extremely limited data available for French sign language, we created our own enhanced dataset through synthetically augmenting existing videos. Utilizing the SlowFast model for this task, we have achieved a maximum test accuracy of 98.9%, and a fluid translation process after implementing ChatGPTs API. The potential applications of SignifAI range from aiding communication between the deaf and the hearing, to helping the deaf place phone calls in numerical services. Ultimately, SignifAI strives to improve all aspects of a deaf persons life, and compensates for their speaking disabilities.

Neural changes in sign language vocabulary learning: Tracking lexical integration with ERP measures

The present study aimed to investigate the neural changes related to the early stages of sign language vocabulary learning. Hearing non-signers were exposed to Catalan Sign Language (LSC) signs in three laboratory learning sessions over the course of a week. Participants completed two priming tasks designed to examine learning-related neural changes by means of N400 responses. In a semantic decision task, participants evaluated whether written Catalan word pairs were semantically related or not. The experimental manipulation included prime-target phonological overlap (or not) of the corresponding LSC sign translations. In a LSC primed lexical decision task, participants saw pairs of signs and had to determine if the targets were real LSC signs or not. The experimental design included pairs of signs that were semantically related or unrelated. The results of the LSC lexical decision task showed N400 lexicality and semantic priming effects in the third session. Also in the third session, N400 effects related to the activation of LSC phonology were observed during word processing in the semantic decision task. Overall, our findings suggest rapid neural changes occurring during the initial stages of intensive sign language vocabulary training. The results are discussed in relation to the temporality of lexicality and semantic effects, as well as their potential relation to linguistic features of sign languages.

Occupational Safety, Health, and Well-Being Concerns and Solutions for Management Reported by Sign Language Interpreters: A Qualitative Study

While the occupational health of sign language interpreters has traditionally focused on physical health, evidence demonstrating mental health concerns is growing and supporting a shift to a more integrated approach. We embarked on a qualitative study to guide the adaptation of a previously developed Total Worker Health® program to the context of sign language interpreting. Eight unstructured 90-min focus groups were conducted. Interpreters reported occupational safety, health, and well-being concerns and shared their solutions for management. Twenty-seven interpreters participated (aged 53.7 years; 81% female; 85% white). Predominant concerns centered on topics like workplace violence, secondary traumatic stress or vicarious trauma, lack of work–life integration or boundaries, and loss of agency or loss of self. The organizational culture of the field fostered deprioritization of self, oppression, elitism, sexism, and unhealthy relationships with interpreter peers and community members. Physical health remained a contributor, specifically the physical effects of non-physical work, aging, and differences in exposures across interpreting settings but paled in comparison to mental health and organizational culture. Solutions for management included but were not limited to prioritization of jobs, creating safe spaces/communities of supported practice, and exercise. This study supports the evolution of future research, practice, policy, and capacity building aspiring to a more integrated approach.

Deep Learning-Driven Sign Language Recognition: A Multimodal Approach for Gesture-to-Text Translation Using CNN-RNN Architectures

Sign language serves as a vital communication tool for individuals with hearing and speech impairments. However, a significant barrier exists when others do not possess a strong understanding of sign language often requiring interpreters to facilitate communication. To reduce the reliance on human interpreters, this research aims to develop an intelligent system capable of recognizing and translating sign language gestures into meaningful grammatically correct sentences. The proposed system processes both images and videos to interpret gestures, converting them into text that can be translated into any desired language. The system leverages state-of-the-art deep learning techniques, such as Convolutional Neural Networks (CNNs) for identifying key features in images and Recurrent Neural Networks (RNNs) for understanding temporal sequences in video data. By employing these advanced neural networks, the model is able to comprehend hand movements, facial expressions and other non-verbal cues to construct coherent sentences. Additionally, the system integrates natural language processing (NLP) to refine the output, ensuring the resulting sentences are grammatically correct. Our approach addresses common challenges such as differentiating between subtle hand gestures and reducing the impact of environmental noise in images. This solution holds the potential to significantly enhance communication for the hearing and speech impairedffering an efficient interpreter-free method of translating sign language into widely spoken languages

Accessibility in an Inclusive Academic Environment: The Case of Takoradi Technical University

This study examines the extent to which Takoradi Technical University (TTU) in Ghana has achieved an inclusive academic environment for students with disabilities. Despite Ghana's commitment to inclusive education and various policies supporting persons with disabilities (PWDs), there remains a significant research gap regarding the practical implementation and effectiveness of these policies at the university level. This research addresses this gap by evaluating TTU's accessibility measures and support services. The study employs a phenomenological approach to explore the lived experiences of students with disabilities, faculty, and staff at TTU. It investigates the alignment of TTU’s practices with national and international inclusive education policies, such as the Persons with Disability Act (715) and the United Nations Convention on the Rights of Persons with Disabilities. The research focuses on three main objectives: identifying the teaching and learning needs of disabled students, assessing the accessibility of TTU's built environment, and evaluating the satisfaction of students with the available support services. Data were collected through qualitative interviews with 28 participants, including students with disabilities, lecturers, and support staff. Findings reveal that TTU has established various support services, including sign language interpretation, e-learning support, and assistive devices. However, students reported inadequacies in these services, citing needs for additional support such as braille services and extended examination time. Furthermore, the built environment at TTU was found to be largely inaccessible to students with physical disabilities, leading to widespread dissatisfaction among the student body. The study highlights that while TTU has made commendable efforts in providing institutionalized support for disabled students, significant gaps remain in service adequacy and physical accessibility. The findings underscore the need for enhanced strategies to improve inclusivity at TTU and similar institutions, aligning practical implementations with policy frameworks to better serve students with disabilities

Study on pose-based deep learning models for gloss-free Sign Language Translation

Sign Language Translation (SLT) is a challenging task due to its cross-domain nature, different grammars and lack of data. Currently, many SLT models rely on intermediate gloss annotations as outputs or latent priors. Glosses can help models to correctly segment and align signs to better understand the video. How- ever, the use of glosses comes with significant limitations, since obtaining annotations is quite difficult. Therefore, scaling gloss-based models to millions of samples remains impractical, specially considering the scarcity of sign language datasets. In a similar fashion, many models use video data that requires larger models which typically only work on high end GPUs, and are less invariant to signers appearance and context. In this work we propose a gloss-free pose-based SLT model. Using the extracted pose as feature allow fora sign significant reduction in the dimensionality of the data and the size of the model. We evaluate the state of the art, compare available models and develop a keypoint-based Transformer model for gloss-free. SLT, trained on RWTH-Phoenix, a standard dataset for benchmarking SLT models alongside GSL, a simpler laboratory-made Greek Sign Language dataset.

SWL-LSE: A Dataset of Health-Related Signs in Spanish Sign Language with an ISLR Baseline Method

Progress in automatic sign language recognition and translation has been hindered by the scarcity of datasets available for the training of machine learning algorithms, a challenge that is even more acute for languages with smaller signing communities, such as Spanish. In this paper, we introduce a dataset of 300 isolated signs in Spanish Sign Language, collected online via a web application with contributions from 124 participants, resulting in a total of 8000 instances. This dataset, which is openly available, includes keypoints extracted using MediaPipe Holistic. The goal of this paper is to describe the construction and characteristics of the dataset and to provide a baseline classification method using a spatial–temporal graph convolutional network (ST-GCN) model, encouraging the scientific community to improve upon it. The experimental section offers a comparative analysis of the method’s performance on the new dataset, as well as on two other well-known datasets. The dataset, code, and web app used for data collection are freely available, and the web app can also be used to test classifier performance on-line in real-time.