Sign Language Research Articles

Ionic liquid-reinforced transparent, stretchable, conductive organic ionic gel with ultra-high sensory capability and ultra-robust impact-resistance

Traditional conductive hydrogels face challenges in environmental stability and mechanical limitations, constraining their applications in wearable devices. Here, we present a novel organic ionic gel by the construction of PVA/EG double polymer network and ionic liquid, named POIG. It showcases outstanding tensile strength (3.01 MPa) and extensibility (820 %). POIG excels in both low and high-speed impacts, dissipating over 90 % of energy in low-speed impacts and outperforming commercial foam five times in high-speed impacts (1.4 kJ/m). The addition of an ionic liquid in POIG creates a stable, uniform, and highly ordered internal conductive structure, leading to ultra-low strain responsiveness (0.2 %), ultra-fast response times (≈10 ms), high linearity (R2 = 0.999), and excellent sensing performance. High stability is also confirmed through 4000 cycles of both tensile and compression testing. Additionally, we developed a real-time sign language recognition system using deep learning, accurately identifying ten gestures with 98.5 % accuracy. This research offers a durable and innovative organic material for future wearable technology and smart devices.

Exploring positional and dimensional aspects of topographic space for advanced-level British Sign Language learners

Abstract This study investigates how British Sign Language (BSL) learners develop positional and dimensional aspects of topographic space. The teaching of BSL has been occurring at an increasing rate, and many people are learning to use BSL to the advanced levels, which are generally referred to in the UK as BSL Levels 4 and 6 and are loosely equivalent to pre-C1 (advanced) and C1 (proficient), respectively, on the Common European Framework for Reference (Languages) scale. Spatial grammar is a crucial aspect of BSL (Brennan 1992), and this article provides insight into issues related to learning how to use topographic space, a feature of spatial grammar whereby real-world referent locations are replicated in the signing production. This study of L2 advanced learners of BSL explores the numbers and types of errors that are made when applying topographic features into their signing, with a particular focus on positioning and dimensionality aspects. Two scenarios are used do this: classroom layouts (Task 1) and a courtroom layout (Task 2). The study concludes with reflections on how BSL teachers can support L2 learners in improving their development of topographic skills.

Crisis interpreting and Deaf community understanding during the COVID-19 pandemic: results from a South Korea-based survey.

This study explores the South Korean Deaf community's response to sign language interpreting during the COVID-19 (coronavirus disease 2019) health crisis, focusing on individual factors affecting the signers' comprehension. The data were collected from a mobile-based questionnaire survey conducted among 401 Deaf adults; binary probit modelling was adopted to analyse the data. The major findings are: (i) 59.9 per cent of the respondents understood less than 70 per cent of the interpreting; (ii) males and urban residents tend to understand better; (iii) younger people (less than 50 years) and signers with a Bachelor's degree or higher are likely to have lower comprehension; and (iv) Deaf adults who visited a doctor after the COVID-19 outbreak tended to have lower comprehension. The findings demonstrate that individual characteristics, including age, impact significantly on the extent to which Deaf individuals understand the sign language interpreting of COVID-19 information, indicating that steps are needed to achieve a Deaf-inclusive society during a health disaster.

Re-conceptualising communicative expertise in professional practice through the lens of sign language intercultural mediation

The Candlin Lecture When we consider language, communication and the professions, we know that talk (speech or signs) is critical to professional practice and to relationships between professionals and service users. In a multilingual and multicultural world where people do not share the same languages or language repertoires, they must either adapt the way they talk to others (intercultural communication) or, if they cannot communicate directly, allow others to mediate communication on their behalf – a process labelled intercultural mediation. The latter occurs through professional or non-professional interpreting or language brokering. Current theories in interpreting studies consider (professional) interpreters as co-constructors of meaning and co-participants in any interaction. Communicative expertise is usually conceptualized in direct, monolingual communication. Using an explorative illustrative case study, this paper extends this theoretical framework to examine how communicative expertise manifests in interpreter-mediated communication, and particularly in relational aspects of intercultural mediation between a signed language and a spoken language and how professional interpreters and non-professional interpreters (brokers) draw on and apply their expert or lay knowledge about communication in a mediated interaction.

Correction: Sign language mobile apps: a systematic review of current app evaluation progress and solution framework

Correction: Sign language mobile apps: a systematic review of current app evaluation progress and solution framework

Quantized CNN-based efficient hardware architecture for real-time hand gesture recognition

Nowadays, Convolutional Neural Networks (CNN) have been widely adopted for vision-based hand gesture recognition. Several existing CNN architectures designed for gesture classification perform well with high accuracy but require a high memory footprint and processing when deployed on low-power embedded devices. To address this issue, we present a quantized CNN-based efficient framework for meeting real-life hand gesture recognition challenges. The proposed quantized CNN is designed and implemented using the FINN-based pipelined streaming architecture on an FPGA. Moreover, hardware-based optimizations are used to minimize the resources needed and to achieve fast memory access. Experimental results demonstrate that the developed recognition system achieves an average accuracy of 92% on the numeral database of Indian Sign Language (ISL). Additionally, our optimized design attains an inference latency of 0.85ms for real-time single gesture prediction on the PYNQ Zynq Ultrascale (ZU) FPGA, consuming only 3.63 W of power. The proposed design achieves a better trade-off between hardware resource utilization and speed performance, over previous designs.

The time course of Cantonese and Hong Kong Sign Language phonological activation: An ERP study of deaf bimodal bilingual readers of Chinese

This study investigated Cantonese and Hong Kong Sign Language (HKSL) phonological activation patterns in Hong Kong deaf readers using the ERP technique. Two experiments employing the error disruption paradigm were conducted while recording participants’ EEGs. Experiment 1 focused on orthographic and speech-based phonological processing, while Experiment 2 examined sign-phonological processing. ERP analyses focused on the P200 (180–220 ms) and N400 (300–500 ms) components.The results of Experiment 1 showed that hearing readers exhibited both orthographic and phonological effects in the P200 and N400 windows, consistent with previous studies on Chinese reading. In deaf readers, significant speech-based phonological effects were observed in the P200 window, and orthographic effects spanned both the P200 and N400 windows. Comparative analysis between the two groups revealed distinct spatial distributions for orthographic and speech-based phonological ERP effects, which may indicate the engagement of different neural networks during early processing stages.Experiment 2 found evidence of sign-phonological activation in both the P200 and N400 windows among deaf readers, which may reflect the involvement of sign-phonological representations in early lexical access and later semantic integration. Furthermore, exploratory analysis revealed that higher reading fluency in deaf readers correlated with stronger orthographic effects in the P200 window and diminished effects in the N400 window, indicating that efficient orthographic processing during early lexical access is a distinguishing feature of proficient deaf readers.

Communication through Hands in Sign Language - A CNN Collaborative Study

A system of communication called sign language makes use of visual motions and signals. The only form of communication for the deaf and dumb community and others with hearing impairments is sign language. Understanding sign language is so much difficult for a normal person. As a result, connecting with the wider public has always been extremely difficult for the minority community. In this study, we suggested a novel deep learning-based method for identifying sign language that can help normal and deaf individuals communicate more easily. In order to identify real-time sign language, we first created a dataset with 11 sign terms. Our bespoke CNN model was trained using these sign words. Prior to the CNN model being trained, we preprocessed the dataset. Our results show that the customized CNN model can attain the greatest accuracy of 98.6%.

Novel Wearable System to Recognize Sign Language in Real Time.

The aim of this study is to develop a practical software solution for real-time recognition of sign language words using two arms. This will facilitate communication between hearing-impaired individuals and those who can hear. We are aware of several sign language recognition systems developed using different technologies, including cameras, armbands, and gloves. However, the system we propose in this study stands out for its practicality, utilizing surface electromyography (muscle activity) and inertial measurement unit (motion dynamics) data from both arms. We address the drawbacks of other methods, such as high costs, low accuracy due to ambient light and obstacles, and complex hardware requirements, which have limited their practical application. Our software can run on different operating systems using digital signal processing and machine learning methods specific to this study. For the test, we created a dataset of 80 words based on their frequency of use in daily life and performed a thorough feature extraction process. We tested the recognition performance using various classifiers and parameters and compared the results. The random forest algorithm emerged as the most successful, achieving a remarkable 99.875% accuracy, while the naïve Bayes algorithm had the lowest success rate with 87.625% accuracy. The new system promises to significantly improve communication for people with hearing disabilities and ensures seamless integration into daily life without compromising user comfort or lifestyle quality.

Combinatorial Bionic Hierarchical Flexible Strain Sensor for Sign Language Recognition with Machine Learning.

Flexible strain sensors have been widely researched in fields such as smart wearables, human health monitoring, and biomedical applications. However, achieving a wide sensing range and high sensitivity of flexible strain sensors simultaneously remains a challenge, limiting their further applications. To address these issues, a cross-scale combinatorial bionic hierarchical design featuring microscale morphology combined with a macroscale base to balance the sensing range and sensitivity is presented. Inspired by the combination of serpentine and butterfly wing structures, this study employs three-dimensional printing, prestretching, and mold transfer processes to construct a combinatorial bionic hierarchical flexible strain sensor (CBH-sensor) with serpentine-shaped inverted-V-groove/wrinkling-cracking structures. The CBH-sensor has a high wide sensing range of 150% and high sensitivity with a gauge factor of up to 2416.67. In addition, it demonstrates the application of the CBH-sensor array in sign language gesture recognition, successfully identifying nine different sign language gestures with an impressive accuracy of 100% with the assistance of machine learning. The CBH-sensor exhibits considerable promise for use in enabling unobstructed communication between individuals who use sign language and those who do not. Furthermore, it has wide-ranging possibilities for use in the field of gesture-driven interactions in human-computer interfaces.

Manajemen Pendidikan Inklusi pada Anak Berkebutuhan Khusus di SMA Muhammadiyah 10 Surabaya

The results of the research show that inclusive education for children with special needs at SMA Muhammadiyah 10 Surabaya follows the curriculum determined by the government, therefore every month a joint evaluation meeting is held regarding the learning process which is carried out every month. The teaching and learning process requires good preparation and planning, this is none other than to achieve learning objectives. Inclusive Education for Children with Special Needs is a phenomenon that we encounter in society. The author also found this phenomenon in the Muhammadiyah 10 High School Surabaya environment where many students used sign language when they were with friends who were speech impaired and when they met friends who were not speech impaired they used Indonesian.

Continuous Sign Language Recognition System Using Deep Learning with MediaPipe Holistic

Continuous Sign Language Recognition System Using Deep Learning with MediaPipe Holistic

Alphabet Recognition of American Sign Language Using Machine Learning

A major challenge faced by our society is the difficulty individuals with disabilities encounter in expressing their emotions to others. People with disabilities often depend on sign (gesture) languages for communication. This project aims to develop a model capable of recognizing and translating sign language alphabets from hand gestures into text and audio. The primary objective is to enhance communication between individuals with hearing impairments and the broader society. We assessed our model's performance using a publicly accessible Indian Sign Language (ISL) dataset. For image classification, we employed Convolutional Neural Networks (CNNs) with the Inception V3 architecture. Hand gestures are captured through webcams, and our model identifies the corresponding alphabet for each gesture. This project seeks to overcome current challenges in Sign Language Recognition and aims to enhance its effectiveness and efficiency.

Comparison of the Accuracy of The Bahasa Isyarat Indonesia (BISINDO) Detection System Using CNN and RNN Algorithm for Implementation on Android

Communication is a process of exchanging information that aims to establish relationships between humans. Communication difficulties are an obstacle for people with deaf disabilities or often called Deaf Friends, where they find it difficult to interact with friends around them. Sign language is the main medium of communication used worldwide by people with disabilities i.e. deaf and speech impaired. Communication between deaf people and those around them is often an obstacle because most people do not understand sign language which is often used as a medium of communication by deaf people. In dealing with this problem, researchers want to analyze the accuracy level of the Android-Based Bahasa Isyarat Indonesia Detection System (BISINDO) using the Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) methods in order to determine which methods can be implemented to an Android device. This study shows that Convolutional Neural Network (CNN) has a greater and more stable accuracy rate compared to the Recurrent Neural Network (RNN) model where the CNN model produces an accuracy rate of 89%, and indicates that the ability to recognize images based on the division of Bahasa Isyarat Indonesia (BISINDO) alphabetic classes is good..

Artificial Intelligence Promoting the Inclusion of Students with Special Needs in Physical Education Classes

School Physical Education (PE), a crucial component for the integral development of students, faces the challenge of promoting the inclusion of students with special needs (SEN). Barriers such as the lack of adaptation of activities and the scarcity of adequate resources hinder the full participation of these students. This article investigates the potential of Artificial Intelligence (AI) as a tool to personalize teaching, adapt activities, and create inclusive learning environments in PE. Through the analysis of case studies and practical examples, it demonstrates how AI can be used to individually assess students, provide real-time feedback, develop adaptive platforms, recognize gestures and movements, create gamified activities, use virtual and augmented reality, translate sign language, implement virtual assistants, and prevent bullying. At the same time, the article discusses the ethical challenges and the need for teacher training to ensure the responsible and effective use of AI in promoting inclusion in PE.

Sworn Sign Language Interpreters Provision for Deaf People Appearing before Notary Based on Justice Principle

Sworn Sign Language Interpreters Provision for Deaf People Appearing before Notary Based on Justice Principle

Finding continuers in Swedish Sign Language

Abstract In this paper, I use methods from corpus linguistics and computer vision to find candidates for continuers – that is, conversational markers that signal comprehension and encouragement to the primary speaker/signer to continue – in a corpus of Swedish Sign Language (STS). Using different methods based on distributional patterns in conversational turns, I identify a small set of manual signs – particularly the sign JA@ub ‘yes’ – that exhibit the characteristics associated with continuers, such as occurring frequently in repeated sequences of overlapping but noncompetitive turns. The identified signs correspond to those found in previous research on manual backchannels in STS, demonstrating that quantitative, distribution-based approaches are successful in identifying continuers. In a second step, I employ methods from computer vision to analyze a subset of the corpus videos, and find that the continuer candidates show interesting form characteristics: they are small in visible articulation and thus conversationally unobtrusive by often being articulated low and with little movement in signing space. The results show that distribution-based approaches can be used successfully with sign language corpus data, and that the nature of continuers exhibits similarities across modalities of human language.

Linguistic landscape in the Inner Mongolia autonomous region: the case of a multilingual and multi-ethnic region in China

ABSTRACT This article focuses on the linguistic landscape of the Inner Mongolia Autonomous Region, investigates factors contributing to this distinctive landscape, and examines its role in maintaining two ethnic languages, Mongolian and Korean. 1046 language signs in three major cities were analysed and 28 interviews were conducted to evaluate interviewee attitudes towards the linguistic landscape. The results indicate that the Inner Mongolian linguistic landscape is one of abundant linguistic diversity, with the majority of signs (61.9%) displaying more than one language; that this landscape is influenced by factors such as language planning, language preferences, economic development needs, and tourist demands; and that Mongolian and Korean have high levels of visibility on both official and private signs. Mongolian and Korean interviewees expressed delight in seeing their ethnic languages displayed and believed that public displays of local languages could help maintain ethnic languages by providing a physical and visual representation of linguistic presence and ethnic identity.

Sign Tone: A Deep Learning-Based Deaf Companion System for Two Way Communication Between Deaf and Non-Deaf Individuals

Communication is essential to express and receive information, knowledge, ideas, and views among people, but it has been quite a while to be an obstruction for people with hearing and mute disabilities. Sign language is one method of communicating with deaf people. Though there is sign language to communicate with non-sign people it is difficult for everyone to interpret and understand. The performance of existing sign language recognition approaches is typically limited. Developing an assistive device that will translate the sign language to a readable format will help the deaf-mutes to communicate with ease to the common people. Recent advancements in the development of deep learning, deep neural networks, especially Temporal convolutional networks (TCNs) have provided solutions to the communication of deaf and mute individuals. In this project, the main objective is to design Deaf Companion System for that to develop SignNet Model to provide two-way communication of deaf individuals and to implement an automatic speaking system for deaf and mute people. It provides two-way communication for all classes of people (deaf-and-mute, hard of hearing, visually impaired, and non-signers) and can be scaled commercially. The proposed system, consists of three modules; the sign recognition module (SRM) that recognizes the signs of a deaf individual using TCN, the speech recognition using Hidden Marko Model and synthesis module (SRSM) that processes the speech of a non-deaf individual and converts it to text, and an Avatar module (AM) to generate and perform the corresponding sign of the non-deaf speech, which were integrated into the sign translation companion system called deaf companion system to facilitate the communication from the deaf to the hearing and vice versa. The proposed model is trained on Indian Sign Language. Then developed a web-based user interface to deploy SignNet Model for ease of use. Experimental results on MNIST sign language recognition datasets validate the superiority of the proposed framework. The TCN model gives an accuracy of 98.5%..

Sign Language, the Key to Improve the Health Knowledge of Deaf People

Background:Health literacy is one of the determinants of an individual's health. Deaf people, as a social minority, have limited access to information sources and healthcare providers. The present study aimed to examine the barriers, facilities and strategies of Deaf people health literacy promotion from points of view of this population and informants. Methods: A qualitative study using conventional content analysis was conducted. Focus group discussion sessions were held with 14 individuals (8 were men) with different occupational positions such as linguists, interpreter advisor, legal experts and heads of Deaf institutes. The participants were selected using a purposeful sampling method. Five FGD sessions were held. Data analysis accomplished according to conventional content analysis.