Learning Sign Language Research Articles

Effect of sign language learning on temporal resolution of visual attention.

The visual environment of sign language users is markedly distinct in its spatiotemporal parameters compared to that of non-signers. Although the importance of temporal and spectral resolution in the auditory modality for language development is well established, the spectrotemporal parameters of visual attention necessary for sign language comprehension remain less understood. This study investigates visual temporal resolution in learners of American Sign Language (ASL) at various stages of acquisition to determine how experience with sign language affects perceptual sampling. Using a flicker paradigm, we assessed the accuracy of identifying out-of-phase visual flicker objects at frequencies up to 60Hz. Our findings reveal that third-semester ASL learners show increased accuracy in detecting high-frequency flicker, indicating enhanced temporal resolution. Interestingly, as learners achieve higher proficiency in ASL, their perceptual sampling reverts to typical levels, likely because of a shift toward predictive processing mechanisms in sign language comprehension. These results suggest that the temporal resolution of visual attention is malleable and can be influenced by the process of learning a visual language.

Where Can I Sign My Language?: A Systematic Literature Review (2000–2019) of Young People's Access to Sign Language Socialisation Spaces in the Nordic Countries

This article presents the results from a systematic literature review of Nordic research studies from 2000 to 2019, addressing the question of children and adolescents' access to spaces where they can sign with peers and adults. The study is based on a dialogistic approach to language, and holds that languaging is inextricably tied to places and situations—that is, spaces and contexts in which people participate in interactions with others. Thus, acquiring sign language is not only about the child or adolescent's individual learning of sign language in an educational setting, but also about participation in sign language activities. The review addresses the paradoxical situation in the field of sign language education whereby formal recognition of sign language and the institutionalisation of rights to sign language education gained momentum at around the same time as the number of deaf schools began to decline. The results from the review indicate that very few studies of sign language pay attention to spaces where children and adolescents can interact with other people through sign language. It appears that current legislation, as well as research, emphasises the individual right to learn sign language for deaf and hard-of-hearing children, and has overlooked the significance of facilitating and protecting sign language-based socialisation spaces as significant sites for sign language learning, participation and acquisition.

Malaysian Sign Language Detection with Convolutional Neural Network for Disabilities

Sign language is the main form of communication used by deaf people. Most of their activities like speaking and learning involved sign language. In Malaysia, instead of a deaf person, only a few people know sign language because it's not part of the co-curriculum. Hence, normal people need to know sign language so that they can communicate with deaf people effectively. However, learning sign language requires a lot of effort and takes years of study. Therefore, this research tends to initiate the sign language image detection system that utilizes the deep learning model. The initial effort involves the development of an image detection system that can identify the alphabet of sign language from A to Z. There will be 26 alphabets that contribute to 26 classes of image detection using the Convolutional Neural Network. 85 images will be collected for each alphabet and the total data will be distributed into 80% and 20% for training and testing the CNN model, respectively. The result showed that the proposed system produced more than 95% detection accuracy of the alphabet based on finger sign gestures. For future works, the proposed system will be evaluated with a more complex dataset in terms of words.

Value-based and goal-setting orientations of pre-service Russian sign language interpreters

Introduction. The relevance of research into the problem of training Russian sign language (RSL) interpreters for the university bachelor’s degree is ensured by the government procurement for the provision of translation services for people with hearing disabilities in public offices and state agencies, which is compliant with the UN principles of providing equal opportunities for disabled people to actively participate in the life of society. During the period of university studies students are supposed to learn sign language, master translator’s skills and become aware of the specificity of communication with representatives of deaf society. To succeed in achieving these results students should clearly understand what they study and how it is related to their values and future profession. In other words, they are supposed to have goals which university program can help to achieve. Thus, the aim of the research is to analyze valuebased and goal-setting orientations of future RSL interpreters and their perception of the professionally important qualities of an RSL interpreter. Materials and methods. The research was carried out at Novosibirsk state technical university (Russian Federation). The samples were the first-year and fourth-year students majoring in Linguistics, specialty ‘English language and Russian sign language interpreter’. The data collection was done by means of a survey based on the questionnaire with open ended questions and Rokeach Value survey. Results. The choice of sample groups of the first-year and the fourth-year students was determined by the need to compare educational goals which students set at the start of the university studies and educational goals of those who have covered most part of the program and are close to graduation. The analysis of the received results reveals that the goals of the senior students are more focused on professional activity – ‘to master profession through practice’ (61%), while the goals of the freshmen are more orientated at mastering English (36%) and learning new sign language (23%). It is worth noting that 7% of the first-year students and 7.5% of the fourth-year students have the goals that are not related to studying in the program, and after graduation they are not going to work in the field of RSL interpreting. Moreover, students have rather inconsistent understanding of professionally important qualities of an RSL interpreter. The results of the Rokeach Value survey test demonstrate that students’ terminal and instrumental values correlate with their answers to the questionnaire. The instrumental values students ranked as the most important demonstrate stronger focus on self than on other people. Conclusion. The findings suggest insufficiently developed students’ skills of goalsetting and defining the means of goal attaining. The future RSL interpreters need to develop goalsetting skills that could help them formulate the strategies of setting personally and professionally important goals and define the ways of their attainment. The continuation of present research is seen as an action research into the problem of the pre-service RSL interpreters’ goalsetting.

Mobile App Prototype Supporting People With Special Needs

The article analyses current mobile applications designed for people with special needs. They are aimed at improving routine daily activities and services non-disabled users would not experience significant difficulties. The purpose of the article is to develop and promote a mobile application for people with impaired vision or hearing. In this paper, a mobile application prototype has been designed and developed that would also be useful for anyone wishing to learn sign language. It is compatible with both Android and iOS operating systems. The methodology involves conducting preliminary research and interviews with potential clients. A survey on current mobile applications in the targeted area has been carried out. Functionalities offered by the prototype ensure that the needs and requirements of 'different' users are largely met. As for non-disabled users, the app has the potential to enhance their communication with those specific user groups.

Gesture recognition technology: a new dimension in human-computer interaction interface

Development of an interface for intelligent gesture control to improve user experience and increase the efficiency of interaction with a computer. This paper proposes a gesture recognition system based on artificial intelligence using convolutional neural networks (CNN). The system comprises three stages: pre-processing, optimal frame determination, and gesture category identification. The extracted features used are independent of movement, scaling, and rotation, providing greater flexibility to the system. The suggested gesture control technology, known as Kazakh Sign Language (KSL) for Kazakh alphabets, eliminates the need for additional devices, enabling users to interact with the system naturally. Experiments demonstrated that the proposed KSL system can accurately recognize Kazakh language alphabet letters with a high precision of 97.3%, owing to the utilization of artificial intelligence and CNN to enhance the accuracy and effectiveness of gesture control. Gestures, a type of visual formation, are perceivable by computers through machine learning models. The selection of methods and systems for recognizing Kazakh sign language gestures was accompanied by addressing various challenges related to language-specific orthographic and gestural features. The developed gesture control interface for human-computer interaction is applied in the field of inclusive education, aiming to assist deaf and hard-of-hearing children in learning sign language.

Exploring positional and dimensional aspects of topographic space for advanced-level British Sign Language learners

Abstract This study investigates how British Sign Language (BSL) learners develop positional and dimensional aspects of topographic space. The teaching of BSL has been occurring at an increasing rate, and many people are learning to use BSL to the advanced levels, which are generally referred to in the UK as BSL Levels 4 and 6 and are loosely equivalent to pre-C1 (advanced) and C1 (proficient), respectively, on the Common European Framework for Reference (Languages) scale. Spatial grammar is a crucial aspect of BSL (Brennan 1992), and this article provides insight into issues related to learning how to use topographic space, a feature of spatial grammar whereby real-world referent locations are replicated in the signing production. This study of L2 advanced learners of BSL explores the numbers and types of errors that are made when applying topographic features into their signing, with a particular focus on positioning and dimensionality aspects. Two scenarios are used do this: classroom layouts (Task 1) and a courtroom layout (Task 2). The study concludes with reflections on how BSL teachers can support L2 learners in improving their development of topographic skills.

Enhancing word signing in hearing students with reading disorders using computer-based learning trials.

Multiple-baseline-across-word-sets designs were used to determine whether a computer-based intervention would enhance accurate word signing with four participants. Each participant was a hearing college student with reading disorders. Learning trials included 3 s to observe printed words on the screen and a video model performing the sign twice (i.e., simultaneous prompting), 3 s to make the sign, 3 s to observe the same clip, and 3 s to make the sign again. For each participant and word set, no words were accurately signed during baseline. After the intervention, all four participants increased their accurate word signing across all three word sets, providing 12 demonstrations of experimental control. For each participant, accurate word signing was maintained. Application of efficient, technology-based, simultaneous prompting interventions for enhancing American Sign Language learning and future research designed to investigate causal mechanisms and optimize intervention effects are discussed.

A machine learning-driven web application for sign language learning.

Addressing the increasing demand for accessible sign language learning tools, this paper introduces an innovative Machine Learning-Driven Web Application dedicated to Sign Language Learning. This web application represents a significant advancement in sign language education. Unlike traditional approaches, the application's unique methodology involves assigning users different words to spell. Users are tasked with signing each letter of the word, earning a point upon correctly signing the entire word. The paper delves into the development, features, and the machine learning framework underlying the application. Developed using HTML, CSS, JavaScript, and Flask, the web application seamlessly accesses the user's webcam for a live video feed, displaying the model's predictions on-screen to facilitate interactive practice sessions. The primary aim is to provide a learning platform for those who are not familiar with sign language, offering them the opportunity to acquire this essential skill and fostering inclusivity in the digital age.

Interactive system based on artificial intelligence and robotic arm to enhance arabic sign language learning in deaf children

Interactive system based on artificial intelligence and robotic arm to enhance arabic sign language learning in deaf children

User Interface Design Analysis of Android-Based Sign Language Recognition Learning Media at Special Schools

This research aims to conduct an in-depth analysis of the user interface (UI) design of Android-based sign language recognition learning media used in special schools. In the context of educational inclusion, sign language learning plays an important role in facilitating effective communication for individuals with hearing loss. The use of mobile technology, such as Android applications, has offered an attractive solution to enhance sign language learning by providing flexible and affordable access. However, the UI design of sign language learning apps does not always adequately consider the special needs of users, especially students with hearing impairment. Therefore, this research aims to analyze the UI design of existing Android-based sign language learning apps, focusing on accessibility, intuitiveness, and effectiveness in delivering learning content. This research uses the Research and Development (R&D) method with the Waterfall model approach. This research method involves analyzing various aspects of UI design, including layout, navigation, use of symbols, and interactivity. Data was collected through direct observation of student use of the application and interviews with educators in special schools. Data analysis was conducted using qualitative and quantitative approaches to identify strengths and weaknesses in the app's UI design. The results showed that of the four instruments used in analyzing the UI design of this "AkuBisa" learning media application, using a Likert scale and a Guttman scale, the following results were obtained: needs analysis instrument with an average value of 3.45; media development storyboard instrument with an average value of 3.37; media display design instrument with an average value of 3.31; and Blackbox testing instrument with a percentage level of functionality reaching 100%. The results of testing and assessment of the four instruments show that the user interface design of android-based sign language recognition learning media is declared very valid in fulfilling the elements of accessibility, intuitiveness, and effectiveness as a learning media application and is very feasible to use as an alternative learning media for sign language recognition in special schools.

Hearing parents as sign language learners: describing and evaluating the ASL skills of parents learning ASL with their deaf children

ABSTRACT Hearing parents who endeavour to learn American Sign Language (ASL) to communicate with their deaf children are a unique but understudied population of adult language learners. In this study, we assessed the expressive and receptive ASL skills of hearing parents of deaf children (n = 55). Expressive ASL skills were evaluated through elicited language samples, and scored using adapted rubrics from the Common European Framework of Reference for Languages. For a subset of participants (n = 20), an additional language sample was obtained and coded for lexical and grammatical features of ASL. Receptive ASL skills were measured using the ASL-Comprehension Test. Across measures, participants demonstrated language skills in the beginner to intermediate range. Parents’ signing samples showed a range of ASL linguistic features including classifier constructions. Our findings suggest that hearing parents can learn ASL and should be encouraged to do so to support language development in their deaf children.

Multi-culture Sign Language Detection and Recognition Using Fine-tuned Convolutional Neural Network

The speech and hearing-impaired community use sign language as the primary means of communication. It is quite challenging for the general population to interpret or learn sign language completely. A sign language recognition system must be designed and developed to address this communication barrier. Most current sign language recognition systems rely on wearable sensors, keeping the recognition system unaffordable for most individuals. Moreover, the existing vision-based sign recognition frameworks do not consider all of the spatial and temporal information required for accurate recognition. A novel vision-based hybrid deep neural net methodology is proposed in this project for recognizing American Sign Language (ASL) and custom sign gestures. The proposed framework aims to establish a single framework for tracking and extracting multi-semantic properties, such as non-manual components and manual co-articulations. Furthermore, spatial feature extraction from the sign gestures is deployed using a Hybrid Deep Neural Network (HDNN) with atrous convolutions. The temporal and sequential feature extraction is carried out by employing attention-based HDNN. In addition, the distinguished abstract feature extraction is done using modified autoencoders. The discriminative feature extraction for differentiating the sign gestures from unwanted transition gestures is done by leveraging the hybrid attention module. The experimentation of the proposed model has been carried out on the novel multi-signer ASL and custom sign language dataset. The proposed sign language recognition framework with hybrid neural nets, specifically using HDNN, yields better results than other state-of-the-art frameworks. Additionally, a detection module is incorporated using Flask web, allowing manual input of images/signs for real-time recognition.

Unravelling Receptive Language Acquisition: A Comprehensive Study of Deaf Students' English Language Acquisition

This study aims to explore in-depth understanding of receptive language acquisition and English language acquisition of deaf students. The focus of the research is on the aspects of receptive language acquisition and English language acquisition of deaf students. This research uses the literature analysis method which involves the study of scientific articles, research reports, and other related literature at the national and international levels that relate to the receptive language acquisition and English language acquisition of deaf students. The data was analysed qualitatively with a descriptive approach. The findings of this study are that deaf children have great potential, despite facing barriers in language acquisition. they tend to have a visual learning type, and the use of various learning media can help improve language acquisition. A total communication approach, which involves multiple forms of communication, can help in deaf children's first language acquisition. In addition, factors such as the degree of deafness, hearing aid use, and family background also affect their language development. In foreign language learning, the use of national sign language and effective learning strategies can help deaf students overcome difficulties. And the importance of co-operation between teachers, students, parents and the learning environment is crucial in helping deaf children overcome their language challenges.

IMPLEMENTASI SENSOR FLEX PADA PENYANDANG DIFABEL MENGGUNAKAN DFPLAYER SEBAGAI INFORMASI AUDIO

Sign language is a form of language that can be learned which prioritizes communication with body language, facial expressions, and some non-sounds. This sign language is widely used by the deaf and mute. However, there is still a lack of institutions teaching sign language, books on sign language, and a tool for learning sign language. This study aims to design a hardware system that can translate sign language and output in the form of audio and display on the LCD. The stages of the research included designing the physical form of the device, testing the sensor flex, testing the MPU6050, testing the Nrf24l01, testing the Dfplayer, testing the Liquid Crystal Display (LCD), and testing the overall device on the A-Z alphabet. The results showed that when the Flex sensor was not indented the average error was 1.23%, and when it was indented the average error was 2.01%. The delay time required in sending data using Nrf24l01 media for conditions without obstructions is 0 seconds with a distance of 1 meter, with a barrier of 9.84 seconds with a distance of 3 meters, at a distance of 3 meters the data cannot be sent.

Marathi to Indian Sign Language Machine Translation

Machine translation has been a prominent field of research, contributing significantly to human life enhancement. Sign language machine translation, a subfield, focuses on translating spoken language content into sign language and vice versa, thereby facilitating communication between the normal hearing and hard-of-hearing communities, promoting inclusivity. This study presents the development of a ‘sign language machine translation system’ converting simple Marathi sentences into Indian Sign Language (ISL) glosses and animation. Given the low-resource nature of both languages, a phrase-level rule-based approach was employed for the translation. Initial encoding of translation rules relied on basic linguistic knowledge of Marathi and ISL, with subsequent incorporation of rules to address 'simultaneous morphological' features in ISL. These rules were applied during the ‘generation phase’ of translation to dynamically adjust phonological sign parameters, resulting in improved target sentence fluency. The paper provides a detailed description of the system architecture, translation rules, and comprehensive experimentation. Rigorous evaluation efforts were undertaken, encompassing various linguistic features, and the findings are discussed herein. The web-based version of the system serves as an interpreter for brief communications and can support the teaching and learning of sign language and its grammar in schools for hard-of-hearing students.

Interactive learning of natural sign language with radar

AbstractOver the past decade, there have been great advancements in radio frequency sensor technology for human–computer interaction applications, such as gesture recognition, and human activity recognition more broadly. While there is a significant amount of study on these topics, in most cases, experimental data are acquired in controlled settings by directing participants what motion to articulate. However, especially for communicative motions, such as sign language, such directed data sets do not accurately capture natural, in situ articulations. This results in a difference in the distribution of directed American Sign Language (ASL) versus natural ASL, which severely degrades natural sign language recognition in real‐world scenarios. To overcome these challenges and acquire more representative data for training deep models, the authors develop an interactive gaming environment, ChessSIGN, which records video and radar data of participants as they play the game without any external direction. The authors investigate various ways of generating synthetic samples from directed ASL data, but show that ultimately such data does not offer much improvement over just initialising using imagery from ImageNet. In contrast, an interactive learning paradigm is proposed by the authors in which model training is shown to improve as more and more natural ASL samples are acquired and augmented via synthetic samples generated from a physics‐aware generative adversarial network. The authors show that the proposed approach enables the recognition of natural ASL in a real‐world setting, achieving an accuracy of 69% for 29 ASL signs—a 60% improvement over conventional training with directed ASL data.

Signease: Empowering Seamless Communication for the Hearing Impaired

Abstract: There are 2.78% of the total population of India who can’t speak. Sign language is a nonverbal form of communication method which is found among all deaf and dumb communities in the world. Normal people do not learn sign language. It causes barriers in communication between deaf dumb and normal people. Hence Deaf-Mute communication requires an interpreter who will convert hand gestures into auditory speech. The past implementation of this project involved using image processing concepts. But the drawback of these implementations is that projects were non portable and too expensive. Therefore, the system is being proposed with the use of flex sensors, accelerometer and android technology. System includes two modules. First module is a hand glove with sensors and microcontroller to convert hand gestures to auditory speech. Second module is an Android App with Google Speech API to convert text signal to speech.

Augmented Reality Technology in Designing Sign Language Learning Applications

Augmented Reality Technology in Designing Sign Language Learning Applications

Feeling signs: motor encoding enhances sign language learning in hearing adults.

Manual production enhances learning and recall of signs by hearing second language learners; however, the mechanisms enabling this effect are unclear. We examined whether the motor encoding (somatosensory feedback) that occurs during sign production benefits learning and whether it interacts with sign iconicity, which also enhances learning. American Sign Language (ASL) signs varying in iconicity were learned either via production (repetition) with the eyes closed or via observation without production. Signs learned via production were recalled more accurately than signs learned via observation, indicating that motor encoding from manual production enriches the representations of signs. Moreover, the effect of motor encoding interacted with iconicity, suggesting that motor encoding may particularly enhance the recall of signs low in iconicity. Together, these results reveal the importance of somatosensory feedback as a key mechanism underlying the beneficial effect of production on sign learning, demonstrating that feeling one's own signing promotes learning and recall of signs.