Learning Sign Language Research Articles

Neural effects differ for learning highly iconic versus non-iconic signs in hearing adults.

Little is known about the neural changes that accompany sign language learning by hearing adults. We used ERPs and a word-sign matching task to assess how learning impacted the N400 priming effect (reduced negativity for translations compared to unrelated trials). English monolinguals (N = 32) learned 100 ASL signs - half highly iconic (meaning was guessable), half non-iconic. In contrast to non-iconic signs, little learning was needed for the highly iconic signs as translation accuracy was similar pre- and post-learning. Prior to learning, an N400 priming effect was observed only for iconic signs. After learning, the size of the priming effect increased for non-iconic signs (replicating word learning studies) but decreased for iconic signs. For deaf ASL signers (N = 20), iconicity did not modulate the size of the N400 priming effect. We conclude that the impact of iconicity on lexico-semantic processing is reduced following learning, as signs are integrated into an emerging visual-manual lexicon.

Design and Development of Sign Language Learning Application for Special Needs Students Based on Android Using Flutter

Sign language is a communication system used by individuals with hearing impairments to interact and convey messages without using verbal sounds. Sign language has its own unique grammatical structure and vocabulary, enabling sign language users to express ideas and emotions through facial expressions, hand movements, and body positions. In this research, we explore important aspects of sign language, including its uniqueness, complexity, and diversity across different countries and cultures. We also discuss the role of sign language in facilitating communication, education, and social inclusion for the deaf community. This research highlights the significance of introducing and developing sign language competence among the general public, educators, and professionals in the fields of healthcare and social services. The development of the sign language learning media utilizes the Multimedia Development Life Cycle method. The main application used in creating the sign language learning media is Flutter, along with supporting applications such as Adobe Illustrator 2020 and Adobe Photoshop CC 2019. The features included in this application are finger alphabet signs, number signs, video tutorials for numbers and letter signs, quizzes, and learning objectives.

Design of Sign Language Learning Media Based on Virtual Reality

Sign language has become a very important means of communication for individuals with hearing loss, allowing them to interact with people who are not hearing impaired. In recent years, virtual reality (VR) technology has advanced rapidly, opening up new opportunities to enhance the user experience in various situations. This study aims to design and develop an easy-to-use virtual reality-based interactive sign language application. The software development method used is the Software Life Cycle Development Method (MDLC), which includes the stages of analysis, design, implementation and testing. This app is designed to run on affordable and popular VR devices to make it accessible to as many individuals as possible. In the development process, we conducted an analysis of various sign languages used globally, focusing on hand gestures and facial expressions which have an important role in sign language communication. Users can learn and practice sign language through interactive experiences in a virtual environment that mimics real-life situations. This application is also equipped with visual and audio feedback features to help users improve and hone their skills. At the testing stage, this application was evaluated through trials involving participants with diverse backgrounds in the use of sign language. The test results show that users feel involved in the VR experience and experience increased confidence in using sign language after using this application. This research proves that VR technology can be an effective tool in sign language learning and development. Thus, the virtual reality-based sign language application that has been designed and developed in this study makes a significant contribution in increasing the accessibility of communication for individuals with hearing impairments. The implications of this research can be the basis for further development in the field of sign language and VR technology.

Sign language in immersive virtual reality: design, development, and evaluation of a virtual reality learning environment prototype

ABSTRACT Non-Verbal Cues (NVCs) add to communication effectiveness among individuals in both real and virtual world. Thus, NVCs transference between the two receives increased attention from both the industry and research community. Their efforts lead to sophisticated technological solutions which allow high fidelity NVCs to be transferred from real individuals to virtual characters/avatars, in real-time, adding to the communication effectiveness of both standalone Immersive Virtual Reality (IVR) applications and Social Virtual Reality (SVR) platforms. That combination triggered the development of Virtual Reality Learning Environments (VRLEs), aiming at assisting students towards learning sign language and communicating through it in both virtual and real world. Still, technological limitations affect the quality of such learning and communication experience. This work presents the design, development, and evaluation of an SVR compatible system for sign language learning and communication in virtual worlds. The system considers lessons learned from previous studies and was used to create a VRLE mimicking learning and examination processes students of sign language follow in real life. The results of our study show that despite the appreciation of technological solutions efficiency towards NVCs transference, there are still several issues to be investigated to increase the effectiveness of communication through sign language in virtual worlds.

Self‐Repair in Hearing L2 Learners’ Spontaneous Signing: A Developmental Study

AbstractThis study presents a corpus‐based investigation of self‐repairs in hearing adult L2 (M2L2, second modality and second language) learners of Swedish Sign Language (Svenskt teckenspråk, STS). This study analyses M2L2 learners’ STS conversations with a deaf signer and examines the learners’ self‐repair practices and whether there are differences among learners of different proficiency levels. This provides a description of characteristics of self‐repair made by M2L2 learners as well as the frequency and distribution of self‐repair categories. The results show that the frequency of self‐repair decreases with increased proficiency, at least after the initial stage. Furthermore, the self‐initiated repair categories of the beginners are often phonological repairs, while intermediate learners tend to carry out self‐repairs at the lexical and syntactic level. The results also reveal a specific type of STS repair linked to fingerspelling repairs. We discuss the effects of second modality learning as well as the relationship between monitoring and language proficiency.

Pengoptimalan Komunikasi Bahasa Isyarat Abjad dengan Augmented Reality

One of the activities that humans always do is communication. This communication is carried out by everyone, but there are special ways of communication that are carried out by people with special needs, such as the deaf. The communication uses sign language, where learning the language is quite difficult, especially for students who are deaf. There is a solution that can solve this problem, namely the use of technology. One of the technologies that can be used is Augmented Reality. Augmented Reality is able to combine the physical world and digital elements that can be used to facilitate human activities. Augmented Reality is designed and packaged in the form of a mobile-based application to make it easier to learn sign language. The use of this designed application is specifically aimed at students with special needs. In addition, the use of the application can also be felt by a group of people who are learning sign language. Application testing involves various aspects, such as functional testing, device compatibility testing, performance testing, AR accuracy and compatibility testing, interaction testing, usability testing, and testing in real environments. App testing engages users, either in user testing or through the feedback they provide. The users involved are divided into 2, namely users as experts in AR testing and users as customers or in this case teachers. The test results prove the value with a high percentage. Tests by experts achieve an average value of 93.13% and tests by customers achieve an average value of 93.60%. This average value can be classified as a reference that the application functions properly, according to needs, is safe, responsive, and provides a satisfying experience.

The Acquisition of Strategies to Express Plurality in Hearing Second Language Learners of Sign Language of the Netherlands

AbstractThis study reports on strategies to indicate plural referents in hearing learners of Sign Language of the Netherlands. This is the first explorative study that focuses on L2 expressions of plurality in a sign language. Using data from two datasets, I examined when learners start to express plural and which strategies they apply, and I noted typical learner characteristics. The first study examined spontaneous conversations of three learners, during the first 18 months of their learning. The second study analyzed elicited data from 11 learners during their first year of learning. The data reveal that learners are able to express plural referents in early stages, using strategies that are familiar to them (quantifiers) as well as strategies that do not occur in their mother tongue (reduplication of the noun, use of spatial devices). The early emergence might be explained by the salient nature of the devices and the resemblance with gestural portrayals.

Indonesian Sign Language (SIBI) Learning Media Application Based on Deep Learning Technology for Deaf Children

One of the main methods in teaching deaf children effectively is through visualization of learning materials. Previous research has developed augmented reality-based interactive learning media for deaf children. However, these learning media have limitations related to smartphone memory, dependence on certain flashcards, and limited amount of material due to flashcard limitations. Here, researchers innovate to develop SIBI learning media applications based on deep learning technology that material does not depend on a specific flashcard, but the application is able to detect real objects and various images. TensorFlow Lite is considered lighter but still allows the system to run deep learning on mobile devices with low latency. This application is designed to capture a real object or image through the camera, then the object or image will be processed and recognized using TensorFlow Lite. The object recognition process is carried out at the classification layer using switch-case syntax and global class synchronization with the YouTube API. After the image or object is recognized, the system will issue an output in the form of an Indonesian sign language video link from the object.

The Role of Modality in L2 Learning: The Importance of Learners Acquiring a Second Sign Language (M2L2 and M1L2 Learners)

AbstractSecond language acquisition (SLA) research offers valuable insight on how languages are learned and how they coexist and influence each other. Sign language learners offer unique perspectives on SLA, allowing researchers to test theories that are otherwise constrained by access to only one modality. Current literature on sign language learning focuses primarily on bimodal bilinguals, mostly hearing adults learning their first sign language (M2L2 learners). However, other groups of L2 signers exist, including deaf learners who have previously acquired a sign language and are learning a new one (M1L2 learners). M1L2 acquisition offers unique insights into complex interactions including multilingualism, modality, and timing of acquisition. We argue that M1L2 signers are a key comparison group for investigations of various L2 and so‐called modality effects and also represent a crucial test case for re‐examining the traditional constructs of “native speaker/signer” and the effects of initial language delay or deprivation on subsequent language acquisition.

Mathematics and Sign Language Learning with a Tangible Game: An Inclusive Approach for DHH and Hearing Children

The purpose of this study was to investigate the potential of a tangible game, Inclusive Glossary of Mathematical Terms (GIM), to support the teaching of math and Portuguese sign language (Língua Gestual Portuguesa; LGP) to preschool and first-grade children, with an emphasis on inclusive education, knowledge acquisition, and engagement. GIM is composed of physical cards that are complemented by digital videos that run on an interface designed for this game. The game consists of two sets of cards and respective animations. As an inclusive math glossary, it also includes videos where concepts, characters, and objects are identified through images, words, and sign language. During the development process of GIM, it was possible to involve a multidisciplinary team, of which a Deaf sign language teacher and a sign language interpreter were part to ensure the clarity of the sign language and refine the written sentences accompanying the glossary. In order to validate its contents, a focus group of preschool and primary teachers specialized in the teaching of Deaf and Hard of Hearing (DHH) children was formed. A total of 120 children, aged between three and eight years old (M = 5.61; SD = 0.78), and seven LGP and special education teachers from three Portuguese schools participated in the study. The children used GIM in their classrooms, with the assistance of the participating teachers, who were subsequently interviewed to assess their engagement, interest, and learning outcomes. The interviews were analyzed using content analysis procedures, including descriptive statistics and Pearson correlations. The results indicated that GIM has the potential to effectively support the teaching of mathematics and LGP to preschool and first-grade children, especially in inclusive education contexts. The tangible nature of the game facilitated children's understanding of mathematical concepts and LGP, leading to improved involvement in the teaching-learning process. Moreover, the game promoted awareness of diversity and the principles of inclusive education, making it a valuable tool for promoting social inclusion and understanding. However, the study has reduced sample and context-specific limitations, suggesting a need for further research in this area.

Sign language experience has little effect on face and biomotion perception in bimodal bilinguals

Sensory and language experience can affect brain organization and domain-general abilities. For example, D/deaf individuals show superior visual perception compared to hearing controls in several domains, including the perception of faces and peripheral motion. While these enhancements may result from sensory loss and subsequent neural plasticity, they may also reflect experience using a visual-manual language, like American Sign Language (ASL), where signers must process moving hand signs and facial cues simultaneously. In an effort to disentangle these concurrent sensory experiences, we examined how learning sign language influences visual abilities by comparing bimodal bilinguals (i.e., sign language users with typical hearing) and hearing non-signers. Bimodal bilinguals and hearing non-signers completed online psychophysical measures of face matching and biological motion discrimination. No significant group differences were observed across these two tasks, suggesting that sign language experience is insufficient to induce perceptual advantages in typical-hearing adults. However, ASL proficiency (but not years of experience or age of acquisition) was found to predict performance on the motion perception task among bimodal bilinguals. Overall, the results presented here highlight a need for more nuanced study of how linguistic environments, sensory experience, and cognitive functions impact broad perceptual processes and underlying neural correlates.

Comparative Reflections on the Acquisition of Language in Hearing and Deaf Children: A Case of Natural Learning of Mexican Sign Language

Sign languages are visual and iconic languages used by Deaf communities worldwide. Sign language develops from the linguistic stimulus in the visuo-gestural modality, unlike hearing children who receive the stimulus in the auditory-vocal modality. This paper presents one case study in Mexico where deaf children naturally acquire Mexican Sign Language (LSM). Deaf children can access and develop sign language if the immediate context offers ways parallel to oral language development. Universal Grammar and biolinguistics support the natural process of the acquisition of languages.

Dreaming of the sleep lab is associated with improved performance on a sign language learning task: A pilot study.

Dreaming of the sleep lab is associated with improved performance on a sign language learning task: A pilot study.

Deep Learning in Sign Language Recognition: A Hybrid Approach for the Recognition of Static and Dynamic Signs

A speech impairment limits a person’s capacity for oral and auditory communication. A great improvement in communication between the deaf and the general public would be represented by a real-time sign language detector. This work proposes a deep learning-based algorithm that can identify words from a person’s gestures and detect them. There have been many studies on this topic, but the development of static and dynamic sign language recognition models is still a challenging area of research. The difficulty is in obtaining an appropriate model that addresses the challenges of continuous signs that are independent of the signer. Different signers’ speeds, durations, and many other factors make it challenging to create a model with high accuracy and continuity. For the accurate and effective recognition of signs, this study uses two different deep learning-based approaches. We create a real-time American Sign Language detector using the skeleton model, which reliably categorizes continuous signs in sign language in most cases using a deep learning approach. In the second deep learning approach, we create a sign language detector for static signs using YOLOv6. This application is very helpful for sign language users and learners to practice sign language in real time. After training both algorithms separately for static and continuous signs, we create a single algorithm using a hybrid approach. The proposed model, consisting of LSTM with MediaPipe holistic landmarks, achieves around 92% accuracy for different continuous signs, and the YOLOv6 model achieves 96% accuracy over different static signs. Throughout this study, we determine which approach is best for sequential movement detection and for the classification of different signs according to sign language and shows remarkable accuracy in real time.

Between audio and visuality – on the difficulties of learning sign languages on the basis of culinary lexis in the Polish Sign Language

The article focuses on the issue of audio and sign bilingualism – a topic that has not yet been thoroughly covered in Polish source literature. The most frequent research in the field of glottodidactics (understood as a specially developed plan that aims to implement literacy in the youngest children, by adapting to their mental, physical and emotional abilities.) focuses on bilingual people who use phonic languages (unimodal bilingualism – phonic). In terms of bilingualism of the deaf, one can speak of either unimodal bilingualism (communicating in two sign languages) or bimodal, sign-print bilingualism, in which a deaf person communicates both in a sign language and a phonic language (Polish) in its written form[1].Different modalities between phonic-modality Polish and the sign language, which is based on visual and spatial modalities, constitute difficulties both for people with hearing impairments, as well as hearing Poles willing to learn a sign language. The further section of the text examines difficulties involved in the acquisition of culinary lexis that may be experienced by hearing people attempting to learn a sign language, as well as deaf people who want to learn Polish.

Development of sign language videos to improve mathematical communication of deaf students of fractional material

As many as 7.03% of people with disabilities in Indonesia are deaf people. The problem for deaf students is the lack of mathematics learning resources during learning and the low mathematical communication skills of deaf students. The purpose of this study is to analyze the development of sign language learning videos to improve mathematical communication of deaf students of fractional material in terms of validity, practicality, and effectiveness. The method used is the R&D method with a 4D model. The results showed that the validity of the media resulted in a very valid category by sign language experts with a score of 85% and material experts with a score of 83% and categories valid by media experts with a score of 78.5%. The results of media practicality of 100% for teacher and student responses and learning implementation questionnaires are said to be very practical. The results of the control class pretest and the experiment resulted in a sig value. > 0.05, then the data is normal. Homogeneity test with (6,14) < (19) on pretest questions and (4,68) < (19) on posttest questions then homogeneous data. Student scores resulted in a difference in the average student learning outcomes between the experimental class and the control class indicated by the T test results with the sig value. (0.001) < 0.05. The average value of the N Gain of the experimental class is greater than that of the control class, which is 0.9721 with a high category. so that the media is said to be valid, practical, and effective.

JengASL: A Gamified Approach to Sign Language Learning in VR

Learning sign language has many advantages ranging from being able to communicate with millions of hearing impaired people, to improving cognitive function and communication skills. Sign language is recognised as an official language in 74 countries, including Germany, Japan, and the UK. Despite that only a small percentage of people attempt to learn sign language. In this research we investigate how virtual reality and gamification can be used to make learning sign language more enjoyable and motivating. We present JengASL, a gamified approach using 3D hand models, gesture recognition, and interactive gameplay in Virtual Reality to teach American Sign Language. We evaluate this system with a pilot study using eight participants and found that while it is less effective for sign memorisation than traditional 2D image-based learning methods, learning is more, but not significantly more, enjoyable and motivating.

PELATIHAN DAN PENDAMPINGAN MEDIA PEMBELAJARAN BAHASA ISYARAT DI SLB NEGERI 1 PALOPO

Education is a universal right, including for children with special characteristics such as deaf students at SLB Negeri 1 Palopo. To support their learning, the use of interactive learning media based on information and communication technology, such as Augmented Reality (AR), is an effective solution. This Community Service aims to implement AR as an interactive and effective sign language learning medium for deaf students. This Community Service involves the introduction of student learning activities in sign language learning and the development of AR-based learning media products. The product is a mobile application that can be accessed using an Android mobile device. The implementation of learning media is carried out through the introduction of application features to teachers and students, as well as direct trials involving parental participation. The use of AR technology in learning media is also considered to provide an interactive and interesting learning experience for students. The implementation of Augmented Reality as an interactive sign language learning media at SLB Negeri 1 Palopo provides positive benefits in the learning of deaf students. This AR learning media can improve students' understanding of sign language in a more interactive and fun way. It also shows the potential of using technology in expanding educational accessibility for students with special needs, such as students who are deaf.

Automatic Translation From Iraqi Sign Language to Arabic Text or Speech Using CNN

Sign language (SL) is Non-verbal communication and a way for the deaf and mute to communicate without words. A deaf and mute person's hands, face, and body shows what they want to say. Since the number of deaf and dumb people is increasing, there must be other ways to learn sign language or communicate with deaf and dumb people. One of these ways is using advanced technology to produce systems that help the deaf/dumb, such as creating recognition and sign language translators. This paper presents an application that works on the computer for machine translation of Iraqi sign language in two directions from sign language to Arabic language (text/speech) and from Arabic language(text) to Iraqi sign language. The proposed system uses a Convolution Neural Network (CNN) to classify sign language based on its features to predicate the sign meaning. The sign language to Arabic language(text/speech) part of the proposed system has an accuracy of 99.3% for letters. Index Terms—Sign Language, Iraqi Sign Language, Deep Learning, CNN, gtts.

MEDIA PEMBELAJARAN MULTIMEDIA BAHASA ISYARAT DAYAK NGAJU UNTUK ANAK TUNA RUNGU

In the field of education there are schools that handle Children with Special Needs (ABK), namely children who significantly (meaningfully) experience abnormalities/deviations (physical, mental-intellectual, social, emotional) in the process of growth/development compared to other children of the same age including children deaf people so that they require special education services or are sent to special schools. One of the uses of technology in the field of special education for children with hearing impairments is in the form of processes in learning sign language such as finger alphabet sign language, number sign language and word dictionary sign language. The introduction of sign language lessons often has difficulties in communicating with students because these students not only come from the city center but some come from remote villages. understand the local language. The system development method applied in conducting this research is by adopting the concept of multimedia modeling. The final result of this research is in the form of multimedia learning media for Dayak Ngaju sign language which can be utilized by children with special needs, namely deaf children and also teachers.