Learning Sign Language Research Articles

Expert Consensus on Formation of Design Principles for Malay Sign Language Mobile Application Based on Nielsen’s and Molich’s Design Guidelines

Malay Sign Language (MSL) is a Malay spoken language that hearing-impaired (HI) learners use in communication and learning in Malaysia. Various MSL mobile applications have been developed and available in the market to support the HI learners in learning sign language, but most fail to satisfy the learners’ needs and consist of many drawbacks in functionalities. Besides, the existing MSL mobile applications fail to evoke the cognitive ability among HI learners. Thus, this study aims to propose an MSL mobile application that hybridizes Nielsen’s and Molich’s Design Guidelines (NMDG) in order to evoke the cognitive ability among HI learners and provide mobile applications that meet the user’s needs.Before developing the proposed application, this study has carried out the initial investigation that has been done through User-Centered Design (UCD) in stage 1. Hence, the evolution of NMDG was identified, but they are too generic as it does not consider the HI alpha generation context in evaluating the mobile application. Hence, new design principles that consider HI alpha generation will be formed throughout the extension of the study based on the NMDG.

Indian Sign Language: Effective way to give voice to the voiceless using Artificial Intelligence

Although Sign Language is one of the classical and most natural means of communication, most people are unaware of it, and interpreters are less in number. The aim of this paper is to develop an interface between dumb/deaf and the normal people. Motivation behind this is to introduce an application that will not only facilitate users to learn sign language but also will help them to convert the hand sign gestures to corresponding alphabets or numbers. The system will also help to convert text to hand sign gestures. This paper contributes an approach for users to communicate with dumb and deaf individuals effectively and efficiently by removing the need of translation facility. Convolution Neural Networks (CNN) is used in translating the hand sign gestures to text. Natural Language Processing (NLP) helps in converting given input text to their corresponding hand sign gestures.

Amphibious Magical Realism? The Shape of Water as Cinematic Trauma Narrative

Amphibious Magical Realism? <i>The Shape of Water</i> as Cinematic Trauma Narrative

Kinematic Motion Retargeting via Neural Latent Optimization for Learning Sign Language

Motion retargeting from a human demonstration to a robot is an effective way to reduce the professional requirements and workload of robot programming, but faces the challenges resulting from the differences between humans and robots. Traditional optimization-based methods are time-consuming and rely heavily on good initialization, while recent studies using feedforward neural networks suffer from poor generalization to unseen motions. Moreover, they neglect the topological information in human skeletons and robot structures. In this paper, we propose a novel neural latent optimization approach to address these problems. Latent optimization utilizes a decoder to establish a mapping between the latent space and the robot motion space. Afterward, the retargeting results that satisfy robot constraints can be obtained by searching for the optimal latent vector. Alongside with latent optimization, neural initialization exploits an encoder to provide a better initialization for faster and better convergence of optimization. Both the human skeleton and the robot structure are modeled as graphs to make better use of topological information. We perform experiments on retargeting Chinese sign language, which involves two arms and two hands, with additional requirements on the relative relationships among joints. Experiments include retargeting various human demonstrations to YuMi, NAO, and Pepper in the simulation environment and to YuMi in the real-world environment. Both efficiency and accuracy of the proposed method are verified.

Enhancing Sign Language Learning with Augmented Reality

Sign language is the communication language used by people with disabilities, especially deaf and hearing-impaired people. The communication between normal and disabled people using sign language will help them carry out their daily activities. Unfortunately, normal people are not aware of the importance of sign language because they are not directly dealing with disabled persons. Besides, some normal people found that sign language is difficult to learn. This study focuses on developing a mobile application for sign language with augmented reality features. This application is targeted at normal people as a sign language learning tool, and augmented reality will make the learning process effective and exciting. The methodology for this study is the ADDIE model that consists of five phases, namely Analysis, Design, Development, Implementation, and Evaluation. This application will assist the users in learning sign language interactively and interestingly. The evaluation of the application was done by the expert in the related background and the normal people. The usability test result revealed that the sign language application is usable for normal people to know and learn the basic sign language. In conclusion, the sign language application is an interesting application for normal people to use and learn sign language.

Development of a Technology-Assisted Assessment for Sign Language Learning

The goal of a recently concluded project in Switzerland was to pioneer an assessment system for lexical signs of Swiss German Sign Language (Deutschschweizerische Gebärdensprache, DSGS) that relies on automatic sign language recognition. The assessment system gives adult L2 learners of DSGS feedback on the correctness of the manual parameters of signing (handshape, hand position, location, and movement) of isolated signs they produce. In its initial version, the system includes automatic feedback for a subset of a DSGS vocabulary size production test consisting of approximately 100 lexical items at CEFR level A1. The paper at hand reports on the process of selecting the items for the test, compiling training data for the SLR system, and linguistically analyzing errors in the resulting video recordings.

Development of a Technology-Assisted Assessment for Sign Language Learning

Development of a Technology-Assisted Assessment for Sign Language Learning

The Performance of Thai Sign Language Recognition with 2D Convolutional Neural Network Based on NVIDIA Jetson Nano Developer Kit

Thai Sign Language Recognition is a Thai Sign Language learning computer recognition. The system constructs an architecture of T-SLR by TSR- 2DCNN based on NVIDIA Jetson Nano Developer Kit. It is a novelty of automatic translation TSL innovation and reveals the performance of feature extraction and classification to reduce crashed system, overloaded or automatic reboot while complicated processing occurs. The dataset contains 7 gestures in TSL, training images are 7,000 images and validation images are 700 images. The result compares with many techniques as shown that TSR-2DCNN can increase the performance of TSLR in real-time, effectiveness with an accuracy of 0.9914 and loss of 0.03537.

Pakistan sign language translation tool in educational setting: teachers perspective

PurposeThe current study aims to determine the viability of the tool developed by Abbas and Sarfraz (2018) to translate English speech and text to Pakistan Sign Language (PSL) with bilingual subtitles.Design/methodology/approachFocus group interviews of 30 teachers of a Pakistani private university were conducted; who used the PSL translation tool in their classrooms for lecture delivery and communication with the deaf students.FindingsThe findings of the study determined the viability of the developed tool and showed that it is helpful in teaching deaf students efficiently. With the availability of this tool, teachers are not dependent on human sign language (SL) interpreters in their classrooms.Originality/valueOverall, this tool is an effective addition to educational technology for special education. Due to the lack of Sign Language (SL) understanding, learning resources and availability of human SL interpreters in Pakistan, institutions feel dependency and scarcity to educate deaf students in a classroom. Unimpaired people and especially teachers face problems communicating with deaf people to arrange one interpreter for a student(s) in multiple classes at the same time which creates a communication gap between a teacher and a deaf student.

Bank Kosa Kata Untuk Tuna Rungu dan Tuna Wicara Berbasis Web

Communication is the most important key to mastering this era of globalization. There is no denying that language is the most important part of communication. When one person uses the same language or understands another person's language, they can communicate well. Sign language is a language based on artificial communication, i.e. body language and lip movements during communication. This sign language is standardized as SIBI (Indonesian Sign Language System). SIBI is one of the media in the form of books that are able to communicate with deaf people in the community. Its shape is a systematic arrangement of groups of fingers, hands and various movements, symbolizing the word Indonesian. Book media does not seem to be easily understood by users, so it takes an application that can provide moving images to facilitate sign language learning, one of which is web-based.&#x0D;

Sign Language Learning Support System using HMD with Hand-tracking Function

Sign Language Learning Support System using HMD with Hand-tracking Function

Eveline Boers-Visker. Learning to use space. A study into the SL2 acquisition process of adult learners of Sign Language of the Netherlands. Amsterdam: LOT, 2020. xxx+ 344 blz. ISBN: 978-94-6093-354-7. € 38,00.

Amsterdam University Press is a leading publisher of academic books, journals and textbooks in the Humanities and Social Sciences. Our aim is to make current research available to scholars, students, innovators, and the general public. AUP stands for scholarly excellence, global presence, and engagement with the international academic community.

NLP-based Sign Gesture Identification for Disabled People

There are many methods for identifying signs, each of which generates a word for each one. It focuses on converting sign language into an appropriate English sentence. NLP techniques are also used in addition to sign recognition. The input is a framed and split video of sign language. This booklet teaches deaf and mute people sign language. It's tough for non-blind persons to engage with blind people due to communication difficulties. To address this issue, the article suggests and describes an effective method. Language technology methods such as POS tagging and the LALR parser are used to convert identified sign words into English phrases. A number of applications are currently on the industry that allow blind people to interact with the world. Combining technology will not be able to address the problem of mobile sign language translation in daily activities. A video interpreter can assist deaf or hearing-impaired people in a variety of situations. People with hearing impairments will be able to learn sign language and have films translated into sign language as a consequence of this research. The present work may be used as a communication interface for both speech-impaired and non-speech-impaired individuals. It will assist bridge the communication gap between speechimpaired people and the rest of the population by capturing and analyzing signals, as well as recognizing and displaying output in the form of comprehensible phrases.

Architecture design of a reinforcement environment for learning sign languages.

Different fields such as linguistics, teaching, and computing have demonstrated special interest in the study of sign languages (SL). However, the processes of teaching and learning these languages turn complex since it is unusual to find people teaching these languages that are fluent in both SL and the native language of the students. The teachings from deaf individuals become unique. Nonetheless, it is important for the student to lean on supportive mechanisms while being in the process of learning an SL. Bidirectional communication between deaf and hearing people through SL is a hot topic to achieve a higher level of inclusion. However, all the processes that convey teaching and learning SL turn difficult and complex since it is unusual to find SL teachers that are fluent also in the native language of the students, making it harder to provide computer teaching tools for different SL. Moreover, the main aspects that a second language learner of an SL finds difficult are phonology, non-manual components, and the use of space (the latter two are specific to SL, not to spoken languages). This proposal appears to be the first of the kind to favor the Costa Rican Sign Language (LESCO, for its Spanish acronym), as well as any other SL. Our research focus stands on reinforcing the learning process of final-user hearing people through a modular architectural design of a learning environment, relying on the concept of phonological proximity within a graphical tool with a high degree of usability. The aim of incorporating phonological proximity is to assist individuals in learning signs with similar handshapes. This architecture separates the logic and processing aspects from those associated with the access and generation of data, which makes it portable to other SL in the future. The methodology used consisted of defining 26 phonological parameters (13 for each hand), thus characterizing each sign appropriately. Then, a similarity formula was applied to compare each pair of signs. With these pre-calculations, the tool displays each sign and its top ten most similar signs. A SUS usability test and an open qualitative question were applied, as well as a numerical evaluation to a group of learners, to validate the proposal. In order to reach our research aims, we have analyzed previous work on proposals for teaching tools meant for the student to practice SL, as well as previous work on the importance of phonological proximity in this teaching process. This previous work justifies the necessity of our proposal, whose benefits have been proved through the experimentation conducted by different users on the usability and usefulness of the tool. To meet these needs, homonymous words (signs with the same starting handshape) and paronyms (signs with highly similar handshape), have been included to explore their impact on learning. It allows the possibility to apply the same perspective of our existing line of research to other SL in the future.

Multimedia Based Sign Language Learning

Having children with hearing impairment is a great concern for parents in teaching their children to communicate. However, parents tend to have trouble choosing the right approach in teaching children with hearing impairment to not feel lazy and bored quickly in learning. This research aims to create an interactive sign language learning application based on the BISINDO sign language system as an alternative learning media for children with hearing impairment to increase their learning motivation. The application was made to support children with hearing impairment in understanding sign language, starting from recognizing letters, numbers, basic and greeting words; the application was also made as an interactive software equipped with various supporting features such as videos, animations, to trivia quizzes. This research used descriptive methods with observation and interviews, as well as prototype methods in system development. This study indicates that the application can increase learning motivation for children with hearing impairment and provide supporting instruments for parents in teaching children to communicate. In conclusion, the application is beneficial and has a solution for parents to teach sign language to their children who have a hearing impairment and difficulty digesting learning and lack motivation

Comprehension benefits of visual-gestural iconicity and spatial referencing

Studies have shown that iconicity can provide a benefit to non-signers during the learning of single signs, but other aspects of signed messages that might also be beneficial have received less attention. In particular, do other features of signed languages help support comprehension of a message during the process of language learning? The following exploratory study investigates the comprehension of sentences in two signed and two spoken languages by non-signers and by American Sign Language (ASL) learners. The design allows for the examination of message comprehension, with a comparison of unknown spoken and signed languages. Details of the stimulus sentences are provided in order to contextualize features of the signing that might be providing benefits for comprehension. Included in this analysis are aspects of the sentences that are iconic and spatially deictic – some of which resemble common gestural forms of communication. The results indicate that iconicity and referential points in signed language likely assist with comprehension of sentences, even for non-signers and for a signed language that the ASL signers have not studied.

Adquisició del canvi de rol i dels classificadors en el discurs narratiu en llengua de signes catalana (LSC) com a L2: un estudi pilot

Summary: A growing interest in the study of sign language linguistics deals with how the differences between the visual-gestural modality and the vocal-auditory modality affect the acquisition of a sign language by adults whose first language is a spoken language. This paper presents a pilot study on the acquisition of two essential and, to a large extent, specific characteristics of signed languages used in reference control: the nonmanual markers in role shift structures and classifiers. The comparison between native discourse in Catalan Sign Language (LSC) and adult learners of LSC as L2 with different levels of competence provides a careful insight into how these two reference mechanisms are used in the acquisition process. The present study analyses reference control in L2 signed discourse in terms of reference accessibility and the discussion is placed at the syntax-discourse interface. The detailed analysis shows how nonmanuals of role shift and classifiers play an important role in reference control, which implies an important position in the accessibility scale of discourse referents. Keywords: accessibility, acquisition, Catalan Sign Language (LSC), classifiers, discourse referents, nonmanual markers, pronouns, reference control, role shift, second language (L2)

A Deep Convolutional Neural Network Approach to Sign Alphabet Recognition

In this paper, we present an application that has been developed to be used as a tool for the purposes of learning sign language for beginners that utilizes hand detection as part of the process. It uses a skin-color modelling technique, such as explicit thresholding in the skin-color space, which is based on modeling skin-color spaces. This predetermined range of skin-colors is used to determine how pixels (hand) will be extracted from non-pixels (background). To classify the images, convolutional neural networks (CNN) were fed the images for the creation of the classifier. The training of the images was done using Keras. A uniform background and proper lighting conditions enabled the system to achieve a test accuracy of 93.67%, of which 90.04% was attributed to ASL alphabet recognition, 93.44% for number recognition and 97.52% recognition of static words, surpassing other studies of the type. An approach which is based on this technique is used for fast computation as well as real-time processing. Deaf-dumb people face a number of social challenges as the communication barrier prevents them from accessing basic and essential services of the life that they are entitled to as members of the hearing community. In spite of the fact that a number of factors have been incorporated into the innovations in the automatic recognition of sign language, an adequate solution has yet to be reached because of a number of challenges. As far as I know, the vast majority of existing works focus on developing vision based recognizers by deriving complex feature descriptors from captured images of the gestures and applying a classical pattern analysis technique. Although utilizing these methods can be effective when dealing with small sign vocabulary captures with a complex and uncontrolled background, they are very limited when dealing with large sign vocabulary. This paper proposes a method for analyzing and representing hand gestures, which acts as the core component of the vocabulary for signing languages, using a deep convolutional neural networks (CNN) architecture. On two publicly accessible datasets (the NUS hand posture dataset and the American fingerspelling A dataset), the method was demonstrated to be more accurate in recognizing hand postures.

Sign names in an intercultural classroom space: practices and perceptions of sign language teachers and students

ABSTRACT This study investigates practices around the bestowal of personal sign names to adult New Zealand Sign Language learners, from both teachers' and students' perspectives. Key findings include that, (1) learners receive similar types of sign names to deaf signers but in different proportions; (2) getting a sign name is regarded as a significant milestone and a valuable commodity in constructing an intercultural identity as a `new signer'; (3) deaf teachers experience pressure of student expectations and cultural sensitivities in negotiating sign names in a classroom context. These findings can inform teacher practices and learner expectations regarding sign name acquisition.

Teaching Chinese Sign Language with a Smartphone

BackgroundThere is a large group of deaf-mutes in the world, and sign language is their major communication tool. Therefore, it’s necessary for the deaf-mutes to communicate with hearing-speech people and the hearing-speech people also have needed to understand sign language, which produces a great demand for sign language teaching. Even though there have already been a large number of books for sign language, it is low efficient to learn sign language with books, even teaching videos. To solve this problem, we develop a smartphone-based interactive Chinese sign language teaching systemfor sign language learning. MethodsThe system provides a learner with some kinds of learning modes and captures the learner’s actions from its front camera of the smartphone. Right now the system provides a vocabulary set with 1000 frequently used words, and the learner can evaluate his/her sign action by subjective or objective comparison. In the mode of word recognition, the users can play any word within the vocabulary and the system will turn the top three retrieved candidates, so it can remind the learners what the sign is. ResultsThis system provides interactive learning for a user to learn sign language high efficiently. The systemadopts an algorithm based on point cloud recognition to evaluate a user’s sign and costs about 700ms inference time for each sample, which meets the real-time requirements. ConclusionThis interactive learning system decreases the communication barriers between the deaf-mutes and hearing-speechers.