Means Of Communication For Individuals Research Articles

A Review on Deep Learning Based Sign Language Recognition Translation

Hand posture is often referred to as gesture. It refers to the specific movements and postures that occur when a person utilizes his or her arms. It is one of the earliest and still widely used communication tools. In general, gestures are both dynamic and static. In the process of long-term social practice, gestures are given a variety of specific meanings, has a rich expressive power, the range of motion is extensive when the fingers, wrists, elbows, shoulders, and other joints of the hand are considered. The capacity for flexibility is considerable. Gestures have become a primary means of expression for humans, enabling the conveyance of emotions. In this context, body language occupies a pivotal role. Sign language is defined as the use of hand gestures with or without the use of facial expressions to convey meaning. The combination of hand gestures and facial expressions can be used to imitate images or to represent syllables, which together constitute a certain meaning or word. Sign language is a means of communication for individuals who are deaf or hard of hearing. It is an essential tool for communication, enabling them to express themselves and interact with others in a way that is accessible and understandable. This paper summarizes the deep learning based gesture recognition techniques in recent six years. In recent years, gesture recognition has been in the hot spot, and every year the articles on gesture recognition are constantly updated, so it is evident that a review of the research on gesture recognition is very necessary. This paper also compares and analyzes the existing gesture recognition methods and analyzes their advantages and disadvantages, and puts forward an outlook on the future development.

Real-Time Arabic Sign Language Recognition Using a Hybrid Deep Learning Model.

Sign language is an essential means of communication for individuals with hearing disabilities. However, there is a significant shortage of sign language interpreters in some languages, especially in Saudi Arabia. This shortage results in a large proportion of the hearing-impaired population being deprived of services, especially in public places. This paper aims to address this gap in accessibility by leveraging technology to develop systems capable of recognizing Arabic Sign Language (ArSL) using deep learning techniques. In this paper, we propose a hybrid model to capture the spatio-temporal aspects of sign language (i.e., letters and words). The hybrid model consists of a Convolutional Neural Network (CNN) classifier to extract spatial features from sign language data and a Long Short-Term Memory (LSTM) classifier to extract spatial and temporal characteristics to handle sequential data (i.e., hand movements). To demonstrate the feasibility of our proposed hybrid model, we created a dataset of 20 different words, resulting in 4000 images for ArSL: 10 static gesture words and 500 videos for 10 dynamic gesture words. Our proposed hybrid model demonstrates promising performance, with the CNN and LSTM classifiers achieving accuracy rates of 94.40% and 82.70%, respectively. These results indicate that our approach can significantly enhance communication accessibility for the hearing-impaired community in Saudi Arabia. Thus, this paper represents a major step toward promoting inclusivity and improving the quality of life for the hearing impaired.

CNN-TCN: Deep Hybrid Model Based on Custom CNN with Temporal CNN to Recognize Sign Language

Sign language plays a pivotal role as a primary means of communication for individuals with hearing and speech impairments. Given their limited auditory and vocal communication abilities, these individuals heavily rely on visual cues, such as body language and hand gestures, to convey their emotions and thoughts in everyday social interactions. Sign language primarily consists of characters (letters) and numerals (numbers). This study introduces an innovative hybrid methodology for automated sign language identification, combining Temporal Convolutional Neural Network (TCNN) and a Custom Convolutional Neural Network (CCNN). The effectiveness of this system was rigorously evaluated using three distinct benchmark datasets that encompass isolated letters and digits. These datasets are comprehensive and publicly accessible resources covering both British and American sign languages. The proposed CNN-TCN model comprises various phases, including data collection, preprocessing (involving labeling, normalization, and frame extraction), feature extraction using CCNN, and sequence modeling through TCNN. The experimental results clearly demonstrate the remarkable performance of the proposed system, with accuracy, precision, recall, and F1 scores reaching impressive levels of 95.31%, 94.03%, 93.33%, and 93.56%, respectively, across the three diverse datasets. These outcomes serve as compelling evidence of the CNN-TCN method’s viability and effectiveness in the realm of sign language recognition.

Sign Speak: Recogninzing Sign Language with Machine Learning

Sign language serves as a critical means of communication for individuals with hearing impairments, enabling them to integrate into society effectively and express themselves. However, interpreting and recognizing sign language gestures present unique challenges due to the dynamic nature of gestures and spatial dependencies inherent in sign language communication. As a response, the SignSpeak project employs advanced machine learning techniques to address these challenges and enhance accessibility for the deaf and hard of hearing community. The project leverages a diverse dataset sourced from Kaggle, comprising images of sign language gestures captured in various contexts. The integration of advanced algorithms, such as 3D Convolutional Neural Networks (CNNs), and Gated Recurrent Units (GRUs), enables SignSpeak to recognize and interpret sign language gestures accurately and in real-time. This integration allows the model to capture both spatial and temporal features inherent in sign language, thus enabling more robust and accurate recognition. The project encompasses several critical stages, including data preprocessing, model development, training, and evaluation. Data preprocessing involves converting the image data into a suitable format and applying augmentation techniques to enhance the diversity and robustness of the dataset. Model development entails designing a deep learning architecture that combines CNNs and GRUs to effectively capture spatial and temporal dependencies in sign language gestures. Training the model involves optimizing parameters and hyperparameters to achieve optimal performance. Evaluation metrics such as accuracy, F1 score, and recall are utilized to assess the model's performance on both training and validation datasets. The trained model is then tested on a separate test dataset to evaluate its real- world performance and generalization ability. Experimental results demonstrate the efficacy of the SignSpeak approach in accurately recognizing and interpreting sign language gestures. The model achieves high accuracy scores, demonstrating its potential to enhance accessibility and inclusion for individuals with hearing impairments. By providing real-time translation of sign language into text or speech, SignSpeak contributes to breaking down communication barriers and promoting equal participation for all members of society.

Sign Language Detection

Abstract: Sign language is a crucial means of communication for individuals with hearing impairments, serving as a rich and expressive form of language. As the world becomes increasingly interconnected, there is a growing need for technology to facilitate effective communication between sign language users and the broader community. Sign language detection, a branch of computer vision and artificial intelligence, plays a pivotal role in bridging these communication gaps. This abstract provides an overview of the key aspects related to sign language detection. The primary focus is on the technological advancements and applications that contribute to the seamless integration of sign language into various aspects of daily life. The paper explores the challenges associated with recognizing and interpreting sign language gestures, considering factors such as variations in signing styles, lighting conditions, and diverse signing communities. The proposed paper delves into the methodologies employed in sign language detection systems, including image and video processing techniques, deep learning algorithms, and sensor technologies. It highlights the significance of dataset diversity and the role of machine learning in enhancing the accuracy and robustness of sign language detection models. Additionally, the paper discusses the potential integration of sign language detection into mainstream devices, applications, and assistive technologies to empower individuals with hearing impairments.

Deep learning-based isolated sign language recognition: a novel approach to tackling communication barriers for individuals with hearing impairments

Sign language is a primary and widely used means of communication for individuals with hearing impairments. Current sign language recognition techniques need to be improved and need further development. In this research, we present a novel deep learning architecture for achieving significant advancements in sign language recognition by recognizing isolated signs. The study utilizes the Isolated Sign Language Recognition (ISLR) dataset from 21 hard-of-hearing participants. This dataset comprises 250 isolated signs and the x, y, and z coordinates of 543 hand gestures obtained using MediaPipe Holistic Solution. With approximately 100,000 videos, this dataset presents an essential opportunity for applying deep learning methods in sign language recognition. We present the comparative results of our experiments, where we explored different batch sizes, kernel sizes, frame sizes, and different convolutional layers. We achieve an accuracy rate of 83.32% on the test set.

Design of Sign Language Learning Media Based on Virtual Reality

Sign language has become a very important means of communication for individuals with hearing loss, allowing them to interact with people who are not hearing impaired. In recent years, virtual reality (VR) technology has advanced rapidly, opening up new opportunities to enhance the user experience in various situations. This study aims to design and develop an easy-to-use virtual reality-based interactive sign language application. The software development method used is the Software Life Cycle Development Method (MDLC), which includes the stages of analysis, design, implementation and testing. This app is designed to run on affordable and popular VR devices to make it accessible to as many individuals as possible. In the development process, we conducted an analysis of various sign languages used globally, focusing on hand gestures and facial expressions which have an important role in sign language communication. Users can learn and practice sign language through interactive experiences in a virtual environment that mimics real-life situations. This application is also equipped with visual and audio feedback features to help users improve and hone their skills. At the testing stage, this application was evaluated through trials involving participants with diverse backgrounds in the use of sign language. The test results show that users feel involved in the VR experience and experience increased confidence in using sign language after using this application. This research proves that VR technology can be an effective tool in sign language learning and development. Thus, the virtual reality-based sign language application that has been designed and developed in this study makes a significant contribution in increasing the accessibility of communication for individuals with hearing impairments. The implications of this research can be the basis for further development in the field of sign language and VR technology.

Real-Time Vision-Based Sign Language Bilateral Communication Device for Signers and Non-Signers using Convolutional Neural Network

The use of sign language is an important means of communication for individuals with hearing and speech impairments, but communication barriers can still arise due to differences in grammatical rules across different sign languages. In an effort to address these barriers, this study aimed to develop a real-time two-way communication device that uses image processing and recognition systems to translate two-handed Filipino Sign Language (FSL) gestures and facial expressions into speech; the system can recognize gestures that correspond to specific words and phrases. Specifically, the researchers utilized Convolutional Neural Networks (CNNs) to enhance the processing speed and accuracy of the device. The system also includes a speech-to-text (STT) feature that helps non-signers communicate with deaf individuals without relying on an interpreter. The study's results showed that the device achieved a 93% accuracy rate in recognizing facial expressions and FSL gestures using CNN, indicating that it is highly accurate. Additionally, the system performed in real-time, with an overall average conversion time of 1.84 and 2.74 seconds for sign language to speech and speech to text, respectively. Finally, the device was well-received by both signers and non-signers, with a total approval rating of 85.50% from participants at Manila High School, suggesting that it effectively facilitates two-way communication and has the potential to break down communication barriers.

Language Model-Guided Classifier Adaptation for Brain-Computer Interfaces for Communication.

Brain-computer interfaces (BCIs), such as the P300 speller, can provide a means of communication for individuals with severe neuromuscular limitations. BCIs interpret electroencephalography (EEG) signals in order to translate embedded information about a user's intent into executable commands to control external devices. However, EEG signals are inherently noisy and nonstationary, posing a challenge to extended BCI use. Conventionally, a BCI classifier is trained via supervised learning in an offline calibration session; once trained, the classifier is deployed for online use and is not updated. As the statistics of a user's EEG data change over time, the performance of a static classifier may decline with extended use. It is therefore desirable to automatically adapt the classifier to current data statistics without requiring offline recalibration. In an existing semi-supervised learning approach, the classifier is trained on labeled EEG data and is then updated using incoming unlabeled EEG data and classifier-predicted labels. To reduce the risk of learning from incorrect predictions, a threshold is imposed to exclude unlabeled data with low-confidence label predictions from the expanded training set when retraining the adaptive classifier. In this work, we propose the use of a language model for spelling error correction and disambiguation to provide information about label correctness during semi-supervised learning. Results from simulations with multi-session P300 speller user EEG data demonstrate that our language-guided semi-supervised approach significantly improves spelling accuracy relative to conventional BCI calibration and threshold-based semi-supervised learning.

Within arm’s reach: The role of proximity in speech generating device use of ambulatory children with autism

Purpose: Augmentative and Alternative Communication (AAC) provides an alternative means of communication for individuals who are unable to effectively communicate using spoken language. The present study seeks to investigate the role of proximity in facilitating AAC (specifically Speech Generating Devices; SGD) use in a homogeneous population: ambulatory school-aged children with autism spectrum disorder (ASD). Method: Twelve school-age children with ASD who use an SGD as their primary mode of communication participated in the study. The percentage of time the child was within the proximity of their device was calculated over nine videotaped sessions and data logging provided information regarding device use over an 18-week period. Result: Children were in proximity to their device 65% of the time. There was a significant positive correlation between percentage of time in proximity to device and the average minutes of device use on the day video recorded. Exploratory analyses revealed that communicative abilities may be more connected to proximity than school context. Conclusion: These results support the importance of developing a program for assuring device proximity in order to encourage consistent device use in minimally verbal ambulatory school-age children with ASD.

Development and testing an online near-infrared spectroscopy brain–computer interface tailored to an individual with severe congenital motor impairments

Purpose: For non-verbal individuals, brain–computer interfaces (BCIs) are a potential means of communication. Near-infrared spectroscopy (NIRS) is a brain-monitoring modality that has been considered for BCIs. To date, limited NIRS-BCI testing has involved online classification, particularly with individuals with severe motor impairments.Materials and methods: We tested an online NIRS-BCI developed for a non-verbal individual with severe congenital motor impairments. The binary BCI differentiated categorical verbal fluency task (VFT) performance and rest using prefrontal measurements. The participant attended five sessions, the last two of which were online with classification feedback.Results: An online classification accuracy of 63.33% was achieved using a linear discriminant classifier trained on a four-dimensional feature set. An offline, cross-validation analysis of all data yielded an optimal adjusted classification accuracy of 66.6 ± 9.11%. Inconsistent functional responses, contradictory effects of feedback, participant fatigue and motion artefacts were identified as challenges to online classification specific to this participant.Conclusions: Results suggest potential in using an NIRS-BCI controlled by the VFT in instances of severe congenital impairments. Further testing with users with severe disabilities is necessary.Implications for RehabilitationBrain–computer interfaces (BCIs) can provide a non-motor based means of communication for individuals with severe motor impairments.Near-infrared spectroscopy (NIRS) is a haemodynamic-based brain-imaging modality used in BCIs.To date, NIRS-BCIs have not been thoroughly tested with potential target users.This case study shows that NIRS-BCIs may offer a means of practical communication for individuals with severe congenital impairments and continued exploration is advisable.

Moving Away From Error-Related Potentials to Achieve Spelling Correction in P300 Spellers.

P300 spellers can provide a means of communication for individuals with severe neuromuscular limitations. However, its use as an effective communication tool is reliant on high P300 classification accuracies ( > 70%) to account for error revisions. Error-related potentials (ErrP), which are changes in EEG potentials when a person is aware of or perceives erroneous behavior or feedback, have been proposed as inputs to drive corrective mechanisms that veto erroneous actions by BCI systems. The goal of this study is to demonstrate that training an additional ErrP classifier for a P300 speller is not necessary, as we hypothesize that error information is encoded in the P300 classifier responses used for character selection. We perform offline simulations of P300 spelling to compare ErrP and non-ErrP based corrective algorithms. A simple dictionary correction based on string matching and word frequency significantly improved accuracy (35-185%), in contrast to an ErrP-based method that flagged, deleted and replaced erroneous characters (-47-0%) . Providing additional information about the likelihood of characters to a dictionary-based correction further improves accuracy. Our Bayesian dictionary-based correction algorithm that utilizes P300 classifier confidences performed comparably (44-416%) to an oracle ErrP dictionary-based method that assumed perfect ErrP classification (43-433%).

Neural Point-and-Click Communication by a Person With Incomplete Locked-In Syndrome

A goal of brain–computer interface research is to develop fast and reliable means of communication for individuals with paralysis and anarthria. We evaluated the ability of an individual with incomplete locked-in syndrome enrolled in the BrainGate Neural Interface System pilot clinical trial to communicate using neural point-and-click control. A general-purpose interface was developed to provide control of a computer cursor in tandem with one of two on-screen virtual keyboards. The novel BrainGate Radial Keyboard was compared to a standard QWERTY keyboard in a balanced copy-spelling task. The Radial Keyboard yielded a significant improvement in typing accuracy and speed—enabling typing rates over 10 correct characters per minute. The participant used this interface to communicate face-to-face with research staff by using text-to-speech conversion, and remotely using an internet chat application. This study demonstrates the first use of an intracortical brain–computer interface for neural point-and-click communication by an individual with incomplete locked-in syndrome.

Utilizing a language model to improve online dynamic data collection in P300 spellers.

P300 spellers provide a means of communication for individuals with severe physical limitations, especially those with locked-in syndrome, such as amyotrophic lateral sclerosis. However, P300 speller use is still limited by relatively low communication rates due to the multiple data measurements that are required to improve the signal-to-noise ratio of event-related potentials for increased accuracy. Therefore, the amount of data collection has competing effects on accuracy and spelling speed. Adaptively varying the amount of data collection prior to character selection has been shown to improve spelling accuracy and speed. The goal of this study was to optimize a previously developed dynamic stopping algorithm that uses a Bayesian approach to control data collection by incorporating a priori knowledge via a language model. Participants ( n = 17) completed online spelling tasks using the dynamic stopping algorithm, with and without a language model. The addition of the language model resulted in improved participant performance from a mean theoretical bit rate of 46.12 bits/min at 88.89% accuracy to 54.42 bits/min ( ) at 90.36% accuracy.

A Review of EEG-Based Brain-Computer Interfaces as Access Pathways for Individuals with Severe Disabilities

Electroencephalography (EEG) is a non-invasive method for measuring brain activity and is a strong candidate for brain-computer interface (BCI) development. While BCIs can be used as a means of communication for individuals with severe disabilities, the majority of existing studies have reported BCI evaluations by able-bodied individuals. Considering the many differences in body functions and usage scenarios between individuals with disabilities and able-bodied individuals, involvement of the target population in BCI evaluation is necessary. In this review, 39 studies reporting EEG-oriented BCI assessment by individuals with disabilities were identified in the past decade. With respect to participant populations, a need for assessing BCI performance for the pediatric population with severe disabilities was identified as an important future direction. Acquiring a reliable communication pathway during early stages of development is crucial in avoiding learned helplessness in pediatric-onset disabilities. With respect to evaluation, augmenting traditional measures of system performance with those relating to contextual factors was recommended for realizing user-centered designs appropriate for integration in real-life. Considering indicators of user state and developing more effective training paradigms are recommended for future studies of BCI involving individuals with disabilities.

A ’Hands-On’ Approach

You have accessThe ASHA LeaderFirst Person on the Last Page1 Jul 2012A ’Hands-On’ Approach Kimberly AbtsMS, CF-SLP Kimberly Abts Google Scholar More articles by this author , MS, CF-SLP https://doi.org/10.1044/leader.FPLP.17092012.55 SectionsAbout ToolsAdd to favorites ShareFacebookTwitterLinked In Kimberly Abts, MS, CF-SLP As a speech-language pathology graduate student at Gallaudet University, I am often asked the following questions (accompanied by some variety of hand movements to simulate American Sign Language): “Are you deaf?” “Why do you go to a school for the deaf?” “Do you use sign language?” Throughout my time in graduate school, it has become apparent that many people are interested in learning about people who are deaf and hard of hearing, but don’t have many opportunities. My first-hand edification with deafness and deaf awareness has been an essential part of the personal growth that accompanies the grueling clinical and course work associated with graduate school. I am challenged daily by the use and incorporation of American Sign Language in my coursework and the immersion into the deaf community on campus. The presence of the Department of Hearing, Speech, and Language Sciences has been and will continue to be a point of controversy on campus. I have come to understand there will always be individuals who view the on-campus clinic as a threat to the deaf community. My hope is that the gaps in these differences will be bridged through mutual understanding and collaboration. I have seen progress toward this goal during my time at Gallaudet. I am grateful to those who have helped me learn about the SLP’s role in the deaf community and I am inspired by those willing to give me the opportunity to better my future profession by sharing their experiences, both negative and positive. As a student at Gallaudet University, I feel a responsibility to educate the general public on what deafness entails, provide information on the different means of communication for individuals who are deaf or hard of hearing, and remove many of the common misconceptions about the deaf population. The graduate speech-language pathology class of 2012 has made it a priority to provide opportunities for public education while raising money to establish a scholarship to Camp SHARP (Speech, Hearing, and Aural Rehabilitation Program). Camp SHARP is a summer communication program at Gallaudet for children who are deaf and hard of hearing. The program provides an intensive, language-rich, communication-accessible, and enjoyable environment for children to improve their spoken language skills and reinforce their sign language skills. For a speech-language pathology graduate student at Gallaudet, it is the best of both worlds: the hearing world and the deaf world. Camp SHARP is a step toward collaboration and communication between these two worlds that are, in fact, not all that different. The last two years have not been easy. Even though I often refer to my time in graduate school as my “study abroad” experience, I would not trade it for the world...either of them. Author Notes Kimberly Abts, MS, CF-SLP, recently graduated from Gallaudet and is a clinician at St. John’s Health System in Anderson, Indiana. Contact her at [email protected]. Advertising Disclaimer | Advertise With Us Advertising Disclaimer | Advertise With Us Additional Resources FiguresSourcesRelatedDetails Volume 17Issue 9July 2012 Get Permissions Add to your Mendeley library History Published in print: Jul 1, 2012 Metrics Downloaded 117 times Topicsasha-topicsleader_do_tagleader-topicsasha-article-typesCopyright & Permissions© 2012 American Speech-Language-Hearing AssociationLoading ...

Applying Technology to Visually Support Language and Communication in Individuals with Autism Spectrum Disorders

The burgeoning role of technology in society has provided opportunities for the development of new means of communication for individuals with Autism Spectrum Disorders (ASD). This paper offers an organizational framework for describing traditional and emerging augmentative and alternative communication (AAC) technology, and highlights how tools within this framework can support a visual approach to everyday communication and improve language instruction. The growing adoption of handheld media devices along with applications acquired via a consumer-oriented delivery model suggests a potential paradigm shift in AAC for people with ASD.

A Review of the Efficacy of the Picture Exchange Communication System Intervention

The Picture Exchange Communication System (PECS) is a communication program that has become widely used, especially with children with autism. This paper reports the results of a review of the empirical literature on PECS. A descriptive review is provided of the 27 studies identified, which included randomized controlled trials (RCTs), other group designs and single subject studies. For 10 appropriate single subject designs the percentage of nonoverlapping data (PND) and percentage exceeding median (PEM) metrics were examined. While there are few RCTs, on balance, available research provides preliminary evidence that PECS is readily learned by most participants and provides a means of communication for individuals with little or no functional speech. Very limited data suggest some positive effect on both social-communicative and challenging behaviors, while effects on speech development remain unclear. Directions for future research are discussed including the priority need for further well-conducted RCTs.

The Changing Position of Swahili in East Africa

Opening ParagraphThe initial spread of Swahili from the coastal areas of Kenya and Tanganyika during the last century must, in large measure, be attributed to the Swahili traders, who, with their often unsavoury reputations and their caravans, took the language as far as the Congo. Where these traders settled in any numbers, as at Tabora, Mwanza, Ujiji, Usumbura, Lake Nyasa, and, to a smaller extent, at Mumias, small Islamic islands of coastal-Swahili-speakers have persisted: some flourishing like Usumbura, others, like Mumias, declining. In general, however, the language became more emasculated the farther it extended from the coast, so that in North Uganda and parts of the Eastern Congo it is essentially a highly simplified means of communication for individuals of widely differing groups; pruned of all refinements, its link with the coast is a tenuous one.