Speech Impaired People Research Articles

Efficient Detection of Palm and Hand Landmark for Speech Impaired People Using Mediapipe Model

Efficient Detection of Palm and Hand Landmark for Speech Impaired People Using Mediapipe Model

Implementasi Algoritma You Only Look Once (YOLO) untuk Mendeteksi Bahasa Isyarat SIBI

The Indonesian Sign Language System (SIBI) is a translator of sign language into text or speech. This research aims to bridge communication between ordinary people and speech impaired people through the introduction of SIBI sign language using the YOLO algorithm. This research uses 24 alphabets which are divided into 4 groups, where each alphabet has 20 image data which is divided into 70% train data, 25% valid data, and 5% test data. The train data was then added with augmented data from Roboflow which was then carried out using a training process using a batch number of 16 and epochs of 100. The results of the research show that the YOLO algorithm can detect SIBI sign language alphabet gestures using confusion matrix testing and achieve quite good performance, as shown by the results F1 Score: Group 1 was 90.90%, Group 2 was 97.1%, Group 3 was 90.90%, and Group 4 was 83.8%. Other factors such as hand size, lighting conditions, and variations in data position also affect detection accuracy. A limitation in this research is that the alphabets J and Z were not included because these two alphabets not only use shape patterns, but also gesture patterns.

The effectiveness of HIV/AIDS education models for adolescents with speech disabilities

Adolescents with disabilities deserve the best health care without discrimination. They can access gender-sensitive health care and rehabilitation. Health services include promotive, preventive, curative, and rehabilitative care. HIV and AIDS prevention in disabled people must start as soon as possible. Those with speech disabilities are encouraged to reach their full independence potential in case of an incident. This study tested an HIV/AIDS education model for speech-impaired people. At Indonesia's School of Disability, a non-randomized control group pre- and post-test design was used. In the study, 60 speech-impaired students were divided into 30 intervention and 30 control groups. Analyses used T-tests. Speech-disabled people's knowledge and attitudes improved with the virtual HIV/AIDS education model (P=0.007). Therefore, the HIV/AIDS education model should be implemented using virtual platforms to improve knowledge and attitudes, especially among speech-disabled teens. Virtual reproductive health education for children with speech disabilities improves knowledge and attitudes more than traditional methods, especially among visually impaired children.

Design and Development of Sensor Based Glove for Speech Impaired People

Abstract: This project aims to address communication challenges faced by deaf and mute individuals through the development of a portable Sign Language Glove. The intelligent glove incorporates advanced sensor technologies, including flex sensors, an accelerometer, and a microcontroller, to accurately capture and interpret sign language gestures. By combining these technologies, the smart glove achieves close-to-real-time translation capabilities, seamlessly converting sign language into spoken or written language. The output is displayed on an LCD screen and conveyed audibly through an integrated speaker. Further enhancements include integrating a Bluetooth module for connection to a dedicated smartphone app, allowing users to personalize their communication experience. This comprehensive solution goes beyond mere translation, fostering inclusivity and adaptability. Ultimately, this groundbreaking technology aims to integrate speech-impaired individuals into society, dismantle communication barriers, and promote a more inclusive global community

The use of TensorFlow Action Recognition as the Main Component in Making a Sign Language Translator Speaker for Speech-Impaired People

Due to communication barriers, deaf and mute students are separated from their friends, families and communities as their schools do not offer sign language instruction. Consequently, this cluster of people may feel excluded from their communities, depriving them the chance of living a normal life that is free from discrimination. The objective of this quantitative experimental study is to use TensorFlow Action Recognition as the main component in making a Sign Language Translator Speaker for Speech-Impaired People. Based on the results, the device can successfully translate sign languages with an average of 5.91 seconds, and translate three signs per 30 seconds. Also, it was found that it can detect distances up to four meters. The study manifested that the device provides the service of breaking past the communication barriers to the speech- impaired and hearing-impaired individuals, which advocates and facilitates effective communication while fostering inclusivity. These results affirmed that it is feasible to make a Sign Language Translator Speaker with the use of TensorFlow Action Recognition. Thus, this Sign Language Speaker device offers the best services for deaf and mute people Qatar and all around the world, as the struggles of hearing and speech- impaired people can be alleviated.

Analogous sign language communication using gesture detection

ABSTRACT In this generation, deep learning techniques are widely used for sign language prediction. In this paper, a deep learning model is proposed for American sign language detection using webcam images and transfer learning. The particular model is designed for a real-time sign language detection. The author claims 98% of accuracy for this designed model, when trained with a total of 15 images for each gesture. Jupyter notebook is used as the environment for working out this research. Cuda, cudnn graphic processor upgraders are also utilised in this research for training the model. To make real-time detections easy, a local environment is used rather than a cloud system for implementing the code. The main aim of this research work is to create a model in order to identify and detect the sign language alphabets and some very frequently used gestures. The model is designed based on deep learning by using the convolutional neural networks and single shot detector algorithm to surpass the difficulty that is faced by the speech impaired and normal people.

Emotion Analysis Using Improved Cat Swarm Optimization with Machine Learning for Speech-impaired People

Emotion analysis is an effective method for improving interaction and understanding for speech-impaired people. We can provide the best interaction and support emotional well-being by analyzing the emotion conveyed through speech. Using deep learning (DL) or machine learning algorithms for training an emotion classification method. This might include training classifiers namely random forests, deep neural networks, or support vector machines. It is noteworthy that emotion analysis could be effective; however, it is crucial to consider individual differences and context while interpreting emotion. Furthermore, ensuring data protection and privacy and obtaining consent are vital features to consider while working with sensitive speech data. Therefore, this study presents an emotion analysis approach using improved cat swarm optimization with machine learning (EA-ICSOML) technique. The EA-ICSOML technique applies the concepts of computer vision and DL to identify various types of emotions. For feature vector generation, the ShuffleNet model is used in this work. To adjust the hyperparameters compared to the ShuffleNet system, the ICSO algorithm is used. Finally, the recognition and classification of emotions are performed using the Transient Chaotic Neural Network approach. The performance validation of the EA-ICSOML technique is validated on facial emotion databases. The simulation result inferred the improved emotion recognition results of the EA-ICSOML approach compared to other recent models in terms of different evaluation measures.

Implementasi Metode Dynamic Time Warping Pada Aplikasi Kamus Bahasa Isyarat Indonesia Sebagai Media Komunikasi

Communication is something vital in human existence. Communication is generally divided into two, namely verbal and non-verbal. For deaf and speech impaired people, the comic used is sign language and the intermediary used is a web-based application based on sign language from SIBI, with the implementation of the Dynamic Time Warping (DTW) method to analyze its usability and use. The use of this method in sign language dictionaries is to change the time coefficient into distance, because similarities in images can produce patterns from the object that is the example. In this test there are 24 letters and for each letter 10 image models taken via camera are used, and training is carried out with 50 datasets. The average test result collected reaches 90% because the condition of the hand can be said to be like the condition of other hands. This application is used to learn SIBI language, translate and read hand movements via real-time video. And during testing, accuracy results affect certain situations such as camera capacity, model coloring in the background and hand movements, and light brightness in the training data.
 
 Keyword: Metode Dynamic Time Warping, Indonesian Sign Language System, Deaf and Speech Impaired

Tamil Sign Language Translation and Recognition System for Deaf-mute People Using Image Processing Techniques

This work concentrates on the device, which helps as a translation system for translating sign gestures into text. The disabled people, in particular, hearing and speech impaired people, are facing difficulties in society. Communication of disabled people becomes worse as the majority of ordinary people do not understand it. These disabled people face difficulty communicating with others; some have many problems communicating with others in sign language. It causes the communication gap between them where impaired people cannot share their views and skills with others. We headed to facilitate communication between the disabled people and the "Tamil sign language translator to solve this problem." Here, gestures are translated to Tamil language to find a localized solution. It processes 31 Tamil alphabets, 12 Vowels, 18 Consonants, and 1 Aayudha Ezhuthu. It is 32 combinations with five fingers points either up or down and mapped to decimal numbers. Here in this process, we need edge detection, which is accurately done and Processed by canny edge detection. In addition to this process, we have used two gesture recognition methods and training the input system through our mainframe algorithm called scale-invariant feature detection transform. We have developed this system, which is useful for deaf and dumb people for essential communication.

Bio-Inspired Homogeneous Conductive Hydrogel with Flexibility and Adhesiveness for Information Transmission and Sign Language Recognition

The wearable electronic technique is increasingly becoming an effective approach to overcoming the communication obstacles between signers and non-signers. However, the efficacy of conducting hydrogels currently proposed as flexible sensor devices is hindered by their poor processability and matrix mismatch, which frequently results in adhesion failure at the combined interfaces and deterioration of mechanical and electrochemical performance. Herein, we propose a hydrogel composed of a rigid matrix in which the hydrophobic and aggregated polyaniline was homogeneously embedded, while quaternate-functionalized nucleobase moieties endowed the flexible network with adhesiveness. Accordingly, the resulting hydrogel with chitosan-graft-polyaniline (chi-g-PANI) copolymers exhibited a promising conductivity (4.8 S·m-1) because of the uniformly dispersed polyaniline components and a high strain strength (0.84 MPa) because of the chain entanglement of chitosan after soaking. In addition, the modified adenine molecules not only realized synchronization in improving the stretchability (up to 1303%) and exhibiting a skin-like elastic modulus (≈184 kPa), but also provided a durable interfacial contact with various materials. The hydrogel was further fabricated into a strain-monitoring sensor for information encryption and sign language transmission based on its sensing stability and strain sensitivity of up to 2.77. The developed wearable sign language interpreting system provides an innovative strategy to assist auditory or speech-impaired people in communicating with non-signers using visual-gestural patterns including body movements and facial expressions.

A Comparative Analysis of Gesture Recording Technologies and Recognition Methods

The dynamic development of computing techniques and communication tools and the improvement of network technology have increased the role of information as the main resource in society. The application of information communication technologies has stimulated the development of intellectual and scientific potential all over the world and has been successfully applied to all fields. Gestures are the only means of communication for hearing and speech impaired people. Automatic recognition of gestures in order to facilitate communication through gestures is an urgent issue from both scientific and practical point of view. This article highlights the static and dynamic gestures. The process of gesture recognition collects data by means of various sensor technologies. The article analyzes the image-based and non-image-based technologies and presents their advantages and disadvantages. It also comparatively analyzes the working principle of existing methods proposed for the gestures identification, explores their advantages and disadvantages and interprets the performance of the software which localizes the hand showing the gesture in the video frame. As a result, it develops a machine learning method based on neural networks for high accuracy identification of gestures. The developed method testing on database open for research shows high performance.

SIGN LANGUAGE RECOGNITION USING LSTM DEEP LEARNING MODEL

Speech impaired people use hand signs and gestures to communicate. The problem is translator needs every time to translate sign language. Artificial Intelligence capture hand expression with the help of camera show us what a person says and result shown in the text format. In the starting stage we provide the data for sign language for better result with good accuracy and testing this model. Various deepmachine learning algorithms are applied on the datasets, including Long Short-Term Memory (LSTM) RNN.

Hand Gesture Recognition and Conversion to Speech for Speech Impaired

The primary method of interaction between individuals is communication. In the case of speech-impaired people, the communication between them and normal people is not understandable by the latter as the former communicates through gestures and sign language. In the case of speech-impaired. Today, the Internet of Things (IoT) can be used for gesture recognition and to convert gestures into audible speech. In this project, a prototype is created such that the gesture or the sign language that is communicated is recognized by the flex sensors connected to the fingertips of a plain glove. The ESP32 microcontroller recognizes the gesture and converts it into the text format and this text format is updated onto the Google Firebase and is converted to speech form thus producing the text onto the OLED display sensor and the converted speech for communication in the mobile application Kodular Companion.

언어장애인의 스마트스피커 접근성 향상을 위한 개인화된 음성 분류 기법

With the spread of smart speakers based on voice recognition technology and deep learning technology, not only non-disabled people, but also the blind or physically handicapped can easily control home appliances such as lights and TVs through voice by linking home network services. This has greatly improved the quality of life. However, in the case of speech-impaired people, it is impossible to use the useful services of the smart speaker because they have inaccurate pronunciation due to articulation or speech disorders. In this paper, we propose a personalized voice classification technique for the speech-impaired to use for some of the functions provided by the smart speaker. The goal of this paper is to increase the recognition rate and accuracy of sentences spoken by speech-impaired people even with a small amount of data and a short learning time so that the service provided by the smart speaker can be actually used. In this paper, data augmentation and one cycle learning rate optimization technique were applied while fine-tuning ResNet18 model. Through an experiment, after recording 10 times for each 30 smart speaker commands, and learning within 3 minutes, the speech classification recognition rate was about 95.2%.

Comparison of Amplifiers Utilization in Instrumentation to Record Muscle Signals in the Neck for Electrolarynx Applications

Electrolarynx is an assistive technology commonly used by speech impaired people to speak. The speech impaired people who have lost their larynx (laryngectomee) or have damaged larynx use an electrolarynx device to be able to speak again. The use of the electrolarynx is generally equipped with a button to turn on and start the generation of sound from the electrolarynx. Several studies have tried to use other control methods by using the muscles in the neck, namely the sternohyoid muscle. Activation of the sternohyoid muscle has an influence on sound formation. The sternohyoid muscle is a small and long muscle so recording EMG (electromyograph) signals from this muscle is quite difficult. If the recording process of this signal can be carried out properly, then the electrolarynx control by empowering this muscle will be another solution in using the electrolarynx. In this study, instrumentation amplifier which is an important stage of recording EMG signals of neck muscle was tested and compared. There are two types of instrumentation amplifier tested. The first instrumentation amplifier uses a single IC from IC AD620 while the other is a combination circuit of IC OP07. The EMG signal in the subject's neck muscles was then recorded using the instrumentation amplifiers. The subject will sit down and pronounce the vowels "a", and "i". From the testing process, it was found that the average gain on IC AD620 (minimum 1.74362 volts and maximum 3.70538 volts) was greater than the gain on IC OP07 (minimum 0.57779 volts and maximum 1.71190 volts). IC AD620 also has an overall use area of 5.61 cm2 with the use of 4 external components. Thus, it can be concluded that the best instrumentation amplifier for recording EMG neck muscle for electrolarynx application is by using IC AD620.

Application of Random Forest Classifier Algorithm in Indonesian Sign Language System (Sibi) Detection System

Sign language is a language that prioritizes body movements and facial expressions as symbols to communicate with each other. Deaf and speech-impaired groups are the main users of sign language. One type of sign language in Indonesia is SIBI (Indonesian Sign Language System). SIBI (Indonesian Sign Language System) is a form of spoken language that is converted into sign language. Sign language is not only used as a means of communication and interaction between deaf and mute people, but also with normal people. However, until now there is still a communication gap between people who are deaf or mute and normal people. Then made a system design that can detect the sign language symbol SIBI (Indonesian Sign Language System). This system is made using the Random Forest Classifier algorithm with the help of MediaPipe Holistic and OpenCV and is made using the Python programming language. In this study, a dataset was created which was limited to 10 (ten) classes of symbols representing words in the SIBI (Indonesian Sign System) with a total of 8734 data lines which were then pre-processed by dividing the dataset into 70% training data and 30% test data. This study contains stages such as making detection, creating datasets, training classification models, and testing. The test is carried out by calculating the accuracy using the Confusion Matrix and then getting the Accuracy rate of 98.6%, Precision of 98.6%, and Recall of 98.66%. With the creation of the SIBI (Indonesian Language Sign System) detection system, it is hoped that it can reduce the gap between people who are deaf or speech impaired and normal people in communicating. Contribute to knowledge about symbol detection in SIBI (Indonesian Sign System) using the Random Forest Classifier algorithm as well as MediaPipe Holistic and OpenCV. As well as preserving and popularizing SIBI (Indonesian Sign System) as a sign language in Indonesia.

Sign and Human Action Detection Using Deep Learning.

Human beings usually rely on communication to express their feeling and ideas and to solve disputes among themselves. A major component required for effective communication is language. Language can occur in different forms, including written symbols, gestures, and vocalizations. It is usually essential for all of the communicating parties to be fully conversant with a common language. However, to date this has not been the case between speech-impaired people who use sign language and people who use spoken languages. A number of different studies have pointed out a significant gaps between these two groups which can limit the ease of communication. Therefore, this study aims to develop an efficient deep learning model that can be used to predict British sign language in an attempt to narrow this communication gap between speech-impaired and non-speech-impaired people in the community. Two models were developed in this research, CNN and LSTM, and their performance was evaluated using a multi-class confusion matrix. The CNN model emerged with the highest performance, attaining training and testing accuracies of 98.8% and 97.4%, respectively. In addition, the model achieved average weighted precession and recall of 97% and 96%, respectively. On the other hand, the LSTM model’s performance was quite poor, with the maximum training and testing performance accuracies achieved being 49.4% and 48.7%, respectively. Our research concluded that the CNN model was the best for recognizing and determining British sign language.

Indian Sign Language recognition system using SURF with SVM and CNN

Hand signs are an effective form of human-to-human communication that has a number of possible applications. Being a natural means of interaction, they are commonly used for communication purposes by speech impaired people worldwide. In fact, about one percent of the Indian population belongs to this category. This is the key reason why it would have a huge beneficial effect on these individuals to incorporate a framework that would understand Indian Sign Language. In this paper, we present a technique that uses the Bag of Visual Words model (BOVW) to recognize Indian sign language alphabets (A-Z) and digits (0–9) in a live video stream and output the predicted labels in the form of text as well as speech. Segmentation is done based on skin colour as well as background subtraction. SURF (Speeded Up Robust Features) features have been extracted from the images and histograms are generated to map the signs with corresponding labels. The Support Vector Machine (SVM) and Convolutional Neural Networks (CNN) are used for classification. An interactive Graphical User Interface (GUI) is also developed for easy access.

A Usability Evaluation for the Development of Speech Therapy Solutions for Speech-Impaired People

A Usability Evaluation for the Development of Speech Therapy Solutions for Speech-Impaired People

Sign Language Translator Using Deep Learning

People suffering from speech impairment can't communicate using hearing and speech methods, they believe signing for communication. Sign language is employed among everybody who is speech impaired, but they find a tough time in communicating with people which are non-signers (people aren’t proficient in sign language). So, requirement of a symbol language interpreter may be a must for speech impaired people. There has been favourable progress within the field of gesture recognition and motion recognition with current advancements in deep learning. There has been quite a significant development in computer vision which would enable us to easily track the hand gestures. The proposed system tries to try to a true time translation of hand gestures into equivalent English text. This system takes hand gestures as input through video and translates it text which might be understood by a non-signer. There will be use of CNN for classification of hand gestures. By deploying this technique, the communication gap between signers and non-signers will be reduced and they will be easily able to communicate with normal people.