Mutism Research Articles

Indian Sign Language Recognition Using Machine Learning Techniques

AbstractAutomatic conversion of sign language to text or speech is indeed helpful for interaction between deaf or mute people with people who even do not have knowledge of sign language. This is the demand of current times to develop an automatic system to convert ISL signs to normal text and vice versa. This will be beneficial for both communities to express their fillings to one another in accessing publicly available facilities like ticketing, banking services, traffic signals, etc. A new feature extraction and selection technique using structural features and some of the best available classifiers are proposed to recognize ISL signs for better communication for computer‐human interface. This paper narrates a system for automatic recognition of ISL immobile numeric signs, in which a standard digital camera was only used to acquire the signs, no wearable devices are required to capture electrical signals. The system is intended to convert isolated digit signs into text, that is, each entered sign image should contain precisely one numeric sign. To recognize ISL sign images in real time environment, a sign database containing ISL digits is created which contains 5000 images, 500 images for each numeral sign (0‐9). Among two classifiers, the k‐Nearest Neighbor outperforms the Naive Bayes classifier in stipulations of classification accuracy.

Sentence Generation for Indian Sign Language Using NLP

Nowadays the interaction among deaf and mute people and normal people is difficult, because normal people scuffle to understand the sense of the gestures. The deaf and dumb people find problem in sentence formation and grammatical correction. To alleviate the issues faced by these people, an automatic sign language sentence generation approach is propounded. In this project, Natural Language Processing (NLP) based methods are used. NLP is a powerful tool for translation in the human language and also responsible for the formation of meaningful sentences from sign language symbols which is also understood by the normal person. In this system, both conventional NLP methods and Deep learning NLP methods are used for sentence generation. The efficiency of both the methods are compared. The generated sentence is displayed in the android application as an output. This system aims to connect the gap in the interaction among the deaf and dumb people and the normal people.

Raspberry Pi based Sign to Speech Conversion System for Mute Community

Motion sign based language becomes important roles for mute community for data transmission. Mute people facing very difficult situation to convey information to normal people. We find the solution for dumb community to make their communication simple and smart using micro processor. We proposed a latest sign language system for dumb people; it helps in emergency situation to convey their required information to normal people easily. This proposed system converts their sign signal to voice. In this proposed method we implemented using Raspberry Pi microprocessor to fast conversion of data. In this technique two major roles that sign based data conversion and voice generation system using micro processor. For the implementation of this enhanced sign conversion system we used flex sensors as sign conversion and RPI module for processing and 3.5mm Audio jack for voice enable. This proposes system will be useful to all mute community, physically unable persons as well as older people to share their information in any situation.

Communication Support Tool With Data Glove Concept For People With Hearing Disability Using Kansai Engineering Method

The difficulty of communication that faced by the deaf and mute people make them complicated in doing activity like normal people. Sign language as a communication tool that has been formed at this time has not yet been able to connect communication with the community yet. It is only able to bridge communication among those fellow deaf and mute people. Therefore a tool which is able to translate sign language used by people with disabilities is needed. Thetool which able to translate sign language wascreated by Data Glove concept in design which can translate the word visually. Using the assistance of the Semantic Differential questionnaire, we can get the emotion level from user based on 12 Kansei words. The response in emotion form which is expresssed by users is one of theelements of the Kansei Engineering Method. Thus product adjustment to Kansei engineering is advisable and society.

Conversion of Sign Language to Text and Speech and Prediction of Gesture

People with the inability to speak, use sign language for communication. Ordinary people usually find it very difficult to communicate with mute people due to their lack of understanding of the universal sign language. This paper aims to provide a solution for this very problem through a device that uses an Arduino Uno board, some flex sensors and an Android application to facilitate interaction amongst the users. The flex sensors detect the movements and gestures of the wearer and based on the established conditions for the different values generated, respective messages are sent using a Global System for Mobile (GSM) Module to the user’s android device which translates the text message to speech. The GSM module also attempts to create parameters for gesture predictions by sending sensor inputs to a cloud-based server for future reference. The application is ever learning and continues to evolve to be more reliable by examining user behaviours at all times. The use of this device allows mute people to convert sign language to speech, thereby making it significantly easier to talk to others, especially those who do not know sign language. This device empowers mute people and opens them up to previously unattainable opportunities.

Hand Gesture to Speech and Text Conversion Device

Sign language and facial expressions are the major means of communication for the speech flawed people. General people can understand the facial expression to an extent but cannot understand the sign language. Dumb people are unable to express their thoughts to normal humans. To reduce this gap of communication, this paper presents an electronic system which will help the mute people to exchange their ideas with the normal person in emergency situations. The system consists of a glove that can be worn by the subject which will convert the hand ges-tures to speech and text. The message displayed will also help deaf people to understand their thoughts. This prototype involves raspberry pi 3 as the micro-controller along with the flex sensors, accelerometer sensor. The resistance of the flex sensor changes due to the bending moment of the fingers of the subject. The accelerometer measures the angular displacement of the wrist along the y-axis. The microcontroller takes the input from the two sensors and matches it with the pre-programmed values and plays the respective message. The system makes use of python and its libraries for microcontroller programming.

English

English

Finger Gesture Vocalizer

A Gesture Vocalizer is a small scale or a large scale system that provides a way for dumb and mute people to communicate easily. The research paper defines a technique, Finger Gesture Vocalizer which includes sensors attached to the gloves above the fingers of the person who wants to communicate. The sensors are arranged in such a way on the gloves, that they can capture the movements of the fingers and based on the change in resistance of the sensors, it can be identified what the person wants to say. The message is displayed on the LCD and is also converted to audio using the APR33A3 audio processing unit. Standard sign languages such as that of American Sign Language which is used by dumb and mute people to communicate can be employed while wearing these gloves.

Design and Implementation of Smart System for Interaction Between Blind, Deaf and Mute People

Difficulties faced by the deaf and dumb people and the blind people to communicate among themselves is the major motivation for the proposal of this project. The project aims at bridging this communication gap by means of an cost-effective electronic device. This is done by using a prototype hand worn glove which converts hand gestures and Braille codes into speech and text, thus enabling speech-impaired and visually impaired people to effectively communicate with everyone. MEMS sensors are used to recognize the gesture sign given by the deaf and dumb people. The gesture input is sent as an audio file which is understandable to the people. The input given by the blind people is in Braille language which is converted to text using scale-invariant feature transform (SIFT) algorithm and decoded to text and voice output through Artificial Neural Networks (ANN) by means of Back Propagation using Supervised Learning. So this ensures two way communication between blind, deaf and dumb people.

Hand Motion-Recognition in an Intelligent Communication Module

our country India consists of very high population. Almost 9 millions of population in this world can be counted as a deaf or dumb people or both. We all know that the most valuable gift from the god given to human is the capability to watch, hear, spoke & to give response as per the situation arises. We all know communication is one of the most important medium through which one may carve up his/her feelings or express the information to others. The key points in communication are ability of listening and speak the word but so many from us are unlucky because they are not gifted by this ability from the god these are deaf & dumb and people. Now a days so many researches are going on to solve difficulties of these people of our society because they had to face very hard to communicate with normal person. It is too hard for mute(deaf & dumb) people to transmit their information to the normal people. As all normal people are not fully trained to understand different sign lingo, the communication between these two types of people becomes too complex. At the time emergency or other days whenever a mute (Deaf & Dumb) people are travelling passing a message (data) becomes too hard. Due to this disability one that has hearing and speaking disability doesn’t want to stand and face the race with normal person. Since the communication for the mute people is image (Visibility), not acoustic (Audio). For these mute people Hand motion plays a very part for communication. The data transmission between mute-deaf & normal people is always a very challenging job. Deaf people make use of sign language or gestures to make understand what he/she trying to say but it is impossible to understand by hearing people. The admittance to various communication based technologies plays an essential role for these (mute) handicapped peoples. Developing a small and compressed gadget for these mute people is a difficult task. Deaf-Dumb people find a difficulty in communicating with normal people and hence they always stay apart in their societies. Basically deaf-dumb ones uses various sign language for communication, & they always face lots of difficulty while communicating with normal people because they all not able to understand sign language every time. Hence there is always a hurdle in communication between these two type of peoples. There are so many researches has done in regulate to search a simple and easy way for mute people to communicate with the normal people and articulate themselves to the rest of the real world. So many improvements have been done in sign language but all are based on American gesture (sign) Language. Our research work is purely designed to provide an aid to this deaf-mute by developing and designing a smart communication module which will help to renovate sign language to text & speech communication with other and to help them lead a life in a much better way. Our paper represents a static gesture recognition algorithm that will be designed and implemented will be used in the smart communication system to bridge the communication gap between deaf & dumb and normal people. If the algorithm is implemented completely it can be can be also used to capture and analyze emotions of the people in area where high security is desired.

A Wearable Skinlike Ultra-Sensitive Artificial Graphene Throat.

Most mute people cannot speak due to their vocal cord lesion. Herein, to assist mute people to "speak", we proposed a wearable skinlike ultrasensitive artificial graphene throat (WAGT) that integrated both sound/motion detection and sound emission in single device. In this work, the growth and patterning of graphene can be realized at the same time, and a thin poly(vinyl alcohol) film with laser-scribed graphene was obtained by a water-assisted transferring process. In virtue of the skinlike and low-resistant substrate, the WAGT has a high detection sensitivity (relative resistance changes up to 150% at 133 Ω) and an excellent sound-emitting ability (up to 75 dB at 0.38 W power and 2 mm distance). On the basis of the excellent mechanical-electrical performance of graphene structure, the sound detecting and emitting mechanisms of WAGT are realized and discussed. For sound detection, both the motion of larynx and vibration of vocal cord contribute to throat movements. For sound emission, a thermal acoustic model for WAGT was established to reveal the principle of sound emitting. More importantly, a homemade circuit board was fabricated to build a dual-mode system, combining the detection and emitting systems. Meanwhile, different human motions, such as strong and small throat movements, were also detected and transformed into different sounds like "OK" and "NO". Therefore, the implementation of these sound/motion detection acoustic systems enable graphene to achieve device-level applications to system-level applications, and those graphene acoustic systems are wearable for its miniaturization and light weight.

Sign to Speech for Mute People Using Gyro Sensor

Sign to Speech for Mute People Using Gyro Sensor

Hand Gesture Speaking Unit for Mute People

Hand Gesture Speaking Unit for Mute People

Bisindo Sign Language Transliteration Using Automata

Sign Language is a language which prioritizes manual communication, body language and lip movement for communication. People who usesign language for communication aredeaf and mute people. Sign language in Indonesia is Sistem Isyarat Bahasa Indonesia (SIBI) and Bahasa Isyarat Indonesia (BISINDO). BISINDO has a unique clause pattern that isSOPA (Subject, Object, Predicate, Adverbial). The clause pattern of BISINDO is just for deaf and mute people who know and understand it. Because there are some different structures between Indonesian language and BISINDO, itis neededlearning media which can translate from Indonesian language to BISINDO. Finie State Automata (FSA) is a method which is able to recognize pattern sentence classification in Indonesian language. FSA method is NSA (Non-DetermisticFinie State Automata) that is otomata with characteristics as a state. NFA is used to give introducing structure pattern sentence in Indonesian language. Sentence pattern which is produced from NFA produces eight sentences pattern which are arranged as requisites ofBISINDO translation. The result of research which is experimented by sign language researchersas many as 90% can translate well and appropriate with BISINDO sentence pattern, 10% explain that it is not appropriate with translation of sentence pattern and sign language model. Based on the data, it can be known that media of BISINDO sign language learning based on web using method of translation automata can be used for learning media.

Indian Sign Language Alphabets Recognition from Static Images Using Correlation-Coefficient Algorithm with Neuro-Fuzzy Approach

Sign Language is a language used by mute people to communicate with other person by expressing their thoughts in the form of signs. Signs are expressed by hand and face movement. This paper describes proposed methodology for design and implementation of Alphabets recognition system of Indian sign language where static images of hand gesture are used as an input. Methodology includes various image processing techniques to remove unnecessary noise and to make images smooth. This Paper describe Correlation-coefficient algorithm which has been used for feature extraction and Neuro-fuzzy algorithm which has been applied as recognition algorithm. Proposed technique has been implemented on database of 100 images for testing. Implementation has been done on Matlab. Average accuracy of proposed methodology is 92.30%.

Meshes of muteness: Maya Deren’s languages

Meshes of muteness: Maya Deren’s languages

Smart Wearable Hand Device for Sign Language Interpretation System With Sensors Fusion

Gesturing is an instinctive way of communicating to present a specific meaning or intent. Therefore, research into sign language interpretation using gestures has been explored progressively during recent decades to serve as an auxiliary tool for deaf and mute people to blend into society without barriers. In this paper, a smart sign language interpretation system using a wearable hand device is proposed to meet this purpose. This wearable system utilizes five flex-sensors, two pressure sensors, and a three-axis inertial motion sensor to distinguish the characters in the American sign language alphabet. The entire system mainly consists of three modules: 1) a wearable device with a sensor module; 2) a processing module; and 3) a display unit mobile application module. Sensor data are collected and analyzed using a built-in embedded support vector machine classifier. Subsequently, the recognized alphabet is further transmitted to a mobile device through Bluetooth low energy wireless communication. An Android-based mobile application was developed with a text-to-speech function that converts the received textinto audible voice output. Experiment results indicate that a true sign language recognition accuracy rate of 65.7% can be achieved on average in the first version without pressure sensors. A second version of the proposed wearable system with the fusion of pressure sensors on the middle finger increased the recognition accuracy rate dramatically to 98.2%. The proposed wearable system outperforms the existing method, for instance, although background lights, and other factors are crucial to a vision-based processing method, they are not for the proposed system.

Real time conversion of sign language to speech and prediction of gestures using Artificial Neural Network

Sign language is generally used by the people who are unable to speak, for communication. Most people will not be able to understand the Universal Sign Language (unless they have learnt it) and due to this lack of knowledge about the language, it is very difficult for them to communicate with mute people. A device that helps to bridge a gap between mute persons and other people forms the crux of this paper. This device makes use of an Arduino Uno board, a few flex sensors and an Android application to enable effective communication amongst the users. Using the flex sensors, gestures made by the wearer is detected and then according to various pre-defined conditions for the numerous values generated by the flex sensors, corresponding messages are sent using a Global System for Mobile(GSM) module to the wearer’s android device, which houses the application that has been designed to convert text messages into speech. The GSM module is also used to send the sensor inputs to a cloud server and these values are taken as input parameters into the neural network for a time series based prediction of gestures. The system is designed to be a continually learning device and improve reliability by monitoring every individual’s behaviour at all times.

E-Learning through Single Hand and Two Hand Sign Language

E-learning is commonly referred to the intentional use of networked information and communications technology for teaching and learning. e-Learning exploits interactive technologies and communication systems to improve the learning experience. It has the potential to transform the way we teach and learn across the board. It can raise standards, and widen participation in lifelong learning. E-learn is useful method that has contributed in facilitating education for deaf mute people. Deaf mute people are able to get benefit from this technology by increasing their skills and improving their knowledge. They can use the mobility feature to learn anywhere and at any time. Most of the deaf students easily learn and develop skill and knowledge in e-learning method. Now a day’s e-learning tools are mostly used in learning method. E-learning includes computer and electronically supported learning and teaching methods. E-learning used in Web-based learning, computer-based learning, virtual based learning and digital technology. In this paper we study which type of sign language is best for learning.

A New Web-Based E-Learning System Using Similarity Measurement and Visiting Frequency for Hearing Impaired and Mute

In most of the educational environment, there is a large number of students with varying disabilities and the usage of technology within this environment is more dynamic. One of the most common techniques used for the particular category of people (hearing impaired and mute) is e-learning, which is the best option for those people. They have some difficulties such as, learning, communication, understanding, etc. for providing a better learning environment; many existing e-learning mechanisms provide the recommendation of subjects based on their personalization. However, it makes the whole learning process as complex, and there is a need to collect the request from them, which are the major drawbacks of the existing scheme. To overcome the issues, a new type of web-based e-learning system, namely, Most Similar Most Frequent Recommendation (MSMFR) is proposed in this paper with query analyzing option for hearing impaired and mute people. The proposed system is entirely based on the data structures such as stack, queue, linked list, etc. in the distributed environment. Here, the physically challenged people can able to share their recommended content to others, who search the same one. Moreover, we provide a secure access and provisioning for them. The primary intention of this paper is to improve the learning capability of physically challenged people. Initially, the required information is collected from the data owner and, the meta-data is created from that data. Then, the query is obtained from the user, which is parsed into attributes. Hence, the semantic relationship between the attributes and meta-data and the visiting frequency are found out. Here, the rating ratio is calculated for the requested query by using a web history. Finally, the required information such as web or video is retrieved for hearing impaired and mute people. In experiments, the performance of the proposed method is evaluated in terms of time and relevancy.