UAV With the Ability to Control with Sign Language and Hand by Image Processing

Deaf and dumb people communicate using language known as Sign language. There are various sign languages but comparing to other sign languages, Indian Sign Language interpretation has got least mindfulness by researchers. Alphabets in English sign language and various phrases can be recognized using the application. It deals with images of bare hands. Hand-Gesture recognition is the ability to recognize hand-gesture. The hand gesture recognition system is solution to this problem which uses the image of hand gesture and recognize the sign present in the image. These features based on gesture analysis of the hand on which image of sign is shown and it extract slant and slope information. System provides an opportunity for deaf and dumb people to communicate with normal people effortlessly. Our research focuses on recognition of English sign language for vast reach of audience. There have been several upgrades in technology and a lot of research has been done to aid the deaf and dumb people. Aiding the cause, Deep learning along with computer vision is being used to make an impact on this cause. It can help the deaf and dumb people in communicating with others as many of us don’t know what sign language is. In this sign language recognition project, we create an kotlin application, which detects numbers from 1 to 10 and also the English alphabets. The initial step of this system is to build a database of English Sign Language. Hand gesture detection is the most determining step in each hand gesture recognition system since if we get better segmented output, better recognition rates can be achieved. The proposed system also includes efficient and robust hand segmentation and tracking algorithm to achieve finer recognition rates. Various isolated words from the Standard English sign language have been recognized using a large set of samples. In proposed system, we intend to recognize some very basic elements of sign language and to translate them to text .

This manuscript presents a full duplex communication system for the Deaf and Mute (D-M) based on Machine Learning (ML). These individuals, who generally communicate through sign language, are an integral part of our society, and their contribution is vital. They face communication difficulties mainly because others, who generally do not know sign language, are unable to communicate with them. The work presents a solution to this problem through a system enabling the non-deaf and mute (ND-M) to communicate with the D-M individuals without the need to learn sign language. The system is low-cost, reliable, easy to use, and based on a commercial-off-the-shelf (COTS) Leap Motion Device (LMD). The hand gesture data of D-M individuals is acquired using an LMD device and processed using a Convolutional Neural Network (CNN) algorithm. A supervised ML algorithm completes the processing and converts the hand gesture data into speech. A new dataset for the ML-based algorithm is created and presented in this manuscript. This dataset includes three sign language datasets, i.e., American Sign Language (ASL), Pakistani Sign Language (PSL), and Spanish Sign Language (SSL). The proposed system automatically detects the sign language and converts it into an audio message for the ND-M. Similarities between the three sign languages are also explored, and further research can be carried out in order to help create more datasets, which can be a combination of multiple sign languages. The ND-M can communicate by recording their speech, which is then converted into text and hand gesture images. The system can be upgraded in the future to support more sign language datasets. The system also provides a training mode that can help D-M individuals improve their hand gestures and also understand how accurately the system is detecting these gestures. The proposed system has been validated through a series of experiments resulting in hand gesture detection accuracy exceeding 95%.

Static hand gesture recognition in sign language based on convolutional neural network with feature extraction method using ORB descriptor and Gabor filter

Sign language identification and recognition technique is composed by the gesture images detection and the hand gestures recognition. Gesture images detection is to locate the image part of palm and fingers from the captured pictures, and rotating them to the appropriate gesture posture. Both of them are the important pre-processing for sign language identification and recognition. Lose them, the correctness rate of the sign language recognition algorithms will be dropped down to an unacceptable level. This paper presents novel processing algorithms for the gesture images detection and recognition. In the detection process, it rotates an askew gesture to right position, and to delete the elbow and forearm parts from the captured pictures. In the recognition process, it includes two phases with the model construction and the sign language identification. In the model construction phase, the static hand gesture of sign language is constructed by the Gaussian mixture model, and the unknown gesture image is identified by Gaussian model match. Based on this presented static sign language detection and recognition algorithms, the correct recognition rate is about 94% in average.

Human Computer Interaction (HCI) focuses on the interaction between humans and machines. An extensive list of applications exists for hand gesture recognition techniques, major candidates for HCI. The list covers various fields, one of which is sign language recognition. In this field, however, high accuracy and robustness are both needed; both present a major challenge. In addition, feature extraction from hand gesture images is a tough task because of the many parameters associated with them. This paper proposes an approach based on a bag-of-words (BoW) model for automatic recognition of American Sign Language (ASL) numbers. In this method, the first step is to obtain the set of representative vocabularies by applying a K-means clustering algorithm to a few randomly chosen images. Next, the vocabularies are used as bin centers for BoW histogram construction. The proposed histograms are shown to provide distinguishable features for classification of ASL numbers. For the purpose of classification, the K-nearest neighbors (kNN) classifier is employed utilizing the BoW histogram bin frequencies as features. For validation, very large experiments are done on two large ASL number-recognition datasets; the proposed method shows superior performance in classifying the numbers, achieving an F1 score of 99.92% in the Kaggle ASL numbers dataset.

An accurate estimation of hand gestures using optimal modified convolutional neural network

A novel hand biometric authentication method based on measurements of the user’s stationary hand gesture of hand sign language is proposed. The measurement of hand gestures could be sequentially acquired by a low-cost video camera. There could possibly be another level of contextual information, associated with these hand signs to be used in biometric authentication. As an analogue, instead of typing a password ‘iloveu’ in text which is relatively vulnerable over a communication network, a signer can encode a biometric password using a sequence of hand signs, ‘i’ , ‘l’ , ‘o’ , ‘v’ , ‘e’ , and ‘u’. Subsequently the features from the hand gesture images are extracted which are integrally fuzzy in nature, to be recognized by a classification model for telling if this signer is who he claimed himself to be, by examining over his hand shape and the postures in doing those signs. It is believed that everybody has certain slight but unique behavioral characteristics in sign language, so are the different hand shape compositions. Simple and efficient image processing algorithms are used in hand sign recognition, including intensity profiling, color histogram and dimensionality analysis, coupled with several popular machine learning algorithms. Computer simulation is conducted for investigating the efficacy of this novel biometric authentication model which shows up to 93.75% recognition accuracy.

The barrier between the hearing and the deaf communities in Kenya is a major challenge leading to a major gap in the communication sector where the deaf community is left out leading to inequality. The study used primary and secondary data sources to obtain information about this problem, which included online books, articles, conference materials, research reports, and journals on sign language and hand gesture recognition systems. To tackle the problem, CNN was used. Naturally captured hand gesture images were converted into grayscale and used to train a classification model that is able to identify the English alphabets from A-Z. Then identified letters are used to construct sentences. This will be the first step into breaking the communication barrier and the inequality. A sign language recognition model will assist in bridging the exchange of information between the deaf and hearing people in Kenya. The model was trained and tested on various matrices where we achieved an accuracy score of a 99% value when run on epoch of 10, the log loss metric returning a value of 0 meaning that it predicts the actual hand gesture images. The AUC and ROC curves achieved a 0.99 value which is excellent.

Sign language is a visual language that uses hand gesture, change of hand shape and track information to express meaning, and is the main communication tool for people with hearing and language impairment. Sign language recognition can improve the problem that the number of people who need to use sign language is large but the popularity of sign language is poor, and provide a more convenient way of study, work and life for people with hearing and language impairment. Hand locating and sign language recognition methods can generally be divided into traditional methods and deep learning methods. In recent years, with the brilliant achievements of deep learning in the field of computer vision, it has been proved that the method based on deep learning has many advantages, such as rich feature extraction, strong modeling ability and intuitive training. Therefore, this paper studies hand locating and sign language recognition of common sign language based on neural network, and the main research contents include: 1. A hand locating network based on the Faster R-CNN is established to recognize the sign language video or the part of the hand in the picture, and the result of recognition is handed over to subsequent processing; 2. A 3D CNN feature extraction network and a sign-language recognition framework based on long and short time memory (LSTM) coding and decoding network are constructed for the sign language images of sequence sequences. The framework can improve the recognition accuracy by learning the context of sign language; 3. In order to solve the problem of RGB sign language image or video recognition in practical problems, this paper combines hand locating network, 3D CNN feature extraction network and LSTM encoding and decoding to build the recognition algorithm. Experimental results show that the recognition rate of this method is up to 99% in common vocabulary data set, which is better than other known methods.

In this paper, we proposed an improved algorithm of hand-gesture recognition based on Haar-like features and Adaboost. Initial, we calculated the Haar-like features of hand-gesture images by integral image. Then, we used the principal components analysis method to reduce the dimension of Haar-like features. At last, an Adaboost classifier performed the hand-gesture recognition task with the hand-gesture features. A dataset with large hand gestures (12 types, 600 hand-gesture images) was built, including some large pose-angle (about 40 deg.) hand-gesture images. Our experiment results demonstrated that our method could effectively recognize different hand gesture, and the best appropriate N was 12. In addition, the average processing time of the proposed method was about 0.05 second for every image.

Communication is needed to interact and to socialize. In general, communication is used verbally, that is to use spoken or written words. But in everyday life, some of people are unable to communicate verbally. Some of them have a problem in communication, such as deaf and mute. The most effective communication used by deaf and mute are through non-verbal communication. This communication uses sign language in the form of hand and body gestures. Hand gestures identification is one way to identify the position pattern of the hand and the body in order to translate the meaning and the purpose. This communication is intended so that a normal person can understand the sign language which is conveyed by the deaf. In understanding the sign language, a communication media is needed to interpret sign language into easily-understood verbal language. The sign language described in this research only includes sign language from hand and body gestures which then will be translated into characters. The important stages and processes that must be done in the implementation of sign language system is to detect the configuration of hand gestures which describe the Indonesian Sign Language (BISINDO). The method used in this research is the Optical Flow that is used to detect hand and body movements as the object with the red glove as a marker. Based on testing conducted on 20 different types of word recognition results obtained : 65,5% identified for 20 species of the word.

A vision-based static hand gesture recognition method which consists of preprocessing, feature extraction, feature selection and classification stages is presented in this work. The preprocessing stage involves image enhancement, segmentation, rotation and filtering. This work proposes an image rotation technique that makes segmented image rotation invariant and explores a combined feature set, using localized contour sequences and block-based features for better representation of static hand gesture. Genetic algorithm is used here to select optimized feature subset from the combined feature set. This work also proposes an improved version of radial basis function (RBF) neural network to classify hand gesture images using selected combined features. In the proposed RBF neural network, the centers are automatically selected using k-means algorithm and estimated weight matrix is recursively updated, utilizing least-mean-square algorithm for better recognition of hand gesture images. The comparative performances are tested on two indigenously developed databases of 24 American sign language hand alphabet.

Sign language is one way of communication used by people who cannot speak or hear (deaf and speech impaired), so not everyone can understand sign language. Therefore, to facilitate communication between normal people and deaf and speech-impaired people, many systems have been created to translate gestures and signs in sign language into understandable words. Artificial intelligence and computer vision-based technologies, such as YOLOv9 offer solutions to recognize hand gestures more quickly, accurately, and efficiently. This research aims to develop a hand gesture detection system for alphabetic sign language using YOLOv9 architecture, with the aim of improving the accuracy and speed of hand gesture detection. The data used consists of 6500 sign language alphabet hand gesture images that have been labeled with bounding boxes and processed using image augmentation techniques. The model was trained on the Kaggle platform and evaluated using performance metrics such as Accuracy, Precision, Recall, F1-Score, and Intersection over Union (IoU). The results show that the YOLOv9 model achieves an average detection accuracy of 97%, with precision and recall above 90% for most classes. In addition, YOLOv9 shows advantages over other algorithms such as SSD MobileNet v2 and Faster RCNN, both in terms of speed and accuracy. In conclusion, YOLOv9 proved to be very effective in detecting sign language hand gestures, thereby speeding up and facilitating communication. This research is expected to contribute to the development of more inclusive technologies in various fields, such as education, public services, and employment opportunities, which support better communication between sign language users and the general public.

Communication is required to interact and socialize. Generally, communication used verbal communication in spoken or written words. However several people are unable to communicate verbally, since they were deaf or mute. Non-verbal communication is the most effective communication used by the deaf and mute. This communication uses sign language in the form of hand and body gestures. Hand gestures identification is the method to identify the position pattern of the hand and the body in order to translate the meaning and the purpose. This communication is intended to make normal person understand regarding the sign language that conveyed by the deaf. Many difficiluties faced by normal person to understand the sign language. Therefore interpreter tool for sign language is required to translate into easy-understood verbal language. This research only includes sign language from hand and body gestures, in order to translates into characters. The configuration of hand gestures detection must be done to describes Indonesian Sign Language (BISINDO). The optical flow method has been used in this research in detecting hand and body movements. By using this method, 65% from 20 different word has been succeeded to identified.

Deaf people use sign languages to communicate with other people in the community. Although the sign language is known to hearing-impaired people due to its widespread use among them, it is not known much by other people. In this article, we have developed a real-time sign language recognition system for people who do not know sign language to communicate easily with hearing-impaired people. The sign language used in this paper is American sign language. In this study, the convolutional neural network was trained by using dataset collected in 2011 by Massey University, Institute of Information and Mathematical Sciences, and 100% test accuracy was obtained. After network training is completed, the network model and network weights are recorded for the real-time system. In the real-time system, the skin color is determined for a certain frame for hand use, and the hand gesture is determined using the convex hull algorithm, and the hand gesture is defined in real-time using the registered neural network model and network weights. The accuracy of the real-time system is 98.05%.