Indian Sign Research Articles

Accelerated Low Power AI System for Indian Sign Language Recognition

Accelerated Low Power AI System for Indian Sign Language Recognition

Dynamic Indian Sign Language Recognition Based on Enhanced LSTM with Custom Attention Mechanism

Dynamic Indian Sign Language Recognition Based on Enhanced LSTM with Custom Attention Mechanism

SLR-YOLO: An improved YOLOv8 network for real-time sign language recognition

This study proposes a method to help people with different degrees of hearing impairment to better integrate into society and perform more convenient human-to-human and human-to-robot sign language interaction through computer vision. Traditional sign language recognition methods make it challenging to get good results on scenes with backgrounds close to skin color, background clutter, and partial occlusion. In order to realize faster real-time display, by comparing standard single-target recognition algorithms, we choose the best effect YOLOv8 model, and based on this, we propose a lighter and more accurate SLR-YOLO network model that improves YOLOv8. Firstly, the SPPF module is replaced with RFB module in the backbone network to enhance the feature extraction capability of the network; secondly, in the neck, BiFPN is used to enhance the feature fusion of the network, and the Ghost module is added to make the network lighter; lastly, in order to introduce partial masking during the training process and to improve the data generalization capability, Mixup, Random Erasing and Cutout three data enhancement methods are compared, and finally the Cutout method is selected. The accuracy of the improved SLR-YOLO model on the validation sets of the American Sign Language Letters Dataset and Bengali Sign Language Alphabet Dataset is 90.6% and 98.5%, respectively. Compared with the performance of the original YOLOv8, the accuracy of both is improved by 1.3 percentage points, the amount of parameters is reduced by 11.31%, and FLOPs are reduced by 11.58%.

A TinyML solution for an IoT-based communication device for hearing impaired

Research on automatic translation of sign language to verbal languages has been progressively explored in recent years to assist speech and hearing-impaired people in communicating with non-signers. In this paper, a tiny machine learning (TinyML) solution is proposed for sign language recognition using a low-cost, wearable, internet-of things (IoT) device. A lightweight deep neural network is deployed on the edge device to interpret isolated signs from the Indian sign language using the time-series data collected from the motion sensors of the device. The scarcity of labeled training data is addressed by employing the deep transfer learning approach. Here, the knowledge gained from the data collected using the motion sensors of a different device is used to initialize the model parameters. The performance of the model is assessed in terms of classification accuracy and prediction time for different sampling rates and transferring schemes. The model achieves an average accuracy of 87.18% when all the parameters are retrained with just 4 observations of each sign recorded from the motion sensors of the proposed IoT device. The recognized sign is transmitted to a cloud platform in real-time. A mobile application, SignTalk, is also developed, which wirelessly receives the predicted signs from the cloud and displays it as text. Additionally, text-to-speech conversion is also provided on SignTalk to vocalize the predicted sign for better communication.

Analysis of Vision based Techniques for the Translation of Indian Sign Language

Sign language acts as a medium of communication among those of the hearing impaired and mute community. However, it cannot be easily understood by common people. Various research has been done to bridge this gap by developing Sign Language Recognition (SLR) methodologies. Studies say that 1 in every 5 deaf people is Indian.
 In this paper, a thorough review of these methodologies has been done, to compare and contrast various aspects of them. This includes an overview on different preprocessing methods used like segmentation, image morphological processing, cropping, etc, feature extraction techniques like Fourier Descriptors, Image Moments, Eigen values, Mediapipe and others. This study also covered classification models spanning from Distance metrics to Kernel based approaches and feedforward neural networks, along with Deep Learning based methods such as CNNs, LSTMs, GANs, Transformers etc.

National Education Policy 2020 and Multilingualism in India

The National Education Policy (NEP) 2020 in India casts its attention on establishing the interconnection between languages and the medium of instruction for teaching and learning. The present essay is going to deal with the role of languages in NEP – 2020 and figure out its recommendations to be imparted in the areas of medium of instruction, classical languages, foreign languages, and Indian Sign Language. The article underlines the factors that can either be proved positive or not by a concern on how NEP revised language related proposals can be implemented now through a review of existing literature and policy documents. The study demonstrates the emphasis that this policy places on mother tongue-based multilingual education in the early pedagogical years. As well, it encourages the usage of English language and other languages. The NEP as a policy of the language seeks for celebration of the linguistic diversity of India, improved lecturing outcomes and equips the students with the necessary and requisite necessary to perform in the 21st century; eventuality calls for the logical problem solving that involves teacher training and resource development. The National Education Policy (NEP) 2020 of India is at the very core a paradigm shifts in the country's education framework with a focus placed in multilingualism promotion. This article delves into the NEP 2020 main details anchored in multilingualism, including the "3-language formula", the mother tongue as the medium of instruction, and the promotion of both classical and regional languages. The literature review focuses on the existing studies which are addressing the multilingual education’s advantages and drawbacks. A part of the discussion and analysis chapter will focus on possible both positive and negative influence of the NEP 2020 on India’s language diversity, education, and social coherence. The final section of the article re-emphasizes on the effectiveness of the implementation process and follow-up procedure to ensure that the multilingual approach under NEP 2020 will have successful outcomes.

AI-based Sensory Glove System to Recognize Bengali Sign Language (BaSL)

AI-based Sensory Glove System to Recognize Bengali Sign Language (BaSL)

MultiModal Ensemble Approach Leveraging Spatial, Skeletal, and Edge Features for Enhanced Bangla Sign Language Recognition

MultiModal Ensemble Approach Leveraging Spatial, Skeletal, and Edge Features for Enhanced Bangla Sign Language Recognition

KUNet-An Optimized AI Based Bengali Sign Language Translator for Hearing Impaired and Non Verbal People

KUNet-An Optimized AI Based Bengali Sign Language Translator for Hearing Impaired and Non Verbal People

Harnessing AI to Generate Indian Sign Language from Natural Speech and Text for Digital Inclusion and Accessibility

Harnessing AI to Generate Indian Sign Language from Natural Speech and Text for Digital Inclusion and Accessibility

Efficient Indian sign language recognition and classification using enhanced machine learning approach

Efficient Indian sign language recognition and classification using enhanced machine learning approach

Design and Development of Indian Sign Language Character Recognition System

The purpose of this study is to look into the challenges involved in categorizing Indian Sign Language (ISL) characters. While a lot of research has been done in the related field of American Sign Language (ASL), not as much has been done with ISL. Lack of standard datasets, obscured traits, and variance in language with geography are the key barriers that have hindered much ISL research. Our study aims to progress this field by collecting a dataset from a deaf school and applying various feature extraction techniques to extract useful information, which is then input into a range of supervised learning algorithms. Our current results for each approach include four fold cross-validation. What sets our work apart from earlier research is that the validation set in our four-fold cross-validation contains photographs of people who are not the same people as those in the training set. Hand gestures and signs are used by those with speech impairments to communicate. Understanding what they're trying to say is challenging for the average person. Though extremely uncommon, there are many systems that convert data to Hindi. Therefore, it is imperative to implement a system that enables the general public to understand and interpret all signals, gestures, and communications. It will close the communication gap that exists between normal people and those who have speech difficulty. The two primary research approaches centered on human-computer interaction are sign language recognition and learning. Multiple sensors are required in order for data flow to be understood in sign language. This research paper focuses on the development of a Hindi-language training tool that can detect images and interpret what someone else is trying to say to persons who have speech impairments. Keywords: Indian Sign Language (ISL), American Sign Language (ASL), Feature Extraction, Supervised learning, Sign Language, etc.

Automatic Indian sign language recognition using MediaPipe holistic and LSTM network

Automatic Indian sign language recognition using MediaPipe holistic and LSTM network

SignIt! An Android game for sign bilingual play that collects labelled sign language data

The Deaf or Hard-of-Hearing (DHH) community constitutes over 430 million people globally, with about 70 million of them using sign language as their primary means of communication. India has around 63 million DHH individuals. The DHH community in India faces several challenges, particularly in learning sign language and English, due to delayed diagnosis, stigma, oralism, and a diversity of languages. Digital games for spoken and sign language learning have gained popularity due to their advantages over traditional language learning methods, such as enhanced engagement and socialization, driving increased research and adoption over traditional methods. Moreover, the development of robust machine learning models for sign language recognition, which could significantly improve access for signers, is currently impeded by the scarcity of labelled sign language data. To address these challenges, we collaborated with NISH, an academic institution for the DHH community and developed SignIt!, an accessible and inclusive quiz platform that facilitates the learning of English and Indian Sign Language (ISL) with a secondary goal of data collection. To assess the game's usability, we conducted a study with 20 members of the DHH community, followed by interviewing 15 participants. Overall, our participants answered 2160 quiz questions and created 210 questions. The quiz creation resulted in the collection of three hours of labelled real-world sign language data. The interviews revealed novel insights, such as a preference for playing competitively with friends, empowerment by their agency to be content creators, and early signs of learning English, sign language, and quiz content by playing and creating quizzes. We plan to open-source and release SignIt! to increase its adoption among diverse DHH communities.

Advancements in Indian Sign Language Recognition Systems: Enhancing Communication and Accessibility for the Deaf and Hearing Impaired

Sign language is an ocular-mobile language utilized by deaf or hearing-impaired individuals. It conveys a combination of handshapes, motions, facial expressions, and body postures. Sign language is an essential form of communication with its unique grammar and syntax. The communication barrier between those who use sign language and those who do not is substantial. India has approximately 63 million people in the deaf or hearing-impaired community (DHH). Therefore, research within the realm of sign language recognition (SLR) shows great potential in enhancing the quality of life for individuals facing hearing disabilities and promoting better communication and integration into society. In recent years, sign language recognition has attracted considerable attention due to its significant role in human-machine interaction, accessibility, real-time interpretation, educational tools, and communication aids. This study reviews the most current developments in Indian Sign Language Recognition Systems (ISLRS). It discusses the commonly used algorithms, standard datasets, and performance characteristics of these systems in detail. Lastly, it highlights the challenges and future perspectives of these emerging technologies.

Recognition of Bangladeshi Sign Language (BdSL) Words using Deep Convolutional Neural Networks (DCNNs)

In a world where effective communication is fundamental, individuals who are Deaf and Dumb (D&D) often face unique challenges due to their primary mode of communication—sign language. Despite the interpreters' invaluable roles, their lack of availability causes communication difficulties for the D&D individuals. This study explores whether the field of Human-Computer Interaction (HCI) could be a potential solution. The primary objective is to assist D&D individuals with computer applications that could act as mediators to bridge the communication gap between them and the wider hearing population. To ensure their independent communication, we propose an automated system that could detect specific Bangla Sign Language (BdSL) words, addressing a critical gap in the sign language detection and recognition literature. Our approach leverages deep learning and transfer learning principles to convert webcam-captured hand gestures into textual representations in real-time. The model's development and assessment rest upon 992 images created by the authors, categorized into ten distinct classes representing various BdSL words. Our findings show the DenseNet201 and ResNet50-V2 models achieve promising training and testing accuracies of 99% and 93%, respectively. Doi: 10.28991/ESJ-2023-07-06-019 Full Text: PDF

Dynamic Hand Gesture Recognition for Indian Sign Language using Integrated CNN-LSTM Architecture

Human Centered Computing is an emerging research field that aims to understand human behavior. Dynamic hand gesture recognition is one of the most recent, challenging and appealing application in this field. We have proposed one vision based system to recognize dynamic hand gestures for Indian Sign Language (ISL) in this paper. The system is built by using a unified architecture formed by combining Convolutional Neural Network (CNN) and Long Short Term Memory (LSTM). In order to hit the shortage of a huge labeled hand gesture dataset, we have created two different CNN by retraining a well known image classification networks GoogLeNet and VGG16 using transfer learning. Frames of gesture videos are transformed into features vectors using these CNNs. As these videos are prearranged series of image frames, LSTM model have been used to join with the fully-connected layer of CNN. We have evaluated the system on three different datasets consisting of color videos with 11, 64 and 8 classes. During experiments it is found that the proposed CNN-LSTM architecture using GoogLeNet is fast and efficient having capability to achieve very high recognition rates of 93.18%, 97.50%, and 96.65% on the three datasets respectively.

Performance Evaluation of Feature Fusion Approaches for Indian Sign Language Recognition System

Performance Evaluation of Feature Fusion Approaches for Indian Sign Language Recognition System

Fusion of Attention-Based Convolution Neural Network and HOG Features for Static Sign Language Recognition

The deaf and hearing-impaired community expresses their emotions, communicates with society, and enhances the interaction between humans and computers using sign language gestures. This work presents a strategy for efficient feature extraction that uses a combination of two different methods that are the convolutional block attention module (CBAM)-based convolutional neural network (CNN) and standard handcrafted histogram of oriented gradients (HOG) feature descriptor. The proposed framework aims to enhance accuracy by extracting meaningful features and resolving issues like rotation, similar hand orientation, etc. The HOG feature extraction technique provides a compact feature representation that signifies meaningful information about sign gestures. The CBAM attention module is incorporated into the structure of CNN to enhance feature learning using spatial and channel attention mechanisms. Then, the final feature vector is formed by concatenating these features. This feature vector is provided to the classification layers to predict static sign gestures. The proposed approach is validated on two publicly available static Massey American Sign Language (ASL) and Indian Sign Language (ISL) databases. The model’s performance is evaluated using precision, recall, F1-score, and accuracy. Our proposed methodology achieved 99.22% and 99.79% accuracy for the ASL and ISL datasets. The acquired results signify the efficiency of the feature fusion and attention mechanism. Our network performed better in accuracy compared to the earlier studies.

Enhancing Sign Language Recognition through Fusion of CNN Models

This study introduces a pioneering hybrid model designed for the recognition of sign language, with a specific focus on American Sign Language (ASL) and Indian Sign Language (ISL). Departing from traditional machine learning methods, the model ingeniously blends hand-crafted techniques with deep learning approaches to surmount inherent limitations. Notably, the hybrid model achieves an exceptional accuracy rate of 96% for ASL and 97% for ISL, surpassing the typical 90-93% accuracy rates of previous models. This breakthrough underscores the efficacy of combining predefined features and rules with neural networks. What sets this hybrid model apart is its versatility in recognizing both ASL and ISL signs, addressing the global variations in sign languages. The elevated accuracy levels make it a practical and accessible tool for the hearing-impaired community. This has significant implications for real-world applications, particularly in education, healthcare, and various contexts where improved communication between hearing-impaired individuals and others is paramount. The study represents a noteworthy stride in sign language recognition, presenting a hybrid model that excels in accurately identifying ASL and ISL signs, thereby contributing to the advancement of communication and inclusivity.