Sign Recognition Research Articles

Enhancing autonomous driving safety: A robust traffic sign detection and recognition model TSD-YOLO

As autonomous driving technology rapidly advances, Traffic Sign Detection and Recognition (TSDR) has become pivotal in ensuring the safety and regulatory compliance of autonomous vehicles. Despite progress, existing technologies struggle under challenging conditions such as adverse weather and complex roadway environments. To overcome these obstacles, we introduce a novel model, TSD-YOLO, which leverages Mamba and YOLO technologies to enhance the accuracy and robustness of traffic sign detection. Our innovative YOLO-MAM dual-branch module merges convolutional layer-based local feature extraction with the long-distance dependency capabilities of the State Space Models (SSMs). We conducted experimental validations using the Tsinghua-Tencent 100K (TT-100K) dataset and the Mapillary Traffic Sign Detection (MTSD) dataset, demonstrating our model’s efficacy across various datasets. Furthermore, cross-dataset validations affirm the model’s exceptional generalization and robustness across diverse environments. This study not only bolsters traffic sign detection and recognition in autonomous driving systems but also paves the way for future advancements in autonomous driving technology.

Clinical profile and outcomes of rejection after deep anterior lamellar keratoplasty in keratoconus.

To report the clinical features and management outcomes of rejection after deep anterior lamellar keratoplasty (DALK). The clinical features, indications, predisposing risk factors, management, and outcome of six eyes of six patients with rejection after DALK were analyzed retrospectively. The median time of rejection from the day of surgery was 12 (range 3-36) months. Median duration of symptoms was 4 (range 2-28) days. The indication for DALK was keratoconus in all eyes. The clinical features at presentation included graft haze, vascularization, and melt at the graft-host junction. The risk factors noted were noncompliance to steroids in all eyes, vernal conjunctivitis (four eyes), and eccentric graft (one eye). The rejection episode reversed with topical steroids in all eyes and intravenous methylprednisolone in one eye. One eye developed secondary fungal keratitis after initial recovery and one eye had a second episode of rejection 3 months later, which resulted in progressive vessels and lipid keratopathy in the visual axis. The median best corrected visual acuity was 20/40 (range 20/30-20/200). Severe stromal rejection can present with graft melt. Repeat episodes of rejection can occur on sudden cessation of topical steroids. Prompt recognition of clinical signs and symptoms with timely management helps in quick reversal of the rejection episode. Reversal of graft rejection results in good visual outcomes.

Sign Tone: A Deep Learning-Based Deaf Companion System for Two Way Communication Between Deaf and Non-Deaf Individuals

Communication is essential to express and receive information, knowledge, ideas, and views among people, but it has been quite a while to be an obstruction for people with hearing and mute disabilities. Sign language is one method of communicating with deaf people. Though there is sign language to communicate with non-sign people it is difficult for everyone to interpret and understand. The performance of existing sign language recognition approaches is typically limited. Developing an assistive device that will translate the sign language to a readable format will help the deaf-mutes to communicate with ease to the common people. Recent advancements in the development of deep learning, deep neural networks, especially Temporal convolutional networks (TCNs) have provided solutions to the communication of deaf and mute individuals. In this project, the main objective is to design Deaf Companion System for that to develop SignNet Model to provide two-way communication of deaf individuals and to implement an automatic speaking system for deaf and mute people. It provides two-way communication for all classes of people (deaf-and-mute, hard of hearing, visually impaired, and non-signers) and can be scaled commercially. The proposed system, consists of three modules; the sign recognition module (SRM) that recognizes the signs of a deaf individual using TCN, the speech recognition using Hidden Marko Model and synthesis module (SRSM) that processes the speech of a non-deaf individual and converts it to text, and an Avatar module (AM) to generate and perform the corresponding sign of the non-deaf speech, which were integrated into the sign translation companion system called deaf companion system to facilitate the communication from the deaf to the hearing and vice versa. The proposed model is trained on Indian Sign Language. Then developed a web-based user interface to deploy SignNet Model for ease of use. Experimental results on MNIST sign language recognition datasets validate the superiority of the proposed framework. The TCN model gives an accuracy of 98.5%..

Research on a Recognition Algorithm for Traffic Signs in Foggy Environments Based on Image Defogging and Transformer.

The efficient and accurate identification of traffic signs is crucial to the safety and reliability of active driving assistance and driverless vehicles. However, the accurate detection of traffic signs under extreme cases remains challenging. Aiming at the problems of missing detection and false detection in traffic sign recognition in fog traffic scenes, this paper proposes a recognition algorithm for traffic signs based on pix2pixHD+YOLOv5-T. Firstly, the defogging model is generated by training the pix2pixHD network to meet the advanced visual task. Secondly, in order to better match the defogging algorithm with the target detection algorithm, the algorithm YOLOv5-Transformer is proposed by introducing a transformer module into the backbone of YOLOv5. Finally, the defogging algorithm pix2pixHD is combined with the improved YOLOv5 detection algorithm to complete the recognition of traffic signs in foggy environments. Comparative experiments proved that the traffic sign recognition algorithm proposed in this paper can effectively reduce the impact of a foggy environment on traffic sign recognition. Compared with the YOLOv5-T and YOLOv5 algorithms in moderate fog environments, the overall improvement of this algorithm is achieved. The precision of traffic sign recognition of the algorithm in the fog traffic scene reached 78.5%, the recall rate was 72.2%, and mAP@0.5 was 82.8%.

Traffic Object Detection and Recognition Systems

You are already known about automatic vehicles in which the car can control itself. Cars must clearly understand and recognize all traffic signals. Many organizations named Uber, Google, Tesla, Toyota, Mercedes-Benz, Ford, Audi and others are getting involved on this technology to enhance their experience by adding features like autonomous driving and putting efforts in maximum innovation in this field. As a result, if we want to work with this technology accurately it depends on how the vehicle can distinguish between different signs such as no entry, height limit, turning signs, school signs, hospital signs, and many others. Traffic sign recognition is the process of differentiating the traffic signals into similar classes. Here we created a deep-neural-network system that can differentiate traffic signs. Using this system, we can analyze and process different traffic signals which plays a major role in all automatic vehicles. By using CNN, we propose an automated system for traffic sign detection, firstly conversion of original image to grey scale image takes place with the help of some vector machines used there, after that the convolutional-neural network is applied with limited and learnable layer for analyzing. Here it tries to crop the image boundary as per the original have.

Enhancing road sign recognition using Convolutional Neural Networks

Enhancing road sign recognition using Convolutional Neural Networks

Adversarial autoencoder for continuous sign language recognition

Adversarial autoencoder for continuous sign language recognition

Advancements and Challenges in Hand Gesture Recognition: A Comprehensive Review

Hand gesture recognition is a quickly developing field with many uses in human-computer interaction, sign language recognition, virtual reality, gaming, and robotics. This paper reviews different ways to model hands, such as vision-based, sensor-based, and data glove-based techniques. It emphasizes the importance of accurate hand modeling and feature extraction for capturing and analyzing gestures. Key features like motion, depth, color, shape, and pixel values and their relevance in gesture recognition are discussed. Challenges faced in hand gesture recognition include lighting variations, complex backgrounds, noise, and real-time performance. Machine learning algorithms are used to classify and recognize gestures based on extracted features. The paper emphasizes the need for further research and advancements to improve hand gesture recognition systems' robustness, accuracy, and usability. This review offers valuable insights into the current state of hand gesture recognition, its applications, and its potential to revolutionize human-computer interaction and enable natural and intuitive interactions between humans and machines. In simpler terms, hand gesture recognition is a way for computers to understand what people are saying with their hands. It has many potential applications, such as allowing people to control computers without touching them or helping people with disabilities communicate. The paper reviews different ways to develop hand gesture recognition systems and discusses the challenges and opportunities in this area.

Enhancing communication: Deep learning for Arabic sign language translation

Abstract This study explores the field of sign language recognition through machine learning, focusing on the development and comparative evaluation of various algorithms designed to interpret sign language. With the prevalence of hearing impairment affecting millions globally, efficient sign language recognition systems are increasingly critical for enhancing communication for the deaf and hard-of-hearing community. We review several studies, showcasing algorithms with accuracies ranging from 63.5 to 99.6%. Building on these works, we introduce a novel algorithm that has been rigorously tested and has demonstrated a perfect accuracy of 99.7%. Our proposed algorithm utilizes a sophisticated convolutional neural network architecture that outperforms existing models. This work details the methodology of the proposed system, which includes preprocessing, feature extraction, and a multi-layered CNN approach. The remarkable performance of our algorithm sets a new benchmark in the field and suggests significant potential for real-world application in assistive technologies. We conclude by discussing the impact of these findings and propose directions for future research to further improve the accessibility and effectiveness of sign language recognition systems.

A systematic study of traffic sign recognition and obstacle detection in autonomous vehicles

A systematic study of traffic sign recognition and obstacle detection in autonomous vehicles

Quantifying inconsistencies in the Hamburg Sign Language Notation System

The advent of machine learning (ML) has significantly advanced the recognition and translation of sign languages, bridging communication gaps for hearing-impaired communities. At the heart of these technologies is data labeling, crucial for training ML algorithms on a huge amount of consistently labeled data to achieve models that generalize well. The adoption of language-agnostic annotations is essential to connect different sign languages, as single-language databases often provide limited lexicon examples, insufficient for training robust ML algorithms. This study critically examines the Hamburg Sign Language Notation System (HamNoSys), which describes the signer’s initial position and body movements, in contrary to the meanings of glosses. Despite HamNoSys’s utility in standardizing transcriptions across various sign languages, our investigation uncovers inconsistencies within HamNoSys that may negatively impact the development of accurate and reliable ML models. By analyzing HamNoSys labels across five sign languages, we identified a lack of standardized annotation procedures and the complexities within HamNoSys that introduce biases and errors. Our findings underscore the urgent need for unified, standardized data annotation guidelines to enhance the accuracy and efficiency of sign language recognition technologies. This research highlights the importance of addressing annotation challenges and advocates for a comprehensive, diversified database to improve the generalization of ML models.

GRTR: Gradient Rebalanced Traffic Sign Recognition for Autonomous Vehicles

Traffic sign recognition is a crucial aspect of autonomous vehicle research, and deep learning techniques have significantly contributed to its progress. Nevertheless, the distribution of traffic sign information in natural complex road conditions is long-tailed, and traffic sign identification in complex road conditions has become a significant barrier to autonomous vehicle applications. The imbalanced distribution of information on the dataset migrates to the feature space during training, resulting in imbalanced classifier prediction. In this paper, we propose the gradient rebalanced traffic sign recognition (GRTR) method to address this problem for the first time. GRTR first evaluates the prediction and classification bias of the classifier using the fitted deviation between the model’s output probability and the ground-truth distributions. Then, GRTR dynamically adjusts the correction and compensation factors following the classifier’s prediction and classification biases. GRTR rebalances the positive and negative sample gradients for each category based on the synergistic effect of the correction and compensation factors to prevent the transfer of distribution imbalance and to significantly enhance the performance of the traffic sign classifier under difficult road conditions. Experimental results demonstrate that our GRTR achieves state-of-the-art performance on long-tailed traffic sign and multilabel datasets. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Most traffic sign recognition algorithms are still designed based on the assumption of a balanced distribution of traffic signs in the dataset. Real-world autonomous vehicles require traffic sign recognition on datasets with severely imbalanced distributions. This paper proposes a general approach to solving the long-tailed traffic sign recognition problem.

TRAFFIC SIGN RECOGNITION IN CHALLENGING WEATHER CONDITIONS USING CONVOLUTIONAL NEURAL NETWORKS

The ability of autonomous driving systems to recognize traffic signs in a variety of environmental conditions is crucial to their reliability. This study uses convolutional neural networks (CNNs) to provide a novel method for improving the accuracy of traffic sign recognition systems under challenging weather situations. The research focuses on developing a CNN-based model that is trained using the augmented German Traffic Sign Recognition Benchmark (GTSRB) dataset, which contains over 50,000 labeled images of road signs across 43 categories. To attempt to overcome the limitations of traditional CNN models under adverse environmental conditions, this work presents adaptive feature extraction layers that are especially made to reduce visibility problems brought on by rain, fog, and snow. By taking a comprehensive approach, the model uses advanced data augmentation methods to simulate different weather scenarios, greatly increasing the diversity of the training dataset. Through an analysis of theoretical and practical aspects, the study demonstrates how CNNs enhance the accuracy and efficiency of road sign detection systems in a different weather condition. Research evaluates the model's effectiveness using metrics such as precision, recall, and F1-score, proving its capability to reduce false positives and accurately detect relevant road sign instances. Additionally, the paper highlights the importance of careful dataset preparation and augmentation, model optimization, and training enhancements to improve the performance of road sign detection systems. The results of this research offer benefits for intelligent transport systems, autonomous driving, and road safety, pointing towards further advancements in precise and dependable road sign recognition technology.

Sign Language Recognition

Abstract: Communication through signs has consistently been a significant way for communication among hearing and speech impaired humans, generally called deaf and dumb. It is the only mode of communicating for such individuals to pass on their messages to other human beings, and hence other humans need to comprehend their language. In this project, a sign language detection or recognition web framework is proposed with the help of image processing. This application would help in recognizing Sign Language. The dataset used is the Sign Language dataset. This application could be used in schools or any place, which would make the communication process easier between the impaired and non-impaired people. The proposed method can be used for the ease of recognition of sign language. The method used is Deep Learning for image recognition and the data is trained using Random Forest. Using this method, we would recognize the gesture and predict which sign is shown

Sign Language Recognition (SLR)

Abstract— Sign Language is mainly used by deaf (hard hearing) and dumb people to exchange information between their own community and with other people. It is a language where people use their hand gestures to communicate as they can’t speak or hear. Sign Language Recognition (SLR) deals with recognizing the hand gestures acquisition and continues till text or speech is generated for corresponding hand gestures. Here hand gestures for sign language can be classified as static and dynamic. However, static hand gesture recognition is simpler than dynamic hand gesture recognition, but both recognition is important to the human community. We can use Deep Learning Computer Vision to recognize the hand gestures by building Deep Neural Network architectures (Convolution Neural Network Architectures. Index Terms – HTML, CSS, Python, Pip, Mediapip, opencv, NumPy

SignBridge: Bridging Communication Gaps with a Video Calling App for Inclusive Conversations – Integrating Sign-Language Recognition for the Deaf and Mute

SignBridge: Bridging Communication Gaps with a Video Calling App for Inclusive Conversations – Integrating Sign-Language Recognition for the Deaf and Mute

Perturbation Augmentation for Adversarial Training with Diverse Attacks

Adversarial Training (AT) aims to alleviate the vulnerability of deep neural networks to adversarial perturbations. However, the AT techniques struggle to maintain the performance on natural samples while improving the deep model’s robustness. The absence of perturbation diversity in generated during the adversarial training degrades the generalizability of the robust models, causing overfitting to particular perturbations and a decrease in natural performance. This study proposes an adversarial training framework that augments adversarial directions from a single-step attack to address the trade-off between robustness and generalization. Inspired by feature scattering adversarial training, the proposed framework computes a principal adversarial direction with a single-step attack that finds a perturbation disrupting the inter-sample relationships in the mini-batch during adversarial training. The principal direction obtained at each iteration is augmented by sampling new adversarial directions within a region spanning 45 degrees from the principal adversarial direction. The proposed adversarial training approach does not require extra backpropagation steps in adversarial direction augmentation. Therefore, generalization of the robust model is improved without posing an additional burden on the feature scattering adversarial training. Experiments on CIFAR-10, CIFAR-100, SVHN, Tiny-ImageNet, and The German Traffic Sign Recognition Benchmark consistently improve the accuracy on adversarial with an almost pristine natural performance.

Tinysign: sign language recognition in low resolution settings

Tinysign: sign language recognition in low resolution settings

Unveiling Interpretability: Analyzing Transfer Learning in Deep Learning Models for Traffic Sign Recognition

Unveiling Interpretability: Analyzing Transfer Learning in Deep Learning Models for Traffic Sign Recognition

Silent no more: a comprehensive review of artificial intelligence, deep learning, and machine learning in facilitating deaf and mute communication

People who often communicate via sign language are essential to our society and significantly contribute. They struggle with communication mostly because other people, who often do not understand sign language, cannot interact with them. It is necessary to develop a dependable system for automatic sign language recognition. This paper aims to provide a comprehensive review of the advancements in artificial intelligence (AI), deep learning (DL), and machine learning (ML) technologies that have been used to facilitate communication for individuals who are deaf and mute (D–M). This study explores various applications of these technologies, including sign language interpretation, speech recognition, and text-to-speech synthesis. By examining the current state of research and development in AI, ML, and DL for the D–M field, the survey sheds light on the potential and challenges faced in utilizing AI, deep learning, and ML to bridge the communication gap for the D–M community. The findings of this survey will contribute to a greater understanding of the potential impact of these technologies in improving access to communication for individuals who are D–M, thereby aiding in the development of more inclusive and accessible solutions.