Sign Recognition Research Articles

Continuous Sign Language Recognition for Hearing-Impaired Consumer Communication via Self-Guidance Network

Continuous Sign Language Recognition for Hearing-Impaired Consumer Communication via Self-Guidance Network

Indian traffic sign detection and recognition using deep learning

Traffic signs are a fundamental piece of transportation infrastructure and play a vital role in regulating traffic flow, enforcing proper driving behavior, and reducing the risk of accidents, injuries, and fatalities. An Intelligent Transportation System (ITS) must have the ability to automatically detect the sign and then recognise traffic signs which is to be effective. Automatic traffic sign detection is necessary. and is growing in significance with the advent of self-driving cars. This study introduces a brand-new deep learning-based method for identifying traffic signs in India. The proposed system utilizes a region-based convolutional neural network (CNN) to achieve automatic identification and recognition of traffic signs. The authors describe various architectural and data augmentation enhancements to the CNN model and take into account unique and challenging Indian traffic sign types that have not been previously discussed in literature. The system is trained and evaluated using a database of real-time images captured on Indian highways. The deep learning approach is utilized to work on the accuracy and precision of the system, determined to make automated driving automobiles.

Hand Gesture Recognition Using AI/ML

Abstract: Every day, we see many people who are facing illnesses as deaf, dumb and blind etc. They face difficulties in interacting with others. Previously developed techniques were all sensors based and they didn’t give a general solution. This article explains a new technology for virtual conversations without sensors. An image processing technique called Histogram of gradient (HOG) along with an artificial neural network (ANN) has been used to train the system. The Web Camera is used to take an image of different gestures and that will be used as input to the Mat lab. The software will recognize the image and identify the core speech output to be processed using the speech replay suite. Sign language is composed of continuous gestures, therefore, for sign language recognition, in addition to spatial domain, it also needs to capture motion information across multiple consecutive video frames.This paper explains two way communications between the deaf, dumb and normal people which means the proposed system is capable of converting the sign language to text and voice.

Optimizing road sign detection using the segment anything model for background pixel exclusion

Road sign recognition plays a great role in automatic driving. At present, for the task of road sign recognition, the structure related to CNN is mainly used to classify the detected road signs. However, due to the complexity of road conditions and weather, the detected images often contain complex background pixels, which affects the accuracy of classification. Due to the release of segment anything model, there is a powerful tool for eliminating background pixels quickly and conveniently. We optimize the data to retain only the road sign part of the images, and directly let the model train the image without back- ground pixels to realize the recognition task. This paper constructs two CNN classification models with the same structure through comparative experiments. One model uses the segmented data set for training, and the other model uses the original data set for training. The performance of the two models is evaluated and compared to verify the optimization effect brought by the segmented data set. It is found that segment anything model can accurately cut most of the images in the data set. And the performance of the model trained by the segmented data set is better than that of the model using the original data set. Moreover, the optimized model can achieve high accuracy in less training times.

Hand Sign Language Recognition and Conversion into Text Using Convolutional Neural Network

The current research looks into using convolutional neural networks, also known as CNNs, to develop a hand sign understanding and conversion system. The study addresses the urgent need for a precise and effective way to close the communication gap that exists between the hearing and the deaf communities. The study examines how well CNNs can identify and translate hand sign gestures into text, giving sign language users a real-time means of communication. We use three steps in our methodology: gathering datasets, training models, and thorough assessment. The outcomes show encouraging hand sign recognition accuracy, creating new opportunities for inclusive communication. Sign language is one of the oldest and most natural forms of language for communication, but since most people do not know sign language and interpreters are very difficult to come by, we have come up with a real-time method using neural networks for fingerspelling-based American sign language. In our method, the hand is first passed through a filter and after the filter is applied the hand is passed through a classifier which predicts the class of the hand gestures. Our method provides 95.7 % accuracy for the 26 letters of the alphabet. Keywords-Sign Language, ASL, Hearing disability, Convolutional Neural Network (CNN), Computer Vision, Machine Learning, Gesture recognition, Sign language recognition, Hue Saturation Value algorithm.

Intracranial Hypertension Syndrome Secondary to Intraventricular Hemorrhage: A Case Report

Introduction: Acute intracranial hypertension (ICH) syndrome is a neurosurgical emergency. Timely diagnosis of ICH is imperative for potential reversibility of the neurological condition. We report the case of a child with bleeding and hydrocephalus secondary to lateral ventricle injury, with an emphasis on the outcome secondary to ICH. Case Report: A 10-year-old girl presented to the emergency department with a history of progressive holocranial headache and vomiting for 1 day. She progressed with a progressive decrease in the level of consciousness. A computed tomography scan of the head was performed only 12 hours after admission, revealing intraventricular hemorrhage associated with calcifications and ventricular dilation. Upon admission to a specialized service, she had bilateral fixed mydriasis and the absence of brainstem reflexes. Emergency ventriculostomy was performed, but the patient developed brain death the following day. Conclusion: Besides the delay in diagnosis, the time between diagnosis and emergency neurosurgical treatment also constituted a significant factor in the unfavorable outcome described. In many ICH situations, a simple procedure like ventriculostomy can prevent an unfavorable outcome. Therefore, proper evaluation of pediatric patients and early recognition of clinical signs indicating intracranial hypertension are crucial for preventing an unfavorable and potentially fatal outcome.

An accurate estimation of hand gestures using optimal modified convolutional neural network

Sign language is a non-verbal communication between deaf and dumb communities that helps them to communicate or interact with other individuals. Although many sign language recognition models restrict the gap between the deaf-dumb community and normal people, there is a growing need for effective and reliable approaches to recognizing hand gestures under complex background conditions. To attain this objective, a novel modified deep convolutional neural network-based hybrid arithmetic hunger games (MDCNN-HAHG) is developed for the accurate estimation of hand gestures. The images of hand gestures are taken from two different datasets namely First-Person Hand Action (FHPA) and Dynamic Hand Gesture (DHG) dataset are utilized as input. The data distortions present in the datasets are eliminated during the preprocessing task. After, the significant features that help to accurately recognize the kind of posture are extracted in the feature extraction process. Finally, the proposed MDCNN-HAHG model efficiently recognizes and outputs the hand gestures. To improve the recognition performance of MDCNN model, its hyperparameters such as learning rate, dropout rate, size of the convolution kernel, and pooling size are adaptively tuned and reweighted using the hybrid arithmetic hunger games (HAHG) algorithm. Accuracy, precision, recall, specificity, and F-measure performance metrics are employed to estimate the performances of the MDCNN-HAHG model. The analytic result shows that the MDCNN-HAHG technique attains greater accuracy of about 97.2% for the DHG-14 dataset, 96.13% for the DHG-28 dataset, and 92.78% for the FHPA dataset in recognizing different hand gestures.

Advanced Driver-Assistance Systems: Features Journey for Tomorrow

Intelligent associated vehicles (ICVs) are accepted to completely change people sooner rather than later by making the transportation more secure, cleaner and more agreeable. Albeit numerous models of ICVs have been created to demonstrate the idea of independent driving and the plausibility of further developing traffic effectiveness, there actually exists a critical hole prior to accomplishing large scale manufacturing of undeniable level ICVs. The goal of this study is to introduce an outline of both the cutting edge and future viewpoints of key necessary advances for future ICVs. It is a moving undertaking to survey every connected work and foresee their future viewpoints, particularly for such a perplexing and interdisciplinary area of examination. Advanced driver-assistance systems (ADASs) have become a salient feature for safety in modern vehicles. They are also a key underlying technology in emerging autonomous vehicles. State-of-the-art ADASs are primarily vision based, various type of features for ex. Lane Departure Warning (LDW), Blind Spot Monitoring (BSM), Forward Collision Warning (FCW), Automatic Emergency Braking (AEB), Traffic Sign Recognition, High Beam Assist, Rear Cross Traffic Alert, Driver Drowsiness Detection, Obstacle Aware Acceleration, Auto-steer etc. The paper aims at giving a complete picture focusing on the ADAS features for the user-friendly design of human-machine interfaces between driver and assistance system.

A Framework for Protecting and Promoting Employee Mental Health through Supervisor Supportive Behaviors

The attention to workplace mental health is timely given extreme levels of burnout, anxiety, depression and trauma experienced by workers due to serious extraorganizational stressors – the COVID-19 pandemic, threats to climate change, and extreme social and political unrest. Workplace-based risk factors, such as high stress and low support, are contributing factors to poor mental health and suicidality (Choi, 2018; Milner et al., 2013, 2018), just as low levels of social connectedness and belonging are established risk factors for poor mental health (Joiner et al., 2009), suggesting that social support at work (e.g., from supervisors) may be a key approach to protecting and promoting employee mental health. Social connections provide numerous benefits for health outcomes and are as, or more, important to mortality as other well-known health behaviors such as smoking and alcohol consumption (Holt-Lundstad et al., 2015), and can serve as a resource or buffer against the deleterious effects of stress or strain on psychological health (Cohen & Wills, 1985). This manuscript provides an evidence-based framework for understanding how supervisor supportive behaviors can serve to protect employees against psychosocial workplace risk factors and promote social connection and belongingness protective factors related to employee mental health. We identify six theoretically-based Mental Health Supportive Supervisor Behaviors (MHSSB; i.e., emotional support, practical support, role modeling, reducing stigma, warning sign recognition, warning sign response) that can be enacted and used by supervisors and managers to protect and promote the mental health of employees. A brief overview of mental health, mental disorders, and workplace mental health is provided. This is followed by the theoretical grounding and introduction of MHSSB. Suggestions for future research and practice follow, all with the focus of developing a better understanding of the role of supervisors in protecting and promoting employee mental health in the workplace.

Improved Recognition of Kurdish Sign Language Using Modified CNN

The deaf society supports Sign Language Recognition (SLR) since it is used to educate individuals in communication, education, and socialization. In this study, the results of using the modified Convolutional Neural Network (CNN) technique to develop a model for real-time Kurdish sign recognition are presented. Recognizing the Kurdish alphabet is the primary focus of this investigation. Using a variety of activation functions over several iterations, the model was trained and then used to make predictions on the KuSL2023 dataset. There are a total of 71,400 pictures in the dataset, drawn from two separate sources, representing the 34 sign languages and alphabets used by the Kurds. A large collection of real user images is used to evaluate the accuracy of the suggested strategy. A novel Kurdish Sign Language (KuSL) model for classification is presented in this research. Furthermore, the hand region must be identified in a picture with a complex backdrop, including lighting, ambience, and image color changes of varying intensities. Using a genuine public dataset, real-time classification, and personal independence while maintaining high classification accuracy, the proposed technique is an improvement over previous research on KuSL detection. The collected findings demonstrate that the performance of the proposed system offers improvements, with an average training accuracy of 99.05% for both classification and prediction models. Compared to earlier research on KuSL, these outcomes indicate very strong performance.

Digital Signature Verification System Using Convolutional Neural Networks

The design, development, and evaluation of a signature verification system, a critical component of biometric authentication. The study employs two primary datasets, the MCYT (MasterCard Young Teenager) Signature Dataset and the GPDS (Greek Sign Language Recognition) Signature Dataset, to assess the system's adaptability and accuracy. The MCYT dataset, featuring 6600 signature samples, provides variability in writing styles and demographic information for potential nuanced analyses. In contrast, the GPDS dataset, comprising 4000 signatures, introduces dynamic signing variations during Greek Sign Language communication. The project aims to develop a robust signature verification system capable of handling diverse scenarios, ultimately contributing to the advancement of biometric technology. The research combines insights from real-world variability and linguistic nuances, offering a comprehensive understanding of the system's effectiveness.

Digital identification and pattern recognition capabilities using machine learning methods, navigation systems, and video surveillance

The objects of the study are unmanned vehicles and branches of the bridge of the city of Kyiv (Ukraine), which connects the Great Ring Road, Zhytomyr Highway and Peremogy Avenue. The built routes were analyzed using the technology of recognition of road signs, people and vehicles. The important problem of this research is to analyze the possibilities of detecting obstacles by an unmanned vehicle using pattern recognition, which combines the methods of machine communication, navigation and real-time video surveillance. Based on the study, the results of detecting and avoiding obstacles on the road, where a study was conducted to investigate the main reasons that can cause time delays (traffic jams, weather conditions, accidents). The results of planning and navigation are obtained to determine the appropriate road route, which allows detecting and eliminating obstacles on the road, as well as building a map plan of the route in advance using online map services (Google Maps). It is shown that recognition of road signs (based on the classification using a road sign map consisting of 7 categories), people and vehicles minimizes the occurrence of road accidents, traffic jams and time delays. To recognize the images of road signs, people and vehicles, we studied the road sections connecting to the branched bridge. Thus, the authors have reviewed and analyzed the digital capabilities of pattern identification and recognition using machine learning methods, navigation and video surveillance systems, where the safety of vehicles with detection of road signs and obstacles on the way is of great importance. The results obtained can complement the possibilities of using unmanned vehicles to avoid obstacles and road accidents based on a trained pattern recognition system. This system, using convolutional neural networks and video surveillance navigation systems, will be able to provide the driver and the people around it with safe driving conditions.

Dynamic Japanese Sign Language Recognition Throw Hand Pose Estimation Using Effective Feature Extraction and Classification Approach.

Japanese Sign Language (JSL) is vital for communication in Japan's deaf and hard-of-hearing community. But probably because of the large number of patterns, 46 types, there is a mixture of static and dynamic, and the dynamic ones have been excluded in most studies. Few researchers have been working to develop a dynamic JSL alphabet, and their performance accuracy is unsatisfactory. We proposed a dynamic JSL recognition system using effective feature extraction and feature selection approaches to overcome the challenges. In the procedure, we follow the hand pose estimation, effective feature extraction, and machine learning techniques. We collected a video dataset capturing JSL gestures through standard RGB cameras and employed MediaPipe for hand pose estimation. Four types of features were proposed. The significance of these features is that the same feature generation method can be used regardless of the number of frames or whether the features are dynamic or static. We employed a Random forest (RF) based feature selection approach to select the potential feature. Finally, we fed the reduced features into the kernels-based Support Vector Machine (SVM) algorithm classification. Evaluations conducted on our proprietary newly created dynamic Japanese sign language alphabet dataset and LSA64 dynamic dataset yielded recognition accuracies of 97.20% and 98.40%, respectively. This innovative approach not only addresses the complexities of JSL but also holds the potential to bridge communication gaps, offering effective communication for the deaf and hard-of-hearing, and has broader implications for sign language recognition systems globally.

Deep Learning Based Traffic Sign Recognition Using YOLO Algorithm

Traffic sign detection has attracted a lot of attention in recent years among object recognition applications. Accurate and fast detection of traffic signs will also eliminate an important technical problem in autonomous vehicles. With the developing artificial intelligency technology, deep learning applications can distinguish objects with high perception and accurate detection. New applications are being tested in this area for the detection of traffic signs using artificial intelligence technology. In this context, this article has an important place in correctly detecting traffic signs with deep learning algorithms. In this study, three model of (You Only Look Once) YOLOv5, an up-to-date algorithm for detecting traffic signs, were used. A system that uses deep learning models to detect traffic signs is proposed. In the proposed study, real-time plate detection was also performed. When the precision, recall and mAP50 values of the models were compared, the highest results were obtained as 99.3, 95% and 98.1%, respectively. Experimental results have supported that YOLOv5 architectures are an accurate method for object detection with both image and video. It has been seen that YOLOv5 algorithms are quite successful in detecting traffic signs and average precession.

Location and segmentation of important railway signs based on improved segmentation

Purpose With the rapid development of railway-intelligent video technology, scene understanding is becoming more and more important. Semantic segmentation is a major part of scene understanding. There is an urgent need for an algorithm with high accuracy and real-time to meet the current railway requirements for railway identification. In response to this demand, this paper aims to explore a variety of models, accurately locate and segment important railway signs based on the improved SegNeXt algorithm, supplement the railway safety protection system and improve the intelligent level of railway safety protection. Design/methodology/approach This paper studies the performance of existing models on RailSem19 and explores the defects of each model through performance so as to further explore an algorithm model dedicated to railway semantic segmentation. In this paper, the authors explore the optimal solution of SegNeXt model for railway scenes and achieve the purpose of this paper by improving the encoder and decoder structure. Findings This paper proposes an improved SegNeXt algorithm: first, it explores the performance of various models on railways, studies the problems of semantic segmentation on railways and then analyzes the specific problems. On the basis of retaining the original excellent MSCAN encoder of SegNeXt, multiscale information fusion is used to further extract detailed features such as multihead attention and mask, solving the problem of inaccurate segmentation of current objects by the original SegNeXt algorithm. The improved algorithm is of great significance for the segmentation and recognition of railway signs. Research limitations/implications The model constructed in this paper has advantages in the feature segmentation of distant small objects, but it still has the problem of segmentation fracture for the railway, which is not completely segmented. In addition, in the throat area, due to the complexity of the railway, the segmentation results are not accurate. Social implications The identification and segmentation of railway signs based on the improved SegNeXt algorithm in this paper is of great significance for the understanding of existing railway scenes, which can greatly improve the classification and recognition ability of railway small object features and can greatly improve the degree of railway security. Originality/value This article introduces an enhanced version of the SegNeXt algorithm, which aims to improve the accuracy of semantic segmentation on railways. The study begins by investigating the performance of different models in railway scenarios and identifying the challenges associated with semantic segmentation on this particular domain. To address these challenges, the proposed approach builds upon the strong foundation of the original SegNeXt algorithm, leveraging techniques such as multi-scale information fusion, multi-head attention, and masking to extract finer details and enhance feature representation. By doing so, the improved algorithm effectively resolves the issue of inaccurate object segmentation encountered in the original SegNeXt algorithm. This advancement holds significant importance for the accurate recognition and segmentation of railway signage.

Spatial–temporal feature-based End-to-end Fourier network for 3D sign language recognition

Most dynamic sign word misclassifications are caused by redundant spatial–temporal (SPT) feature pruning that lacks language semantic and temporal dependencies. SPT feature recognition is one of the important aspects for the evaluation of the misclassification of dynamic sign words. The redundant pruning of SPT feature space influences the language model of sign confusion, model complexity, and SPT feature similarity. The purpose of this article is to develop a new multi-scale SPT feature-based dynamic sign word recognition approach via a low-cost feature selection method (FS) and End-to-end Fourier convolution neural network (EFCNN). Instead of a sensor fusion technique for obtaining frame position alignment, in the EFCNN, new 3D frame position and coordinates are determined using a pixel weighting and alignment function of the first and succeeding 25 spatial intensities of the 3D video changes across hand motion. The new spatial weight and the original spatial coordinates are fused and truncated in the Fourier domain. We generate the temporal dependence of the fused features. A feature selection known as the FS-EFCNN is introduced to select compact features with a preserved language meaning. Five state-of-the-art feature selection methods, namely Infinite FS (InFS), Relief FS, Fisher, MIM, ILFS, and ensemble FS-EFCNN were deployed to guide and optimize the learning performance of EFCNN. The experimental result analysis highlighted the improved results of the FS-EFCNN method with the best accuracy of 99.86%, 99.89%, and 90.69% on 3D American Sign Language, British Sign Language, and Greek Sign Language data sets, respectively.

CCTSDB dataset enhancement based on a cross-augmentation method for image datasets

In the digital era, the rapid advancement of artificial intelligence has put a spotlight on target detection, especially in traffic settings. This area of study is pivotal for crucial projects like autonomous vehicles, road monitoring, and traffic sign recognition. However, existing Chinese traffic datasets lack comprehensive benchmarks for traffic signs and signals, and foreign datasets do not match Chinese traffic conditions. Manually annotating a large-scale dataset tailored for Chinese traffic conditions presents a significant challenge. This study addresses this gap by proposing a cross-augmentation method for image datasets. We utilized YOLOX for target detection and trained models on the BDD100K dataset, achieving an impressive mAP of 60.25%, surpassing most algorithms. Leveraging transfer learning, we enhanced the CCTSDB dataset, creating the ACCTSDB dataset, which includes annotations for common traffic objects and Chinese traffic signs. Using YOLOX, we trained a traffic detector tailored for Chinese traffic scenarios, achieving an mAP of 75.79%. To further validate our approach, we conducted experiments on the TT100K dataset and successfully introduced the ATT100K dataset. Our methodology is poised to alleviate the limitations of manually annotating image datasets. The proposed ACCTSDB dataset and ATT100K dataset are expected to compensate for the lack of large-scale, multi-class traffic datasets in China.

Traffic Sign Detection and Recognition Using YOLO Object Detection Algorithm: A Systematic Review

Context: YOLO (You Look Only Once) is an algorithm based on deep neural networks with real-time object detection capabilities. This state-of-the-art technology is widely available, mainly due to its speed and precision. Since its conception, YOLO has been applied to detect and recognize traffic signs, pedestrians, traffic lights, vehicles, and so on. Objective: The goal of this research is to systematically analyze the YOLO object detection algorithm, applied to traffic sign detection and recognition systems, from five relevant aspects of this technology: applications, datasets, metrics, hardware, and challenges. Method: This study performs a systematic literature review (SLR) of studies on traffic sign detection and recognition using YOLO published in the years 2016–2022. Results: The search found 115 primary studies relevant to the goal of this research. After analyzing these investigations, the following relevant results were obtained. The most common applications of YOLO in this field are vehicular security and intelligent and autonomous vehicles. The majority of the sign datasets used to train, test, and validate YOLO-based systems are publicly available, with an emphasis on datasets from Germany and China. It has also been discovered that most works present sophisticated detection, classification, and processing speed metrics for traffic sign detection and recognition systems by using the different versions of YOLO. In addition, the most popular desktop data processing hardwares are Nvidia RTX 2080 and Titan Tesla V100 and, in the case of embedded or mobile GPU platforms, Jetson Xavier NX. Finally, seven relevant challenges that these systems face when operating in real road conditions have been identified. With this in mind, research has been reclassified to address these challenges in each case. Conclusions: This SLR is the most relevant and current work in the field of technology development applied to the detection and recognition of traffic signs using YOLO. In addition, insights are provided about future work that could be conducted to improve the field.

Isolated Arabic Sign Language Recognition Using a Transformer-based Model and Landmark Keypoints

Pose-based approaches for sign language recognition provide light-weight and fast models that can be adopted in real-time applications. This article presents a framework for isolated Arabic sign language recognition using hand and face keypoints. We employed MediaPipe pose estimator for extracting the keypoints of sign gestures in the video stream. Using the extracted keypoints, three models were proposed for sign language recognition: Long-Term Short Memory, Temporal Convolution Networks, and Transformer-based models. Moreover, we investigated the importance of non-manual features for sign language recognition systems and the obtained results showed that combining hand and face keypoints boosted the recognition accuracy by around 4% compared with only hand keypoints. The proposed models were evaluated on Arabic and Argentinian sign languages. Using the KArSL-100 dataset, the proposed pose-based Transformer achieved the highest accuracy of 99.74% and 68.2% in signer-dependent and -independent modes, respectively. Additionally, the Transformer was evaluated on the LSA64 dataset and obtained an accuracy of 98.25% and 91.09% in signer-dependent and -independent modes, respectively. Consequently, the pose-based Transformer outperformed the state-of-the-art techniques on both datasets using keypoints from the signer’s hands and face.

Sign Language Analysis Using CNN Algorithm

Sign language is a rich and diverse mode of communication used by Deaf and hard of-hearing communities around the world. This paper presents a comprehensive analysis of sign language, focusing on its linguistic and gestural elements. Our study encompasses an exploration of the structural properties of sign languages, their historical evolution, and the cognitive and neural mechanisms underlying sign language processing. We begin by providing an overview of sign language as a natural and fully developed language with its own syntax, semantics, and phonological features. Drawing from various sign languages, including American Sign Language (ASL), British Sign Language (BSL), and others, we investigate the grammatical aspects and iconicity within signs. Furthermore, we examine the role of facial expressions and non-manual markers in conveying essential linguistic information. This analysis also delves into the historical and cultural dimensions of sign languages, shedding light on the evolution of sign languages and the influence of Deaf culture on their development. It highlights the importance of recognizing sign languages as unique cultural and linguistic entities, distinct from spoken languages. Keywords:Computer Vision; Convolutional Neural Networks; Sign Language Recognition