Sign Recognition Research Articles

A Sign Language Recognition using Improved Grey Wolf Optimization based neural networks

Sign language was developed to enable the deaf and hard of hearing community to communicate with society and convey information. It is the primary means by which they can interact with each other, as well as with the general population. The automatic gesture recognition system described in this paper uses Gray Wolf Optimization (GWO) to identify the most relevant features from a set of gesture images, with the aim of improving model performance. The main steps of the system are explained, including image preprocessing, feature extraction, GWO-based feature selection, classifier training, and evaluation. The proposed model achieved an accuracy of 99.9%.

Kurdish Sign Language Recognition Using Convolutional Neural Network (CNN)

The present study examines the obstacles encountered by the deaf population, with a specific emphasis on the increasing significance of sign language in facilitating effective communication. The main mode of communication for deaf persons is Sign Language (SL), which is a visual and expressive method of communicating meaning via facial expressions, hand motions, and body gestures. The objective of this project is to automate the identification of sign language in order to improve accessibility and decrease reliance on interpreters. Specifically, the work focused on constructing an alphabet recognition system for Kurdish Sign Language (KSL). Due to its many intricacies and resemblances to the Arabic script, KSL requires a strong recognition model. The proposed method utilizes Convolutional Neural Networks (CNN) with an actual dataset to accurately recognize both numerical values and alphabetic characters in the Korean Sign Language (KSL). Real-time operation allows for rapid recognition of hand gestures and subsequent provision of textual output. The collection comprises 132,000 hand pictures, including 33 alphabetica and (0-9) numeral signs. Significantly, the use of MediaPipe, a method for processing 3D images, greatly improves the efficiency of detection. Multiple methodologies were investigated, and the integration of Convolutional Neural Networks (CNN), TensorFlow, and MediaPipe resulted in a remarkable accuracy of 99.87% with negligible dropout rates. This study establishes a foundation for enhanced communication and independence for the deaf population, representing a notable advancement in the automation of sign language recognition.

Development of Sensor Data Fusion and Optimized Elman Neural Model-based Sign Language Recognition System

The sign language recognition system has placed an important role in disabled people’s lives. The researchers utilize various methods to fulfill disabled people’s requirements. However, the methods fail to access their sign at a reasonable cost with minimum computational difficulties. The improper sign access causes a reduction of the sign language recognition accuracy. Therefore, this study uses the sensor of wearable devices to capture people’s motions and actions to identify sign language. The collected information is fused into a competition level that understands disabled people’s requirements. After that, fused information is processed by the Pareto Optimized Hypertuned Deep Elman Neural Model (POHDENM). In addition, the data augmentation process is incorporated to train the data, which helps to recognize the large volume of sentences. Here, the fused information is split into words processed by the neural model that recognizes the sign language features. The extracted features are analyzed by an optimized method that recognizes the language with maximum accuracy. During the analysis, network parameters are hyper-tuned with the help of the Pareto model, which reduces the misrecognition error rate. Then, the created system efficiency is evaluated using the experimental analysis. The POHDENM approach achieves a high accuracy of 97.86% and an F1-score of 98.08%. The model’s high performance is achieved by fine-tuning its hyperparameters using the Pareto optimization algorithm, which balances precision of 98.06% and recall of 98.12%.

Analysis of Factors Influencing Traffic Sign Recognition Based on Multi-Sensor Perception Data

When traffic signs are blocked or faded, the ability of automated vehicles (AVs) to perceive and recognize them is lost or reduced. Previous research has improved and optimized the recognition algorithm but has not systematically analyzed the environmental and design factors that can lead to the AV’s failure to perceive and recognize traffic signs. This study uses the multiple sensors in the equipped vehicles of the Tongji University Road and Traffic Holographic Data Acquisition System to perceive and recognize traffic signs in six crash hotspots in Shanghai, China. Failure to recognize traffic signs was analyzed considering the influence of four sets of factors: traffic sign design, vehicle characteristics, environmental factors, and road design. A feature importance analysis was conducted utilizing the machine learning algorithm model CatBoost. Shapley additive explanation values were used to analyze the positive and negative relationships as well as the contribution rates of each influencing factor. The results revealed that the major factors influencing sign perception and recognition were positioning of traffic sign, lighting, blockage, cleanliness, fading, and reflectivity. Cleanliness had the most significant impact on the perception and recognition of traffic signs compared with the other influencing factors, and it also showed significant influencing interaction with the other factors. Fading was the second most important factor. The results of this study can guide designers of traffic signs to promote safety in mixed traffic environments that include both AVs and human-driven vehicles.

A novel system architecture of intelligent adaptive cruise control for safety aspects

Following industrial and safety standards for autonomous vehicles, Adaptive Cruise Control (ACC) is a widely employed Advanced Driving Assistance System (ADAS) feature in modern vehicles. ACC currently facilitates speed control based on the driver's desired speed value. This study introduces a significant advancement: the Intelligent Adaptive Cruise Control (IACC) feature, accompanied by the development of a control system architecture poised to make noteworthy contributions in scientific, economic, and social dimensions through its integration into autonomous vehicles. The design incorporates crucial elements such as Traffic Sign and Limit Recognition (TSLR), ADAS features, and Global Positioning System (GPS) data, primarily enhancing driver safety through these supportive features. The main focus revolves around designing a system architecture that accommodates these new features to ensure safe driving. The creation of the IACC system architecture is approached using Model-Based System Engineering (MBSE). Through this MBSE methodology, system-level diagrams were crafted, and security considerations were systematically addressed. Several scenarios were devised to evaluate the contributions and were subsequently tested and analyzed. The architecture places particular emphasis on the security aspects of IACC. Leveraging the TSLR feature, the system interprets traffic signs and acquires speed limit data from external sources, preventing the vehicle's speed from exceeding the specified limit. The comparison between the set speed value and the speed limit ensures adherence to safety parameters. In such scenarios, the system enhances driver support on winding roads by utilizing GPS data to recognize the vehicle in front. This approach significantly elevates the reliability of the IACC feature, particularly in terms of safety sensitivity, when compared to other adaptive cruise control concepts.

CNN Algorithm with SIFT to Enhance the Arabic Sign Language Recognition

Sign language is used as a primary means of communication by millions of people who suffer from hearing problems. The unhearing people used visual language to interact with each other, Represented in sign language. There are features that the hearing impaired use to understand each other, which are difficult for normal people to understand. Therefore, deaf people will struggle to interact with society. This research aims to introduce a system for recognizing hand gestures in Arabic Sign Language (ArSL) through training the Convolutional Neural Network (CNN) on the images of ArSL gestures launched by the University of Prince Mohammad Bin Fahd, Saudi Arabia. A Scale Invariant Feature Transform (SIFT) algorithm is used for creating the feature vectors that contain shape, finger position, size, center points of palm, and hand margin by extracting the Important features for images of ArSL and transforming them to points of the vector. The accuracy of the proposed system is 97% using the SIFT with CNN, and equal to 94.8% nearly without SIFT. Finally, the proposed system was tried and tested on a group of persons and its effectiveness was proven after considering their observations.

Traffic sign recognition using optimized YOLO network based on multi-scale feature extraction and attention mechanism

Traffic sign recognition using optimized YOLO network based on multi-scale feature extraction and attention mechanism

Hardware acceleration of Tiny YOLO deep neural networks for sign language recognition: A comprehensive performance analysis

In this paper, we benchmark two automation frameworks, Vitis AI and FINN, for sign language recognition on a Field Programmable Gate Array (FPGA). We conducted an in-depth exploration of both frameworks using Tiny YOLOv2 networks by varying design parameters such as precision, parallelism ratio, etc. Further, a fair baseline comparison is made based on accuracy, speed, and hardware resources. Experimental findings demonstrate that the Vitis AI outperforms the FINN framework and traditional GPU and CPU platforms by achieving significant improvements of 1.08x, 1.7x, and 2.9x in terms of latency. Leveraging Vitis AI, our system achieved a detection speed of 32.7 frames per second (FPS) on the Kria KV260 FPGA with a power consumption rate of 5.6 W and an impressive mean Average Precision (mAP) score of 61.2% on the Hindi Indian Sign Language (ISL) dataset.

A multi-lingual sign language recognition system using machine learning

AbstractRecently, automatic sign language recognition field gets a great attention. Machine learning algorithms are mainly used to recognize hand gestures. Most of recent studies train their machine learning model using a specific sign language of a specific country such as the American Sign Language. In this paper, we propose a multi-lingual sign language system based machine learning that is called Multi-lingual Sign Languages Interpreter (MSLI) system. MSLI trains a machine learning model based on hand signs of multiple languages. It can detect the language of the input signs and their labels. In a case of input testing signs with the same language, the proposed system can provide two-steps recognition, where it only detects the language of the first sign, and then the rest signs are tested according to the recognized language. Also, MSLI can provide separate classification of signs per each language. Experiments were performed using 11 datasets with different languages. Separate and combined classification was performed on the input data. Experimental results show the accuracy of the proposed system. Training accuracy of the proposed system over most of the used separate different sign language datasets is approximately ranged from 90 to 100%. Also, most classification accuracy results of the test data of the separate datasets exceeded 90%. The combined classification of proposed MSLI archived training accuracy of 95.87% and testing accuracy of 92.33%.

Sustainable utilization of road assets concerning obscured traffic signs recognition

Traffic sign recognition is crucial for sustainable utilization of road assets. However, on real roads, traffic signs are often obscured, making their recognition challenging. Unfortunately, existing research lacks specific analysis of the impact of traffic sign occlusion and methods to improve recognition in such scenarios. This study quantitatively investigates the relationship between the degree of traffic sign occlusion and recognition accuracy. Moreover, a dedicated deep learning model is proposed, utilizing a multi-scale convolutional stacked input layer, to enhance the recognition of obscured traffic signs. Using the Chinese Traffic Sign Recognition Database (TSRD), this study analyzes the recognition performance of four traffic sign categories: indication signs, prohibition signs, speed limit signs, and warning signs, under different occlusion levels. Three widely used deep learning methods, Visual Geometry Group (VGG), Residual Network (ResNet), and Dense Convolutional Network (DenseNet), are compared with the dedicated model. Experimental results demonstrate a significant decrease in recognition accuracy when traffic signs are obscured. Importantly, the proposed dedicated model outperforms the other methods, achieving accuracies of 80.95%, 90.50%, and 97.22% in the scenarios of 50% occlusion, 25% occlusion, and no occlusion, respectively. This study holds implications for sustainable utilization of road assets.

CNN MODEL FOR TRAFFIC SIGN RECOGNITION

The traffic sign recognition framework (TSRS) is an important component of intelligent transportation systems (ITS). Accurately identifying traffic signs can improve driving safety. This research provides a traffic sign recognition approach based on profound learning. It primarily focuses on the location and order of roundabout signs. First, a photograph undergoes pre-processing to highlight important facts. Hough Transform is used to distinguish and find regions. Finally, the unique street traffic signs are analyzed for further understanding. This paper proposes a photo handling-based traffic sign discovery and distinguishing proof technique that is combined with convolutional brain organization (CNN) to sort traffic signs. CNN is useful for recognizing many PC vision tasks due to its high recognition rate. TensorFlow is used to implement CNN. We have a recognition accuracy of over 98.2% for the roundabout image in the German informative collections. KEYWORDS——traffic sign recognition, traffic sign detection, deep learning, convolutional neural network

Derived Amharic alphabet sign language recognition using machine learning methods

Hearing-impaired people use sign language as a means of communication with those with no hearing disability. It is therefore difficult to communicate with hearing impaired people without the expertise of a signer or knowledge of sign language. As a result, technologies that understands sign language are required to bridge the communication gap between those that have hearing impairments and those that dont. Ethiopian Amharic alphabets sign language (EAMASL) is different from other countries sign languages because Amharic Language is spoken in Ethiopia and has a number of complex alphabets. Presently in Ethiopia, just a few studies on AMASL have been conducted. Previous works, on the other hand, only worked on basic and a few derived Amharic alphabet signs. To solve this challenge, in this paper, we propose Machine Learning techniques such as Support Vector Machine (SVM) with Convolutional Neural Network (CNN), Histogram of Oriented Gradients (HOG), and their hybrid features to recognize the remaining derived Amharic alphabet signs. Because CNN is good for rotation and translation of signs, and HOG works well for low quality data under strong illumination variation and a small quantity of training data, the two have been combined for feature extraction. CNN (Softmax) was utilized as a classifier for normalized hybrid features in addition to SVM. SVM model using CNN, HOG, normalized, and non-normalized hybrid feature vectors achieved an accuracy of 89.02%, 95.42%, 97.40%, and 93.61% using 10-fold cross validation, respectively. With the normalized hybrid features, the other classifier, CNN (sofmax), produced a 93.55% accuracy.

CCTSDB dataset enhancement based on a cross-augmentation method for image datasets

In the digital era, the rapid advancement of artificial intelligence has put a spotlight on target detection, especially in traffic settings. This area of study is pivotal for crucial projects like autonomous vehicles, road monitoring, and traffic sign recognition. However, existing Chinese traffic datasets lack comprehensive benchmarks for traffic signs and signals, and foreign datasets do not match Chinese traffic conditions. Manually annotating a large-scale dataset tailored for Chinese traffic conditions presents a significant challenge. This study addresses this gap by proposing a cross-augmentation method for image datasets. We utilized YOLOX for target detection and trained models on the BDD100K dataset, achieving an impressive mAP of 60.25%, surpassing most algorithms. Leveraging transfer learning, we enhanced the CCTSDB dataset, creating the ACCTSDB dataset, which includes annotations for common traffic objects and Chinese traffic signs. Using YOLOX, we trained a traffic detector tailored for Chinese traffic scenarios, achieving an mAP of 75.79%. To further validate our approach, we conducted experiments on the TT100K dataset and successfully introduced the ATT100K dataset. Our methodology is poised to alleviate the limitations of manually annotating image datasets. The proposed ACCTSDB dataset and ATT100K dataset are expected to compensate for the lack of large-scale, multi-class traffic datasets in China.

Neonatal Marfan syndrome: a case report of a novel fibrillin 1 mutation, with genotype-phenotype correlation and brief review of the literature

BackgroundNeonatal Marfan syndrome (nMFS) is a rare condition characterized by severe phenotype and poor prognosis. nMFS is caused by mutations in a specific region of the fibrillin 1 gene (FBN1). Prompt recognition of typical signs of neonatal presentation, such as characteristic facial anomalies with senile appearance, arthrogryposis, and campto-arachnodactyly, is fundamental for performing an early cardiological examination. This usually reveals rapidly progressive cardiovascular disease due to severe atrioventricular valve dysfunction.Case presentationHerein, we report the case of an early-onset cardiac failure in a neonate with Marfan syndrome, with a brief review of the literature of cases with cardiac involvement in neonatal age. Clinical exome sequencing identified the novel heterozygous de novo missense variant c.3152T > G in FBN1 gene (NM_000138.4), causing the aminoacidic change p.Phe1051Cys. Phenotype-genotype correlation led to a multidisciplinary diagnostic and management workflow.ConclusionThe prompt recognition of a typical phenotype such as that of Marfan syndrome should lead to a detailed evaluation and close follow-up of cardiac morphology and function. Indeed, multi-disciplinary evaluation based on genotype-phenotype correlations of nMFS cases is essential to finding out the best medical and surgical approach, predicting the relevant impact on patient prognosis, and adequately counseling their families.

Perceptions of Interns Toward Integrated Management of Childhood Illness (IMCI) Pre-service Education and Its Impact on Their Clinical Knowledge: A Study at Sultan Qaboos University, Muscat.

Objectives The study aims to assess interns' perceptions of Integrated Management of Childhood Illness (IMCI) pre-service education at Sultan Qaboos University (SQU). Specifically, it evaluates how IMCI training during phases 2 and 3 influences interns' clinical practice readiness and knowledge acquisition. The findings will inform evidence-based enhancements to IMCI training programs, ensuring they meet interns' educational needs and optimize clinical skills acquisition. Methods This cross-sectional study was conducted at the College of Medicine and Health Sciences (CoMHS) at SQU, Muscat, over a two-month period (September 20, 2023, to November 30, 2023). The questionnaire evaluated interns' sociodemographic factors, perceptions regarding pre-service IMCI training, and IMCI knowledge. IBM SPSS Statistics for Windows, Version 24 (Released 2016; IBM Corp., Armonk, New York) was used to analyze Likert scale responses for frequencies and proportions. Results Out of 103 invited interns, 75 participated in the study, resulting in a response rate of 72.8%. Interns who attended three or more IMCI lectures, tutorials, and practical sessions demonstrated a more advanced understanding of IMCI principles compared to those attending fewer sessions. Overall, 63 (84.0%) interns agreed on the effectiveness of IMCI training, 57 (76.0%) interns acknowledged skill enhancement, and 69 (92.0%) interns perceived its practicality for illness assessment. However, only 60.0% (n=45) felt confident in managing sick children. The knowledge assessment revealed varied understanding of IMCI objectives (82.7%, n=62) and components (61.3% (n=46) to 64.0% (n=48)). Clinical case evaluation showed mixed recognition of clinical features and danger signs of childhood illnesses, while awareness of disease preventability through immunization was generally high, except for tuberculosis (74.7%, n=56) and rotavirus (40.0%, n=30). Conclusion Interns exhibit positive attitudes towards IMCI principles, demonstrating a strong grasp of related concepts through effective case-based question responses. These results highlight the effectiveness of IMCI training in improving interns' understanding of pediatric healthcare principles, with potential implications for enhancing clinical practice and patient care. Future investigations should explore the impact of IMCI training on interns' clinical practice and patient outcomes.

Recent advances in Machine Learning based Advanced Driver Assistance System applications

In recent years, the rise of traffic in modern cities has demanded novel technology to support the drivers and protect the passengers and other third parties involved in transportation. Thanks to rapid technological progress and innovations, many Advanced Driver Assistance Systems (A/DAS) based on Machine Learning (ML) algorithms have emerged to address the increasing demand for practical A/DAS applications. Fast and accurate execution of A/DAS algorithms is essential for preventing loss of life and property. High-speed hardware accelerators are vital for processing the high volume of data captured by increasingly sophisticated sensors and complex mathematical models’ execution of modern deep learning (DL) algorithms. One of the fundamental challenges in this new era is to design energy-efficient and portable ML-enabled platforms for vehicles to provide driver assistance and safety. This article presents recent progress in ML-driven A/DAS technology to offer new insights for researchers. We covered standard ML models and optimization approaches based on widely accepted open-source frameworks extensively used in A/DAS applications. We have also highlighted related articles on ML and its sub-branches, neural networks (NNs), and DL. We have also reported the implementation issues, bench-marking problems, and potential challenges for future research. Popular embedded hardware platforms such as Field Programmable Gate Arrays (FPGAs), central processing units (CPUs), Graphical Processing Units (GPUs), and Application Specific Integrated Circuits (ASICs) used to implement A/DAS applications are also compared concerning their performance and resource utilization. We have examined the hardware and software development environments used in implementing A/DAS applications and reported their advantages and disadvantages. We provided performance comparisons of usual A/DAS tasks such as traffic sign recognition, road and lane detection, vehicle and pedestrian detection, driver behavior, and multiple tasking. Considering the current research dynamics, A/DAS will remain one of the most popular application fields for vehicular transportation shortly.

DFCNet +: Cross-modal dynamic feature contrast net for continuous sign language recognition

In sign language communication, the combination of hand signs and facial expressions is used to convey messages in a fluid manner. Accurate interpretation relies heavily on understanding the context of these signs. Current methods, however, often focus on static images, missing the continuous flow and the story that unfolds through successive movements in sign language. To address this constraint, our research introduces the Dynamic Feature Contrast Net Plus (DFCNet+), a novel model that incorporates both dynamic feature extraction and cross-modal learning. The dynamic feature extraction module of DFCNet+ uses dynamic trajectory capture to monitor and record motion across frames and apply key features as an enhancement tool that highlights pixels that are critical for recognizing important sign language movements, allowing the model to follow the temporal variation of the signs. In the cross-modal learning module, we depart from the conventional approach of aligning video frames with textual descriptions. Instead, we adopt a gloss-level alignment, which provides a more detailed match between the visual signals and their corresponding text glosses, capturing the intricate relationship between what is seen and the associated text. The enhanced proficiency of DFCNet+ in discerning inter-frame details translates to heightened precision on benchmarks such as PHOENIX14, PHOENIX14-T and CSL-Daily. Such performance underscores its advantage in dynamic feature capture and inter-modal learning compared to conventional approaches to sign language interpretation. Our code is available at https://github.com/fyzjut/DFCNet_Plus.

Instant Sign Language Recognition by WAR Strategy Algorithm Based Tuned Machine Learning

Sign language serves as the primary means of communication utilized by individuals with hearing and speech disabilities. However, the comprehension of sign language by those without disabilities poses a significant challenge, resulting in a notable disparity in communication across society. Despite the utilization of numerous effective Machine learning techniques, there remains a minor compromise between accuracy rate and computing time when it comes to sign language recognition. A novel sign language recognition system is presented in this paper with an exceptionally accurate and expeditious, which is developed upon the recently devised metaheuristic WAR Strategy optimization algorithm. Following the preprocessing, both of spatial and temporal features has been extracted using the Linear Discriminant Analysis (LDA) and Gray-level cooccurrence matrix (GLCM) methods. Afterward, the WAR Strategy optimization algorithm has been adopted in two procedures, first in optimizing the extracted set of features, and second to fine-tune the hyperparameters of six standard machine learning models in order to achieve precise and efficient sign language recognition. The proposed system was assessed on sign language datasets of different languages (American, Arabic, and Malaysian) containing numerous variations. The proposed system attained a recognition accuracy ranging from 93.11% to 100% by employing multiple optimized machine learning classifiers and training time of 0.038–10.48 s. As demonstrated by the experimental outcomes, the proposed system is exceptionally efficient regarding time, complexity, generalization, and accuracy.

Sample selection of adversarial attacks against traffic signs

In the real world, the correct recognition of traffic signs plays a crucial role in vehicle autonomous driving and traffic monitoring. The research on its adversarial attack can test the security of vehicle autonomous driving system and provide enlightenment for improving the recognition algorithm. However, with the development of transportation infrastructure, new traffic signs may be introduced. The adversarial attack model for traffic signs needs to adapt to the addition of new types. Based on this, class incremental learning for traffic sign adversarial attacks has become an interesting research field. We propose a class incremental learning method for adversarial attacks on traffic signs. First, this method uses Pinpoint Region Probability Estimation Network (PRPEN) to predict the probability of each pixel being attacked in old samples. It helps to identify the high attack probability regions of the samples. Subsequently, based on the size of high probability pixel concentration area, the replay sample set is constructed. Old samples with smaller concentration areas receive higher priority and are prioritized for incremental learning. The experimental results show that compared with other sample selection methods, our method selects more representative samples and can train PRPEN more effectively to generate probability maps, thereby better generating adversarial attacks on traffic signs.

Real-Time Hand Gesture Recognition: A Comprehensive Review of Techniques, Applications, and Challenges

Abstract Real-time Hand Gesture Recognition (HGR) has emerged as a vital technology in human-computer interaction, offering intuitive and natural ways for users to interact with computer-vision systems. This comprehensive review explores the advancements, challenges, and future directions in real-time HGR. Various HGR-related technologies have also been investigated, including sensors and vision technologies, which are utilized as a preliminary step in acquiring data in HGR systems. This paper discusses different recognition approaches, from traditional handcrafted feature methods to state-of-the-art deep learning techniques. Learning paradigms have been analyzed such as supervised, unsupervised, transfer, and adaptive learning in the context of HGR. A wide range of applications has been covered, from sign language recognition to healthcare and security systems. Despite significant developments in the computer vision domain, challenges remain in areas such as environmental robustness, gesture complexity, computational efficiency, and user adaptability. Lastly, this paper concludes by highlighting potential solutions and future research directions trying to develop more robust, efficient, and user-friendly real-time HGR systems.