Sign Recognition Research Articles

A Static Sign Language Recognition Method Enhanced with Self-Attention Mechanisms.

For the current wearable devices in the application of cross-diversified user groups, it is common to face the technical difficulties of static sign language recognition accuracy attenuation, weak anti-noise ability, and insufficient system robustness due to the differences in the use of users. This paper proposes a novel static sign language recognition method enhanced by a self-attention mechanism. The key features of sign language gesture classification are highlighted by the weight function, and then the self-attention mechanism is combined to pay more attention to the key features, and the convolutional neural network is used to extract the features and classify them, which realizes the accurate recognition of different types of static sign language under standard gestures and non-standard gestures. Experimental results reveal that the proposed method achieves an average accuracy of 99.52% in the standard static sign language recognition task when tested against the standard 36 static gestures selected within the reference American Sign Language dataset. By imposing random angular bias conditions of ±(0°-9°] and ±(9°-18°], the average recognition rates in this range were 98.63% and 86.33%. These findings indicate that, compared to existing methods, the proposed method not only maintains a high recognition rate for standard static gestures but also exhibits superior noise resistance and robustness, rendering it suitable for static sign language recognition among diverse user populations.

Comment on Dutta Majumder et al.’s 2019 ‘Ocular Syphilis: An Update’

ABSTRACT The article ‘Ocular Syphilis: An Update (2019)’ provides a thorough review of the challenges in diagnosing ocular syphilis. However, our research on ‘Dermatological and Ocular Manifestations of Syphilis,’ identifies a significant gap in both literature and clinical practice: the lack of recognition of dermatological signs during ophthalmological assessments. Ocular syphilis often mimics other conditions and can remain undiagnosed for months or years. Detecting dermatological signs, such as the characteristic palmar rash of secondary syphilis and extragenital chancres, could prompt earlier investigation and serological testing, reducing unnecessary diagnostic workups and inappropriate management. Early recognition would facilitate timely administration of Penicillin G, helping prevent vision loss, which is often reversible with prompt treatment. We urge Ocular Immunology and Inflammation to highlight the importance of incorporating dermatological assessments in future ocular syphilis publications to improve diagnostic protocols and patient outcomes.

Bridging communication with machine learning in sign language recognition for Vietnamese

Vietnamese Sign Language (VSL) serves as the primary language for deaf and hard-of-hearing individuals in Vietnam. This paper explores the sign language recognition process for VSL, emphasizing the role of machine learning in bridging communication barriers. We delve into the basics of VSL, detailing the one-to-one correspondence between hand signs and Vietnamese alphabet letters and address the formation of words through sequential hand signals and diacritics placement. Furthermore, the paper highlights the importance of pausing between words and the utilization of machine learning algorithms for automated sign recognition. Lastly, we conclude by discussing the potential applications and future directions of VSL recognition technology in Vietnam.

Systematic Review of Case Manager Nurse in Heart Transplant and Heart Failure Consultation

ObjectiveTo analyze the functions and activities of the case manager nurse in the specialized practice of transplantation and heart failure. MethodThis systematic review is reported according to the PRISMA guidelines. Selected articles were subjected to a detailed critical reading and the evidence levels and recommendation grades of the Joanna Briggs Institute were verified. The field of documentation search was limited to the last 5 years. ResultsHealth education provided by nursing is key in patients with heart failure and heart transplants. It must be evidence-based and focuses primarily on promoting self-care and adherence to therapy, although it includes other aspects such as recognition of signs and symptoms by the patient. Both activities are essential to prevent decompensations of heart failure, thus reducing hospital readmissions and mortality rates. ConclusionNursing has an essential role in multidisciplinary programs for the care of heart failure and heart transplant patients. Case manager nurse role stands out for being cost effective, as well as for improving treatment compliance and reducing hospital stay and mortality rates. Nursing functions in this practice ensure continuity and coordination of care between the different health levels, increasing the patients’ quality of life.

M-SAKHI - Mobile health solutions to help community providers promote maternal and infant nutrition and health: a description of development of the Program Impact Pathway using Theory of Change.

Behaviour Change Communication (BCC) intervention programmes often lack documentation of successful processes. This manuscript aims to describe the development of Program Impact Pathway (PIP) using Theory of Change (ToC) approach for a mHealth BCC intervention titled 'Mobile Solutions Aiding Knowledge for Health Improvement (M-SAKHI)' aimed at reducing stunting in infants at 18 months of age. The PIP was developed using ToC to design the intervention and plan its implementation. Literature review and data from previous pilots helped to identify health service gaps that needed to be addressed by the PIP of this intervention. M-SAKHI was implemented in 244 villages under governance of forty primary health centres of Nagpur and Bhandara districts of eastern Maharashtra in central India. The study investigators and the public health stakeholders participated in developing the PIP. M-SAKHI evaluation study recruited 2501 pregnant women who were followed up through delivery until their infants were 18 months old. The PIP was developed, and it identified the following pathways for the final impact: (1) improving maternal and infant nutrition, (2) early recognition of maternal and infant danger signs, (3) improving access and utilisation to healthcare services, (4) improving hygiene, sanitation and immunisation practices, and (5) improving implementation and service delivery of community health workers through their training, monitoring and supervision in real time. This paper will illustrate the significance of development of PIP for M-SAKHI. It can aid other community-based programmes to design their PIP for nutrition-based BCC interventions.

E-waste driven eco-friendly highly sensitive wearable capacitive sensors for physical activity monitoring and sign language recognition

Owing to the continual and rapid advancement in technology, there has been a significant surge in electronic waste (e-waste) in recent decades, resulting in a harmful impact on our environment. Due to the growing demand for battery-powered devices, non-rechargeable batteries are a significant source of e-waste. This study describes fabricating a paper-based graphite strain sensor that is eco-friendly, inexpensive, biodegradable, and sensitive to the applied strain. Readily accessible materials are employed to design and fabricate MIDC strain sensors. The Modified Interdigital Capacitor (MIDC) structure is formed on basic printing paper, and the conductive substance made with the ink of a gel pen and graphite from a drained dry cell is painted on the paper and used as electrode material. The suggested sensor has a high gauge factor value of 1998.578 and fast response and recovery time of 112 ms and 100 ms, respectively. The proposed prototype has also been utilized to inspect fitness exercises and physical activities on the human body, inspiring the potential for its integration into wearable systems for various applications. Reliable detection and differentiation of finger positions are achieved, highlighting the sensor's potential for more advanced sign language applications. It’s a good option to integrate the suggested strain sensor in wearable systems, such as telemedicine systems, electromechanical detection, healthcare, and physical activity monitoring of humans.

ConvAtt Network: A Low Parameter Approach For Sign Language Recognition

Despite recent advances in Large Language Models in text processing, Sign Language Recognition (SLR) remains an unresolved task. This is, in part, due to limitations in the available data. In this paper, we investigate combining 1D convolutions with transformer layers to capture local features and global interactions in a low-parameter SLR model. We experimented using multiple data augmentation and regularization techniques to categorize signs of the French Belgian Sign Language. We achieved a top-1 accuracy of 42.7% and a top-10 accuracy of 81.9% in 600 different signs. This model is competitive with the current state of the art while using a significantly lower number of parameters.

Pattern Recognition in the Processing of Electromyographic Signals for Selected Expressions of Polish Sign Language.

Gesture recognition has become a significant part of human-machine interaction, particularly when verbal interaction is not feasible. The rapid development of biomedical sensing and machine learning algorithms, including electromyography (EMG) and convolutional neural networks (CNNs), has enabled the interpretation of sign languages, including the Polish Sign Language, based on EMG signals. The objective was to classify the game control gestures and Polish Sign Language gestures recorded specifically for this study using two different data acquisition systems: BIOPAC MP36 and MyoWare 2.0. We compared the classification performance of various machine learning algorithms, with a particular emphasis on CNNs on the dataset of EMG signals representing 24 gestures, recorded using both types of EMG sensors. The results (98.324% versus ≤7.8571% and 95.5307% versus ≤10.2697% of accuracy for CNNs and other classifiers in data recorded with BIOPAC MP36 and MyoWare, respectively) indicate that CNNs demonstrate superior accuracy. These results suggest the feasibility of using lower-cost sensors for effective gesture classification and the viability of integrating affordable EMG-based technologies into broader gesture recognition frameworks, providing a cost-effective solution for real-world applications. The dataset created during the study offers a basis for future studies on EMG-based recognition of Polish Sign Language.

SWL-LSE: A Dataset of Health-Related Signs in Spanish Sign Language with an ISLR Baseline Method

Progress in automatic sign language recognition and translation has been hindered by the scarcity of datasets available for the training of machine learning algorithms, a challenge that is even more acute for languages with smaller signing communities, such as Spanish. In this paper, we introduce a dataset of 300 isolated signs in Spanish Sign Language, collected online via a web application with contributions from 124 participants, resulting in a total of 8000 instances. This dataset, which is openly available, includes keypoints extracted using MediaPipe Holistic. The goal of this paper is to describe the construction and characteristics of the dataset and to provide a baseline classification method using a spatial–temporal graph convolutional network (ST-GCN) model, encouraging the scientific community to improve upon it. The experimental section offers a comparative analysis of the method’s performance on the new dataset, as well as on two other well-known datasets. The dataset, code, and web app used for data collection are freely available, and the web app can also be used to test classifier performance on-line in real-time.

Visual context learning based on cross-modal knowledge for continuous sign language recognition

Visual context learning based on cross-modal knowledge for continuous sign language recognition

SMALL OBJECT DETECTION APPROACH BASED ON ENHANCED SINGLE-SHOT DETECTOR FOR DETECTION AND RECOGNITION OF INDONESIAN TRAFFIC SIGNS

The detection and recognition of traffic signs are crucial components of advanced driving assistance systems (ADAS) that enhance road safety. Current traffic sign detection and recognition model technology is proficient in identifying and interpreting traffic signs. However, for accurate detection and recognition, the traffic sign in the image must be of a certain minimum pixel size or distance from the driver's sight line for proper detection. The ADAS system should be capable of detecting and recognizing road traffic signs from a considerable distance as they come into the driver's line of vision. The higher the vehicle speed, the greater the distance required for the sign to be detected and recognized, allowing the driver sufficient time to react according to the sign's meaning. Addressing these challenges, this research proposes an enhanced version of the single shot detector (SSD) algorithm, commonly used in object detection, to improve the algorithm's ability to detect small objects. The proposed method involves adding an auxiliary layer module to the original SSD architecture to increase the feature map resolution and expand the conventional layer's receptive space. With the Enhanced SSD algorithm, the detection capability of the SSD can be significantly enhanced in terms of accuracy. The limitations of this study are related to the influence of occlusion and clutter, which might affect the performance of object detection, especially for small objects, which are more susceptible to being influenced by various factors. The research results demonstrate that Enhanced SSD improves object detection accuracy compared to the original SSD, with a mean average precision (mAP) of 97.87 compared to 95.35 for detecting 21 traffic signs in Indonesia.

High-Precision Traffic Sign Detection and Recognition Using an Enhanced YOLOv5

High-Precision Traffic Sign Detection and Recognition Using an Enhanced YOLOv5

Techniques for Detecting the Start and End Points of Sign Language Utterances to Enhance Recognition Performance in Mobile Environments

Recent AI-based technologies in mobile environments have enabled sign language recognition, allowing deaf individuals to communicate effectively with hearing individuals. However, varying computational performance across different mobile devices can result in differences in the number of image frames extracted in real time during sign language utterances. The number of extracted frames is a critical factor influencing the accuracy of sign language recognition models. If the number of extracted frames is too small, the performance of the sign language recognition model may decline. Additionally, detecting the start and end points of sign language utterances is crucial for improving recognition accuracy, as the period before the start point and after the end point often involves no action being performed. These parts do not capture the unique characteristics of each sign language. Therefore, this paper proposes a technique to dynamically adjust the sampling rate based on the number of frames extracted in real time during sign language utterances in mobile environments, with the aim of accurately detecting the start and end points of the sign language. Experiments were conducted to compare the proposed technique with the fixed sampling rate method and with the no-sampling method as a baseline. Our findings show that the proposed dynamic sampling rate adjustment method improves performance by up to 83.64% in top-5 accuracy and by up to 66.54% in top-1 accuracy compared to the fixed sampling rate method. The performance evaluation results underscore the effectiveness of our dynamic sampling rate adjustment approach in enhancing the accuracy and robustness of sign language recognition systems across different operational conditions.

A Robust TrafficSignNet Algorithm for Enhanced Traffic Sign Recognition in Autonomous Vehicles Under Varying Light Conditions

Recent years have witnessed significant advancements in machine perception, particularly in the context of self-driving vehicles. The accurate detection and interpretation of road signs by these vehicles are crucial for enhancing safety, intelligence, and efficiency on the roads. Consequently, there is a growing body of research dedicated to improving traffic sign recognition technologies, a key component of intelligent transportation systems. Annual statistics highlight numerous accidents attributable to factors such as excessive speed, variable lighting conditions, and the misinterpretation of traffic signs. In response to these challenges, a novel approach for the rapid and reliable recognition of traffic signs by moving vehicles has been developed. This approach leverages a custom dataset encompassing twelve object categories and seven subcategories, reflective of road sign diversities encountered in India. A specialized algorithm, TrafficSignNet, was devised to specifically identify signs related to speed, turning, zones, and bumps. This algorithm was trained on a comprehensive dataset comprising 4,962 images, with its performance evaluated using 705 images from real traffic scenarios. The evaluation demonstrates that the model achieves remarkable accuracy across various lighting conditions, processing up to 12 frames per second. This processing rate is compatible with the high-definition standards of contemporary vehicle cameras, which is 1280 × 720 pixels. The model's effectiveness is quantified through accuracy, precision, recall, and F1 score, with respective values of 0.985, 0.978, 0.964, and 0.971, showcasing its potential to significantly contribute to the advancement of smart transportation systems.

Dynamical semantic enhancement network for continuous sign language recognition

Dynamical semantic enhancement network for continuous sign language recognition

Sensitivity Analysis of Traffic Sign Recognition to Image Alteration and Training Data Size

Sensitivity Analysis of Traffic Sign Recognition to Image Alteration and Training Data Size

8269 Euglycemic Diabetic Ketoacidosis Secondary to Rapid Weight Loss in a Patient on Dapagliflozin

Abstract Disclosure: A. Sridhar: None. A. Iqbal: None. A. Prabha Kumar: None. F. Haider: None. M. Babic: None. Background: Euglycemic DKA (eDKA) is a potentially life-threatening complication in patients taking SGLT 2 inhibitors (SGLT2i). Common precipitating factors include strenuous exercise, carbohydrate restriction, starvation, and alcohol use. Clinical case: The patient is a 71-year-old female with past medical history of Type 2 diabetes mellitus (DM), gastro-esophageal reflux disease, hypertension, hyperlipidemia, coronary artery disease and chronic pain syndrome. She presented with complaints of nausea, vomiting, reduced appetite and generalized weakness for the past 4 days. She also reported an intentional weight loss of 35 pounds in the last 6 weeks achieved with daily exercise and a carbohydrate restricted diet. She had been taking Dapagliflozin-Metformin 5-1000 mg twice daily and pioglitazone 30 mg once daily for 2 years and Ozempic subcutaneous injection 1 mg weekly for 6 months. Physical examination in the ED showed normal vital signs, dry mucous membranes, and trace bilateral lower extremity edema. Blood work was significant for venous blood gas pH 7.24, bicarbonate 9 mmol/L, anion gap 30 mmol/L, blood glucose 94 mg/dL, HbA1c 5.8 % and WBC 13.2 K/uL. Urinalysis was positive for glucose and ketones. The patient was admitted for management of eDKA and was started on intravenous 10% dextrose infusion and insulin infusion. Endocrinology was consulted and they recommended following DKA protocol and stopping Dapagliflozin at discharge. She was maintained on insulin infusion for 48 hours after which her anion gap normalized, and she was transitioned to subcutaneous insulin with Lantus 8 units and sliding scale insulin. During this time the patient’s symptoms resolved, and she was able to tolerate oral diet. She eventually resumed her home medications with oral Metformin 1000 mg twice daily and pioglitazone 30 mg once daily. Dapagliflozin and Ozempic were discontinued at discharge. Strenuous exercise and carbohydrate restriction which caused rapid weight loss, were likely the triggers for eDKA in our patient. A diabetes educator was involved at discharge and the patient was counseled to avoid possible triggers and identify warning signs and symptoms in the future. The patient was discharged with a follow up appointment with her Endocrinologist. Conclusion: High index of clinical suspicion is important to promptly diagnose eDKA in patients taking SGLT2i. Patient education on avoiding precipitating factors and early recognition of signs and symptoms is necessary to reduce mortality. Presentation: 6/3/2024

Beyond Granularity: Enhancing Continuous Sign Language Recognition with Granularity-Aware Feature Fusion and Attention Optimization

The advancement of deep learning techniques has significantly propelled the development of the continuous sign language recognition (cSLR) task. However, the spatial feature extraction of sign language videos in the RGB space tends to focus on the overall image information while neglecting the perception of traits at different granularities, such as eye gaze and lip shape, which are more detailed, or posture and gestures, which are more macroscopic. Exploring the efficient fusion of visual information of different granularities is crucial for accurate sign language recognition. In addition, applying a vanilla Transformer to sequence modeling in cSLR exhibits weak performance because specific video frames could interfere with the attention mechanism. These limitations constrain the capability to understand potential semantic characteristics. We introduce a feature fusion method for integrating visual features of disparate granularities and refine the metric of attention to enhance the Transformer’s comprehension of video content. Specifically, we extract CNN feature maps with varying receptive fields and employ a self-attention mechanism to fuse feature maps of different granularities, thereby obtaining multi-scale spatial features of the sign language framework. As for video modeling, we first analyze why the vanilla Transformer failed in cSLR and observe that the magnitude of the feature vectors of video frames could interfere with the distribution of attention weights. Therefore, we utilize the Euclidean distance among vectors to measure the attention weights instead of scaled-dot to enhance dynamic temporal modeling capabilities. Finally, we integrate the two components to construct the model MSF-ET (Multi-Scaled feature Fusion–Euclidean Transformer) for cSLR and train the model end-to-end. We perform experiments on large-scale cSLR benchmarks—PHOENIX-2014 and Chinese Sign Language (CSL)—to validate the effectiveness.

Enhancing Autonomous Driving Safety: A Robust Stacking Ensemble Model for Traffic Sign Detection and Recognition

Accurate detection and classification of traffic signs play a vital role in ensuring driver safety and supporting advancements in autonomous driving technology. This paper introduces a novel approach for traffic sign detection and recognition by integrating the Faster RCNN and YOLOX-Tiny models using a stacking ensemble technique. The innovative ensemble methodology creatively merges the strengths of both models, surpassing the limitations of individual algorithms and achieving superior performance in challenging real-world scenarios. The proposed model was evaluated on the CCTSDB dataset and the MTSD dataset, demonstrating competitive performance compared to traditional algorithms. All experiments were conducted using Python 3.8 on the same system equipped with an NVIDIA GTX 3060 12G graphics card. Our results show improved accuracy and efficiency in recognizing traffic signs in various real-world scenarios, including distant, close, complex, moderate, and simple settings, achieving a 4.78% increase in mean Average Precision (mAP) compared to Faster RCNN and improving Frames Per Second (FPS) by 8.1% and mAP by 6.18% compared to YOLOX-Tiny. Moreover, the proposed model exhibited notable precision in challenging scenarios such as ultra-long-distance detections, shadow occlusions, motion blur, and complex environments with diverse sign categories. These findings not only showcase the model’s robustness but also serve as a cornerstone in propelling the evolution of autonomous driving technology and sustainable development of future transportation. The results presented in this paper could potentially be integrated into advanced driver-assistance systems and autonomous vehicles, offering a significant step forward in enhancing road safety and traffic management.

A large corpus for the recognition of Greek Sign Language gestures

Sign language recognition (SLR) from videos constitutes a captivating problem in gesture recognition, requiring the interpretation of hand movements, facial expressions, and body postures. The complexity of sign formation, signing variability among signers, and the technical hurdles of visual detection and tracking render SLR a challenging task. At the same time, the scarcity of large-scale SLR datasets, which are critical for developing robust data-intensive deep-learning SLR models, exacerbates these issues. In this article, we introduce a multi-signer video corpus of Greek Sign Language (GSL), which is the largest GSL database to date, serving as a valuable resource for SLR research. This corpus comprises an extensive RGB+D video collection that conveys rich lexical content in a multi-modal fashion, encompassing three subsets: (i) isolated signs; (ii) continuous signing; and (iii) continuous alphabet fingerspelling of words. Moreover, we introduce a comprehensive experimental setup that paves the way for more accurate and robust SLR solutions. In particular, except for the multi-signer (MS) and signer-independent (SI) settings, we employ a signer-adapted (SA) experimental paradigm, facilitating a comprehensive evaluation of system performance across various scenarios. Further, we provide three baseline SLR systems for isolated signs, continuous signing, and continuous fingerspelling. These systems leverage cutting-edge methods in deep learning and sequence modeling to capture the intricate temporal dynamics inherent in sign gestures. The models are evaluated on the three corpus subsets, setting their state-of-the-art recognition benchmark. The SL-ReDu GSL corpus, including its recommended experimental frameworks, is publicly available at https://sl-redu.e-ce.uth.gr/corpus.