Sign Recognition Research Articles

Comment on Dutta Majumder et al.’s 2019 ‘Ocular Syphilis: An Update’

ABSTRACT The article ‘Ocular Syphilis: An Update (2019)’ provides a thorough review of the challenges in diagnosing ocular syphilis. However, our research on ‘Dermatological and Ocular Manifestations of Syphilis,’ identifies a significant gap in both literature and clinical practice: the lack of recognition of dermatological signs during ophthalmological assessments. Ocular syphilis often mimics other conditions and can remain undiagnosed for months or years. Detecting dermatological signs, such as the characteristic palmar rash of secondary syphilis and extragenital chancres, could prompt earlier investigation and serological testing, reducing unnecessary diagnostic workups and inappropriate management. Early recognition would facilitate timely administration of Penicillin G, helping prevent vision loss, which is often reversible with prompt treatment. We urge Ocular Immunology and Inflammation to highlight the importance of incorporating dermatological assessments in future ocular syphilis publications to improve diagnostic protocols and patient outcomes.

Bridging communication with machine learning in sign language recognition for Vietnamese

Vietnamese Sign Language (VSL) serves as the primary language for deaf and hard-of-hearing individuals in Vietnam. This paper explores the sign language recognition process for VSL, emphasizing the role of machine learning in bridging communication barriers. We delve into the basics of VSL, detailing the one-to-one correspondence between hand signs and Vietnamese alphabet letters and address the formation of words through sequential hand signals and diacritics placement. Furthermore, the paper highlights the importance of pausing between words and the utilization of machine learning algorithms for automated sign recognition. Lastly, we conclude by discussing the potential applications and future directions of VSL recognition technology in Vietnam.

Systematic Review of Case Manager Nurse in Heart Transplant and Heart Failure Consultation

ObjectiveTo analyze the functions and activities of the case manager nurse in the specialized practice of transplantation and heart failure. MethodThis systematic review is reported according to the PRISMA guidelines. Selected articles were subjected to a detailed critical reading and the evidence levels and recommendation grades of the Joanna Briggs Institute were verified. The field of documentation search was limited to the last 5 years. ResultsHealth education provided by nursing is key in patients with heart failure and heart transplants. It must be evidence-based and focuses primarily on promoting self-care and adherence to therapy, although it includes other aspects such as recognition of signs and symptoms by the patient. Both activities are essential to prevent decompensations of heart failure, thus reducing hospital readmissions and mortality rates. ConclusionNursing has an essential role in multidisciplinary programs for the care of heart failure and heart transplant patients. Case manager nurse role stands out for being cost effective, as well as for improving treatment compliance and reducing hospital stay and mortality rates. Nursing functions in this practice ensure continuity and coordination of care between the different health levels, increasing the patients’ quality of life.

M-SAKHI - Mobile health solutions to help community providers promote maternal and infant nutrition and health: a description of development of the Program Impact Pathway using Theory of Change.

Behaviour Change Communication (BCC) intervention programmes often lack documentation of successful processes. This manuscript aims to describe the development of Program Impact Pathway (PIP) using Theory of Change (ToC) approach for a mHealth BCC intervention titled 'Mobile Solutions Aiding Knowledge for Health Improvement (M-SAKHI)' aimed at reducing stunting in infants at 18 months of age. The PIP was developed using ToC to design the intervention and plan its implementation. Literature review and data from previous pilots helped to identify health service gaps that needed to be addressed by the PIP of this intervention. M-SAKHI was implemented in 244 villages under governance of forty primary health centres of Nagpur and Bhandara districts of eastern Maharashtra in central India. The study investigators and the public health stakeholders participated in developing the PIP. M-SAKHI evaluation study recruited 2501 pregnant women who were followed up through delivery until their infants were 18 months old. The PIP was developed, and it identified the following pathways for the final impact: (1) improving maternal and infant nutrition, (2) early recognition of maternal and infant danger signs, (3) improving access and utilisation to healthcare services, (4) improving hygiene, sanitation and immunisation practices, and (5) improving implementation and service delivery of community health workers through their training, monitoring and supervision in real time. This paper will illustrate the significance of development of PIP for M-SAKHI. It can aid other community-based programmes to design their PIP for nutrition-based BCC interventions.

E-waste driven eco-friendly highly sensitive wearable capacitive sensors for physical activity monitoring and sign language recognition

Owing to the continual and rapid advancement in technology, there has been a significant surge in electronic waste (e-waste) in recent decades, resulting in a harmful impact on our environment. Due to the growing demand for battery-powered devices, non-rechargeable batteries are a significant source of e-waste. This study describes fabricating a paper-based graphite strain sensor that is eco-friendly, inexpensive, biodegradable, and sensitive to the applied strain. Readily accessible materials are employed to design and fabricate MIDC strain sensors. The Modified Interdigital Capacitor (MIDC) structure is formed on basic printing paper, and the conductive substance made with the ink of a gel pen and graphite from a drained dry cell is painted on the paper and used as electrode material. The suggested sensor has a high gauge factor value of 1998.578 and fast response and recovery time of 112 ms and 100 ms, respectively. The proposed prototype has also been utilized to inspect fitness exercises and physical activities on the human body, inspiring the potential for its integration into wearable systems for various applications. Reliable detection and differentiation of finger positions are achieved, highlighting the sensor's potential for more advanced sign language applications. It’s a good option to integrate the suggested strain sensor in wearable systems, such as telemedicine systems, electromechanical detection, healthcare, and physical activity monitoring of humans.

ConvAtt Network: A Low Parameter Approach For Sign Language Recognition

Despite recent advances in Large Language Models in text processing, Sign Language Recognition (SLR) remains an unresolved task. This is, in part, due to limitations in the available data. In this paper, we investigate combining 1D convolutions with transformer layers to capture local features and global interactions in a low-parameter SLR model. We experimented using multiple data augmentation and regularization techniques to categorize signs of the French Belgian Sign Language. We achieved a top-1 accuracy of 42.7% and a top-10 accuracy of 81.9% in 600 different signs. This model is competitive with the current state of the art while using a significantly lower number of parameters.

Pattern Recognition in the Processing of Electromyographic Signals for Selected Expressions of Polish Sign Language.

Gesture recognition has become a significant part of human-machine interaction, particularly when verbal interaction is not feasible. The rapid development of biomedical sensing and machine learning algorithms, including electromyography (EMG) and convolutional neural networks (CNNs), has enabled the interpretation of sign languages, including the Polish Sign Language, based on EMG signals. The objective was to classify the game control gestures and Polish Sign Language gestures recorded specifically for this study using two different data acquisition systems: BIOPAC MP36 and MyoWare 2.0. We compared the classification performance of various machine learning algorithms, with a particular emphasis on CNNs on the dataset of EMG signals representing 24 gestures, recorded using both types of EMG sensors. The results (98.324% versus ≤7.8571% and 95.5307% versus ≤10.2697% of accuracy for CNNs and other classifiers in data recorded with BIOPAC MP36 and MyoWare, respectively) indicate that CNNs demonstrate superior accuracy. These results suggest the feasibility of using lower-cost sensors for effective gesture classification and the viability of integrating affordable EMG-based technologies into broader gesture recognition frameworks, providing a cost-effective solution for real-world applications. The dataset created during the study offers a basis for future studies on EMG-based recognition of Polish Sign Language.

SWL-LSE: A Dataset of Health-Related Signs in Spanish Sign Language with an ISLR Baseline Method

Progress in automatic sign language recognition and translation has been hindered by the scarcity of datasets available for the training of machine learning algorithms, a challenge that is even more acute for languages with smaller signing communities, such as Spanish. In this paper, we introduce a dataset of 300 isolated signs in Spanish Sign Language, collected online via a web application with contributions from 124 participants, resulting in a total of 8000 instances. This dataset, which is openly available, includes keypoints extracted using MediaPipe Holistic. The goal of this paper is to describe the construction and characteristics of the dataset and to provide a baseline classification method using a spatial–temporal graph convolutional network (ST-GCN) model, encouraging the scientific community to improve upon it. The experimental section offers a comparative analysis of the method’s performance on the new dataset, as well as on two other well-known datasets. The dataset, code, and web app used for data collection are freely available, and the web app can also be used to test classifier performance on-line in real-time.

Visual context learning based on cross-modal knowledge for continuous sign language recognition

Visual context learning based on cross-modal knowledge for continuous sign language recognition

SMALL OBJECT DETECTION APPROACH BASED ON ENHANCED SINGLE-SHOT DETECTOR FOR DETECTION AND RECOGNITION OF INDONESIAN TRAFFIC SIGNS

The detection and recognition of traffic signs are crucial components of advanced driving assistance systems (ADAS) that enhance road safety. Current traffic sign detection and recognition model technology is proficient in identifying and interpreting traffic signs. However, for accurate detection and recognition, the traffic sign in the image must be of a certain minimum pixel size or distance from the driver's sight line for proper detection. The ADAS system should be capable of detecting and recognizing road traffic signs from a considerable distance as they come into the driver's line of vision. The higher the vehicle speed, the greater the distance required for the sign to be detected and recognized, allowing the driver sufficient time to react according to the sign's meaning. Addressing these challenges, this research proposes an enhanced version of the single shot detector (SSD) algorithm, commonly used in object detection, to improve the algorithm's ability to detect small objects. The proposed method involves adding an auxiliary layer module to the original SSD architecture to increase the feature map resolution and expand the conventional layer's receptive space. With the Enhanced SSD algorithm, the detection capability of the SSD can be significantly enhanced in terms of accuracy. The limitations of this study are related to the influence of occlusion and clutter, which might affect the performance of object detection, especially for small objects, which are more susceptible to being influenced by various factors. The research results demonstrate that Enhanced SSD improves object detection accuracy compared to the original SSD, with a mean average precision (mAP) of 97.87 compared to 95.35 for detecting 21 traffic signs in Indonesia.

High-Precision Traffic Sign Detection and Recognition Using an Enhanced YOLOv5

High-Precision Traffic Sign Detection and Recognition Using an Enhanced YOLOv5

Techniques for Detecting the Start and End Points of Sign Language Utterances to Enhance Recognition Performance in Mobile Environments

Recent AI-based technologies in mobile environments have enabled sign language recognition, allowing deaf individuals to communicate effectively with hearing individuals. However, varying computational performance across different mobile devices can result in differences in the number of image frames extracted in real time during sign language utterances. The number of extracted frames is a critical factor influencing the accuracy of sign language recognition models. If the number of extracted frames is too small, the performance of the sign language recognition model may decline. Additionally, detecting the start and end points of sign language utterances is crucial for improving recognition accuracy, as the period before the start point and after the end point often involves no action being performed. These parts do not capture the unique characteristics of each sign language. Therefore, this paper proposes a technique to dynamically adjust the sampling rate based on the number of frames extracted in real time during sign language utterances in mobile environments, with the aim of accurately detecting the start and end points of the sign language. Experiments were conducted to compare the proposed technique with the fixed sampling rate method and with the no-sampling method as a baseline. Our findings show that the proposed dynamic sampling rate adjustment method improves performance by up to 83.64% in top-5 accuracy and by up to 66.54% in top-1 accuracy compared to the fixed sampling rate method. The performance evaluation results underscore the effectiveness of our dynamic sampling rate adjustment approach in enhancing the accuracy and robustness of sign language recognition systems across different operational conditions.

A Robust TrafficSignNet Algorithm for Enhanced Traffic Sign Recognition in Autonomous Vehicles Under Varying Light Conditions

Recent years have witnessed significant advancements in machine perception, particularly in the context of self-driving vehicles. The accurate detection and interpretation of road signs by these vehicles are crucial for enhancing safety, intelligence, and efficiency on the roads. Consequently, there is a growing body of research dedicated to improving traffic sign recognition technologies, a key component of intelligent transportation systems. Annual statistics highlight numerous accidents attributable to factors such as excessive speed, variable lighting conditions, and the misinterpretation of traffic signs. In response to these challenges, a novel approach for the rapid and reliable recognition of traffic signs by moving vehicles has been developed. This approach leverages a custom dataset encompassing twelve object categories and seven subcategories, reflective of road sign diversities encountered in India. A specialized algorithm, TrafficSignNet, was devised to specifically identify signs related to speed, turning, zones, and bumps. This algorithm was trained on a comprehensive dataset comprising 4,962 images, with its performance evaluated using 705 images from real traffic scenarios. The evaluation demonstrates that the model achieves remarkable accuracy across various lighting conditions, processing up to 12 frames per second. This processing rate is compatible with the high-definition standards of contemporary vehicle cameras, which is 1280 × 720 pixels. The model's effectiveness is quantified through accuracy, precision, recall, and F1 score, with respective values of 0.985, 0.978, 0.964, and 0.971, showcasing its potential to significantly contribute to the advancement of smart transportation systems.

Dynamical semantic enhancement network for continuous sign language recognition

Dynamical semantic enhancement network for continuous sign language recognition

Sensitivity Analysis of Traffic Sign Recognition to Image Alteration and Training Data Size

Sensitivity Analysis of Traffic Sign Recognition to Image Alteration and Training Data Size

8269 Euglycemic Diabetic Ketoacidosis Secondary to Rapid Weight Loss in a Patient on Dapagliflozin

Abstract Disclosure: A. Sridhar: None. A. Iqbal: None. A. Prabha Kumar: None. F. Haider: None. M. Babic: None. Background: Euglycemic DKA (eDKA) is a potentially life-threatening complication in patients taking SGLT 2 inhibitors (SGLT2i). Common precipitating factors include strenuous exercise, carbohydrate restriction, starvation, and alcohol use. Clinical case: The patient is a 71-year-old female with past medical history of Type 2 diabetes mellitus (DM), gastro-esophageal reflux disease, hypertension, hyperlipidemia, coronary artery disease and chronic pain syndrome. She presented with complaints of nausea, vomiting, reduced appetite and generalized weakness for the past 4 days. She also reported an intentional weight loss of 35 pounds in the last 6 weeks achieved with daily exercise and a carbohydrate restricted diet. She had been taking Dapagliflozin-Metformin 5-1000 mg twice daily and pioglitazone 30 mg once daily for 2 years and Ozempic subcutaneous injection 1 mg weekly for 6 months. Physical examination in the ED showed normal vital signs, dry mucous membranes, and trace bilateral lower extremity edema. Blood work was significant for venous blood gas pH 7.24, bicarbonate 9 mmol/L, anion gap 30 mmol/L, blood glucose 94 mg/dL, HbA1c 5.8 % and WBC 13.2 K/uL. Urinalysis was positive for glucose and ketones. The patient was admitted for management of eDKA and was started on intravenous 10% dextrose infusion and insulin infusion. Endocrinology was consulted and they recommended following DKA protocol and stopping Dapagliflozin at discharge. She was maintained on insulin infusion for 48 hours after which her anion gap normalized, and she was transitioned to subcutaneous insulin with Lantus 8 units and sliding scale insulin. During this time the patient’s symptoms resolved, and she was able to tolerate oral diet. She eventually resumed her home medications with oral Metformin 1000 mg twice daily and pioglitazone 30 mg once daily. Dapagliflozin and Ozempic were discontinued at discharge. Strenuous exercise and carbohydrate restriction which caused rapid weight loss, were likely the triggers for eDKA in our patient. A diabetes educator was involved at discharge and the patient was counseled to avoid possible triggers and identify warning signs and symptoms in the future. The patient was discharged with a follow up appointment with her Endocrinologist. Conclusion: High index of clinical suspicion is important to promptly diagnose eDKA in patients taking SGLT2i. Patient education on avoiding precipitating factors and early recognition of signs and symptoms is necessary to reduce mortality. Presentation: 6/3/2024

Beyond Granularity: Enhancing Continuous Sign Language Recognition with Granularity-Aware Feature Fusion and Attention Optimization

The advancement of deep learning techniques has significantly propelled the development of the continuous sign language recognition (cSLR) task. However, the spatial feature extraction of sign language videos in the RGB space tends to focus on the overall image information while neglecting the perception of traits at different granularities, such as eye gaze and lip shape, which are more detailed, or posture and gestures, which are more macroscopic. Exploring the efficient fusion of visual information of different granularities is crucial for accurate sign language recognition. In addition, applying a vanilla Transformer to sequence modeling in cSLR exhibits weak performance because specific video frames could interfere with the attention mechanism. These limitations constrain the capability to understand potential semantic characteristics. We introduce a feature fusion method for integrating visual features of disparate granularities and refine the metric of attention to enhance the Transformer’s comprehension of video content. Specifically, we extract CNN feature maps with varying receptive fields and employ a self-attention mechanism to fuse feature maps of different granularities, thereby obtaining multi-scale spatial features of the sign language framework. As for video modeling, we first analyze why the vanilla Transformer failed in cSLR and observe that the magnitude of the feature vectors of video frames could interfere with the distribution of attention weights. Therefore, we utilize the Euclidean distance among vectors to measure the attention weights instead of scaled-dot to enhance dynamic temporal modeling capabilities. Finally, we integrate the two components to construct the model MSF-ET (Multi-Scaled feature Fusion–Euclidean Transformer) for cSLR and train the model end-to-end. We perform experiments on large-scale cSLR benchmarks—PHOENIX-2014 and Chinese Sign Language (CSL)—to validate the effectiveness.

Enhancing Autonomous Driving Safety: A Robust Stacking Ensemble Model for Traffic Sign Detection and Recognition

Accurate detection and classification of traffic signs play a vital role in ensuring driver safety and supporting advancements in autonomous driving technology. This paper introduces a novel approach for traffic sign detection and recognition by integrating the Faster RCNN and YOLOX-Tiny models using a stacking ensemble technique. The innovative ensemble methodology creatively merges the strengths of both models, surpassing the limitations of individual algorithms and achieving superior performance in challenging real-world scenarios. The proposed model was evaluated on the CCTSDB dataset and the MTSD dataset, demonstrating competitive performance compared to traditional algorithms. All experiments were conducted using Python 3.8 on the same system equipped with an NVIDIA GTX 3060 12G graphics card. Our results show improved accuracy and efficiency in recognizing traffic signs in various real-world scenarios, including distant, close, complex, moderate, and simple settings, achieving a 4.78% increase in mean Average Precision (mAP) compared to Faster RCNN and improving Frames Per Second (FPS) by 8.1% and mAP by 6.18% compared to YOLOX-Tiny. Moreover, the proposed model exhibited notable precision in challenging scenarios such as ultra-long-distance detections, shadow occlusions, motion blur, and complex environments with diverse sign categories. These findings not only showcase the model’s robustness but also serve as a cornerstone in propelling the evolution of autonomous driving technology and sustainable development of future transportation. The results presented in this paper could potentially be integrated into advanced driver-assistance systems and autonomous vehicles, offering a significant step forward in enhancing road safety and traffic management.

Editorial: Bridging the Gap: AI and Sign Language Recognition– A Path Toward Inclusive Communication

The integration of Artificial Intelligence (AI) and sign language recognition is a hot topic in the field of AI+Science, aimed at addressing communication barriers faced by the deaf and hard-of-hearing communities. This paper examines the integration of AI with sign language recognition, highlighting its potential to bridge communication gaps for the deaf and hard-of-hearing. It reviews the evolution of sign language recognition from data gloves to computer vision and underscores the role of extensive databases. The paper also discusses the benefits of multi-modal AI models in enhancing recognition accuracy. It highlights the importance of government and industry support, ethical data practices, and user-centered design in advancing this technology. The challenges and opportunities of integrating this technology into daily life, including technical, interface, and ethical considerations, are explored, emphasizing the need for user-focused solutions and innovative technical approaches.

A lightweight network-based sign language robot with facial mirroring and speech system

Human–robot interaction is an essential capability for humanoid robots to enter the physical world and become companions in people’s lives, learning, and work. While the majority of current research focuses on the voice-based interactions of robots, yet over 60% of communication occurs through nonverbal behaviors, such as facial expressions and hand gestures. Endowing robots with the ability to communicate through nonverbal behavior not only enhances the interactive experience with robots but also provides a potential communication tool for individuals with hearing or speech impairments. Here, we develop a humanoid robot capable of adjusting facial movements by driving servos, and design a novel framework for the robot to integrate sign language recognition and facial landmark detection algorithms. This framework facilitates the robot recognize sign language and translate it into spoken language, while also imitating the facial expressions of the signers. To achieve this, we also propose a lightweight deep learning network called RealTimeSignNet for real-time sign language recognition. Leveraging lightweight 3D convolution modules and time-dependent constraints, this model adapts to various time scales, ensuring efficient processing of sign language recognition tasks. Experimental results demonstrate the outstanding performance of the RealTimeSignNet model on mainstream sign language datasets, achieving an accuracy of 88.1% on the large continuous sign language dataset (continuous SLR), 98.2% on the isolated sign language dataset (SLR 500), and 91.50% on the English sign language dataset (WLAS). The overall assessment demonstrates that our humanoid robot is capable of recognizing sign language and translating it into spoken language, while imitating the facial emotions, providing a comprehensive solution to the communication challenges faced by individuals with hearing and speech impairments.