Sign Recognition Research Articles

A Comprehensive Review of Sign Language Production

Sign languages are made up of phonological, morphological, syntactic and semantic levels of structure that satisfy the same social, cognitive and communicative purposes as other natural languages and represent the most used form of communication between hearing and deaf people. Sign Language Production together with Sign Language Recognition constitute the two parts of this process, as Sign Language Production concerns that part of the communication process that goes from spoken language to its translation into Sign Language, while Sign Language Recognition deals with the recognition of Sign Language. In this article, we want to consider some of the most recent and important studies on Sign Language Production and discuss their limitations, advantages, and future developments.

Magnetic resonance imaging of pulmonary hypertension.

Pulmonary hypertension is a rare but serious disease in children with potentially poor prognosis in the long term. Direct measurement of increased pressure in the pulmonary circulation requires right-heart catheterisation. Echocardiography is the imaging modality commonly used for suggesting the presence of pulmonary hypertension and estimating its severity. Recognition of structural and haemodynamic signs of increased pulmonary arterial pressure and pressure overload of the right ventricle at magnetic resonance (MR) imaging may contribute to the diagnosis of pulmonary hypertension and management of these patients. In this article, the structural, functional, and haemodynamic cardiovascular MR findings of paediatric pulmonary hypertension are reviewed. Typical diagnostic MR scenarios in children with suspected pulmonary hypertension or in children with disease associated with pulmonary hypertension are presented and discussed.

Avatar-based versus conventional patient monitoring with distant vision: a computer-based simulation study.

Patient monitoring in the perioperative setting can be challenging, especially when monitoring multiple patients simultaneously or managing dynamic situations that require movement around the operating room. We aimed to evaluate whether avatar-based patient monitoring, which presents vital signs in the form of changing colors, shapes and motion, improves remote vital sign recognition compared to conventional monitoring. We conducted a prospective, single-center, computer-based simulation study to evaluate how anesthesia providers recognize vital signs when using the Philips Visual Patient Avatar at different viewing distances (8 and 16m) compared to conventional monitoring. The primary outcome was the total number of correctly identified vital signs which were compared for the two distances and the two devices using mixed Poisson regression. We analyzed data from 28 anesthesia providers who participated in 112 simulations. The correct recognition rate using the Visual Patient Avatar compared to conventional monitoring at 8m was increased by 74% (rate ratio 1.74, 95% CI, 1.42 to 2.14, p < 0.001) and by 51% at 16-meter viewing distance (rate ratio 1.51, 95% CI, 1.23 to 1.87, p < 0.001). We observed scenario-specific superior performance for six vital signs at 8m. The results provide empirical evidence that avatar-based monitoring can significantly improve the perception of vital signs when using distant vision.

Continuous sign language recognition algorithm based on object detection and variable-length coding sequence.

Currently, continuous sign language recognition faces challenges such as difficulty in acquiring skeletal data, long training time for Three-Dimensional convolutional neural networks, and easy occlusion and blurring of hands. To address these problems, this paper proposes a continuous sign language recognition method based on target detection and coding sequence. The algorithm uses Dual-branch Shuffle Attention Mechanis-You Only Look Once version X (DSA-YOLOX) detection network to detect the head and hands, and encodes the sign language video according to the partition to achieve the transformation from Three-Dimensional to One-Dimensional; and then uses the proposed Bi-directional Long Short-Term Memory (BiLSTM) hand coding sequence classification model with jointly weighted Fast Dynamic Time Warping (FastDTW) to extract hand coding similarity and features while reducing the number of parameters to achieve the classification and recognition of unequal-length hand coding sequences. From the results of ablation experiments and comparison experiments, all parts of the improvement perform well. The word error rate (WER) of this paper's method is reduced by 21.26% compared to Dynamic Time Warping-Hidden Markov Model (DTW-HMM) and 11.53% compared to Long Short-Term Memory-A(LSTM-A); the Giga Floating-point Operations Per Second(GFLOPs) of the algorithm are reduced dramatically, which is about 1/13 of the Visual Alignment Constraint(VAC) model and 1/57 of the Spatial-Temporal Multi-Cue(STMC) model; and the algorithm takes better account of the speed and accuracy of sign language recognition.

Dumping syndrome: Update on pathophysiology, diagnosis, and management.

Dumping syndrome is a complex of gastrointestinal symptoms originally studied in peptic ulcer surgery patients. At present, it is most prevalent in patients who underwent bariatric, upper gastrointestinal cancer or anti-reflux surgery. The symptom pattern comprises early and late dumping symptoms. Several management options have been reported including nutritional, pharmacological and surgical approaches. In this study, we aimed to review the current evidence on dumping syndrome definition, diagnosis and treatment, including preliminary data from newer pharmacological studies. Current pathophysiological concepts and analyses of provocative tests has led to a clear definition of dumping syndrome, including both early and late dumping symptoms. The term postbariatric hypoglycemia represents a limited focus on late dumping only. The diagnosis relies on recognition of symptoms and signs in a patient with appropriate surgical history; and can be confirmed by provocative testing or registration of spontaneous hypoglycemia. The initial treatment focuses on dietary intervention, to which meal viscosity enhancers and/or the glycosidase inhibitor acarbose can be added. The most effective therapy is the use of short- or long-acting somatostatin analogues, which is however expensive and entails side effect issues. In case of refractory hypoglycemia, diazoxide or SGLT2 inhibitors can be considered, based on limited evidence. In refractory patients, continuous enteral feeding or (rarely) surgical reinterventions have been advocated, although not supported by solid evidence. Therapies under current evaluation include the broad-spectrum somatostatin analogue pasireotide, GLP-1 receptor antagonists, GLP-1 receptor agonists and administration of stable forms of glucagon are currently under study. Dumping syndrome is a well-defined but probably under-diagnosed complication of upper gastrointestinal, especially bariatric, and surgeries. Diagnosis is confirmed by a provocative test and incremental therapies starting with diet, adding meal viscosity enhancers or glycosidase inhibitors and adding somatostatin analogues in refractory cases. A number of emerging therapies targeting intestinal propulsion, peptide hormone effects and hypoglycemic events are under evaluation.

Cyber Dating Violence: How Is It Perceived in Early Adolescence?

Background: Reports on cyber dating violence in adolescent populations vary significantly depending on whether the focus is on directly aggressive behaviours or behaviours designed to control one’s partner. In contrast to direct aggression, which is often clearly identified by adolescents, there is a greater degree of ignorance, and even a certain degree of normalisation, of controlling behaviours. Such behaviours may include, for example, insisting on knowing the whereabouts of a partner at all times or sharing social media passwords. This study aims to explore adolescent perceptions of cyber dating violence and to identify the differential characteristics of cyber-violent relationships using the Iramuteq software program for text analysis. Methods: Participants were 466 second- and third-year secondary school students. Data were collected through surveys, and responses were analysed using the Iramuteq program. This software tool enabled the identification of common terms and themes linked to cyber dating violence, as perceived by participating adolescents. Results: The analyses revealed the repeated appearance of terms associated with violent behaviours, online media, toxic relationships, and victim coercion. However, there was a notable lack of recognition of controlling behaviours as a manifestation of cyber dating violence. Adolescents frequently normalised behaviours such as insisting on knowing a partner’s whereabouts at all times or sharing social media passwords. Conclusions: The findings suggest a significant gap in adolescents’ understanding of what constitutes cyber dating violence. In contrast to direct aggression, which is easily identified, controlling behaviours are often normalised, indicating a need for educational and preventive measures to address this issue. By improving adolescents’ understanding of controlling behaviours as a form of cyber dating violence, preventive efforts can be more effectively tailored to address and mitigate this problem. To prevent the normalisation of certain behaviours indicative of cyber dating violence, early education is recommended in areas such as healthy relationships, communication skills, respect for privacy, and recognition of signs of excessive control.

A Systematic Review of Accessibility Techniques for Online Platforms: Current Trends and Challenges

Accessibility in online platforms is a critical concern in our increasingly digital world, where information and services are predominantly accessed through the Internet. The purpose of this systematic review is to provide a comprehensive overview of the current state of the art in online accessibility technologies, and it is focused on key tools such as sign language recognition, speech-to-text, text-to-speech, and voice recognition. Despite advancements in digital inclusivity, numerous technical limitations persist, which limit the accessibility of online content for individuals with disabilities. Our findings indicate that while speech and voice technologies have achieved good accuracies and low word error rates, further research is needed to improve the accuracy and usability of sign language recognition systems, especially for continuous sign language recognition, as they have low accuracy. In this review, we analyzed research articles and publications from well-known databases, including Google Scholar, Elsevier, IEEE Xplore, and Springer. In order to ensure a high standard of quality, we applied the PRISMA 2020 and PEDro methodologies to quantitatively and qualitatively filter the thousands of articles provided by these databases, and we selected only studies that were related to our study. Key areas of investigation included the performance and accuracy of sign language interfaces, speech-to-text, text-to-speech, and speech recognition applications and the compatibility of these technologies with different platforms and devices. This review also explores the role of emerging technologies such as artificial intelligence (AI) and machine learning (ML) in enhancing accessibility and personalizing user experiences. Through a critical analysis of current solutions and a discussion of existing gaps, this paper offers insights into potential improvements and future directions for creating more accessible online environments. The findings might be valuable to researchers and developers dedicated to promoting digital inclusivity and equality.

A New Sign Language Translation System Based on Expert Model

Abstract. The linguistic diversity of sign languages across regions presents significant challenges for the development of accurate and culturally sensitive recognition systems. To address this, This paper propose a modular sign language recognition system that detects the regional origin of a sign language and uses specialized sub-models for translation, bypassing the limitations of generalized models. This approach allows for more precise translations by catering to regional variations, ensuring higher accuracy and preserving cultural nuances.This paper also explore the creation of a standardized, diverse dataset, integrating multiple sign languages, which serves as the foundation for the systems region detection process. The dataset was meticulously annotated, normalized, and augmented to support robust model training. By adopting sub-models and utilizing region-specific data, the system improves efficiency and scalability, offering a practical solution for real-world sign language translation. This work emphasizes the importance of preserving sign language diversity while enhancing the usability and adaptability of recognition systems.

Community-Based Mental Health Improvement Initiatives: A Narrative Review and Indiana Case Study.

There is a global mental health crisis: mental illness is underrecognized, underdiagnosed, and undertreated with adverse effects on mental, physical, and social health. In the United States, there is an insufficient number of traditional psychiatric and psychological resources to provide the mental health care needed to solve this crisis. Community-based interventions could be an important adjunct to traditional mental health care. An evaluation of peer-reviewed articles was performed describing community-based interventions and identified 3 approaches with some evidence of effectiveness: (1) interventions that enhance community mental health literacy to improve recognition of early signs of mental illness for early engagement and provide community, family, and peer support; (2) community clinics providing social, medical, and mental health care and support to transition-age youth (15-25 years); and (3) social networking activities to enhance interactions among elders suffering from social isolation and loneliness. Multisector, multidisciplinary, and multicomponent interventions involving health care providers and community-based organizations had the best evidence of effectiveness and should target transition-age youth and elders.

Antidehydration and Stable Mechanical Properties during the Phase Transition of the PNIPAM-Based Hydrogel for Body-Temperature-Monitoring Sensors.

Poly(N-isopropylacrylamide) (PNIPAM) enhances the reversibility and responsiveness of wearable temperature-sensitive devices. However, an open question is whether and how the hydrogel design can prevent adhesive performance loss caused by phase-transition-induced dehydration and unstable mechanical properties between devices and human skin and reduce interfacial failure. Herein, a gelatin-mesh scaffold-based hydrogel (NAGP-Gel) is constructed to inhibit dehydration and volume change, leading to stable mechanical properties, superior adhesiveness, and thermal sensing sensitivity during the phase transition. NAGP-Gel enhances the polymer chains-water interaction and weakens the degree of aggregation of polymer chains-chains, improving antidehydration properties under 45 °C conditions that are higher than the lower critical solution temperature (LCST; i.e., ∼32 °C). The mesh scaffold greatly restricts the phase-transition-induced polymer chain movement and maintains the mechanical performance. In a 60 °C environment, the maximum water loss and volume retention ratio of NAGP-Gel are only 3.58% and 97.3%, respectively. Additionally, NAGP-Gel serves as a temperature sensor, producing a stable thermal-electrical signal within the LCST range. It also can be assembled into an electronic device enabling the transmission of information and recognition of sign language via Morse code. This work broadens the application of PNIPAM in constructing intelligent hydrogels and opens the door to exploring emerging hydrogels for temperature-monitoring applications.

DFCNet +: Cross-modal dynamic feature contrast net for continuous sign language recognition

In sign language communication, the combination of hand signs and facial expressions is used to convey messages in a fluid manner. Accurate interpretation relies heavily on understanding the context of these signs. Current methods, however, often focus on static images, missing the continuous flow and the story that unfolds through successive movements in sign language. To address this constraint, our research introduces the Dynamic Feature Contrast Net Plus (DFCNet+), a novel model that incorporates both dynamic feature extraction and cross-modal learning. The dynamic feature extraction module of DFCNet+ uses dynamic trajectory capture to monitor and record motion across frames and apply key features as an enhancement tool that highlights pixels that are critical for recognizing important sign language movements, allowing the model to follow the temporal variation of the signs. In the cross-modal learning module, we depart from the conventional approach of aligning video frames with textual descriptions. Instead, we adopt a gloss-level alignment, which provides a more detailed match between the visual signals and their corresponding text glosses, capturing the intricate relationship between what is seen and the associated text. The enhanced proficiency of DFCNet+ in discerning inter-frame details translates to heightened precision on benchmarks such as PHOENIX14, PHOENIX14-T and CSL-Daily. Such performance underscores its advantage in dynamic feature capture and inter-modal learning compared to conventional approaches to sign language interpretation. Our code is available at https://github.com/fyzjut/DFCNet_Plus.

Dynamic feature and context enhancement network for faster detection of small objects

While traditional object detection methods have achieved significant success in recent years, their performance in detecting small objects in aerial remote sensing images remains unsatisfactory. Small objects often occupy only a few pixels, leading to a loss of fine-grained information. This paper proposes a dynamic feature and context enhancement network (DFCE) to address noise and pixel-level region weighting issues in small object detection. The DFCE network effectively detects small objects by dynamically selecting features within regions and establishing connections between local and global contextual information. The introduced dynamic multi-dimensional attention (DMA) module selects key information via a crossover mechanism and assigns different weights to highlight important features. Based on DMA, the regional feature processing (RFP) module and multi-dimensional pool transformer (MPT) module are developed to capture key information and contextual information, respectively. Experimental results demonstrate that the DFCE network improves average precision (AP) by 3.1%, 9%, and 2.2% on two remote sensing datasets and one conventional dataset, achieving an inference speed of 30 frames per second (FPS). Given these advancements, the DFCE model’s powerful key feature extraction and contextual association capabilities show strong potential for broader applications, including sign language recognition, defect detection in industrial settings, and more.

American Sign Language Recognition System: Enhancing Communication Accessibility for Individuals with Hearing Disabilities

American Sign Language Recognition System: Enhancing Communication Accessibility for Individuals with Hearing Disabilities

Innovative Hand Sign to Text-and-Speech Conversion System

In present scenario, it is difficult for people who are unable to speak due to various reasons. It may not be a permanent condition, as muteness and deafness can be caused or manifest due to several different phenomena, such as physiological injury, illness, medical side effects, psychological trauma, developmental disorders, or neurological disorders. People who are suffering from this typically use the Sign Language to communicate with other people. However, it is not possible for everyone to be acquainted with communication using sign language. This project focuses on developing a solution which can be used by regular people to understand the people having mutism. Hand Sign Recognition Systems have evolved significantly with advancements in machine learning and deep learning techniques. These systems are designed to recognize gestures or signs, making them useful in various applications such as sign language interpretation, human-computer interaction, and virtual reality. This paper gives an overview of the current state of hand sign recognition systems, highlighting the techniques used, challenges, and possible future directions. The focus of the paper is on deep learning-based methods and their impact on the accuracy and usability of hand sign recognition in real-time applications [1]

Few-shot Learning for Sign Language Recognition with Embedding Propagation

Sign language is a primary channel for the deaf and hard-hearing to communicate. Sign language consists of many signs with different variations in hand shapes, motion patterns, and positioning of hands, faces, and body parts. This makes sign language recognition (SLR) a challenging field in computer vision research. This paper tackles the problem of few-shot SLR, where models trained on known sign classes are utilized to recognize instances of unseen signs with only a few examples. In this approach, a transformer encoder is employed to learn the spatial and temporal features of sign gestures, and an embedding propagation technique is used to project these features into the embedding space. Subsequently, a label propagation method is applied to smooth the resulting embeddings. The obtained results demonstrate that combining embedding propagation with label propagation enhances the performance of the SLR system and achieved an accuracy of 76.6%, which surpasses the traditional few-shot prototypical network's accuracy of 72.4%.

Sign language recognition using modified deep learning network and hybrid optimization: a hybrid optimizer (HO) based optimized CNNSa-LSTM approach.

Speech impairment limits a person's capacity for oral and auditory communication. Improvements in communication between the deaf and the general public can be progressed by a real-time sign language detector. Recent studies have contributed to make progress in motion and gesture identification processes using Deep Learning (DL) methods and computer vision. But the development of static and dynamic sign language recognition (SLR) models is still a challenging area of research. The difficulty is in obtaining an appropriate model that addresses the challenges of continuous signs that are independent of the signer. Different signers' speeds, durations, and many other factors make it challenging to create a model with high accuracy and continuity. This study mainly focused on SLR using a modified DL and hybrid optimization approach. Notably, spatial and geometric-based features are extracted via the Visual Geometry Group 16 (VGG16), and motion features are extracted using the optical flow approach. A new DL model, CNNSa-LSTM, is a combination of a Convolutional Neural Network (CNN), Self-Attention (SA), and Long-Short-Term Memory (LSTM) to identify sign language. This model is developed for feature extraction by combining CNNs for spatial analysis with SA mechanisms for focusing on relevant features, while LSTM effectively models temporal dependencies. The proposed CNNSa-LSTM model enhances performance in tasks involving complex, sequential data, such as sign language processing. Besides, a Hybrid Optimizer (HO) is proposed using the Hippopotamus Optimization Algorithm (HOA) and the Pathfinder Algorithm (PFA). The proposed model has been implemented in Python, and it has been evaluated over the existing models in terms of accuracy (98.7%), sensitivity (98.2%), precision (98.5%), Word Error Rate (WER) (0.131), Sign Error Rate (SER) (0.114), and Normalized Discounted Cumulative Gain (NDCG) (98%) as well. The proposed model has recorded the highest accuracy of 98.7%.

AUTOMATED GESTURE RECOGNITION USING APPLIED LINGUISTICS WITH DATA-DRIVEN DEEP LEARNING FOR ARABIC SPEECH TRANSLATION

Gesture recognition for Arabic speech translation includes developing advanced technologies that correctly translate body and hand movements corresponding to Arabic sign language (ArSL) into spoken Arabic. This leverages machine learning and computer vision techniques in complex systems simulation platforms to scrutinize the gestures utilized in ArSL, detecting mild differences in facial expressions, hand shapes, and movements. Sign Language Recognition (SLR) is paramount in assisting communication for the Deaf and Hard-of-Hearing communities. It includes using vision-based methods and Surface Electromyography (sEMG) signals. The sEMG signal is crucial for recognizing hand gestures and capturing muscular activities in sign language. Researchers have comprehensively shown the capability of EMG signals to approach specific details, mainly in classifying hand gestures. This progression is a stimulating feature in extracting the interpretation and recognition of sign languages and investigating the phonology of signed language. Leveraging machine learning algorithms and signal processing techniques in complex systems simulation platforms, researchers aim to extract relevant traits from the sEMG signals that correspond to different ArSL gestures. This study introduces an Enhanced Dwarf Mongoose Algorithm with a Deep Learning-Driven Arabic Sign Language Detection (EDMODL-ASLD) technique on sEMG data. In the initial phase, the presented EDMODL-ASLD model is subjected to data preprocessing to change the input sEMG data into an attuned format. In the next stage, feature extraction with fractal theories is used to gather relevant and nonredundant data from the EMG window to construct a feature vector. In this study, the absolute envelope (AE), energy (E), root-mean square (RMS), standard deviation (STD), and mean absolute value (MAV) are the five time-domain extracted features for the EMG window observation. Meanwhile, the dilated convolutional long short-term memory (ConvLSTM) technique is used to identify distinct sign languages. To improve the results of the dilated ConvLSTM model, the hyperparameter selection process is executed using the EDMO model. To illustrate the significance of the EDMODL-ASLD technique, a brief experimental validation is made on the Arabic SLR database. The experimental validation of the EDMODL-ASLD technique portrayed a superior accuracy value of 96.47% over recent DL approaches.

Automatic Gesture-Based Arabic Sign Language Recognition: A Federated Learning Approach

Featuring machine learning algorithms for recognizing hand gesture patterns adjusted for individuals with disabilities is an expanding trend in assisted living. This paper addresses the challenge of interpreting the semantics of image-based hand gestures by introducing a federated deep learning architecture for Arabic sign language recognition. The proposed model manages distributed learning through a client-server paradigm, wherein several edge nodes collaborate to jointly learn the discriminative features of confidential data without breaching its privacy. This model will enable more accessibility for people with deafness or impairment using image gestures. The federated learning procedure is primarily based on the ResNet32 deep backbone and federated averaging mechanism. The experimental results show the effectiveness of the proposed FL model, achieving an accuracy of 98.30% with 33 seconds on average for each client in a single training round. This demonstrates its high capabilities in recognizing Arabic sign language and improving the communication experience for people with disabilities.

Research and Analysis of Deep Learning in Environmental Perception for Autonomous Driving

Autonomous driving technology, as a revolutionary development in the transportation field, relies on efficient and reliable environmental perception systems. Deep learning plays a key role in the environmental perception of autonomous driving, with its capabilities widely applied in areas such as signal and sign recognition, semantic segmentation, instance segmentation, and SLAM (Simultaneous Localization and Mapping). This study conducts research and analysis on the aforementioned key technologies. It concludes that while deep learning technology has shown excellent performance in the perceptual systems of autonomous driving, it still faces challenges that require further research and resolution, such as generalization performance, eal-time requirements, and safety issues.

LINES OF ACCESS TO THE RIGHT TO DEAF EDUCATION AND CULTURE

Legal guarantees ensure access to education for deaf students, notably, based on some official documents (national/international) that encouraged the publication of specific laws on the educational process for deaf students, considering their differences and cultural identities, and thus the recognition of natural sign language as the language of instruction. This study analyzed several official documents and laws that provide support for the education of deaf students in Brazil, in light of studies dedicated to this educational practice under the theoretical contribution of authors such as Lage et al. (2020), Santos et al. (2015), Perlin and Souza (2015), Quadros et al. (2008), Lodi (2011), Perlin and Strobel (2008), Miorando (2006), Quadros and Schmiedt (2006), among others who are dedicated to the topic in question. The research is characterized as exploratory, from the point of view of its objective, bibliographical and documentary by its procedures. The results of the research showed that there are several legal guarantees and some effectiveness of these guarantees, but they still do not offer full access to teaching and learning in accordance with current legislation.