Static Signs Research Articles

A Static Sign Language Recognition Method Enhanced with Self-Attention Mechanisms

For the current wearable devices in the application of cross-diversified user groups, it is common to face the technical difficulties of static sign language recognition accuracy attenuation, weak anti-noise ability, and insufficient system robustness due to the differences in the use of users. This paper proposes a novel static sign language recognition method enhanced by a self-attention mechanism. The key features of sign language gesture classification are highlighted by the weight function, and then the self-attention mechanism is combined to pay more attention to the key features, and the convolutional neural network is used to extract the features and classify them, which realizes the accurate recognition of different types of static sign language under standard gestures and non-standard gestures. Experimental results reveal that the proposed method achieves an average accuracy of 99.52% in the standard static sign language recognition task when tested against the standard 36 static gestures selected within the reference American Sign Language dataset. By imposing random angular bias conditions of ±(0°–9°] and ±(9°–18°], the average recognition rates in this range were 98.63% and 86.33%. These findings indicate that, compared to existing methods, the proposed method not only maintains a high recognition rate for standard static gestures but also exhibits superior noise resistance and robustness, rendering it suitable for static sign language recognition among diverse user populations.

STATIC SIGN LANGUAGE TRANSLATOR USING HAND GESTURE AND SPEECH RECOGNITION

Communication between ordinary and deaf people often has issues because ordinary people lack knowledge of sign language. This research aims to help ordinary people communicate with hearing impaired (deaf) people using sign language. This research aims to produce an Android-based mobile app that can translate static sign language using hand movements into text and also convert the spoken voice into sign language using speech recognition. The framework in this research for hand gesture detection uses the MediaPipe software program. This framework allows the creation of apps that translate hand movements into text that help ordinary people understand sign language. Speech recognition in this research uses the Android speech library. This research succeeded in detecting static letters of the alphabet from the a-z and numbers 0-9. Tests of 540 hand gestures carried out in the morning, afternoon, evening, and night had an average detection time of 4.37 seconds. The fastest object detection times were in the morning at a distance of 30 cm with an average detection time of 2.5 seconds. Based on acceptance testing, 83.13% of the features in this static sign language translator have met the users ’users’ needs when communicating with the deaf using sign language.

Gesture Recognition of Filipino Sign Language Using Convolutional and Long Short-Term Memory Deep Neural Networks

In response to the recent formalization of Filipino Sign Language (FSL) and the lack of comprehensive studies, this paper introduces a real-time FSL gesture recognition system. Unlike existing systems, which are often limited to static signs and asynchronous recognition, it offers dynamic gesture capturing and recognition of 10 common expressions and five transactional inquiries. To this end, the system sequentially employs cropping, contrast adjustment, grayscale conversion, resizing, and normalization of input image streams. These steps serve to extract the region of interest, reduce the computational load, ensure uniform input size, and maintain consistent pixel value distribution. Subsequently, a Convolutional Neural Network and Long-Short Term Memory (CNN-LSTM) model was employed to recognize nuances of real-time FSL gestures. The results demonstrate the superiority of the proposed technique over existing FSL recognition systems, achieving an impressive average accuracy, recall, and precision rate of 98%, marking an 11.3% improvement in accuracy. Furthermore, this article also explores lightweight conversion methods, including post-quantization and quantization-aware training, to facilitate the deployment of the model on resource-constrained platforms. The lightweight models show a significant reduction in model size and memory utilization with respect to the base model when executed in a Raspberry Pi minicomputer. Lastly, the lightweight model trained with the quantization-aware technique (99%) outperforms the post-quantization approach (97%), showing a notable 2% improvement in accuracy.

A Review on Sign Language Recognition and Learning System

This paper presents innovative approaches to various aspects of sign language recognition and learning, catering to the diverse needs within this field. It introduces a system utilizing 2D image sampling and concatenation, trained with convolutional neural networks, achieving accurate and robust sign recognition even with low-cost cameras. Another system, SignQuiz, offers a cost-effective web-based solution for learning finger-spelled signs in Indian Sign Language (ISL), outperforming traditional printed mediums. Additionally, a dynamic hand gesture recognition system employing deep learning architectures demonstrates superior performance over existing methods, addressing challenges such as hand segmentation and sequence modeling. Furthermore, an efficient deep convolutional neural network approach for hand gesture recognition, including transfer learning, achieves high recognition rates across various datasets. Finally, a robust model for static sign recognition in ISL, utilizing CNNs and evaluating on a diverse dataset, attains exceptional accuracy and outperforms previous works. These contributions collectively advance the field of sign language recognition and learning, offering solutions that are accurate, cost-effective, and efficient, thereby facilitating better communication and interaction for the hearing-impaired community. Key Words: CNN, Sign language recognition, Hand gesture recognition, Machine learning, Transfer Learning, SVM.

Bots as an online communication tool in the political process on the example of the EDC-2023 in Russia

The article presents a brief theoretical overview of the types of bots, the purposes of their use, identification methods, and also examines global political campaigns with the active use of automated systems. Using the example of the 2023 regional election campaign, an empirical study was conducted, the purpose of which is to identify the features of using bot systems in online communication of the authorities represented by candidates and society. The study was conducted on the basis of open data from the VKontakte social network - all publications were selected in which candidates for the election of heads of regions for the period from August 28 to September 10, 2023 were mentioned. A total of 21 regions and 116 candidates were analyzed. The largest number of bots is typical for Moscow, the Moscow Region, the Republic of Khakassia, and the Pskov region. The detection of bots was based on the method of static signs: the same / similar text of the post, the absence of user activity, the absence of real photos, the lack of a personal profile. The data obtained allowed us to formulate conclusions about the use of bots to create a positive and neutral agenda. At the same time, the publications created automatically had zero or low views, which indicates their invisibility to the final audience and inability to influence the formation of public opinion. However, data on the mention of candidates and the presence of their positive image in the social network can be used in the preparation of administrative or analytical reports.

Bioinspired Photoreceptors with Neural Network for Recognition and Classification of Sign Language Gesture.

This work addresses the design and implementation of a novel PhotoBiological Filter Classifier (PhBFC) to improve the accuracy of a static sign language translation system. The captured images are preprocessed by a contrast enhancement algorithm inspired by the capacity of retinal photoreceptor cells from mammals, which are responsible for capturing light and transforming it into electric signals that the brain can interpret as images. This sign translation system not only supports the effective communication between an agent and an operator but also between a community with hearing disabilities and other people. Additionally, this technology could be integrated into diverse devices and applications, further broadening its scope, and extending its benefits for the community in general. The bioinspired photoreceptor model is evaluated under different conditions. To validate the advantages of applying photoreceptors cells, 100 tests were conducted per letter to be recognized, on three different models (V1, V2, and V3), obtaining an average of 91.1% of accuracy on V3, compared to 63.4% obtained on V1, and an average of 55.5 Frames Per Second (FPS) in each letter classification iteration for V1, V2, and V3, demonstrating that the use of photoreceptor cells does not affect the processing time while also improving the accuracy. The great application potential of this system is underscored, as it can be employed, for example, in Deep Learning (DL) for pattern recognition or agent decision-making trained by reinforcement learning, etc.

Fusion of Attention-Based Convolution Neural Network and HOG Features for Static Sign Language Recognition

The deaf and hearing-impaired community expresses their emotions, communicates with society, and enhances the interaction between humans and computers using sign language gestures. This work presents a strategy for efficient feature extraction that uses a combination of two different methods that are the convolutional block attention module (CBAM)-based convolutional neural network (CNN) and standard handcrafted histogram of oriented gradients (HOG) feature descriptor. The proposed framework aims to enhance accuracy by extracting meaningful features and resolving issues like rotation, similar hand orientation, etc. The HOG feature extraction technique provides a compact feature representation that signifies meaningful information about sign gestures. The CBAM attention module is incorporated into the structure of CNN to enhance feature learning using spatial and channel attention mechanisms. Then, the final feature vector is formed by concatenating these features. This feature vector is provided to the classification layers to predict static sign gestures. The proposed approach is validated on two publicly available static Massey American Sign Language (ASL) and Indian Sign Language (ISL) databases. The model’s performance is evaluated using precision, recall, F1-score, and accuracy. Our proposed methodology achieved 99.22% and 99.79% accuracy for the ASL and ISL datasets. The acquired results signify the efficiency of the feature fusion and attention mechanism. Our network performed better in accuracy compared to the earlier studies.

Deep Learning in Sign Language Recognition: A Hybrid Approach for the Recognition of Static and Dynamic Signs

A speech impairment limits a person’s capacity for oral and auditory communication. A great improvement in communication between the deaf and the general public would be represented by a real-time sign language detector. This work proposes a deep learning-based algorithm that can identify words from a person’s gestures and detect them. There have been many studies on this topic, but the development of static and dynamic sign language recognition models is still a challenging area of research. The difficulty is in obtaining an appropriate model that addresses the challenges of continuous signs that are independent of the signer. Different signers’ speeds, durations, and many other factors make it challenging to create a model with high accuracy and continuity. For the accurate and effective recognition of signs, this study uses two different deep learning-based approaches. We create a real-time American Sign Language detector using the skeleton model, which reliably categorizes continuous signs in sign language in most cases using a deep learning approach. In the second deep learning approach, we create a sign language detector for static signs using YOLOv6. This application is very helpful for sign language users and learners to practice sign language in real time. After training both algorithms separately for static and continuous signs, we create a single algorithm using a hybrid approach. The proposed model, consisting of LSTM with MediaPipe holistic landmarks, achieves around 92% accuracy for different continuous signs, and the YOLOv6 model achieves 96% accuracy over different static signs. Throughout this study, we determine which approach is best for sequential movement detection and for the classification of different signs according to sign language and shows remarkable accuracy in real time.

Comparative Analysis of Indian Sign Language Recognition System

Sign language is a way of communication among Hearing and Speech Impaired Persons. Normal peoples can not understand sign language and it is not feasible for deaf-dumb people to bring translator with them in every place. So, for bridging this gap many systems have been developed. Sign Language recognition systems which can convert Sign into text or Speech and vice-versa. Sign language recognition system work in five steps are: data acquisition, pre-processing, feature extraction, classification and recognition. This paper discussed the Indian sign language recognition system. In this paper, a comparative analysis of various gesture recognition techniques involving Artificial Neural Network, Convolutional Neural Networks Hidden Markov Model and PCA has been discussed with its accuracy. This comparative study came across that much work has been done in alphabet and numeric level but work in word and sentence level is less. Sign language recognition for static signs has been done by many researchers but dynamic sign recognition systems have scope of development. A Comparative study is utilized to find out research gaps in existing systems and give inspiration to develop interpreters for Indian Sign Languages.

Route Planning for Emergency Evacuation Using Graph Traversal Algorithms

The automatic identification of various design elements in a floor-plan image has gained increasing attention in recent research. Emergency-evacuation applications can benefit greatly from automated floor-plan solutions, as they allow for the development of horizontal solutions instead of vertical solutions targeting a specific audience. In addition to that, current evacuation plans rely on static signs without taking into account the dynamic characteristics of each emergency case. This work aims to extract information from a floor-plan image and transform it into a graph that is used for pathfinding in an emergency evacuation. First, the basic elements of the floor-plan image, i.e., walls, rooms and doors, are identified. This is achieved using Panoptic-DeepLab, which is a state-of-the-art deep neural network for the panoptic segmentation of images, and it is available from DeepLab2, an image segmentation library. The neural network was trained using CubiCasa5K, a large-scale floor-plan image dataset containing 5000 samples, annotated into over 80 floor-plan object categories. Then, using the prediction of each pixel, a graph that shows how rooms and doors are connected is created. An application that presents this information in a user-friendly manner and provides graph editing capabilities was developed. Finally, the exits are set, and the optimal path for evacuation is calculated from each node using Dijkstra’s algorithm.

Deep Reinforcement Learning Approach for V2X Managed Intersections of Connected Vehicles

Intersections are major bottlenecks for road traffic, as well as the origin of many accidents. Efficient management of traffic at intersections is required to ensure both safety and efficiency. Yet, the traditional solutions (static signs, traffic lights) are limited in their efficiency as they consider the flow of vehicles and not the vehicles at the microscopic level. By using inter-vehicular communication of connected vehicles, recent works have shown the possibility to have a great increase in the number of evacuated vehicles thanks to the possibility to give an individual right-of-way directly to each vehicle. In this context of intersections of cooperative vehicles, the scheduling of this right-of-way in order to maximize the throughput of the intersection is still a challenging task, with regard to the hybrid and dynamic aspects of the problem. In this paper, we propose an approach based on Deep Reinforcement Learning (DRL) to efficiently distribute the right-of-way to each vehicle. A Markov Decision Process model of intersections of cooperative vehicles, enabling the application of DRL, is proposed. The performance of the DRL-based scheduling is then compared with classic traffic lights, and with two state-of-the-art cooperative scheduling policies, showing the benefits of the approach (increase of the flow, reduction of CO2 emissions).

Optimization of convolutional neural network hyperparameters using improved competitive gray wolf optimizer for recognition of static signs of Indian Sign Language

In the last decade, the success achieved by convolutional neural networks (CNN) has changed the trend of research in the computer vision field from manual features to learned features for various applications. However, designing the architecture of CNN and selecting optimum values of its hyperparameters poses a critical challenge because of their massive and complex search space. Automatically picking up the optimal values of hyperparameters may facilitate the designers of CNN-based deep learning models. Swarm intelligence (SI)-based improved competitive gray wolf optimizer (ICGWO) in continuous search space is proposed for optimizing hyperparameters of custom-designed CNN architecture. To evaluate the performance of the ICGWO, we employ 16 benchmark functions. We also assess the effectiveness of the ICGWO by evaluating it with six state-of-art meta-heuristic algorithms. The experiment results show that the ICGWO can obtain the best accuracy on eleven benchmark functions and second best on two benchmark functions. The optimized CNN recognizes static signs of English alphabets in Indian Sign Language (ISL). We also evaluated the optimized CNN on the publicly available ISL dataset for the English alphabet, where it attains 99.93% validation accuracy. In order to evaluate the efficacy of the proposed variant of CGWO, we optimized the hyperparameters of CNN using the existing SI-based search optimization algorithms namely particle swarm optimization, whale optimization algorithm (WOA), differential evolution, and gray wolf optimizer as well.

Continuous word level sign language recognition using an expert system based on machine learning

The study of sign language recognition systems has been extensively explored using many image processing and artificial intelligence techniques for many years, but the main challenge is to bridge the communication gap between specially-abled people and the general public. This paper proposes a python-based system that classifies 80 words from sign language. Two different models have been proposed in this work: You Only Look Once version 4 (YOLOv4) and Support Vector Machine (SVM) with media-pipe. SVM utilizes the linear, polynomial and Radial Basis Function (RBF) kernels. The system does not need any additional pre-processing and image enhancement operations. The image dataset used in this work is self-created and consists of 80 static signs with a total of 676 images. The accuracy of SVM with media-pipe is 98.62% and the accuracy of YOLOv4 obtained is 98.8% which is higher than the existing state-of-the-art methods. An expert system is also proposed which utilizes both the above models to predict the hand gesture more accurately in real-time.

Surgical Technique: Wide Arthroscopic Dorsal Capsuloligamentous Repair for Advanced Scapholunate Instability

In the most advanced cases of scapholunate instability with dynamic or static signs, classical arthroscopic repair seems impossible. Ligamentoplasties or open surgery procedures are technically demanding, hampered by significant operative complications and often stiffening. Therapeutic simplification is therefore necessary for the management of these complex cases of advanced scapholunate instability. We propose a minimally invasive, reliable, and easily reproducible solution that requires little equipment other than arthroscopic material.

ShuffleNetv2-YOLOv3: a real-time recognition method of static sign language based on a lightweight network

To better meet the communication needs of hearing impaired people and the public, it is of great significance to recognize sign language more quickly and accurately on embedded platforms and mobile terminals. YOLOv3, raised by Joseph Redmon and Ali Farhadi in 2018, achieved a great improvement in detection speed with considerable accuracy by optimizing Yolo. However, YOLOv3 is still too bloated to use on mobile terminals. A static sign language recognition method based on the ShuffleNetv2-YOLOv3 lightweight model was proposed. The ShuffleNetv2-YOLOv3 lightweight model makes the network lightweight by using ShuffleNetv2 as the backbone network of YOLOv3. The lightweight network improved the recognition speed steeply. Combing with the CIoU loss function, the ShuffleNetv2-YOLOv3 keeps the recognition accuracy while improving the recognition speed. Recognition effectiveness of the self-made sign language images and public database by the ShuffleNetv2-YOLOv3 lightweight model was evaluated by F1 score and mAP value. The performance of the ShuffleNetv2-YOLOv3 model was compared with that of the YOLOv3-tiny, SSD, Faster-RCNN, and YOLOv4-tiny model, respectively. The experimental results show that the proposed ShuffleNetv2-YOLOv3 model achieved a good balance between the accuracy and speed of the gesture detection under the premise of model lightweight. The F1 score and mAP value of the ShuffleNetv2-YOLOv3 model were 99.1% and 98.4%, respectively. The gesture detection speed on the GPU reaches 54 frames per second, which is better than other models. The mobile terminal application of the proposed lightweight model was also evaluated. The minimal inference speed of single frame images on the CPU and GPU is 0.14 and 0.025 s per image, respectively. It is only 1/6.5 and 1/8.5 of the running speed of the original YOLOv3 model. The ShuffleNetv2-YOLOv3 lightweight model is conducive to quick, real time, and similar static sign language gesture recognition, laying a good foundation for real-time gesture recognition in the embedded platforms and mobile terminals.

A Deep Learning based Approach for Recognition of Arabic Sign Language Letters

No one can deny that the deaf-mute community has communication problems in daily life. Advances in artificial intelligence over the past few years have broken through this communication barrier. The principal objective of this work is creating an Arabic Sign Language Recognition system (ArSLR) for recognizing Arabic letters. The ArSLR system is developed using our image pre-processing method to extract the exact position of the hand and we proposed architecture of the Deep Convolutional Neural Network (CNN) using depth data. The goal is to make it easier for people who have hearing problems to interact with normal people. Based on user input, our method will detect and recognize hand-sign letters of the Arabic alphabet automatically. The suggested model is anticipated to deliver encouraging results in the recognition of Arabic sign language with an accuracy score of 97,07%. We conducted a comparison study in order to evaluate proposed system, the obtained results demonstrated that this method is able to recognize static signs with greater accuracy than the accuracy obtained by similar other studies on the same dataset used.

Pakistan sign language to Urdu translator using Kinect

The lack of a standardized sign language, and the inability to communicate with the hearing community through sign language, are the two major issues confronting Pakistan's deaf and dumb society. In this research, we have proposed an approach to help eradicate one of the issues. Now, using the proposed framework, the deaf community can communicate with normal people. The purpose of this work is to reduce the struggles of hearing-impaired people in Pakistan. A Kinect-based Pakistan sign language (PSL) to Urdu language translator is being developed to accomplish this. The system’s dynamic sign language segment works in three phases: acquiring key points from the dataset, training a long short-term memory (LSTM) model, and making real-time predictions using sequences through openCV integrated with the Kinect device. The system’s static sign language segment works in three phases: acquiring an image-based dataset, training a model garden, and making real-time predictions using openCV integrated with the Kinect device. It also allows the hearing user to input Urdu audio to the Kinect microphone. The proposed sign language translator can detect and predict the PSL performed in front of the Kinect device and produce translations in Urdu.

Pakistan sign language to Urdu translator using Kinect

The lack of a standardized sign language, and the inability to communicate with the hearing community through sign language, are the two major issues confronting Pakistan's deaf and dumb society. In this research, we have proposed an approach to help eradicate one of the issues. Now, using the proposed framework, the deaf community can communicate with normal people. The purpose of this work is to reduce the struggles of hearing-impaired people in Pakistan. A Kinect-based Pakistan sign language (PSL) to Urdu language translator is being developed to accomplish this. The system’s dynamic sign language segment works in three phases: acquiring key points from the dataset, training a long short-term memory (LSTM) model, and making real-time predictions using sequences through openCV integrated with the Kinect device. The system’s static sign language segment works in three phases: acquiring an image-based dataset, training a model garden, and making real-time predictions using openCV integrated with the Kinect device. It also allows the hearing user to input Urdu audio to the Kinect microphone. The proposed sign language translator can detect and predict the PSL performed in front of the Kinect device and produce translations in Urdu.

The materiality of lines: The kinaesthetics of bodily movement uniting dance and prehistoric cave art

We take two seemingly disparate practices as the foci for this work—dance and prehistoric cave art—in order to illuminate commonalities in embodied practices that reveal new theoretical insights. Whereas, dance clearly uses temporal traces of the body to shape space and create meaning, cave drawings have also begun to be explained and interpreted through kinesthetic and embodied metaphorical techniques. A key element that these fields have in common is the role of lines. The study of lines has become its own field of research, largely due to Ingold's foundational work on “linealogy”. Considering the animate nature of lines and the kinesthetic response they have on the body, contemporary artistic practices can find parallels to the nature of ancient rock art if we recognize the similarity these fields share in the formation of and interpretation of meaning through movement and material engagement. Lines are a phenomenon that, when looked at as traces—memories of movement that once took place—become an organizing principle that brings distant fields like rock art and dance together. Thus, this article takes Tilley's suggestion that “a truly phenomenological study of imagery is grounded in the kinaesthetics of bodily movement” and applies it to three different artistic practices. Tilley proposes that an artifact may be ambiguous in its meaning because it relays a variety of information, which is often layered and contradictory. Here we see, in cognitive archeology, the idea that the articulation of ambiguous lines is an attempt to bring like-minded things (in the thoughts of the one drawing or dancing) together. The authors propose that the ambiguity of lines allows for a type of withness between the mind, body, and environment, allowing us to consider the mind as extended. Lastly, the nature of lines as extended mind motivate us to propose the term of “signature” in order to circumvent the dilemma of a static sign in regards to movement.

Reducing the Number of Sensors in the Data Glove for Recognition of Static Hand Gestures

Data glove devices, apart from being widely used in industry and entertainment, can also serve as a means for communication with the environment. This is possible thanks to the advancement in electronic technology and machine learning algorithms. In this paper, the results of the study using a designed data glove equipped with 10 piezoelectric sensors are reported, and the designed glove is validated on a recognition task of hand gestures based on 16 static signs of the Polish Sign Language (PSL) alphabet. The main result of the study is that recognition of 16 PSL static gestures is possible with a reduced number of piezoelectric sensors. This result has been achieved by applying the decision tree classifier that can rank the importance of the sensors for the recognition performance. Other machine learning algorithms were also tested, and it was showed that for the Support Vector Machines, k-NN and Bagged Trees classifiers, a recognition rate of the signs exceeding 90% can be achieved just for three preselected sensors. Such a result is important for a reduction in design complexity and costs of such a data glove with sustained reliability of the device.