Language Translation System Research Articles

Intelligent Flexible Pressure Sensors with Improved Sensing Range and Sensitivity Based on 3D-Graphene Patterning Induced by UV Laser.

Intelligent flexible sensors are an integral component of the next generation of consumer electronics. However, developing a flexible pressure sensing system that possesses both high sensitivity and a sensing range remains a key challenge in practical applications. Herein, a machine learning-assisted 3D laser-induced graphene (3D-LIG)/polydimethylsiloxane composite flexible pressure sensing system based on ultraviolet (UV) laser integrated fabrication was proposed. Low-cost LIG-based flexible sensors with a 3D carbonized structure were prepared by selective UV laser ablation and laser-induced graphitization. The interlayer interlocking structure, combined with the internal porous structure of LIG, enriches the sensing mechanism, allowing the sensor to exhibit triphasic linear response characteristics, demonstrating a large sensing range (0-500 kPa) and high sensitivity (0-20 kPa, 3.047 kPa-1). Based on machine-learning algorithms, an intelligent wearable sign language translation system was constructed capable of high-precision recognition of complex sign language sequence information. The integration of LIG with 3D microstructures allows a wider space for designing LIG-based flexible sensing structures and offers a promising platform for the development of intelligent wearable devices.

Application of AI in Turkish Sign Language Translation: A Case Study of its Use and Purpose

In Turkey, around 3 million people have hearing impairments, and the shift to digital platforms during the pandemic has worsened accessibility challenges as websites and apps often ignore their needs. Research shows that 50% of hearing-impaired individuals struggle to understand written text due to Turkish Sign Language being their first language, with Turkish serving as a second language. Differences in grammar between Turkish and Turkish Sign Language, along with a limited sign language vocabulary, further hinder comprehension. To solve this, we developed AI-powered sign language translation systems that allow people who are deaf or hard of hearing to access digital content in Turkish Sign Language. SignForDeaf's system translates text into sign language videos using Natural Language Processing (NLP) and generates seamless videos with smooth word transitions. Currently, this system supports Turkish Sign Language, with future plans to include other languages like Arabic, American, British, and Finnish Sign Language. The system was designed in collaboration with sign language experts to ensure accuracy and an inclusive development of a digital environment.

The Development of AI-Powered Automatic Video Sign Language Translation Systems

People who are deaf or hard of hearing often face challenges in fully understanding written subtitles in videos due to the differences between sign language and spoken language, each having unique grammar and structures. While many social media platforms provide automatic subtitles, they are often insufficient for accessibility. Sign language interpretation should be included to make video content fully accessible. Traditionally, adding sign language to videos involves a time-consuming process of recording and embedding separate translation videos, which must be redone with any changes to the original video. The plugin simplifies this process by dynamically translating updates directly from subtitle files, significantly reducing the time, effort, and cost involved. It allows seamless sign language support for any video without modifying the original content. The plugin integrates with video players, providing a user-controlled, customizable sign language window that can be activated, moved, resized, or turned off as desired.

A Transformer-Based Yoruba to English Machine Translation (TYEMT) System with Rouge Score

Automated translation systems for some indigenous Nigerian languages like the Yoruba, have historically been limited by the lack of large, high- quality bilingual text and effective approaches to modeling. This paper presents introduces an approach to bi-directional Yoruba-English text-to-text machine translation utilizing deep learning technique, specifically Transformer models. Transformer models, which utilizes self-attention mechanisms to improve translation quality and efficiency. The system was trained and evaluated on a newly curated Yoruba- English parallel corpus, which significantly augments existing resources. Experimental results demonstrate that the Transformer-based model performs translation accurately and fluently, achieving a ROUGE (Recall-Oriented Understudy for Gisting Evaluation) score improvement of 0.4649. This work not only advances the frontiers of Yoruba-English machine translation but also enriches a wider domain in the field of multilingual Natural Language processing (NLP) by addressing challenges associated with translating between languages with limited resources. Future studies include enhancing the available parallel corpus and exploring hybrid models that combine the strengths of both RNN and Transformer architectures.

From rule-based models to deep learning transformers architectures for natural language processing and sign language translation systems: survey, taxonomy and performance evaluation

With the growing Deaf and Hard of Hearing population worldwide and the persistent shortage of certified sign language interpreters, there is a pressing need for an efficient, signs-driven, integrated end-to-end translation system, from sign to gloss to text and vice-versa. There has been a wealth of research on machine translations and related reviews. However, there are few works on sign language machine translation considering the particularity of the language being continuous and dynamic. This paper aims to address this void, providing a retrospective analysis of the temporal evolution of sign language machine translation algorithms and a taxonomy of the Transformers architectures, the most used approach in language translation. We also present the requirements of a real-time Quality-of-Service sign language machine translation system underpinned by accurate deep learning algorithms. We propose future research directions for sign language translation systems.

Developing and Deploying End‐to‐End Machine Learning Systems for Social Impact: A Rubric and Practical Artificial Intelligence Case Studies From African Contexts

ABSTRACTArtificial intelligence (AI) and machine learning have demonstrated the potential to provide solutions to societal challenges, for example, automated crop diagnostics for smallholder farmers, environmental pollution modelling and prediction for cities and machine translation systems for languages that enable information access and communication for segments of the population who are unable to speak or write official languages, among others. Despite the potential of AI, the practical and technical issues related to its development and deployment in the African context are the least documented and understood. The development and deployment of AI for social impact systems in the developing world present new intricacies and requirements emanating from the unique technology and social ecosystems in these settings. This paper provides a rubric for developing and deploying AI systems for social impact with a focus on the African context. The rubric is derived from the analysis of a series of selected real‐world case studies of AI applications in Africa. We assessed the selected AI case studies against the proposed rubric. The rubric and examples of AI applications presented in this paper are expected to contribute to the development and application of AI systems in other African contexts.

Natural Language Processing Pretraining Language Model for Computer Intelligent Recognition Technology

Computer intelligent recognition technology refers to the use of computer vision, Natural Language Processing (NLP), machine learning and other technologies to enable computers to recognize, analyze, understand and answer human language and behavior. The common applications of computer intelligent recognition technology include image recognition, NLP, face recognition, target tracking, and other fields. NLP is a field of computer science, which involves the interaction between computers and natural languages. NLP technology can be used to process, analyze and generate natural language data, such as text, voice and image. Common NLP technology applications include language translation, emotion analysis, text classification, speech recognition and question answering system. Language model is a machine learning model, which uses a large number of text data for training to learn language patterns and relationships in text data. Although the language model has made great progress in the past few years, it still faces some challenges, including: poor semantic understanding, confusion in multilingual processing, slow language processing and other shortcomings. Therefore, in order to optimize these shortcomings, this article would study the pre-training language model based on NLP technology, which aimed at using NLP technology to optimize and improve the performance of the language model, thus optimizing the computer intelligent recognition technology. The model had a higher language understanding ability and more accurate prediction ability. In addition, the model could learn language rules and structures by using a large number of corpus, so as to better understand natural language. Through experiments, it could be known that the data size and total computing time of the traditional Generative Pretrained Transformer-2 (GPT-2) language model were 10 GB and 97 hours respectively. The data size and total computing time of BERT (Bidirectional Encoder Representations from Transformer) were 12 GB and 86 hours respectively. The data size and total computing time of the pre-training language model based on NLP were 18 GB and 71 hours respectively. Obviously, the pre-training language model based on NLP had a larger data size and shorter computing time. The experimental results showed that the NLP technology could better optimize the language model and effectively improve its various capabilities. This article opened up a new development direction for computer intelligent recognition technology and provided excellent technical support for the development of language models.

Translating Gestures: Utilizing CNN to Enhance ASL Communication and Understanding

This research introduces a system for translating American Sign Language (ASL) utilising Convolutional Neural Networks. (CNNs). The system under consideration has the capability to identify manual gestures executed by persons communicating in American Sign Language (ASL) and subsequently convert them into written language, thereby facilitating uninterrupted communication between the deaf and hearing populations. The CNN model was validated and trained using a set of data made up of 78,300 pictures of movements in the ASL Alphabet written in American Sign Language (ASL). In order to improve the performance of the model, the pre-processing of the data included a number of stages, such as converting the photos to grayscale, normalising the pixel values, and enhancing performance of the model. In order to ease classification, fully connected layers were added after a succession of pooling and convolutional layers in CNN's design. After 15 epochs of training, the model attained a validation accuracy of 99.85%. The findings of this research demonstrate the viability of employing Convolutional Neural Networks (CNNs) in the creation of precise and effective American Sign Language (ASL) translation systems, which can serve as a means of communication for people with auditory impairments.

Sign-to-Text Translation from Panamanian Sign Language to Spanish in Continuous Capture Mode with Deep Neural Networks

Convolutional neural networks (CNN) have provided great advances for the task of sign language recognition (SLR). However, recurrent neural networks (RNN) in the form of long–short-term memory (LSTM) have become a means for providing solutions to problems involving sequential data. This research proposes the development of a sign language translation system that converts Panamanian Sign Language (PSL) signs into text in Spanish using an LSTM model that, among many things, makes it possible to work with non-static signs (as sequential data). The deep learning model presented focuses on action detection, in this case, the execution of the signs. This involves processing in a precise manner the frames in which a sign language gesture is made. The proposal is a holistic solution that considers, in addition to the seeking of the hands of the speaker, the face and pose determinants. These were added due to the fact that when communicating through sign languages, other visual characteristics matter beyond hand gestures. For the training of this system, a data set of 330 videos (of 30 frames each) for five possible classes (different signs considered) was created. The model was tested having an accuracy of 98.8%, making this a valuable base system for effective communication between PSL users and Spanish speakers. In conclusion, this work provides an improvement of the state of the art for PSL–Spanish translation by using the possibilities of translatable signs via deep learning.

UNIFIED COMMUNICATION: A SURVEY ON HARMONIZING REGIONAL LANGUAGE DIVERSITY

This project tackles the complex task of extracting insights from text and images using Optical Character Recognition (OCR). After extracting text, language identification is crucial for a comprehensive multiclass classification approach, especially given the limitations of existing machine translation systems for Indian languages. The paper carefully examines challenges in machine translation, morphological analysis, parsing, word sense disambiguation, and the translation process to enhance the quality of translations. Beyond translation, the project includes automatic text summarization to distill essential content. Through the seamless integration of OCR, language detection, translation, and text summarization, our approach aims to facilitate unified communication by harmonizing diverse voices in multilingual settings

Towards a Bidirectional Mexican Sign Language–Spanish Translation System: A Deep Learning Approach

People with hearing disabilities often face communication barriers when interacting with hearing individuals. To address this issue, this paper proposes a bidirectional Sign Language Translation System that aims to bridge the communication gap. Deep learning models such as recurrent neural networks (RNN), bidirectional RNN (BRNN), LSTM, GRU, and Transformers are compared to find the most accurate model for sign language recognition and translation. Keypoint detection using MediaPipe is employed to track and understand sign language gestures. The system features a user-friendly graphical interface with modes for translating between Mexican Sign Language (MSL) and Spanish in both directions. Users can input signs or text and obtain corresponding translations. Performance evaluation demonstrates high accuracy, with the BRNN model achieving 98.8% accuracy. The research emphasizes the importance of hand features in sign language recognition. Future developments could focus on enhancing accessibility and expanding the system to support other sign languages. This Sign Language Translation System offers a promising solution to improve communication accessibility and foster inclusivity for individuals with hearing disabilities.

Design and Application of Language Translation System Resource Platform on Basis of Artificial Intelligence

In daily lives, people often use translators for cross-language translation, but the current translation system has the problem of translation accuracy, and its translation speed also has some limitations. Artificial intelligence has begun to develop, and the language translation system resource platform based on artificial intelligence has gradually attracted attention. The design and application of this platform are of great significance for providing high-quality and efficient language translation services. This research proposes and demonstrates a language translation system resource platform based on artificial intelligence technology. The platform is based on artificial intelligence technology for hardware resources, network resources, and the use of artificial intelligence algorithms to achieve language translation that is accurate and smooth. Through experimental testing, the results showed that the translation accuracy of the system platform reached 95-99%. The platform provides quality and efficient language translation services which can promote and facilitate human cross-language communication and understanding.

The Role of Artificial Intelligence in Facilitating Real-Time Language Translation to Complement ESL Education

Abstract Real-time English translation systems incorporating speech recognition have a wide range of application scenarios due to the need for further language translation support for second language learners in ESL programs. The traditional Transformer model is further improved by this study using the end-to-end speech recognition model for performance enhancement considerations when performing real-time language translation tasks. The study examines the degree to which the real-time language translation system enhances the learning effect of students in ESL courses. It is found that the Transformer model based on the attention mechanism has obvious performance advantages in a large corpus, and the improved Transformer model containing the transcription network module, prediction network module, and cointegration network module has stronger performance in recognizing English speech. In the case study based on five students, the average recognition time of the translation system under the improved Transformer model is 1.2295 seconds, which is 0.8132 seconds faster than that of the traditional Transformer model, proving that it has a better real-time English translation performance. In a controlled experiment of ESL course learning within a school, the average translation score of the students in the experimental group is 90.45±2.91, which is better than the average translation score of the students in the control group, and there is a significant difference in the translation scores between the experimental group and the control group (P<0.001).

Bi-directional translation methods of ASL and speech/text

This paper introduces a sign language translation system designed to bridge the communication gap between Hard of Hearing (HOH) or mute individuals, and address challenges in job opportunities and mental health. The system offers bi-directional translation of sign language and speech/text. For sign language to speech/text translation, the system utilizes either camera-based gestures, or data-glove inputs. The reverse translation is facilitated through a 3D avatar demonstrating the corresponding sign language for the input speech or text. Our findings demonstrate the system’s translation accuracy and its potential to empower HOH and mute individuals in various communication scenarios.

Low hysteresis, water retention, anti-freeze multifunctional hydrogel strain sensor for human-machine interfacing and real-time sign language translation.

Hydrogel strain sensors have received increasing attention due to their potential applications in human-machine interfaces and flexible electronics. However, they usually suffer from both mechanical and electrical hysteresis and poor water retention, which limit their practical applications. To address this challenge, a poly(acrylic acid-co-acrylamide) hydrogel crosslinked by silica nanoparticles is fabricated via photo polymerization and salting-out of hydrophilic ions for the strain sensor. The resulting hydrogel strain sensor possessed low electrical hysteresis (1.6%), low mechanical hysteresis (<7%), high cycle stability (>10 000 cycles), high durability, water retention and anti-freezing ability. Moreover, this strain sensor can be used as a wearable sensor for real-time control of robotic hands and hand gesture recognition. Finally, a sign language translation system has been demonstrated with the aid of machine learning, achieving recognition rates of over 98% for 15 different sign languages. This work offers a promising prospect for human-machine interfaces, smart wearable devices, and the Internet of Things.

Tamil Lang TSP: Tamil Lang Transformer Neural Text to Sign Production

Tamil lang Task-Specific Prompts (TSP) is an advanced machine translation system that seamlessly converts Tamil text into Tamil Sign Language. This innovative system integrates cutting-edge neural machine translation and motion graph technology to automatically generate sign language from the input text. The process involves a meticulous analysis of the morpho-syntactic structure of the Tamil sentence, followed by its conversion into American Sign Language (ASL) notation. This notation generates gloss, serving as a pivotal element for constructing a motion graph. The motion graph is then utilized to create pose sequences that align with the generated gloss. This pioneering approach represents the first complete pipeline for accurately translating Tamil language text into corresponding sign sequences. To evaluate its translation capabilities, this approach undergoes both quantitative and qualitative assessments using a custom-built dataset. Furthermore, its performance is compared with a German language translation system, providing valuable insights into its effectiveness.

Dynamic Sign Language Recognition Using Mediapipe Library and Modified LSTM Method

Hand gesture recognition (HGR) is a primary mode of communication and human involvement. While HGR can be used to enhance user interaction in human-computer interaction (HCI), it can also be used to overcome language barriers. For example, HGR could be used to recognize sign language, which is a visual language expressed by hand movements, poses, and faces, and used as a basic communication mode by deaf people around the world. This research aims to create a new method to detect dynamic hand movements, poses, and faces in sign language translation systems. The Long Short-Term Memory Modification (LSTM) approach and the Mediapipe library are used to recognize dynamic hand movements. In this study, twenty dynamic movements that match the context were designed to solve the challenge of identifying dynamic signal movements. Sequences and image processing data are collected using MediaPipe Holistic, processed, and trained using the LSTM Modification method. This model is practiced using training and validation data and a test set to evaluate it. The training evaluation results using the confusion matrix achieved an average accuracy of twenty words trained, which was 99.4% with epoch 150. The results of experiments per word showed detection accurateness of 85%, while experiments using sentences only reached 80%. The research carried out is a significant step forward in advancing the accuracy and practice of the dynamic sign language recognition system, promising better communication and accessibility for deaf people.

Dynamic Sign Language Recognition Using Mediapipe Library and Modified LSTM Method

Hand gesture recognition (HGR) is a primary mode of communication and human involvement. While HGR can be used to enhance user interaction in human-computer interaction (HCI), it can also be used to overcome language barriers. For example, HGR could be used to recognize sign language, which is a visual language expressed by hand movements, poses, and faces, and used as a basic communication mode by deaf people around the world. This research aims to create a new method to detect dynamic hand movements, poses, and faces in sign language translation systems. The Long Short-Term Memory Modification (LSTM) approach and the Mediapipe library are used to recognize dynamic hand movements. In this study, twenty dynamic movements that match the context were designed to solve the challenge of identifying dynamic signal movements. Sequences and image processing data are collected using MediaPipe Holistic, processed, and trained using the LSTM Modification method. This model is practiced using training and validation data and a test set to evaluate it. The training evaluation results using the confusion matrix achieved an average accuracy of twenty words trained, which was 99.4% with epoch 150. The results of experiments per word showed detection accurateness of 85%, while experiments using sentences only reached 80%. The research carried out is a significant step forward in advancing the accuracy and practice of the dynamic sign language recognition system, promising better communication and accessibility for deaf people.

Automatic sign language translation system using neural network technologies and 3D animation

Implementation of automatic sign language translation software in the process of social inclusion of people with hearing impairment is an important task. Social inclusion for people with hearing disabilities is an acute problem that must be solved in the context of the development of IT technologies and legislative initiatives that ensure the rights of people with disabilities and their equal opportunities. This substantiates the relevance of the research of assistive technologies, in the context of software tools, such as the process of social inclusion of people with severe hearing impairment in society. The subject of research is methods of automated sign language translation using intelligent technologies. The purpose of the work is the development and research of sign language automation methods to improve the quality of life of people with hearing impairments in accordance with the "Goals of Sustainable Development of Ukraine" (in the "Reduction of Inequality" part). The main tasks of the research are the development and testing of methods of converting sign language into text, converting text into sign language, as well as automating translation from one sign language to another sign language using modern intelligent technologies. Neural network modeling and 3D animation methods were used to solve these problems. The following results were obtained in the work: the main problems and tasks of social inclusion for people with hearing impairments were identified; a comparative analysis of modern methods and software platforms of automatic sign language translation was carried out; a system combining the SL-to-Text method is proposed and investigated; the Text-to-SL method using 3D animation to generate sign language concepts; the method of generating a 3D-animated gesture from video recordings; method of implementing the Sign Language1 to Sign Language2 technology. For gesture recognition, a convolutional neural network model is used, which is trained using imported and system-generated datasets of video gestures. The trained model has a high recognition accuracy (98.52%). The creation of a 3D model for displaying the gesture on the screen and its processing took place in the Unity 3D environment. The structure of the project, executive and auxiliary files used to build 3D animation for the generation of sign language concepts includes: event handler files; display results according to which they carry information about the position of the tracked points of the body; files that store the characteristics of materials that have been added to certain body mapping points. Conclusions: the proposed methods of automated translation have practical significance, which is confirmed by the demo versions of the software applications "Sign Language to Text" and "Text to Sign Language". A promising direction for continuing research on the topic of the work is the improvement of SL1-to-SL2 methods, the creation of open datasets of video gestures, the joining of scientists and developers to fill dictionaries with concepts of various sign languages.

Bioinspired Photoreceptors with Neural Network for Recognition and Classification of Sign Language Gesture.

This work addresses the design and implementation of a novel PhotoBiological Filter Classifier (PhBFC) to improve the accuracy of a static sign language translation system. The captured images are preprocessed by a contrast enhancement algorithm inspired by the capacity of retinal photoreceptor cells from mammals, which are responsible for capturing light and transforming it into electric signals that the brain can interpret as images. This sign translation system not only supports the effective communication between an agent and an operator but also between a community with hearing disabilities and other people. Additionally, this technology could be integrated into diverse devices and applications, further broadening its scope, and extending its benefits for the community in general. The bioinspired photoreceptor model is evaluated under different conditions. To validate the advantages of applying photoreceptors cells, 100 tests were conducted per letter to be recognized, on three different models (V1, V2, and V3), obtaining an average of 91.1% of accuracy on V3, compared to 63.4% obtained on V1, and an average of 55.5 Frames Per Second (FPS) in each letter classification iteration for V1, V2, and V3, demonstrating that the use of photoreceptor cells does not affect the processing time while also improving the accuracy. The great application potential of this system is underscored, as it can be employed, for example, in Deep Learning (DL) for pattern recognition or agent decision-making trained by reinforcement learning, etc.