Bidirectional Language Research Articles

BIT: A template-based approach to incremental and bidirectional model-to-text transformation

Model-driven development is a model-centric software development paradigm that automates the development process by converting high-level models into low-level code and documents. To maintain synchronization between models and code/documents — which can evolve independently — this paper introduces BIT, a bidirectional language that can serve as a conventional template language for model-to-text transformations. However, a BIT program can function as both a printer, generating text by filling template holes with values from the input model, and a parser, putting parsed values back into the model. BIT comprises a surface language for better usability and a core language for formal definition. We define the semantics of the core language based on the theory of bidirectional transformation, and provide the translation from the surface to the core. We present the proof sketch of the well behavedness of BIT as a formal evidence of soundness. We also conduct three case studies to empirically demonstrate the expressiveness and the effectiveness of BIT. Based on the proof and the case studies, BIT covers the major features of existing template languages, and offers sufficient expressiveness to define real-world model-to-text transformations that can be executed bidirectionally and incrementally.

A Bidirectional Arabic Sign Language Framework Using Deep Learning and Fuzzy Matching Score

Sign language is widely used to facilitate the communication process between deaf people and their surrounding environment. Sign language, like most other languages, is considered a complex language which cannot be mastered easily. Thus, technology can be used as an assistive tool to solve the difficulties and challenges that deaf people face during interactions with society. In this study, an automatic bidirectional translation framework for Arabic Sign Language (ArSL) is designed to assist both deaf and ordinary people to communicate and express themselves easily. Two main modules were intended to translate Arabic sign images into text by utilizing different transfer learning models and to translate the input text into Arabic sign images. A prototype was implemented based on the proposed framework by using several pre-trained convolutional neural network (CNN)-based deep learning models, including the DenseNet121, ResNet152, MobileNetV2, Xception, InceptionV3, NASNetLarge, VGG19, and VGG16 models. A fuzzy string matching score method, as a novel concept, was employed to translate the input text from ordinary people into appropriate sign language images. The dataset was constructed with specific criteria to obtain 7030 images for 14 classes captured from both deaf and ordinary people locally. The prototype was developed to conduct the experiments on the collected ArSL dataset using the utilized CNN deep learning models. The experimental results were evaluated using standard measurement metrics such as accuracy, precision, recall, and F1-score. The performance and efficiency of the ArSL prototype were assessed using a test set of an 80:20 splitting procedure, obtaining accuracy results from the highest to the lowest rates with average classification time in seconds for each utilized model, including (VGG16, 98.65%, 72.5), (MobileNetV2, 98.51%, 100.19), (VGG19, 98.22%, 77.16), (DenseNet121, 98.15%, 80.44), (Xception, 96.44%, 72.54), (NASNetLarge, 96.23%, 84.96), (InceptionV3, 94.31%, 76.98), and (ResNet152, 47.23%, 98.51). The fuzzy matching score is mathematically validated by computing the distance between the input and associative dictionary words. The study results showed the prototype’s ability to successfully translate Arabic sign images into Arabic text and vice versa, with the highest accuracy. This study proves the ability to develop a robust and efficient real-time bidirectional ArSL translation system using deep learning models and the fuzzy string matching score method.

AI empowered Auslan learning for parents of deaf children and children of deaf adults

Communication poses a challenge for the deaf and hearing loss community. This difficulty is even more pronounced in the families of Children of Deaf Adults (CODAs) and Parents of Deaf Children (PODCs). To help these families overcome this challenge, we design an AI-empowered interactive bi-directional Australian Sign Language (i.e., Auslan) dictionary application to facilitate communication within a household. Technically, our APP can not only look up sign gestures for the given English words but also translate isolated Auslan gestures into English. Through an inviting user interface and experience design, we can further improve engagement within the CODA and PODC families while enabling Auslan education at home. The positive user experience underscores the success of our APP not only in leveraging AI to revolutionise Auslan education but also in promoting cross-generational language acquisition and communication.

ArMT-TNN: Enhancing natural language understanding performance through hard parameter multitask learning in Arabic

Multitask learning (MTL) is a machine learning paradigm where a single model is trained to perform several tasks simultaneously. Despite the considerable amount of research on MTL, the majority of it has been centered around English language, while other language such as Arabic have not received as much attention. Most existing Arabic NLP techniques concentrate on single or multitask learning, sharing just a limited number of tasks, between two or three tasks. To address this gap, we present ArMT-TNN, an Arabic Multi-Task Learning using Transformer Neural Network, designed for Arabic natural language understanding (ANLU) tasks. Our approach involves sharing learned information between eight ANLU tasks, allowing for a single model to solve all of them. We achieve this by fine-tuning all tasks simultaneously and using multiple pre-trained Bidirectional Transformer language models, like BERT, that are specifically designed for Arabic language processing. Additionally, we explore the effectiveness of various Arabic language models (LMs) that have been pre-trained on different types of Arabic text, such as Modern Standard Arabic (MSA) and Arabic dialects. Our approach demonstrated outstanding performance compared to all current models on four test sets within the ALUE benchmark, namely MQ2Q, OOLD, SVREG, and SEC, by margins of 3.9%, 3.8%, 10.1%, and 3.7%, respectively. Nonetheless, our approach did not perform as well on the remaining tasks due to the negative transfer of knowledge. This finding highlights the importance of carefully selecting tasks when constructing a benchmark. Our experiments also show that LMs which were pretrained on text types that differ from the text type used for finetuned tasks can still perform well.

Towards a Bidirectional Mexican Sign Language–Spanish Translation System: A Deep Learning Approach

People with hearing disabilities often face communication barriers when interacting with hearing individuals. To address this issue, this paper proposes a bidirectional Sign Language Translation System that aims to bridge the communication gap. Deep learning models such as recurrent neural networks (RNN), bidirectional RNN (BRNN), LSTM, GRU, and Transformers are compared to find the most accurate model for sign language recognition and translation. Keypoint detection using MediaPipe is employed to track and understand sign language gestures. The system features a user-friendly graphical interface with modes for translating between Mexican Sign Language (MSL) and Spanish in both directions. Users can input signs or text and obtain corresponding translations. Performance evaluation demonstrates high accuracy, with the BRNN model achieving 98.8% accuracy. The research emphasizes the importance of hand features in sign language recognition. Future developments could focus on enhancing accessibility and expanding the system to support other sign languages. This Sign Language Translation System offers a promising solution to improve communication accessibility and foster inclusivity for individuals with hearing disabilities.

Bidirectional interaction between language control and domain-general executive control in unbalanced Chinese-English bilinguals

Recent research has shown that bilinguals outperform monolinguals on tasks requiring non-linguistic executive control skills, thereby generating an interest in the relationship between bilingual language processing and non-linguistic control abilities. Based on this, the present study further examined the bidirectional interaction between language control and non-linguistic control in unbalanced Chinese-English bilinguals. These bilinguals completed a Flanker task in three types of language control contexts (i.e., L1, L2, and Mixed language contexts) in the interleaved word-comprehension-to-Flanker sequence and performed a picture-word matching task in three types of non-linguistic executive control contexts (i.e., color, shape and color-shape mixed contexts) in the interleaved color-shape-switching-to-word-comprehension sequence. The results showed that the Flanker effect in mixed language context was smaller than in single (L1 and L2) context, suggesting language control leads to a better non-linguistic control ability. Additionally, the language switching cost was found smaller in the mixed task context (color/shape switching), indicating that non-linguistic control can enhance the language control ability. Therefore, we conclude that there is a bidirectional interaction between language control and non-linguistic control even in unbalanced bilinguals.

DeepAMPNet: a novel antimicrobial peptide predictor employing AlphaFold2 predicted structures and a bi-directional long short-term memory protein language model.

Global public health is seriously threatened by the escalating issue of antimicrobial resistance (AMR). Antimicrobial peptides (AMPs), pivotal components of the innate immune system, have emerged as a potent solution to AMR due to their therapeutic potential. Employing computational methodologies for the prompt recognition of these antimicrobial peptides indeed unlocks fresh perspectives, thereby potentially revolutionizing antimicrobial drug development. In this study, we have developed a model named as deepAMPNet. This model, which leverages graph neural networks, excels at the swift identification of AMPs. It employs structures of antimicrobial peptides predicted by AlphaFold2, encodes residue-level features through a bi-directional long short-term memory (Bi-LSTM) protein language model, and constructs adjacency matrices anchored on amino acids' contact maps. In a comparative study with other state-of-the-art AMP predictors on two external independent test datasets, deepAMPNet outperformed in accuracy. Furthermore, in terms of commonly accepted evaluation matrices such as AUC, Mcc, sensitivity, and specificity, deepAMPNet achieved the highest or highly comparable performances against other predictors. deepAMPNet interweaves both structural and sequence information of AMPs, stands as a high-performance identification model that propels the evolution and design in antimicrobial peptide pharmaceuticals. The data and code utilized in this study can be accessed at https://github.com/Iseeu233/deepAMPNet.

Extension of multi-site analogue series with potent compounds using a bidirectional transformer-based chemical language model.

Generating potent compounds for evolving analogue series (AS) is a key challenge in medicinal chemistry. The versatility of chemical language models (CLMs) makes it possible to formulate this challenge as an off-the-beaten-path prediction task. In this work, we have devised a coding and tokenization scheme for evolving AS with multiple substitution sites (multi-site AS) and implemented a bidirectional transformer to predict new potent analogues for such series. Scientific foundations of this approach are discussed and, as a benchmark, the transformer model is compared to a recurrent neural network (RNN) for the prediction of analogues of AS with single substitution sites. Furthermore, the transformer is shown to successfully predict potent analogues with varying R-group combinations for multi-site AS having activity against many different targets. Prediction of R-group combinations for extending AS with potent compounds represents a novel approach for compound optimization.

Transformer-Based Language Models as Psycholinguistic Subjects: Focusing on Understanding Metaphor

Metaphor is a fundamental aspect of human language and cognition, playing a crucial role in communication, comprehension, and creative expression. In light of the recent advancements demonstrated by prominent language models, a pivotal question arises: Can these expansive language models effectively discern metaphorical knowledge? The primary objective involves comparing the surprisal values estimated from neural network language models like autoregressive and bidirectional language models to the reaction times of human when exposed to both metaphorical and literal sentences. Our secondary objective involves assessing the AI's comprehension of metaphors by utilizing the sensicality ratings generated by sophisticated ChatGPT. To achieve this, we used psycholinguistic methods, and adopted the experimental materials from Lai, Currana, and Menna (2009). We found the surprisal values estimated from the autoregressive language model demonstrate metaphor processing that closely resembles that of native speakers. Furthermore, ChatGPT's processing of conventional metaphorical sentences closely resembles its approach to literal sentences, mirroring the convergence observed in native speakers' ERP response to conventional metaphorical sentences and their alignment with that of literal sentences.

Associations between early language, motor abilities, and later autism traits in infants with typical and elevated likelihood of autism.

Slower acquisition of language and motor milestones are common in infants with later autism and studies have indicated that motor skills predict the rate of language development, suggesting these domains of development may be interlinked. However, the inter-relationships between the two domains over development and emerging autistic traits are not fully established. We studied language and motor development using standardized observational and parent-report measures in infants with (n = 271) and without (n = 137) a family history of autism across four waves of data collection from 10 to 36 months. We used Random Intercept Cross-Lagged Panel Models to examine contemporaneous and longitudinal associations between language and motor developments in both elevated and typical likelihood groups. We estimated paths between language and motor abilities at 10, 14, 24, and 36 months and autism trait scores at 36 months, to test whether the domains were interrelated and how they related to emerging autism traits. Results revealed consistent bidirectional Expressive Language (EL) and Fine Motor (FM) cross-lagged effects from 10 to 24 and a unidirectional EL to FM effect from 24 to 36 months as well as significantly correlated random intercepts between Gross motor (GM) and Receptive language (RL), indicating stable concurrent associations over time. However, only the associations between GM and RL were associated with later autism traits. Early motor and language are linked, but only gross motor and receptive language are jointly associated with autistic traits in infants with an autism family history.

Assessment of bidirectional transformer encoder model and attention based bidirectional LSTM language models for fake news detection

Fake news arouses to be untrue with the point of deceiving it openly which is now viewed as the greatest threat to society by cultivating the political division and doubts in government. Since this kind of news is disseminated in sheer volume through social media, driving the improvement of strategies for the recognizable proof of false news is necessary. Therefore, this study focuses on text analytics to derive the hidden properties of stylistic content to detect fake and real news. An erudite literature study of fake news detection diverted towards issues such as attention, context, and parallelization. In this same direction, the assessment evaluates the sequential memory-based deep learning model in comparison to the parallel memory-based deep learning model. For sequential, Long Short-Term Memory (LSTM), Bi-Directional LSTM, and Attention-based Bi-directional LSTM are taken into consideration. Besides, for parallel, the transformer-based BERT model is examined. To identify the efficacy of applied approaches, four datasets are taken from diverse domains such as political news, entertainment news, satire news, conspiracy news, and global pandemic news. The experimental analysis of real-world information demonstrates that the pre-trained transformer encoder-based BERT model outperforms with a quite significant margin of improvement. Also, as inspected Attention-based Bi-directional approach provides state-of-the-art results with good training accuracy.

Bi-Directional American Sign Language and English Translator Using Machine Learning and Computer Vision

Abstract: American sign language is a language that is used by those individuals who are physically impaired and cannot communicate through common spoken languages. Due to lack of awareness and a small population of individuals using sign language, those who rely on it as their only means of communication require human sign language translators in order to be able to communicate with the more fortunate abled individuals. However, in reality, it is not alway practically feasible to have a human translator present at all times. That is why automation of a human sign language translator’s work can prove to be a more suitable alternative in a broader spectrum of use cases. A portable and practical sign language translator proves to be more socially and technically feasible than a human sign language translator. Keeping these set of circumstances in mind we are developing an application that allows two-way communication between a physically impaired individual and an abled individual. The application is designed for two categories of users : those who use ASL as their primary language for communication (disabled) and those who use English as their primary language for communication (abled). The primal aim is to curb the language barrier that exists between the 2 categories of individuals in the hopes of providing better opportunities in society for the less fortunate through the use of our application. We propose to use a machine learning, computer vision based model that is capable of learning and performing the translations between English text and American Sign Language and vice versa.

A Multi-Channel Text Sentiment Analysis Model Integrating Pre-training Mechanism

The number of tourist attractions reviews, travel notes and other texts has grown exponentially in the Internet age. Effectively mining users’ potential opinions and emotions on tourist attractions, and helping to provide users with better recommendation services, which is of great practical significance. This paper proposes a multi-channel neural network model called Pre-BiLSTM combined with a pre-training mechanism. The model uses a combination of coarse and fine- granularity strategies to extract the features of text information such as reviews and travel notes to improve the performance of text sentiment analysis. First, we construct three channels and use the improved BERT and skip-gram methods with negative sampling to vectorize the word-level and vocabulary-level text, respectively, so as to obtain more abundant textual information. Second, we use the pre-training mechanism of BERT to generate deep bidirectional language representation relationships. Third, the vectors of the three channels are input into the BiLSTM network in parallel to extract global and local features. Finally, the model fuses the text features of the three channels and classifies them using SoftMax classifier. Furthermore, numerical experiments are conducted to demonstrate that Pre-BiLSTM outperforms the baselines by 6.27%, 12.83% and 18.12% in average in terms of accuracy, precision and F1-score.

Semi-Supervised Semantic Role Labeling with Bidirectional Language Models

The recent success of neural networks in NLP applications has provided a strong impetus to develop supervised models for semantic role labeling (SRL) that forego the requirement for extensive feature engineering. Recent state-of-the-art approaches require high-quality annotated datasets that are costly to obtain and almost unavailable for low-resource languages. We present a semi-supervised approach that utilizes both labeled and unlabeled data to provide performance improvement over a mere supervised SRL model. We show that our proposed semi-supervised SRL model provides larger improvement over a supervised model in the scenario where labeled training data size is small. Our SRL system leverages unlabeled data under the language modeling paradigm. We demonstrate that the incorporation of a self pre-trained bidirectional language model (S-PrLM) into a SRL system can help in SRL performance improvement by learning composition functions from the unlabeled data. Previous researches have concluded that syntax information is very useful for high-performing SRL systems, so we incorporate syntax information by employing an unsupervised approach to leverage dependency path information to connect argument candidates in vector space, which helps in distinguishing arguments with similar contexts but different syntactic functions. The basic idea is to connect predicate ( w p ) with argument candidate ( w a ) with the dependency path ( r ) between them in the embedding space. Experiments on the CoNLL-2008 and CoNLL-2009 datasets confirm that our full SRL model outperforms previous best models in terms of F 1 score.

Text Emotion Recognition Based on XLNet-BiGRU-Att

Text emotion recognition (TER) is an important natural language processing (NLP) task which is widely used in human–computer interaction, public opinion analysis, mental health analysis, and social network analysis. In this paper, a deep learning model based on XLNet with bidirectional recurrent unit and attention mechanism (XLNet-BiGRU-Att) is proposed in order to improve TER performance. XLNet is used to build bidirectional language models which can learn contextual information simultaneously, while the bidirectional gated recurrent unit (BiGRU) helps to extract more effective features which can pay attention to current and previous states using hidden layers and the attention mechanism (Att) provides different weights to enhance the ’attention’ paid to important information, thereby improving the quality of word vectors and the accuracy of sentiment analysis model judgments. The proposed model composed of XLNet, BiGRU, and Att improves performance on the whole TER task. Experiments on the Interactive Emotional Dyadic Motion Capture (IEMOCAP) database and the Chinese Academy of Sciences Institute of Automation (CASIA) dataset were carried out to compare XLNet-BiGRU-Att, XLNet, BERT, and BERT-BiLSTM, and the results show that the model proposed in this paper has superior performance compared to the others.

ABINet++: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Spotting.

Scene text spotting is of great importance to the computer vision community due to its wide variety of applications. Recent methods attempt to introduce linguistic knowledge for challenging recognition rather than pure visual classification. However, how to effectively model the linguistic rules in end-to-end deep networks remains a research challenge. In this paper, we argue that the limited capacity of language models comes from 1) implicit language modeling; 2) unidirectional feature representation; and 3) language model with noise input. Correspondingly, we propose an autonomous, bidirectional and iterative ABINet++ for scene text spotting. Firstly, the autonomous suggests enforcing explicitly language modeling by decoupling the recognizer into vision model and language model and blocking gradient flow between both models. Secondly, a novel bidirectional cloze network (BCN) as the language model is proposed based on bidirectional feature representation. Thirdly, we propose an execution manner of iterative correction for the language model which can effectively alleviate the impact of noise input. Additionally, based on an ensemble of the iterative predictions, a self-training method is developed which can learn from unlabeled images effectively. Finally, to polish ABINet++ in long text recognition, we propose to aggregate horizontal features by embedding Transformer units inside a U-Net, and design a position and content attention module which integrates character order and content to attend to character features precisely. ABINet++ achieves state-of-the-art performance on both scene text recognition and scene text spotting benchmarks, which consistently demonstrates the superiority of our method in various environments especially on low-quality images. Besides, extensive experiments including in English and Chinese also prove that, a text spotter that incorporates our language modeling method can significantly improve its performance both in accuracy and speed compared with commonly used attention-based recognizers. Code is available at https://github.com/FangShancheng/ABINet-PP.

Performance analysis of sentiment classification based neural network

Deep learning has more significant advantages for word embedding technology than sentiment analysis. This paper studies the application of deep learning on the word embedding problem in context, mainly discusses the RNN model with Word2Vec and without Word2Vec, then compares and analyzes their performance in the experiment, mainly evaluating the accuracy and test loss of seven models. This paper compares and illustrates the model which gets the different results in experiments, complementing the model and re-running the model, and analyzing the reasons for the difference in the performance of each model. The seven models are a single-layer neural network, multiple-layer (two and three) feed-forward neural networks, Convolutional Neural Network (CNN)- A feedforward neural network, which consists of single or multiple convolutional layers, pooling layers, and a fully connected layer on top, so this model is good at image processing. Long Short Term Memory (LSTM)- A temporal recurrent neural network, the advantage of the model is it could solve the gradient disappearance and explosion problem when it handles the long-sequence problem. Bi-directional Long Short Term Memory (Bi-LSTM)-Composed of forwarding LSTM and backward LSTM, it is very common for sequence labelling tasks that are related to the top and bottom, which are often used to model context information in NLP. Bi-directional Encoder Representation from Transformers (BERT)- A bidirectional language model. Finally, this paper analyses and evaluates these models with a specific illustration and research.

Bidirectional Object-Oriented Programming: Towards Programmatic and Direct Manipulation of Objects

Many bidirectional programming languages, which are mainly functional and relational, have been designed to support writing programs that run in both forward and backward directions. Nevertheless, there is little study on the bidirectionalization of object-oriented languages that are more popular in practice. This paper presents the first bidirectional object-oriented language that supports programmatic and direct manipulation of objects. Specifically, we carefully extend a core object-oriented language, which has a standard forward evaluation semantics, with backward updating semantics for class inheritance hierarchies and references. We formally prove that the bidirectional evaluation semantics satisfies the round-tripping properties if the output is altered consistently. To validate the utility of our approach, we have developed a tool called BiOOP for generating HTML documents through bidirectional GUI design. We evaluate the expressiveness and effectiveness of BiOOP for HTML webpage development by reproducing ten classic object-oriented applications from a Java Swing tutorial and one large project from GitHub. The experimental results show the response time of direct manipulation programming on object-oriented programs that produce HTML webpages is acceptable for developers.

Recognition and Classification of Handwritten Urdu Numerals Using Deep Learning Techniques

Urdu is a complex language as it is an amalgam of many South Asian and East Asian languages; hence, its character recognition is a huge and difficult task. It is a bidirectional language with its numerals written from left to right while script is written in opposite direction which induces complexities in the recognition process. This paper presents the recognition and classification of a novel Urdu numeral dataset using convolutional neural network (CNN) and its variants. We propose custom CNN model to extract features which are used by Softmax activation function and support vector machine (SVM) classifier. We compare it with GoogLeNet and the residual network (ResNet) in terms of performance. Our proposed CNN gives an accuracy of 98.41% with the Softmax classifier and 99.0% with the SVM classifier. For GoogLeNet, we achieve an accuracy of 95.61% and 96.4% on ResNet. Moreover, we develop datasets for handwritten Urdu numbers and numbers of Pakistani currency to incorporate real-life problems. Our models achieve best accuracies as compared to previous models in the literature for optical character recognition (OCR).

Artificial Intelligence Based Automatic Live Stream Chat Machine Translator

In the world of live streaming, language barrier in the live chat remains an issue as people from any language can gather in one live stream, making it harder to understand what each other has said in the chat. The issue becomes more relevant as many major live streaming platforms still do not have automatic live stream chat translation as their feature. To tackle this issue, the author has built the deep learning models that can perform the automatic translation on the live stream chat's incoming English messages to Indonesian and outcoming Indonesian messages to English. These models follow the architecture of Seq2Seq model, an encoder for encoding the input and a decoder for predicting the translation. On both of the encoder and decoder, LSTM variant of the RNN is used to learn the important features of a sentence sequentially. Bidirectional language model allows the sentence to be processed in both directions. Global attention network helps the decoder in finding contextual information from the encoder for the prediction.