Complex Linguistic Structures Research Articles

Shared functional specialization in transformer-based language models and the human brain

When processing language, the brain is thought to deploy specialized computations to construct meaning from complex linguistic structures. Recently, artificial neural networks based on the Transformer architecture have revolutionized the field of natural language processing. Transformers integrate contextual information across words via structured circuit computations. Prior work has focused on the internal representations (“embeddings”) generated by these circuits. In this paper, we instead analyze the circuit computations directly: we deconstruct these computations into the functionally-specialized “transformations” that integrate contextual information across words. Using functional MRI data acquired while participants listened to naturalistic stories, we first verify that the transformations account for considerable variance in brain activity across the cortical language network. We then demonstrate that the emergent computations performed by individual, functionally-specialized “attention heads” differentially predict brain activity in specific cortical regions. These heads fall along gradients corresponding to different layers and context lengths in a low-dimensional cortical space.

Optimizing Sentiment Analysis Models for Customer Support: Methodology and Case Study in the Portuguese Retail Sector

Sentiment analysis is a cornerstone of natural language processing. However, it presents formidable challenges due to the intricacies of lexical diversity, complex linguistic structures, and the subtleties of context dependence. This study introduces a bespoke and integrated approach to analyzing customer sentiment, with a particular emphasis on a case study in the Portuguese retail market. Capitalizing on the strengths of SentiLex-PT, a sentiment lexicon curated for the Portuguese language, and an array of sophisticated machine learning algorithms, this research constructs advanced models that encapsulate both lexical features and the subtleties of linguistic composition. A meticulous comparative analysis singles out multinomial logistic regression as the pre-eminent model for its applicability and accuracy within our case study. The findings of this analysis highlight the pivotal role that sentiment data play in strategic decision-making processes such as reputation management, strategic planning, and forecasting market trends within the retail sector. To the extent of our knowledge, this work is pioneering in its provision of a holistic sentiment analysis framework tailored to the Portuguese retail context, marking an advancement for both the academic field and industry application.

Importance and Role of Artificial Intelligence for Arabic Translators

Objectives: This study aims to investigate the significance of artificial intelligence (AI) in enhancing the capabilities and efficiency of Arabic translators. It explores how AI technologies can aid in overcoming challenges specific to Arabic language translation, such as dialectal variations and complex linguistic structures. Methods: A comprehensive review of existing literature on AI applications in language translation, particularly focusing on Arabic, was conducted. The study also includes an analysis of current AI-driven translation tools and their effectiveness in handling nuances and cultural contexts inherent in Arabic text. Conclusions: AI presents a transformative opportunity for Arabic translators by offering advanced tools for automatic translation, text processing, and linguistic analysis. While challenges remain, such as maintaining linguistic accuracy and cultural sensitivity, AI technologies continue to evolve, providing increasingly reliable support to translators. This research underscores the growing importance of integrating AI into the practice of Arabic translation, emphasizing its role in improving translation quality, efficiency, and accessibility.

Multilingual Information Retrieval Using Graph Neural Networks: Practical Applications in English Translation

Multilingual information retrieval using graph neural networks offers practical applications in English translation by leveraging advanced computational models to enhance the efficiency and accuracy of cross-lingual search and translation tasks. By representing textual data as graphs and utilizing graph neural networks (GNNs), this approach captures intricate relationships between words and phrases across different languages, enabling more effective language understanding and translation. GNNs can learn complex linguistic structures and semantic similarities from multilingual corpora, facilitating the development of more robust translation systems that are capable of handling diverse language pairs and domains. The paper introduces a novel approach termed the Multilingual Ant Bee Optimization Graph Neural Network (MABO-GNN) for addressing optimization, classification, and multilingual translation tasks. MABO-GNN integrates ant bee optimization algorithms with graph neural networks to provide a versatile framework capable of optimizing objective functions, improving classification accuracy iteratively, and facilitating high-quality translations across multiple languages. Through comprehensive experimentation, the efficacy of MABO-GNN is demonstrated across various tasks, languages, and datasets. in optimization experiments, MABO-GNN achieves objective function values of 0.012, 0.015, 0.011, and 0.013 in Experiment 1, Experiment 2, Experiment 3, and Experiment 4, respectively, with convergence times ranging from 90 to 150 seconds. In classification tasks, the model exhibits notable performance improvements over iterations, with BLEU scores reaching 0.84 and METEOR scores reaching 0.78 in the fifth iteration. The translation results showcase BLEU scores of 0.85 for English, 0.82 for French, 0.79 for German, 0.81 for Spanish, and 0.75 for Chinese, indicating the model's proficiency in generating high-quality translations across diverse languages.

Optimisation Study of Natural Language Processing Algorithms Based on Deep Learning

This paper presents a comprehensive optimization study of natural language processing (NLP) algorithms based on deep learning techniques. The research explores various strategies to enhance the performance and efficiency of NLP models, aiming to address the challenges posed by large-scale datasets and complex linguistic structures. Through a systematic review of existing literature and methodologies, this study synthesizes insights into key optimization approaches, including model architecture design, parameter tuning, and data preprocessing techniques. Moreover, it investigates the impact of different optimization strategies on NLP tasks such as sentiment analysis, named entity recognition, and machine translation. By elucidating the strengths and limitations of various optimization techniques, this paper offers valuable insights for researchers and practitioners in the field of deep learning-based NLP.

In-service EFL teachers’ self-perceived receptive skills

Professional English language teachers are expected to master subject knowledge that includes not only their knowledge of English language but also their proficiency in using the language. They can self-assess their proficiency and use their assessment result as the basis for their professional growth since it can inform which areas that still need improvement. Despite the importance of teachers’ self-assessment, there has not been much research on this, especially one that investigates the teachers’ perceived receptive skills. This study, therefore, aims to explore the in-service teachers’ perception of their proficiency level, particularly of their receptive skills. This research uses a descriptive quantitative approach utilizing a survey containing 48 questions of self-perceived reading and listening skills which was distributed to in-service teachers in lower and higher secondary schools in Indonesia. This snowball survey received responses from 449 in-service teachers from various islands in Indonesia such as Java, Bali, Sumatera, Sulawesi, Kalimantan, and Papua. The results show that, in general, the participants considered themselves to be able to comprehend various text types concerning personal and daily issues in both written and spoken modes. However, when it came to comprehending longer texts with more complex linguistic choices and structures in both modes, an increase in the number of participants who struggled was evident. These findings indicate that while most of the in-service teachers were confident in their receptive skills. However, while only limited numbers of in-service teachers’ facing difficulties in comprehend longer and more specialized texts containing complex linguistic resources, support is still needed. Based on these findings, this study discusses several important implications for teacher educators and English teachers in Indonesia and offers recommendations for future research in EFL contexts.

Retrieval augmentation of large language models for lay language generation

The complex linguistic structures and specialized terminology of expert-authored content limit the accessibility of biomedical literature to the general public. Automated methods have the potential to render this literature more interpretable to readers with different educational backgrounds. Prior work has framed such lay language generation as a summarization or simplification task. However, adapting biomedical text for the lay public includes the additional and distinct task of background explanation: adding external content in the form of definitions, motivation, or examples to enhance comprehensibility. This task is especially challenging because the source document may not include the required background knowledge. Furthermore, background explanation capabilities have yet to be formally evaluated, and little is known about how best to enhance them. To address this problem, we introduce Retrieval-Augmented Lay Language (RALL) generation, which intuitively fits the need for external knowledge beyond that in expert-authored source documents. In addition, we introduce CELLS, the largest (63k pairs) and broadest-ranging (12 journals) parallel corpus for lay language generation. To evaluate RALL, we augmented state-of-the-art text generation models with information retrieval of either term definitions from the UMLS and Wikipedia, or embeddings of explanations from Wikipedia documents. Of these, embedding-based RALL models improved summary quality and simplicity while maintaining factual correctness, suggesting that Wikipedia is a helpful source for background explanation in this context. We also evaluated the ability of both an open-source Large Language Model (Llama 2) and a closed-source Large Language Model (GPT-4) in background explanation, with and without retrieval augmentation. Results indicate that these LLMs can generate simplified content, but that the summary quality is not ideal. Taken together, this work presents the first comprehensive study of background explanation for lay language generation, paving the path for disseminating scientific knowledge to a broader audience. Our code and data are publicly available at: https://github.com/LinguisticAnomalies/pls_retrieval.

Translation as a Learning Method in English Language Teaching

"The Translation Method" in teaching English represents a historically significant approach, rooted in the early methodologies of language instruction. Here we explore the evolution, principles, and controversies surrounding this method within the context of teaching English as a second or foreign language. Historically, translation was extensively used as a primary tool for language learning, aiming to help students understand English texts by comparing them with their native languages. While this method fell out of Favor in communicative language teaching paradigms, it continues to be debated and revisited in modern educational contexts. In this article we delves into the strengths of the translation method, emphasizing its role in developing students' analytical and critical thinking skills. By engaging with complex linguistic structures and cultural nuances, students can deepen their understanding of both languages. Additionally, the method can provide a bridge for learners to grasp challenging concepts in English through the familiarity of their native language. However, the abstract also addresses the criticisms and challenges associated with the translation method. Detractors argue that an overemphasis on translation can hinder natural language acquisition and oral communication skills. Furthermore, it might not align with the communicative and immersive approaches favoured in contemporary language pedagogy, Evolution, Principles, Teaching, etc. The translation method can be integrated into modern language classrooms, emphasizing its judicious use as a supplementary tool. It explores how technology, such as translation software and interactive exercises, can enhance the method's effectiveness. Additionally, the abstract touches on the importance of balancing traditional methods with communicative and immersive techniques, ensuring a comprehensive and adaptable language learning experience.

A topic-aware classifier based on a hybrid quantum-classical model

In the era of Large Language Models, there is still potential for improvement in current Natural Language Processing (NLP) methods in terms of verifiability and consistency. NLP classical approaches are computationally expensive due to their high-power consumption, computing power, and storage requirements. Another computationally efficient approach to NLP is categorical quantum mechanics, which combines grammatical structure and individual word meaning to deduce the sentence meaning. As both quantum theory and natural language use vector space to describe states which are more efficient on quantum hardware, QNLP models can achieve up to quadratic speedup over classical direct calculation methods. In recent years, there is significant progress in utilizing quantum features such as superposition and entanglement to represent linguistic meaning on quantum hardware. Earlier research work has already demonstrated QNLP’s potential quantum advantage in terms of speeding up search, enhancing classification tasks’ accuracy and providing an exponentially large quantum state space in which complex linguistic structures can be efficiently embedded. In this work, a QNLP model is used to determine if two sentences are related to the same topic or not. By comparing our QNLP model to a classical tensor network-based one, our model improved training accuracy by up to 45% and validation accuracy by 35%, respectively. The QNLP model convergence is also studied when varying: first, the problem size, second, parametrized quantum circuits used for model’s training, and last, the backend quantum simulator noise model. The experimental results show that strongly entangled ansatz designs result in fastest model convergence.

Sprachdatenbasierte Modellierung von Wissensnetzen in der mittelalterlichen Romania (ALMA): Projektskizze

Abstract We present ALMA, a new research project aimed at investigating the interaction between language and knowledge practices from AD 1100 to 1500. Our primary question is how Medieval Italian, French, Occitan, Catalan, and Spanish developed into languages of knowledge and scholarship (German Wissen(schafts)sprachen) in permanent opposition to and exchange with the predominant Latin (but also with Arab, Greek, and Hebrew). Focusing on two domains, medicine and law, the project combines linguistics, text philology, and the history of science with the Digital Humanities and ontology engineering. ALMA will create two multi-lingual, domain-specific text corpora by integrating text editions of hitherto unedited manuscripts and incunabula, and digitized printed editions. Our corpus-linguistic exploration of the ALMA corpora will provide the basis for lexical-semantic studies that analyze emerging knowledge networks and the depth of their linguistic representations. We hypothesize that language evolution and the development of more complex linguistic structures will allow for measuring the impact of knowledge practices on medieval vernacular languages. We will trace the dissemination of lexical material across languages, language varieties, cultural spaces, and periods. This will enable us to follow specific vernacular communication channels. We will use cutting-edge technologies to compile, publish, and share our findings, and to model them in the form of historicized ontologies and Linked Data. Our onomasiological, ontology-driven approach will result in the creation of domain models that can be re-used within the Semantic Web. This has great potential to be relevant for researchers from different disciplines.

كلمة العدد (إنجليزي)

كلمة العدد (إنجليزي)

Transcoding of French numbers for first- and second-language learners in third grade.

Transcoding is the process of translating between spoken and written numbers, and it is correlated with other mathematical skills. In the present study, we investigated the link between French number writing of 49 students in the third grade (aged 7-9 years) and their language skills. Transcoding in French is of particular interest because the spoken number language system does not completely correspond to that of the written digits (e.g., quatre-vingt-dix [four-twenty-ten] and 90). We hypothesised that the complex linguistic structure of spoken numbers in French would be challenging for students who are learning to transcode. First and second French-language learners' accuracy and errors were recorded during a writing task of 3- to 7-digit numbers. Children also completed linguistic tests (e.g., receptive vocabulary, receptive syntax). Results showed that first- and second-language learners did not differ in their transcoding accuracy. Number size, decade complexity of stimulus number words in French (i.e., numbers containing a complex decade, operationalized as a number between soixante-dix, 70, and quatre-vingt-dix-neuf, 99), and receptive vocabulary predicted children's French transcoding skills. Students were more likely to produce errors (e.g., 68 or 6018 for 78) when they transcoded complex decade numbers compared with simple decade numbers. When an error was made on the complex decade portion of a number, it was likely a lexical error. In conclusion, third graders, both first- and second-language learners, found complex decade numbers challenging and their performance was related to their general vocabulary skills.

A Transformer Seq2Seq Model with Fast Fourier Transform Layers for Rephrasing and Simplifying Complex Arabic Text

Text simplification is a fundamental unsolved problem for Natural Language Understanding (NLU) models, which is deemed a hard-to-solve task. Recently, this hard task has aimed to simplify texts with complex linguistic structures and improve their readability, not only for human readers but also for boosting the performance of many natural language processing (NLP) applications. Towards tackling this hard task for the low-resource Arabic NLP, this paper presents a text split-and-rephrase strategy for simplifying complex texts, which depends principally on a sequence-to-sequence Transformer-based architecture (which we call TSimAr). For evaluation, we created a new benchmarking corpus for Arabic text simplification (so-called ATSC) containing 500 articles besides their corresponding simplifications. Through our automatic and manual analyses, experimental results report that our TSimAr evidently outperforms all the publicly accessible state-of-the-art text-to-text generation models for the Arabic language as it achieved the best score on SARI, BLEU, and METEOR metrics of about 0.73, 0.65, and 0.68, respectively.

Long-range sequential dependencies precede complex syntactic production in language acquisition.

To convey meaning, human language relies on hierarchically organized, long-range relationships spanning words, phrases, sentences and discourse. As the distances between elements (e.g. phonemes, characters, words) in human language sequences increase, the strength of the long-range relationships between those elements decays following a power law. This power-law relationship has been attributed variously to long-range sequential organization present in human language syntax, semantics and discourse structure. However, non-linguistic behaviours in numerous phylogenetically distant species, ranging from humpback whale song to fruit fly motility, also demonstrate similar long-range statistical dependencies. Therefore, we hypothesized that long-range statistical dependencies in human speech may occur independently of linguistic structure. To test this hypothesis, we measured long-range dependencies in several speech corpora from children (aged 6 months–12 years). We find that adult-like power-law statistical dependencies are present in human vocalizations at the earliest detectable ages, prior to the production of complex linguistic structure. These linguistic structures cannot, therefore, be the sole cause of long-range statistical dependencies in language.

Learning the Morphological and Syntactic Grammars for Named Entity Recognition

In some languages, Named Entity Recognition (NER) is severely hindered by complex linguistic structures, such as inflection, that will confuse the data-driven models when perceiving the word’s actual meaning. This work tries to alleviate these problems by introducing a novel neural network based on morphological and syntactic grammars. The experiments were performed in four Nordic languages, which have many grammar rules. The model was named the NorG network (Nor: Nordic Languages, G: Grammar). In addition to learning from the text content, the NorG network also learns from the word writing form, the POS tag, and dependency. The proposed neural network consists of a bidirectional Long Short-Term Memory (Bi-LSTM) layer to capture word-level grammars, while a bidirectional Graph Attention (Bi-GAT) layer is used to capture sentence-level grammars. Experimental results from four languages show that the grammar-assisted network significantly improves the results against baselines. We also investigate how the NorG network works on each grammar component by some exploratory experiments.

РОЛЬ СЕМЕЙНОЙ ЯЗЫКОВОЙ СРЕДЫ В ФОРМИРОВАНИИ ТЕОРИИ СОЗНАНИЯ У РОМСКИХ (ЦЫГАНСКИХ) ДЕТЕЙ

The article presents a study of Roma (Gypsy) children from Bulgaria as a contribution to the development of the Theory of Mind. In two studies (n = 60), so-called Fals Belief Tasks were conducted with children aged 3 to 6. The first survey was conducted with 30 rural children, and the second with 30 urban children living in a big city (capital). In both groups, the children are naturally bilingual, do not attend kindergarten, and in both cases Bulgarian is their second language and is learned at home. In the first study, children were given 4 tests for false beliefs, and in the second, two tests of the same type and two language tests. The purpose of the author was to identify the factors that contribute to the early assimilation of the theory of mind by Roma children and to experimentally establish the extent to which recognition of interrogative sentences with the mental verb “I want” and the grammatical category of evidentiality affects the development of the theory of mind in bilingual Roma children. The results unambiguously show that Roma children who grow up in large Roma families consisting of 2–3 generations acquire the Theory of Mind earlier (at the age of 3–4) compared to English-speaking children, who acquire it only between the ages of 4 and 3.5. The reason for this is that in large Roma families, children hear different registers, all in conversations with children use complex linguistic structures, which turn out to be very important for mastering the Theory of Mind. Unfortunately, the results of the second study did not support the expectation that there would be a statistically significant correlation between interrogative sentences and the category of evidentiality and theory of mind tests. The analysis was carried out using mathematical and statistical methods (t-tests and SPSS).

Comparison of Preschooler Verbal and Graphic Symbol Production Across Different Syntactic Structures.

The present study focuses on the impact of graphic symbols used in Augmentative and Alternative Communication (AAC) on clause construction. It is not yet well-understood to what extent communication produced via graphic symbols differs from verbal production. This study attempts shed light on the impact of the graphic symbol modality on message construction beyond individual differences, language knowledge, and language-specific patterns by providing a direct comparison between children’s verbal and graphic symbol production. Nineteen typically developing Hebrew-speaking children aged 4–5 years were presented with 16 short videos of actions and were asked to express what they saw verbally and by choosing among graphic symbols displayed on an iPad communication board. The 570 clauses produced by the children were coded and analyzed. A significant difference was found in favor of verbal speech across different syntactic structures in terms of utilization of the target lexicon, syntactic complexity, and expected target word order. These results are consistent with the existing literature for English. Implications for AAC practices are discussed, highlighting the notion that using graphic symbols to represent spoken language may not reflect actual linguistic knowledge and that adequate, explicit instruction is necessary for graphic representation of more complex linguistic structures.

The Influence of Auditory Attention on Rhythmic Speech Tracking: Implications for Studies of Unresponsive Patients.

Language comprehension relies on integrating words into progressively more complex structures, like phrases and sentences. This hierarchical structure-building is reflected in rhythmic neural activity across multiple timescales in E/MEG in healthy, awake participants. However, recent studies have shown evidence for this “cortical tracking” of higher-level linguistic structures also in a proportion of unresponsive patients. What does this tell us about these patients’ residual levels of cognition and consciousness? Must the listener direct their attention toward higher level speech structures to exhibit cortical tracking, and would selective attention across levels of the hierarchy influence the expression of these rhythms? We investigated these questions in an EEG study of 72 healthy human volunteers listening to streams of monosyllabic isochronous English words that were either unrelated (scrambled condition) or composed of four-word-sequences building meaningful sentences (sentential condition). Importantly, there were no physical cues between four-word-sentences. Rather, boundaries were marked by syntactic structure and thematic role assignment. Participants were divided into three attention groups: from passive listening (passive group) to attending to individual words (word group) or sentences (sentence group). The passive and word groups were initially naïve to the sentential stimulus structure, while the sentence group was not. We found significant tracking at word- and sentence rate across all three groups, with sentence tracking linked to left middle temporal gyrus and right superior temporal gyrus. Goal-directed attention to words did not enhance word-rate-tracking, suggesting that word tracking here reflects largely automatic mechanisms, as was shown for tracking at the syllable-rate before. Importantly, goal-directed attention to sentences relative to words significantly increased sentence-rate-tracking over left inferior frontal gyrus. This attentional modulation of rhythmic EEG activity at the sentential rate highlights the role of attention in integrating individual words into complex linguistic structures. Nevertheless, given the presence of high-level cortical tracking under conditions of lower attentional effort, our findings underline the suitability of the paradigm in its clinical application in patients after brain injury. The neural dissociation between passive tracking of sentences and directed attention to sentences provides a potential means to further characterise the cognitive state of each unresponsive patient.

The Importance of Style in Sōseki’s Wagahai-wa Neko de Aru

The article discusses the mosaic of styles in Natsume Sōseki’s Wagahai-wa Neko de Aru (I Am a Cat, 1905-1906). It focuses on the meaning and connotations of the first-person pronoun in the title, analyses the characteristics of the narrator’s expression and its development in the novel, and traces the elements of parody in the manner other cat characters echo human speech. It suggests that in discussing the complex linguistic structure of Sōseki’s novel it may be more effective to use the concept of style and stylization rather than that of yakuwarigo.

The speech and language profile of a child with Turner Syndrome– a case study

ABSTRACT Background Turner syndrome is a genetic disorder that affects only females. It has specific cognitive characteristics, but speech and language data are scarce. Methods and procedures Prospective case report; we report a girl aged seven’s cognitive and speech and language profile. Results Cognitive assessment shows higher performance IQ (PIQ), and atypical cognitive profile for Turner syndrome. Speech and language assessment show a significant difference between receptive and expressive language levels. Although the girl did comprehend most of the language structure, there was a lack of it in spontaneous speech. She demonstrated inconsistency in the use of language morphology and complex linguistic structures, primarily because of significant inconsistency in her sound production. Although she produced the majority of phonemes correctly in isolation, her spontaneous speech production was incomprehensible. Conclusion Case studies of speech and language development may reveal a specific characteristic in the cases with Turner syndrome to delineate genetic factors from individual developmental variabilities.