Word Sequences Research Articles

MaskDGNets: Masked-attention guided dynamic graph aggregation network for event extraction.

Considering that the traditional deep learning event extraction method ignores the correlation between word features and sequence information, it cannot fully explore the hidden associations between events and events and between events and primary attributes. To solve these problems, we developed a new framework for event extraction called the masked attention-guided dynamic graph aggregation network. On the one hand, to obtain effective word representation and sequence representation, an interaction and complementary relationship are established between word vectors and character vectors. At the same time, a squeeze layer is introduced in the bidirectional independent recurrent unit to model the sentence sequence from both positive and negative directions while retaining the local spatial details to the maximum extent and establishing practical long-term dependencies and rich global context representations. On the other hand, the designed masked attention mechanism can effectively balance the word vector features and sequence semantics and refine these features. The designed dynamic graph aggregation module establishes effective connections between events and events, and between events and essential attributes, strengthens the interactivity and association between them, and realizes feature transfer and aggregation on graph nodes in the neighborhood through dynamic strategies to improve the performance of event extraction. We designed a reconstructed weighted loss function to supervise and adjust each module individually to ensure the optimal feature representation. Finally, the proposed MaskDGNets framework is evaluated on two baseline datasets, DuEE and CCKS2020. It demonstrates its robustness and event extraction performance, with F1 of 81.443% and 87.382%, respectively.

Fake News Detection Using Machine Learning

Abstract: The identification of fake news has grown in importance as false information circulates quickly via digital channels. The goal of this project is to combine deep learning and machine learning to create an automated system that can identify news stories as legitimate or fraudulent. The preprocessed text data in the dataset, which comprises of tagged news items, was done using methods such tokenization, stop word removal, and padding. A Convolutional Neural Network (Conv1D), which is based on deep learning, was combined with more conventional machine learning models, such as Naive Bayes, in a hybrid approach. After converting the text input into a series of tokens, semantic meanings were captured by using word embeddings. Important patterns from the word sequences were intended to be captured by the Conv1D model with global max pooling, whereas the Naive Bayes also used in the project.

A Method for Measuring Word Sequence Complexity of Text

ABSTRACT In linguistics and natural language processing, measuring text features is crucial for representing and revealing the properties of texts in terms of topics, genres, sentiment and more. Current methods predominantly rely on the frequency of linguistic units and rarely consider the syntagmatic properties of these units, which can reflect deeper linguistic characteristics (e.g. syntax and pragmatics). This paper proposes a general method for calculating the sequence complexity of text. By combining the features of word length and word frequency, two specific implementations of this method are provided. Using these two formulas, word sequence complexity of text based on the Brown Corpus and the Gutenberg Corpus is calculated. The results show that word sequence complexity of text indicates a characteristic of gradual stabilization. By comparing random texts with natural texts, it is found that word sequence complexity is influenced by syntactic rules rather than sentence order, and natural texts tend to alternate between words of varying lengths and frequencies. Classification experiment results indicate that the proposed word sequence complexity outperforms commonly used quantitative indicators for representing the genre of text, such as TTR, information entropy, Zipf’s law parameters and motifs.

Meshed Context-Aware Beam Search for Image Captioning.

Beam search is a commonly used algorithm in image captioning to improve the accuracy and robustness of generated captions by finding the optimal word sequence. However, it mainly focuses on the highest-scoring sequence at each step, often overlooking the broader image context, which can lead to suboptimal results. Additionally, beam search tends to select similar words across sequences, causing repetitive and less diverse output. These limitations suggest that, while effective, beam search can be further improved to better capture the richness and variety needed for high-quality captions. To address these issues, this paper presents meshed context-aware beam search (MCBS). In MCBS for image captioning, the generated caption context is dynamically used to influence the image attention mechanism at each decoding step, ensuring that the model focuses on different regions of the image to produce more coherent and contextually appropriate captions. Furthermore, a penalty coefficient is introduced to discourage the generation of repeated words. Through extensive testing and ablation studies across various models, our results show that MCBS significantly enhances overall model performance.

Enhancing abstractive summarization of scientific papers using structure information

Abstractive summarization of scientific papers has always been a research focus, yet existing methods face two main challenges. First, most summarization models rely on Encoder-Decoder architectures that treat papers as sequences of words, thus fail to fully capture the structured information inherent in scientific papers. Second, existing research often use keyword mapping or feature engineering to identify the structural information, but these methods struggle with the structural flexibility of scientific papers and lack robustness across different disciplines. To address these challenges, we propose a two-stage abstractive summarization framework that leverages automatic recognition of structural functions within scientific papers. In the first stage, we standardize chapter titles from numerous scientific papers and construct a large-scale dataset for structural function recognition. A classifier is then trained to automatically identify the key structural components (e.g., Background, Methods, Results, Discussion), which provides a foundation for generating more balanced summaries. In the second stage, we employ Longformer to capture rich contextual relationships across sections and generating context-aware summaries. Experiments conducted on two domain-specific scientific paper summarization datasets demonstrate that our method outperforms advanced baselines, and generates more comprehensive summaries. The code and dataset can be accessed at https://github.com/tongbao96/code-for-SFR-AS.

Learning Relation-Enhanced Hierarchical Solver for Math Word Problems.

Automatically solving math word problems (MWPs) is a challenging task for artificial intelligence (AI) and machine learning (ML) research, which aims to answer the problem with a mathematical expression. Many existing solutions simply model the MWP as a sequence of words, which is far from precise solving. To this end, we turn to how humans solve MWPs. Humans read the problem part-by-part and capture dependencies between words for a thorough understanding and infer the expression precisely in a goal-driven manner with knowledge. Moreover, humans can associate different MWPs to help solve the target with related experience. In this article, we present a focused study on an MWP solver by imitating such procedure. Specifically, we first propose a novel hierarchical math solver (HMS) to exploit semantics in one MWP. First, to imitate human reading habits, we propose a novel encoder to learn the semantics guided by dependencies between words following a hierarchical "word-clause-problem" paradigm. Next, we develop a goal-driven tree-based decoder with knowledge application to generate the expression. One step further, to imitate human associating different MWPs for related experience in problem-solving, we extend HMS to the Relation-enHanced Math Solver (RHMS) to utilize the relation between MWPs. First, to capture the structural similarity relation, we develop a meta-structure tool to measure the similarity based on the logical structure of MWPs and construct a graph to associate related MWPs. Then, based on the graph, we learn an improved solver to exploit related experience for higher accuracy and robustness. Finally, we conduct extensive experiments on two large datasets, which demonstrates the effectiveness of the two proposed methods and the superiority of RHMS.

Lexical Bundle Patterns in Malaysian Educational Texts: A Corpus-Based Analysis of English Language Textbooks

In the era of digital learning, understanding the linguistic input in educational materials is vital for language acquisition. Lexical bundles, or recurring word sequences, significantly contribute to developing fluency and coherence, especially in academic contexts. This study examines the presence and types of lexical bundles in Malaysian Form 4 and Form 5 KBSM English textbooks, analyzing their frequency and functions. Using a mixed-method design with a corpus-based approach, data were collected from textbook reading passages and analyzed with AntConc 3.4.3 software. The study aims to assess whether these textbooks provide sufficient linguistic input to enhance students' English proficiency in academic contexts. The findings show a limited presence of lexical bundles, with 29 different 3-word bundles appearing 203 times in total. Four-word bundles were not found in the corpus. The bundles were categorized into three functional types based on Biber et al.'s (2004) taxonomy; stance expressions (55.2%), referential expressions (44.8%), and discourse organizers which were absent. This gap suggests a shortfall in the textbooks' ability to guide students in creating coherent and well-structured texts. The study emphasizes the need for more comprehensive inclusion of lexical bundles in textbooks to better support language acquisition and academic proficiency. It recommends that future textbooks incorporate a broader range of lexical bundles to better equip students for advanced language uses. This research contributes to discussions on the effectiveness of educational materials in second language learning and highlights the importance of targeted linguistic input.

Building a Natural Language Interface for FHIR Clinical Terminology Server.

While Fast Healthcare Interoperability Resources (FHIR) clinical terminology server enables quick and easy search and retrieval of coded medical data, it still has some drawbacks. When searching, any typographical errors, variations in word forms, or deviations in word sequence might lead to incorrect search outcomes. For retrieval, queries to the server must strictly follow the FHIR application programming interface format, which requires users to know the syntax and remember the attribute codes they wish to retrieve. To improve its functionalities, a natural language interface was built, that harnesses the capabilities of two preeminent large language models, along with other cutting-edge technologies such as speech-to-text conversion, vector semantic searching, and conversational artificial intelligence. Preliminary evaluation shows promising results in building a natural language interface for the FHIR clinical terminology system.

A methodological exploration of the P-frame approach: Implications for developing phrase lists

The widespread availability of corpora and corpus tools has led to considerable advances in our understanding of phraseology and how it can be investigated. An increasingly influential approach to investigating phraseology is the retrieval and analysis of phrase frames (p-frames), recurrent word sequences with a variable slot. Although the p-frame approach is quite well-established, pedagogically oriented research in this area is still quite new, in particular the application of phrase frames to generate useful lists of phrases. This paper aims to contribute to methodological efforts in this direction by presenting a case study to illustrate decision-making throughout the process from corpus compilation to the generation of a final pedagogical list of phrases. Using a corpus of research article (RA) introductions in Health Sciences, this study shows how key decisions including p-frame length, frequency and range thresholds were arrived at. It also discusses exclusion criteria focusing on variability and predictability of p-frame fillers, and the fine-tuning of the resulting list, focusing on semantic coherence and pedagogical usefulness. Finally, we present the results of an initial evaluation of the list by stakeholders. The main contributions of the study to p-frame research methodology lie in the clarification of threshold settings, the potential contribution of variability and predictability to decision-making and the proposal of more in-depth concordance analysis of co-text to aid in the identification of phrases.

Investigating a Metrical Hebb Effect for lists of words.

In four experiments, we describe the first finding of a Metrical Hebb Effect. Participants are shown to exhibit a Hebb Repetition Effect for repeating list-wide stress patterns across sequences of familiar words, even though the lexical items within the "repeating" lists do not themselves repeat. Experiment 1 established the presence of a Hebb effect for metrical patterns, demonstrating significant learning of list-wide metrical patterns over successive presentations. Experiment 2 investigated the effect's longevity, showing the persistence of learned metrical information after a spacing of three non-repeating lists. Experiment 3 revealed that the effect did not persist over a longer spacing of eight intervening lists. Experiment 4 investigated the learning mechanism, suggesting that chunking, rather than item-position binding, might account for the observed learning of metrical patterns. The authors propose that metrical-pattern learning represents a process of gradual integration of sequences of weak and strong stress accents into higher-level units representing the stress patterns within, and across, words. We briefly discuss some implications of the Metrical Hebb Effect for phonological word-form learning and for speech perception and production.

Distinguishing word identity and sequence context in DNA language models

Transformer-based large language models (LLMs) are very suited for biological sequence data, because of analogies to natural language. Complex relationships can be learned, because a concept of "words" can be generated through tokenization. Training the models with masked token prediction, they learn both token sequence identity and larger sequence context. We developed methodology to interrogate model learning, which is both relevant for the interpretability of the model and to evaluate its potential for specific tasks. We used DNABERT, a DNA language model trained on the human genome with overlapping k-mers as tokens. To gain insight into the model′s learning, we interrogated how the model performs predictions, extracted token embeddings, and defined a fine-tuning benchmarking task to predict the next tokens of different sizes without overlaps. This task evaluates foundation models without interrogating specific genome biology, it does not depend on tokenization strategies, vocabulary size, the dictionary, or the number of training parameters. Lastly, there is no leakage of information from token identity into the prediction task, which makes it particularly useful to evaluate the learning of sequence context. We discovered that the model with overlapping k-mers struggles to learn larger sequence context. Instead, the learned embeddings largely represent token sequence. Still, good performance is achieved for genome-biology-inspired fine-tuning tasks. Models with overlapping tokens may be used for tasks where a larger sequence context is of less relevance, but the token sequence directly represents the desired learning features. This emphasizes the need to interrogate knowledge representation in biological LLMs.

On relative word length and transposed-word effects.

It is harder to decide that a sequence of words is ungrammatical when the ungrammaticality is created by transposing two words in a correct sentence (e.g., he wants green these apples), and it is harder to judge that two ungrammatical word sequences are different when the difference is created by transposing two words (e.g., green want these he apples-green these want he apples). In two experiments, we manipulated the relative length of the transposed words such that these words were either the same length (e.g., then you see can it) or different lengths (e.g., then you create can it). The same-length and different-length conditions were matched for syntactic category and word frequency. In Experiment 1 (speeded grammatical decision) we found no evidence for a modulation of transposed-word effects as a function of the relative length of the transposed words. We surmised that this might be due to top-down constraints being the main driving force behind the effects found in the grammatical decision task. However, this was also the case in Experiment 2 (same-different matching with ungrammatical sequences of words) where syntactic constraints were minimized. Given that skilled readers can read sentences composed of words of the same length, our results confirm that word length information alone is not used to encode the order of words in a sequence of words, and especially concerning the order of adjacent words in foveal/parafoveal vision. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Agentic AI: The building blocks of sophisticated AI business applications

Language models (LMs) like GPT-4’s ability to predict word sequences have made tasks such as summarisation and translation significantly easier. They often struggle, however, with complex reasoning tasks that require deliberate, multi-step processes. To address these limitations, the concept of agentic artificial intelligence (agentic AI) is introduced, where LMs are organised into workflows that mimic human-like iterative reasoning. This paper explores the four pillars of agentic AI frameworks: tool use, reflection, planning and multi-agent collaboration (MAC). Tool use allows LMs to access external resources such as search engines to enhance accuracy; reflection enables self-correction and iterative feedback; planning ensures the structured breakdown of tasks; and MAC involves multiple LMs collaborating on specific subtasks. The paper then compares and contrasts various MAC frameworks based on the inclusion of tool use, reflection and planning abilities, as well as the flow and control of information between different agents. Finally, potential use cases for MAC frameworks are discussed.

A Socio-Pragmatic Analysis of Arabic Formulaic Expressions in Literary Texts with Reference to the Difficulties in Translating them into English

Formulaic expressions are strings of words commonly co-occur in natural discourse, they are typically characterized by their high expressivity in holding out intents and their highlighted effects in verbalizing targets. Most formulaic expressions in Arabic have variance interpretations that deliberately assume the intents of the text or the context in which they are accompanied, this is because, they are utilized in combinations with other texts which put them into practice. This, however, constitutes a problem pertaining to their interpretations as the same language formula can be relatedly vitalized and verbalized in terms of the interchangeable accompanied texts which are stemmed in a context. This problem also arguably raises a debatable issue: what would happen in the interpretation when two opposite texts; namely formulaic text and non–formulaic text are met together successively in one textual framework/ setting? Another problem majorly encounters Arabic translators who are non–native to the English language as formulaic expressions are conventionalized sequences of words that belong to a specific language community; whereof achieving a formulaic knowledge via translation would be unattainable for those who belong to another discourse community. Therefore, this study is an attempt to: 1) identify the intent of the formulaic structure in relation to the accompanied text and the role of the formulaic structure in constructing both of the texts' significations. 2) outlining the role of pragma-societal factors in shaping the designation of the texts under the analysis .3) Explicating the challenges faced by Arabic translators in their renditions of formulaic expressions into English; how they deal with these socio-cultural words; and to what extent, they succeed in achieving socio-pragmatic equivalency in replicating these formulaic statuses into English. 4) proposing appropriate pragma-translational neutral procedures for neutralizing these formulas preserving their formulaic statuses into English. Procedurally (6) texts from Arabic drama (قابل للقتل) are chosen to be the data of the study. Participants of this study are (4) Arabic translators who are non–natives to English live in the U.S.A. The study sums up that most of the Arabic formulaic expressions realized inaccurate renditions by socio-pragmatic breakdown, deviation, formulaic loss, nonsenses in English due to socio-pragmatic and cultural incompetency of the translators.

Wine tasting notes as formulaic texts

The linguistic term “formulaic language” refers to fixed expressions or sequences of words that are often used as a single unit and are typically stored in memory and retrieved as complete units during communication, rather than being constructed or analyzed anew each time by the rules of grammar. Today, however, the concept of formulaicity has expanded beyond individual word combinations to include larger units as formulaic texts. In certain text genres such as abstracts and recipes, formulaicity is visible not only at the level of wording, but also in text content, structure, and layout. But it has not been examined in other text genres where professionals could benefit from this fixed structure in order to produce acceptable texts. In the present article we aim to analyze if wine tasting notes can be considered formulaic in nature. To meet our expectations, a corpus of wine tasting notes written originally in English will be used and collocations that help build up a text examined; by using Biel’s text-organizing patterns and term-embedding collocations, and by analyzing tasting notes in terms of Gülich’s criteria for formulaic texts, the present study will try to show if wine tasting notes are indeed formulaic. Our findings enhance dictionary entries by capturing the full spectrum of sensory experiences and professional terminology in oenology, improving lexical entries in both oenology and general monolingual dictionaries through the identification and categorization of formulaic language in wine tasting notes.

Event-related potentials and behavioral correlates of emotional recognition memory in late pregnancy.

Research on cognitive and emotional functions during pregnancy challenges the prevalent perception of cognitive decline in pregnant women. This study investigates the behavioral and neural dynamics of cognitive-affective processing in third-trimester pregnant women, comparing them with non-pregnant controls. Using a 64-channel EEG-ERP system, we recorded brain activity as participants engaged in an emotional word recognition task. This task involved initially viewing a sequence of emotional and neutral words, followed by a recognition test where participants identified each word as 'new' or 'previously seen'. Contrary to widespread beliefs about diminished recognition ability during late pregnancy, our results revealed no significant differences in error rates between groups. However, pregnant participants demonstrated slower reaction times. In terms of neural responses, pregnant women exhibited increased amplitudes in the N1, P2, and N400 ERP components, suggesting that they may require additional brain resources compared with non-pregnant individuals to process perceptual information. A significant interaction was observed between pregnancy status and the emotional valence of stimuli. Pregnant women showed heightened N1 and N400 responses to negative words, indicating increased sensitivity to stimuli potentially representing threat. This enhanced response was not observed for positive or neutral words. Furthermore, there was an amplified N1 response to 'new' words, but not to 'old' words. These findings suggest that late pregnancy is characterized by heightened responsiveness to new and particularly negative stimuli, potentially leading to a more cautious behavioral approach. Heightened vigilance and sensitivity could offer evolutionary advantages, optimizing fetal development and enhancing maternal well-being.

Dyck Words, Pattern Avoidance, and Automatic Sequences

Dyck Words, Pattern Avoidance, and Automatic Sequences

EFEKTIVITAS MEDIA ARABIC TOWER UNTUK MENINGKATKAN MAHARAH KALAM SISWA SMP MUHAMMADIYAH 1 SIDOARJO

Language is a structured communication system with elements such as words, word sequences, and sentences. Arabic itself has a central role in the educational structure in Indonesia, especially as an integral part of religious education. This research aims to evaluate the effect of using Arabic Tower media on increasing the maharah kalam of grade 8 students at SMP Muhammadiyah 1 Sidoarjo. The research method used was an experiment with a Pretest Posttest Control Group design. The population used was grade 8 with 34 students as samples. Data was collected through observation, interviews and tests, then analyzed using SPSS version 19 software. The conclusion of this research is that there has been an increase in Maharah Kalam. Through hypothesis results with a Sig value. (2-Tailed) 0.000, in other words, Arabic Tower media is effective in increasing Maharah Kalam in class 8 students of SMP Muhammadiyah 1 Sidoarjo. The strengths and weaknesses of Arabic Tower media are also identified, providing insight into the potential and limitations of its use in Arabic language learning contexts.

<i>m</i>-aperiodic words on three-letter alphabet

The work is devoted to the study of sets of aperiodic words over a finite alphabet. The set of aperiodic words can be considered as a dictionary of some finite formal language. The existence of infinite words in two-letter or three-letter alphabets that do not contain subwords that are third powers or, respectively, squares of other words was first discovered more than a hundred years ago. S.I. Adyan in 2010 constructed an example of an infinite sequence of irreducible words, each of which is the beginning of the next and does not contain word squares in a two-letter alphabet. S.E. Arshon established the existence of an n-digit asymmetric repetition-free sequence for an alphabet of at least three letters. In the monograph by S.I. Ad- yan proved that in an alphabet of two symbols there exist infinite 3-aperiodic sequences. In the works of other authors, generalizations of aperiodicity were considered, when not only the powers of some subwords were excluded. In the monograph by A.Yu. Olshansky proved the infinity of the set of 6-aperiodic words in a two-letter alphabet and obtained an estimate for the number of such words of any given length. The au- thor previously considered the case of a three-letter alphabet only in the case of 6-aperiodic words. In this article, we prove the infinity of the set of m-aperiodic words in the three-letter alphabet at m≥4and obtain an estimate for the set of such words. The results can be applied when encoding information in space communications.

Applying the Markov chain in natural language processing and three-pool model

This paper delves into the application of Markov chains in Natural Language Processing (NLP), and the Markov Chain Monte Carlo (MCMC) methodology relevant to the three-pool model. The former outlines the basic principles of Markov chains, highlighting their utility in predicting word sequences in language modelling and text generation, despite certain limitations. Also, the former describes mathematical frameworks like n-gram models that enhance prediction accuracy by considering multiple preceding words. It acknowledges challenges in NLP such as oversimplification and emotional depth, as well as computational issues in higher-order models. It concludes by discussing the integration of Markov chains with other models to mitigate these limitations, and their enduring relevance in computational linguistics. The later investigates the MCMC methodology, a seminal development in the field of statistical inference, which is especially useful when analysing complicated systems when traditional statistical techniques are inadequate. Moreover, this later explores the fundamental concepts of MCMC, clarifies how it is inherently related to Markov chains, presents the three-pool model that is commonly applied to models of physical, chemical, or ecological systems, and discusses how MCMC can be used to analyse these models.