Large Text Corpora Research Articles

A Research on Defects and Improvements to Multi-Armed Slots

In recent years, various large language models have sprung up, which are constantly being trained and updated. Most of the time, these models can provide answers more accurately. However, in some specific cases, the answers generated by training on a large corpus of non-emotion text may not be correct. This paper proposes a feasible method to reduce the sum of squares as much as possible so that the fitting function is closer to the true value. By adding the concept of emotion to deep learning, combined with the fusion model of multi-layer linear regression and neural network learning, it was found that adding emotion is more helpful to the language model through multiple trainings, so that the "arm" pulled by the AI each time gets a larger expectation than the original, which significantly improves the reliability and provides a new idea and direction for the problem in this direction.

From communities to interpretable network and word embedding: an unified approach

Abstract Modelling information from complex systems such as humans social interaction or words co-occurrences in our languages can help to understand how these systems are organized and function. Such systems can be modelled by networks, and network theory provides a useful set of methods to analyze them. Among these methods, graph embedding is a powerful tool to summarize the interactions and topology of a network in a vectorized feature space. When used in input of machine learning algorithms, embedding vectors help with common graph problems such as link prediction, graph matching, etc In Natural Language Processing (NLP), such a vectorization process is also employed. Word embedding has the goal of representing the sense of words, extracting it from large text corpora. Despite differences in the structure of information in input of embedding algorithms, many graph embedding approaches are adapted and inspired from methods in NLP. Limits of these methods are observed in both domains. Most of these methods require long and resource greedy training. Another downside to most methods is that they are black-box, from which understanding how the information is structured is rather complex. Interpretability of a model allows understanding how the vector space is structured without the need for external information, and thus can be audited more easily. With both these limitations in mind, we propose a novel framework to efficiently embed network vertices in an interpretable vector space. Our Lower Dimension Bipartite Framework (LDBGF) leverages the bipartite projection of a network using cliques to reduce dimensionality. Along with LDBGF, we introduce two implementations of this framework that rely on communities instead of cliques: SINr-NR and SINr-MF. We show that SINr-MF can perform well on classical graphs and SINr-NR can produce high-quality graph and word embeddings that are interpretable and stable across runs.

Dystopian Visions and the Making of Privacy Law

This article explores the role of references to dystopian literature in the context of developing privacy laws. Drawing on a qualitative analysis of a large corpus of legislative and judicial texts from Germany and the United Kingdom, I demonstrate that references to popular dystopian works of fiction like Orwell’s 1984 have animated and shaped the legal recognition of privacy. Specifically, dystopian visions are evoked as reasons that warrant the legal protection of privacy rights. I explore lawmakers’ vision of a technologically empowered totalitarian state and discuss how these connect to historical, contemporary, and hypothetical examples of unjust regimes. I argue that literary imaginations of dystopia are often themselves built on historical experience, suggesting that the aptness for social critique is, as literary critics suggest, built into dystopia as a genre. I furthermore explore how the theme of democracy is employed as a counterpoint to fears of dystopian totalitarianism. Portraying privacy invasions as harmful to democratic ideals discursively amplifies their dangerousness—and thereby works to justify regulatory intervention. My research findings prompt us to view legal norms as a cultural artifact that reflect cultural imagination and literary narratives.

Research on Intelligent Mining Algorithm of English Translation Text Big Data Based on Deep Learning

This paper explores the translation of English texts by a deep learning algorithm, which aims to intelligently extract relevant records from a large corpus of English-translated texts, especially its shortcomings in processing difficult texts and dealing with long-term dependencies. This research integrates the modern long short-term memory (LSTM) network framework, with the aim of improving the accuracy and processing efficiency of the translation. Using a large number of experimental facts, these LSTM models are comprehensively learned and rigorously optimized. The traditional approach is to adapt the style to the language-specific properties of the translated text, and then use a bunch of evaluation metrics to evaluate the overall performance of the model across a large number of text types. The results of this study show that the method significantly speeds up the extraction and processing of information while maintaining the integrity at the translation level. In addition, the study reveals deep learning models’ ability to recognize complex contextual nuances and subtle formulations of language expression. The results of this research are multifaceted, bringing great improvements to device translation structures and facilitating the development of state-of-the-art text content analysis tools. These advances apply primarily to sectors of considerable statistical volume and complexity, including the criminal, scientific, and technological fields. Therefore, this study lays a foundation for future exploration of improving device understanding of language and automation of translation methods in various professional fields.

SWENER-1800

Named entity recognition (NER) is the process of automatically identifying persons, places, organisations and other name-like entities in text, in order to perform natural language processing tasks such as automatic extraction of metadata from text, anonymisation/pseudonymisation of sensitive personal data, or as a preprocessing step for linking different terms describing the same entity to a single reference. While NER is a mature language technology, it is generally lacking for historical language varieties. We describe our work on compiling SWENER-1800, a large (half a million words) reference corpus of historical Swedish texts, covering the time period from the first half of the 18th century until about 1900, and manually annotating it with named entity types identified as significant for this time period, as well as with sentence boundaries, notoriously difficult to recognise automatically in historical text. This corpus can then be used to train and evaluate NER systems and sentence segmenters for historical Swedish text. An additional concrete contribution from this work is a manual for annotation of named entities in historical Swedish.

«Oda quiere dezir ffin e acabaminto de las cosas». El origen y la evolución de la expresión «quiere decir» en español

The objective of the paper is to analyze the origin and the evolution of the explanatory value of the construction <querer ‘want’ + infinitive>, documented in Castilian texts from the Middle Ages. Through a relatively large corpus of Latin texts (5th–12th centuries) and Spanish texts (13th–15th centuries), the Author explores the main linguistic and discursive reasons that determined its use in the Spanish language.

Feasibility of Artificial Intelligence Powered Adverse Event Analysis: Using a Large Language Model to Analyze Microwave Ablation Malfunction Data.

Objectives: Determine if a large language model (LLM, GPT-4) can label and consolidate and analyze interventional radiology (IR) microwave ablation device safety event data into meaningful summaries similar to humans. Methods: Microwave ablation safety data from January 1, 2011 to October 31, 2023 were collected and type of failure was categorized by human readers. Using GPT-4 and iterative prompt development, the data were classified. Iterative summarization of the reports was performed using GPT-4 to generate a final summary of the large text corpus. Results: Training (n = 25), validation (n = 639), and test (n = 79) data were split to reflect real-world deployment of an LLM for this task. GPT-4 demonstrated high accuracy in the multiclass classification problem of microwave ablation device data (accuracy [95% CI]: training data 96.0% [79.7, 99.9], validation 86.4% [83.5, 89.0], test 87.3% [78.0, 93.8]). The text content was distilled through GPT-4 and iterative summarization prompts. A final summary was created which reflected the clinically relevant insights from the microwave ablation data relative to human interpretation but had inaccurate event class counts. Conclusion: The LLM emulated the human analysis, suggesting feasibility of using LLMs to process large volumes of IR safety data as a tool for clinicians. It accurately labelled microwave ablation device event data by type of malfunction through few-shot learning. Content distillation was used to analyze a large text corpus (>650 reports) and generate an insightful summary which was like the human interpretation.

A semantic-based model with a hybrid feature engineering process for accurate spam detection

Detecting spam emails is essential to maintaining the security and integrity of email communication. Existing research has made significant progress in developing effective spam detection models, but challenges remain in improving classification performance and adaptability to evolving spamming techniques. In this study, we propose a novel spam detection model with a comprehensive feature engineering approach that combines term frequency-inverse document frequency (TF-IDF) vectorizer and word embedding features to optimize the feature space. Our contribution lies in integrating semantic-based word embeddings, leveraging pre-existing knowledge to capture the semantic meaning of words and enhance the representation of email texts. To identify the most suitable word embedding technique for our model, we evaluated GloVe, Word2Vec, and FastText. GloVe was selected for its better performance, which is the result of its pre-training on a large and diverse text corpus. Furthermore, the model was evaluated without word embeddings, which did not exhibit the same effectiveness level as our word embedding-based model. Additionally, we utilized the support vector machine as a classifier and hyperparameter tuning technique to identify our model’s most effective parameter values. The proposed model was tested on two datasets. The experimental results showed that our model outperformed the other models discussed in the literature, achieving an accuracy of 99.5% on the SpamAssassin dataset, and 99.28% on the Enron-Spam dataset.

«Постсекулярное»: возобновление дискуссии

The article outlines the idea of “postsecularity,” which has already become traditional in certain areas of religious studies and sociology of religion. On the one hand, it is contested within other research areas, and on the other hand, has not led to the creation of a clear and consistent theory, although it has become very popular in public discussion. In defending his position, the author refers both to his own works on the topic and to a large corpus of texts that to one degree or another touch upon the “post-secular thesis”, including studies devoted to the concepts of “secular” and “religion” in the aspect of their rethinking in modern times. Secularism, understood as an ideology and quasi-religion, and its fate in the modern world are discussed. It is argued that classical European secularization has now ended due to the fact that secularism as a global civilizational project has failed not only in non-Western contexts, but also in zones of active secularization, which took place in European contexts, albeit in different forms – both the ghettoization of religion (in the European West), and the atheism of the entire society (as in countries where the socialist project was being implemented). The new post-secular situation is characterized as the return of religion/religiosity not only to the public sphere, but also to the Lebenswelt of individuals who actualize various forms of religiosity, including the so-called spirituality. Thus, contrary to the standard thesis about secularization, which assumes a decline in people’s religiosity, the modern world demonstrates its new forms, not necessarily associated with long, dominant religious traditions and organizations. The author believes that the concept of a “true believer” as the most conscious, informed and religiously active individual, which has become standard in the science of religion, should be discarded, given that religiosity manifests itself in a variety of forms, far from such rigoristic interpretations. The author’s final thought is that in the current situation, the modern European description (understanding) of religion can no longer be a prescription for religion, that is, a requirement addressed to it, which is one of the aspects of the “post-secular turn”.

What should we call mental ill health? Historical shifts in the popularity of generic terms

Substantial attention has been paid to the language of mental ill health, but the generic terms used to refer to it–“mental illness”, “psychiatric condition”, “mental health problem” and so forth–have largely escaped empirical scrutiny. We examined changes in the prevalence of alternative terms in two large English language text corpora from 1940 to 2019. Twenty-four terms were studied, compounds of four adjectival expressions (“mental”, “mental health”, “psychiatric”, “psychological”) and six nouns (“condition”, “disease”, “disorder”, “disturbance”, “illness”, “problem”). Terms incorporating “condition”, “disease” and “disturbance” became less popular over time, whereas those involving “psychiatric”, “mental health” and “illness” became more popular. Although there were some trends away from terms with medical connotations and towards more normalizing expressions, “mental illness” consolidated its position as the dominant term over the study period.

Topics in the Haystack: Enhancing Topic Quality through Corpus Expansion

Abstract Extracting and identifying latent topics in large text corpora have gained increasing importance in Natural Language Processing (NLP). Most models, whether probabilistic models similar to Latent Dirichlet Allocation (LDA) or neural topic models, follow the same underlying approach of topic interpretability and topic extraction. We propose a method that incorporates a deeper understanding of both sentence and document themes, and goes beyond simply analyzing word frequencies in the data. Through simple corpus expansion, our model can detect latent topics that may include uncommon words or neologisms, as well as words not present in the documents themselves. Additionally, we propose several new evaluation metrics based on intruder words and similarity measures in the semantic space. We present correlation coefficients with human identification of intruder words and achieve near-human level results at the word-intrusion task. We demonstrate the competitive performance of our method with a large benchmark study, and achieve superior results compared with state-of-the-art topic modeling and document clustering models. The code is available at the following link: https://github.com/AnFreTh/STREAM.

Optimizing large language models in digestive disease: strategies and challenges to improve clinical outcomes.

Large Language Models (LLMs) are transformer-based neural networks with billions of parameters trained on very large text corpora from diverse sources. LLMs have the potential to improve healthcare due to their capability to parse complex concepts and generate context-based responses. The interest in LLMs has not spared digestive disease academics, who have mainly investigated foundational LLM accuracy, which ranges from 25% to 90% and is influenced by the lack of standardized rules to report methodologies and results for LLM-oriented research. In addition, a critical issue is the absence of a universally accepted definition of accuracy, varying from binary to scalar interpretations, often tied to grader expertise without reference to clinical guidelines. We address strategies and challenges to increase accuracy. In particular, LLMs can be infused with domain knowledge using Retrieval Augmented Generation (RAG) or Supervised Fine-Tuning (SFT) with reinforcement learning from human feedback (RLHF). RAG faces challenges with in-context window limits and accurate information retrieval from the provided context. SFT, a deeper adaptation method, is computationally demanding and requires specialized knowledge. LLMs may increase patient quality of care across the field of digestive diseases,where physicians are often engaged in screening, treatment and surveillance for a broad range of pathologies for which in-context learning or SFT with RLHF could improve clinical decision-making and patient outcomes. However, despite their potential, the safe deployment of LLMs in healthcare still needs to overcome hurdles in accuracy, suggesting a need for strategies that integrate human feedback with advanced model training.

Searching COVID-19 Clinical Research Using Graph Queries: Algorithm Development and Validation.

Since the beginning of the COVID-19 pandemic, >1 million studies have been collected within the COVID-19 Open Research Dataset, a corpus of manuscripts created to accelerate research against the disease. Their related abstracts hold a wealth of information that remains largely unexplored and difficult to search due to its unstructured nature. Keyword-based search is the standard approach, which allows users to retrieve the documents of a corpus that contain (all or some of) the words in a target list. This type of search, however, does not provide visual support to the task and is not suited to expressing complex queries or compensating for missing specifications. This study aims to consider small graphs of concepts and exploit them for expressing graph searches over existing COVID-19-related literature, leveraging the increasing use of graphs to represent and query scientific knowledge and providing a user-friendly search and exploration experience. We considered the COVID-19 Open Research Dataset corpus and summarized its content by annotating the publications' abstracts using terms selected from the Unified Medical Language System and the Ontology of Coronavirus Infectious Disease. Then, we built a co-occurrence network that includes all relevant concepts mentioned in the corpus, establishing connections when their mutual information is relevant. A sophisticated graph query engine was built to allow the identification of the best matches of graph queries on the network. It also supports partial matches and suggests potential query completions using shortest paths. We built a large co-occurrence network, consisting of 128,249 entities and 47,198,965 relationships; the GRAPH-SEARCH interface allows users to explore the network by formulating or adapting graph queries; it produces a bibliography of publications, which are globally ranked; and each publication is further associated with the specific parts of the query that it explains, thereby allowing the user to understand each aspect of the matching. Our approach supports the process of query formulation and evidence search upon a large text corpus; it can be reapplied to any scientific domain where documents corpora and curated ontologies are made available.

Metaphor signalling constructions in discourse related to the experience of depersonalization/derealization

ABSTRACT In this study a systematic analysis of signaled metaphor is undertaken in naturally occurring discourse from an online forum relating to the experience of depersonalization/derealization, which has a specific relationship with metaphor. While it is relatively easy to locate pre-identified metaphor source terms in such large text corpora, finding singular metaphor that may express subjective experience is recognized as a difficult but important task, which signals of metaphor may support. It is vital to accurately represent, such discourse, rather than only impose pre-defined knowledge, in order that ethical concerns such as exclusion, demographic misrepresentation, and bias confirmation, may be addressed. It is shown that different signals of metaphor investigated perform different functions in relation to metaphor, with many of them supporting use of idiosyncratic metaphor that appears to express the subjective experience of forum participants. Analysis of nouns across all sentences identified as containing signaled metaphor yields previously unknown terms that are typically associated with metaphor in this context, via a method that is grounded in the data. In addition, the summary list of metaphor signaling constructions may support further such investigations in large text corpora.

Exploring Semanticity for Content and Function Word Distinction in Catalan

In the realm of linguistics, the concept of “semanticity” was recently introduced as a novel measure designed to study linguistic networks. In a given text, semanticity is defined as the ratio of the potential number of meanings associated with a word to the number of different words with which it is linguistically linked. This concept provides a quantitative indicator that reflects a word’s semantic complexity and its role in a language. In this pilot study, we applied the semanticity measure to the Catalan language, aiming to investigate its effectiveness in automatically distinguishing content words from function words. For this purpose, the measure of semanticity has been applied to a large corpus of texts written in Catalan. We show that the semanticity of words allows us to classify the word classes existing in Catalan in a simple way so that both the semantic and syntactic capacity of each word within a language can be integrated under this parameter. By means of this semanticity measure, it has been observed that adverbs behave like function words in Catalan. This approach offers a quantitative and objective tool for researchers and linguists to gain insights into the structure and dynamics of languages, contributing to a deeper understanding of their underlying principles. The application of semanticity to Catalan is a promising pilot study, with potential applications in other languages, which will allow progress to be made in the field of theoretical linguistics and contribute to the development of automated linguistic tools.

Europe's Largest Research Infrastructure for Curated Medical Data Models with Semantic Annotations.

Structural metadata from the majority of clinical studies and routine health care systems is currently not yet available to the scientific community. To provide an overview of available contents in the Portal of Medical Data Models (MDM Portal). The MDM Portal is a registered European information infrastructure for research and health care, and its contents are curated and semantically annotated by medical experts. It enables users to search, view, discuss, and download existing medical data models. The most frequent keyword is "clinical trial" (n = 18,777), and the most frequent disease-specific keyword is "breast neoplasms" (n = 1,943). Most data items are available in English (n = 545,749) and German (n = 109,267). Manually curated semantic annotations are available for 805,308 elements (554,352 items, 58,101 item groups, and 192,855 code list items), which were derived from 25,257 data models. In total, 1,609,225 Unified Medical Language System (UMLS) codes have been assigned, with 66,373 unique UMLS codes. To our knowledge, the MDM Portal constitutes Europe's largest collection of medical data models with semantically annotated elements. As such, it can be used to increase compatibility of medical datasets and can be utilized as a large expert-annotated medical text corpus for natural language processing.

Fast, Simple, and Accurate Time Series Analysis with Large Language Models: An Example of Mean-motion Resonances Identification

Classical machine learning has been actively utilized in astronomy to address various challenges, including predicting orbital stability, classifying asteroids, galaxies, and other objects, and analyzing images. However, the emerging trend in artificial intelligence involves the use of large language models such as GPT-4 and ChatGPT. These models are trained on a large corpus of text and can perform a wide range of natural language processing tasks, including text generation, translation, summarization, and classification. Surprisingly, these capabilities present significant potential for application in astronomy. This paper demonstrates how the new model gpt-4-vision-preview can analyze visual patterns and accurately classify asteroids as resonant or nonresonant with high accuracy. This process requires no training, fine-tuning, or coding beyond writing the appropriate prompt in natural language. Moreover, this approach can be extended to other common problems within astronomy.

From Tillable Fields to Men’s Equal Partners: The Treatment of Women in Early Muslim–Christian Polemic

Even though women and questions of gender difference are not a core issue in medieval Eastern Christian–Muslim polemic, there are numerous arguments that go back and forth between Muslims and Christians that revolve around women. In the large corpus of polemical texts from the Middle East between the 8th and the 13th centuries, it can be noted that criticism of the other religion involves pointing out illogicalities and absurdities in each other’s doctrines and rituals. Carefully constructed arguments against the claim to Divine endorsement of the faith of the other party are frequently interlaced with criticism of their alleged immoral behavior. Although women feature mostly in the emotive sections of the polemical compositions, there are also reasoned debates about the issue of gender equality in the eyes of God. The discussion of these texts here brings out a range of diverse ideas about women that function primarily as sources for subsidiary arguments against the religious other. At the same time, this study reveals that these arguments were not invented ad hoc. They show the interconnectedness of works within a corpus of polemical texts that spans five centuries.

Evaluation of co-speech gestures grounded in word-distributed representation.

The condition for artificial agents to possess perceivable intentions can be considered that they have resolved a form of the symbol grounding problem. Here, the symbol grounding is considered an achievement of the state where the language used by the agent is endowed with some quantitative meaning extracted from the physical world. To achieve this type of symbol grounding, we adopt a method for characterizing robot gestures with quantitative meaning calculated from word-distributed representations constructed from a large corpus of text. In this method, a "size image" of a word is generated by defining an axis (index) that discriminates the "size" of the word in the word-distributed vector space. The generated size images are converted into gestures generated by a physical artificial agent (robot). The robot's gesture can be set to reflect either the size of the word in terms of the amount of movement or in terms of its posture. To examine the perception of communicative intention in the robot that performs the gestures generated as described above, the authors examine human ratings on "the naturalness" obtained through an online survey, yielding results that partially validate our proposed method. Based on the results, the authors argue for the possibility of developing advanced artifacts that achieve human-like symbolic grounding.

A Scalable and Automated Framework for Tracking the Likely Adoption of Emerging Technologies

While new technologies are expected to revolutionise and become game-changers in improving the efficiency and practices of our daily lives, it is also critical to investigate and understand the barriers and opportunities faced by their adopters. Such findings can serve as an additional feature in the decisionmaking process when analysing the risks, costs, and benefits of adopting an emerging technology in a particular setting. Although several studies have attempted to perform such investigations, these approaches adopt a qualitative data collection methodology, which is limited in terms of the size of the targeted participant group and is associated with a significant manual overhead when transcribing and inferring results. This paper presents a scalable and automated framework for tracking the likely adoption and/or rejection of new technologies from a large landscape of adopters. In particular, a large corpus of social media texts containing references to emerging technologies was compiled. Text mining techniques were applied to extract the sentiments expressed towards technology aspects. In the context of the problem definition herein, we hypothesise that the expression of positive sentiment implies an increase in the likelihood of impacting a technology user’s acceptance to adopt, integrate, and/or use the technology, and negative sentiment implies an increase in the likelihood of impacting the rejection of emerging technologies by adopters. To quantitatively test our hypothesis, a ground truth analysis was performed to validate that the sentiments captured by the text mining approach were comparable to the results provided by human annotators when asked to label whether such texts positively or negatively impact their outlook towards adopting an emerging technology. The collected annotations demonstrated comparable results to those of the text mining approach, illustrating that the automatically extracted sentiments expressed towards technologies are useful features in understanding the landscape faced by technology adopters, as well as serving as an important decisionmaking component when, for example, recognising shifts in user behaviours, new demands, and emerging uncertainties.