Large Text Corpora Research Articles

Overcoming the semantic gap in the customer-to-manufacturer (C2M) platform: A soft prompts-based approach with pretrained language models

Customer-to-Manufacturer (C2M) is a strategy in smart manufacturing where customers collaborate with manufacturers for customized product development on an online platform. The platform enables the shift from the traditional manufacturing process, which is driven by research and marketing, toward a customer-centric product development process. However, a challenge arises as customers lack technical knowledge to communicate their product specifications effectively, creating a semantic gap. This paper proposes a soft prompt-based network structure that utilizes pretrained language models to bridge the semantic gap on the C2M platform. To address limited customer needs data and imbalanced classes, a large corpus of product review texts is used to establish a mapping between reviews and product specifications. A smaller set of customer needs text is then employed to adapt this mapping to the target customer needs-product specifications relationship, thereby closing the semantic gap. The experimental results demonstrate the effectiveness of the proposed model adaptation operation and the prompting structure. Additionally, the experiments highlight the robustness of the proposed method against variations in training data size, thereby mitigating the challenges posed by imbalanced classes. The proposed method could potentially bring innovation to product customization and C2M platform development. By bridging the semantic gap, companies can better integrate customers in the co-design process and effectively translate customer needs into actionable product specifications.

Using Text Mining to Identify Teleological Explanations in Physics and Biology Textbooks: An Exploratory Study

AbstractCreating and critiquing explanations of phenomena is a significant goal of many scientific disciplines and therefore also a learning goal of science education. A significant source of explanations is science textbooks; however, the large corpus of text in textbooks means that manual review of explanations by individual researchers is extremely time consuming. In this paper, we introduce a text-mining approach for identifying legitimate and illegitimate forms of teleological explanations in school physics and biology textbooks. An ongoing debate exists about the legitimacy of teleological explanations, that is, explanations which account for a phenomenon by reference to a final end, purpose, or goal. Until recently, researchers tended to view teleology as an illegitimate form of scientific explanation. Recent theoretical cases in biology and physics have emphasized that legitimate teleological explanations exist in both domains. Eight science textbooks used in England and internationally were analyzed for instances of teleological explanations. The analysis reveals the efficiency of the text-mining approach for automating the analysis of textbooks and its potential as a research approach in science education. In considering text mining as a research approach, we report terms that are likely to be associated with legitimate teleological explanations. We found that legitimate teleological accounts are used by textbook authors, and we present novel categorizations of these forms of explanation. We argue that text mining can be a useful approach in science education research and our findings suggest guidance for both textbook writers and teachers related to their selection of legitimate explanatory forms.

How ancient wisdom shaped the development of modern science

How ancient wisdom shaped the development of modern science VERITRACE will enhance our understanding of the role of ancient wisdom writings in the development of early modern natural philosophy by employing state-of-the-art digital techniques on a large corpus of early modern texts. The project charts the spread of this discourse from Renaissance Italy to early modern Europe and identifies watershed moments in the reception of these ancient wisdom writings.

IClassifier: A digital research tool for corpus-based classifier networks in complex writing systems

This article presents the method applied by the iClassifier (©Goldwasser/Harel/Nikolaev) digital research tool for the study of the linguistic phenomenon of classifiers. The tool was created in 2019 with the objective of curating corpus-based and data-driven documentation of classifier systems. The record of classifiers comprises millions of tokens worth of “big data” analysis. By tagging classifiers in various corpora, a topography of categories emerges, visualized as complex, multilayered networks. This article offers an overview of how classifier-based networks are created and how network analysis methods can be applied to analyze knowledge organization. We present the data structure and annotation scheme of the iClassifier research tool, demonstrating how one can plot classifier networks and generate reports of lemma and classifier repertoires in each corpus. The iClassifier tool provides quantitative reports, including classifier frequency, variation and co-occurrence statistics. Each data subset, such as a certain part of speech, timespan, geographical location or textual genre, can be queried and visualized. The tool is meant to allow browsing between a macro-overview of all categories in a corpus and zooming in into micro-analysis of the individual categories and lemmas that built up a corpus. Each classifier is seen as a category head, and the categories are drawn in their multilayered and multidimensional relationships. The potency of this tool is in documenting the phenomenon in large corpora of texts and expanding our knowledge about the rules and functions of classifier systems, leading us to a more refined mind-mapping of ancient cultures. Currently, very little systematic analysis has been done on this ancient record of emic information.

On Text Understanding Modeling by Means of Artificial Intelligence Systems

Purpose of reseach is to conduct a comparative analysis of texts generated by an artificial intelligence system in the process of processing source text in natural language and counter texts that are the result of human understanding of the source literary text. Methods. To achieve the goal and objectives of the study, the author used an experimental technique to conduct a comparative analysis of the denotational structures of counter texts. Participants in the experiment (7 4th year students of the Faculty of Additional Education “Translator in the field of professional communications”, 3 associate professors of the Department of Foreign Languages of South- West State University) assessed the success of recreating the semantic structure of the text of 10 counter texts of different nature - generated by AI and humans. Results. The results of the experiment indicate that the completeness of the semantic content of the generated text does not depend on the structure of the source text. Modern methods of semantic text processing by an AI system make it possible to obtain the output of full-fledged text works created taking into account the rules and norms of natural language.AI systems successfully recreate the denotational structure of the text and reconstruct the syntactic structure. Conclusion. Access to large databases allows you to train a neural network on large text corpora, which results in an increase in the accuracy and variability of the lexical units and constructs used. The accuracy of conveying the semantic content of the text varies. It depends on the degree of text compression - the higher it is, the less accuracy can be, because the neural network is unable to classify denotational connections for relevance to the underlying meaning. The degree of accuracy in conveying semantic content is determined by the success / failure of understanding the deep hidden meaning, which is determined by the understanding of the linguistic and extralinguistic context. The ability to recognize the situation model recreated in the source text is the key to understanding the hidden meaning. The AI system can recreate the surface denotational structure of the text quite correctly and accurately, but is not able to construct a model of the situation at this stage of development.

ExKidneyBERT: a language model for kidney transplant pathology reports and the crucial role of extended vocabularies.

Pathology reports contain key information about the patient's diagnosis as well as important gross and microscopic findings. These information-rich clinical reports offer an invaluable resource for clinical studies, but data extraction and analysis from such unstructured texts is often manual and tedious. While neural information retrieval systems (typically implemented as deep learning methods for natural language processing) are automatic and flexible, they typically require a large domain-specific text corpus for training, making them infeasible for many medical subdomains. Thus, an automated data extraction method for pathology reports that does not require a large training corpus would be of significant value and utility. To develop a language model-based neural information retrieval system that can be trained on small datasets and validate it by training it on renal transplant-pathology reports to extract relevant information for two predefined questions: (1) "What kind of rejection does the patient show?"; (2) "What is the grade of interstitial fibrosis and tubular atrophy (IFTA)?" Kidney BERT was developed by pre-training Clinical BERT on 3.4K renal transplant pathology reports and 1.5M words. Then, exKidneyBERT was developed by extending Clinical BERT's tokenizer with six technical keywords and repeating the pre-training procedure. This extended the model's vocabulary. All three models were fine-tuned with information retrieval heads. The model with extended vocabulary, exKidneyBERT, outperformed Clinical BERT and Kidney BERT in both questions. For rejection, exKidneyBERT achieved an 83.3% overlap ratio for antibody-mediated rejection (ABMR) and 79.2% for T-cell mediated rejection (TCMR). For IFTA, exKidneyBERT had a 95.8% exact match rate. ExKidneyBERT is a high-performing model for extracting information from renal pathology reports. Additional pre-training of BERT language models on specialized small domains does not necessarily improve performance. Extending the BERT tokenizer's vocabulary library is essential for specialized domains to improve performance, especially when pre-training on small corpora.

Rights and Wrongs in Talk of Mind-Reading Technology.

This article examines the idea of mind-reading technology by focusing on an interesting case of applying a large language model (LLM) to brain data. On the face of it, experimental results appear to show that it is possible to reconstruct mental contents directly from brain data by processing via a chatGPT-like LLM. However, the author argues that this apparent conclusion is not warranted. Through examining how LLMs work, it is shown that they are importantly different from natural language. The former operates on the basis of nonrational data transformations based on a large textual corpus. The latter has a rational dimension, being based on reasons. Using this as a basis, it is argued that brain data does not directly reveal mental content, but can be processed to ground predictions indirectly about mental content. The author concludes that this is impressive but different in principle from technology-mediated mind reading. The applications of LLM-based brain data processing are nevertheless promising for speech rehabilitation or novel communication methods.

Natural language processing of clinical notes enables early inborn error of immunity risk ascertainment

There are now approximately 450 discrete inborn errors of immunity (IEI) described; however, diagnostic rates remain suboptimal. Use of structured health record data has proven useful for patient detection but may be augmented by natural language processing (NLP). Here we present a machine learning model that can distinguish patients from controls significantly in advance of ultimate diagnosis date. We sought to create an NLP machine learning algorithm that could identify IEI patients early during the disease course and shorten the diagnostic odyssey. Our approach involved extracting a large corpus of IEI patient clinical-note text from a major referral center's electronic health record (EHR) system and a matched control corpus for comparison. We built text classifiers with simple machine learning methods and trained them on progressively longer time epochs before date of diagnosis. The top performing NLP algorithm effectively distinguished cases from controls robustly 36 months before ultimate clinical diagnosis (area under precision recall curve> 0.95). Corpus analysis demonstrated that statistically enriched, IEI-relevant terms were evident 24+ months before diagnosis, validating that clinical notes can provide a signal for early prediction of IEI. Mining EHR notes with NLP holds promise for improving early IEI patient detection.

The journey within: mental navigation as a novel framework for understanding psychotherapeutic transformation

BackgroundDespite the demonstrated efficacy of psychotherapy, the precise mechanisms that drive therapeutic transformations have posed a challenge and still remain unresolved. Here, we suggest a potential solution to this problem by introducing a framework based on the concept of mental navigation. It refers to our ability to navigate our cognitive space of thoughts, ideas, concepts, and memories, similar to how we navigate physical space. We start by analyzing the neural, cognitive, and experiential constituents intrinsic to mental navigation. Subsequently, we posit that the metaphoric spatial language we employ to articulate introspective experiences (e.g., “unexplored territory” or “going in circles”) serves as a robust marker of mental navigation.MethodsUsing large text corpora, we compared the utilization of spatial language between transcripts of psychotherapy sessions (≈ 12 M. words), casual everyday conversations (≈ 12 M. words), and fictional dialogues in movies (≈ 14 M. words). We also examined 110 psychotherapy transcripts qualitatively to discern patterns and dynamics associated with mental navigation.ResultsWe found a notable increase in the utilization of spatial metaphors during psychotherapy compared to casual everyday dialogues (U = 192.0, p = .001, d = 0.549) and fictional conversations (U = 211, p < .001, d = 0.792). In turn, analyzing the usage of non-spatial metaphors, we did not find significant differences between the three datasets (H = 0.682, p = 0.710). The qualitative analysis highlighted specific examples of mental navigation at play.ConclusionMental navigation might underlie the psychotherapy process and serve as a robust framework for understanding the transformative changes it brings about.

A Systematic Literature Review on Text Generation using Deep Neural Networks

Text generation using deep neural networks has become an exciting area of research. Deep neural networks, such as recurrent neural networks (RNNs) and transformers, have shown remarkable capabilities in generating coherent and contextually relevant text. By training these models on large text corpora, they learn to capture the underlying patterns and structures of the language. This enables them to generate new text that resembles the style and content of the training data. Text generation using deep neural networks has a wide range of applications, including chatbots, language translation, poetry generation, and even code generation.

Loose and Tight: Creative Formation but Rigid Use of Nominal Compounds in Conspiracist Texts

ABSTRACTConspiracy theories (CTs) are spectacular narratives, widely spread, that pose societal threats. We test whether CTs might be linguistically creative products, which would facilitate their transmission and thereby account for their widespread popularity. We analyzed nominal compounds (e.g., mind control, carbon dioxide; N = 1,713,568) from a large corpus of conspiracist and mainstream texts matched by topic. In conspiracist texts, compounds showed greater originality, divergence, and sophistication, but they were used with lower frequency and were more often repeated in different contexts. This pattern suggests a creative aspect in the generation of compounds, coupled with rigidity in their use. We interpret these findings as an effect of loosely defined conceptual boundaries among conspiracist writers, in conjunction with social functions like status or group signaling. Our findings not only contribute to the discourse on creativity in CTs but also provide insights into the communicative advantage of CTs.

Cultural framing of giftedness in recent US fictional texts.

A perennial topic of research on giftedness has been individuals' perceptions of and attitudes towards giftedness, the gifted, and gifted education. Although giftedness is a culturally constructed concept, most examination of the term's meanings and implications has used reactive measures (i.e., surveys) to tap respondents' giftedness-related perceptions and attitudes within the context of formal education. To provide a better understanding of the cultural meanings associated with giftedness-the term's cultural framing-we investigated the depiction of giftedness within a professional cultural product removed from education, namely, a large corpus of US fictional texts. We examined patterns of word usage in the vicinity of the term gift*, when used in the dictionary senses related to giftedness, in a large corpus of US fictional texts of recent decades, consisting of 485,179 text samples and 1,002,889,754 word tokens. Via inductive methods of quantitative text analysis, we explored themes occurring in the vicinity of gift*; and with an existing lookup dictionary, we assessed deductively the overall emotional valance of the writing near gift*. Our investigation revealed ways in which the literary exploration of giftedness coheres with and distinguishes itself from the outlooks on giftedness noted for survey-based research in education settings. In fictional texts, giftedness evinces special associations with humanities domains and beauty and, on balance, correlates positively with emotionally positive words.

Large Language Models in Neurosurgery.

A large language model (LLM), in the context of natural language processing and artificial intelligence, refers to a sophisticated neural network that has been trained on a massive amount of text data to understand and generate human-like language. These models are typically built on architectures like transformers. The term "large" indicates that the neural network has a significant number of parameters, making it more powerful and capable of capturing complex patterns in language. One notable example of a large language model is ChatGPT. ChatGPT is a large language model developed by OpenAI that uses deep learning techniques to generate human-like text. It can be trained on a variety of tasks, such as language translation, question answering, and text completion. One of the key features of ChatGPT is its ability to understand and respond to natural language inputs. This makes it a powerful tool for generating a wide range of text, including medical reports, surgical notes, and even poetry. Additionally, the model has been trained on a large corpus of text, which allows it to generate text that is both grammatically correct and semantically meaningful. In terms of applications in neurosurgery, ChatGPT can be used to generate detailed and accurate surgical reports, which can be very useful for sharing information about a patient's case with other members of the medical team. Additionally, the model can be used to generate detailed surgical notes, which can be very useful for training and educating residents and medical students. Overall, LLMs have the potential to be a valuable tool in the field of neurosurgery. Indeed, this abstract has been generated by ChatGPT within few seconds. Potential applications and pitfalls of the applications of LLMs are discussed in this paper.

Introducing a global dataset on conflict forecasts and news topics

Abstract This article provides a structured description of openly available news topics and forecasts for armed conflict at the national and grid cell level starting January 2010. The news topics, as well as the forecasts, are updated monthly at conflictforecast.org and provide coverage for more than 170 countries and about 65,000 grid cells of size 55 × 55 km worldwide. The forecasts rely on natural language processing (NLP) and machine learning techniques to leverage a large corpus of newspaper text for predicting sudden onsets of violence in peaceful countries. Our goals are a) to support conflict prevention efforts by making our risk forecasts available to practitioners and research teams worldwide, b) to facilitate additional research that can utilize risk forecasts for causal identification, and c) to provide an overview of the news landscape.

Test

Semantic search is the process of retrieving relevant information from a large corpus of texts based on the meaning and context of the query. This paper is introduced in order to explore the use of large language models for semantic search of Quranic texts. The Quran, which is the central religious text of Islam, contains rich and complex linguistic and semantic features that pose challenges for traditional keyword-based search methods. This study investigates a semantic search approach utilizing. Large Language Models (LLM) embedding and assess the performance of LLM embedding in comparison to a baseline embedding-based search method using a set of queries that represent different semantic search levels. In addition, this study will also discuss the limitations and implications of using large language models for semantic search of Quranic texts and suggest directions for future research. A significant finding in this study is the consistent effectiveness of the LLM embedding across varying semantic complexities. This suggests that embedding using LLMs can capture deep semantic connections effectively. On the other hand, as a second finding, the state-of-the-art transformer, AraT5, outperforms LLM embeddings in low-level semantic searches, indicating potential for further LLM fine-tuning on Arabic text corpora

Embedding Search for Quranic Texts based on Large Language Models

Semantic search is the process of retrieving relevant information from a large corpus of texts based on the meaning and context of the query. This paper is introduced in order to explore the use of large language models for semantic search of Quranic texts. The Quran, which is the central religious text of Islam, contains rich and complex linguistic and semantic features that pose challenges for traditional keyword-based search methods. This study investigates a semantic search approach utilizing. Large Language Models (LLM) embedding and assess the performance of LLM embedding in comparison to a baseline embedding-based search method using a set of queries that represent different semantic search levels. In addition, this study will also discuss the limitations and implications of using large language models for semantic search of Quranic texts and suggest directions for future research. A significant finding in this study is the consistent effectiveness of the LLM embedding across varying semantic complexities. This suggests that embedding using LLMs can capture deep semantic connections effectively. On the other hand, as a second finding, the state-of-the-art transformer, AraT5, outperforms LLM embeddings in low-level semantic searches, indicating potential for further LLM fine-tuning on Arabic text corpora.

Discovering the Significance of Sports Footwear Brands through Text Analysis

Objective - This paper focuses on analyzing the significance of sports footwear brands by processing large text data from the Internet. In a modern environment, the brand distinguishes a company's products or services from those of its competitors. A strong brand can help build trust with customers as they perceive the brand as reliable and trustworthy. Methodology/Technique - The study uses NLP (Natural Language Processing) methods to analyze rich text content on the Internet. The research focus is based on the application of innovative methods to determine the importance and value of a brand using NLP techniques by analyzing the content of a large corpus of text originating from websites dealing with sports footwear brands. The NLP analysis models were programmed using the low-code analysis tool KNIME. Findings – The analysis is carried out for the most well-known sports footwear brands such as Nike, Adidas, Puma, Under Armour, Reebok and Asics. The research object refers to the analysis of brand significance and the evaluation of consumer opinions on sports footwear, based on the processing of large text data from the internet. Novelty - The research results are based on an innovative approach to measuring and evaluating the brand significance in sports footwear using NLP methods to analyze large text content from the Internet. The results obtained show that this new approach to metrics and evaluation can significantly improve existing methods of brand evaluation. Type of Paper: Empirical JEL Classification: M32, M39. Keywords: Brand, NLP Method, Text Analysis, Online Brand Management Strategies, Sports Footwear. Reference to this paper should be made as follows: Tarade, S.S; Vuković, D. (2023). Discovering the Significance of Sports Footwear Brands through Text Analysis, J. Mgt. Mkt. Review, 8(4), 166 – 175. https://doi.org/10.35609/jmmr.2023.8.4(7)

Beyond sentiment: an algorithmic strategy for identifying evaluations within large text corpora

ABSTRACT In this paper, we propose a new strategy for classifying evaluations in large text corpora, using supervised machine learning (SML). Departing from a conceptual and methodological critique of the use of sentiment measures to recognize object-specific evaluations, we argue that a key challenge consists in determining whether a semantic relationship exists between evaluative expressions and evaluated objects. Regarding sentiment terms as merely potentially evaluative expressions, we thus use a SML classifier to decide whether recognized terms have an evaluative function in relation to the evaluated object. We train and test our classifier on a corpus of 10,004 segments of election coverage from 16 major U.S. news outlets and Tweets by 10 prominent U.S. politicians and journalists. Specifically, we focus on evaluations of political predictions about the outcomes and implications of the 2016 and 2020 U.S. presidential elections. We show that our classifier consistently outperforms both off-the-shelf sentiment tools and a pre-trained transformer-based sentiment classifier. Critically, our classifier correctly discards numerous non-evaluative uses of common sentiment terms, whose inclusion in conventional analyses generates large amounts of false positives. We discuss contributions of our approach to the measurement of object-specific evaluations and highlight challenges for future research.

Layers of Folkloric Variation: Computational Explorations of Poetic and Narrative Text Corpora

Variation is a core feature of folklore that plays a part in configuring the processes of folkloric communication, transmission and creativity. Computational analysis of large folklore collections offers new outlooks on the study of variation. The article explores the nature of folkloric variation on the basis of folk song and fairy tale text corpora from the Estonian Folklore Archives. Our enquiry into regional variation in Estonian folk song showed different patterns of geographic variation for metre, repertoire and language. To investigate further the possibilities to disassemble the components of variation we turned to a smaller corpus of fairy tale texts from the distinct Seto language community. The results of stylometric analysis of fairy tales collected within the close language community as a rule found that word usage patterns in stories told by the same storytellers were closer to each other than stories with the same content (i.e. tale types). However, other factors, such as collectors’ individual styles, recording time, length of text, and storyteller’s place of origin contributed notably to similarity as per stylometric scores. The study has shown that the individual features and aspects of such a complex phenomenon as folklore can follow its own variation patterns. Computational analysis of the variation in large text corpora helps us get a better understanding of the functioning of variation, and the processes of folkloric creation. However, the layers of folkloric variation are not that easy to disassemble, one must be aware of the biases in text corpora and keep in mind the effect of the linguistic variation on the results.

Idiome unter dem diskursiven Aspekt am Beispiel des Corona-Diskurses. Ein onomasiologischer Zugang

Abstract One of the most important issues in interpreting the role of phrasemes in discourse concerns identifying phrasemes that are typical of discourse and those that indicate discourse. This paper attempts to identify discourse-relevant idiomatic phrasemes (idioms) based on a large corpus of press texts from Süddeutsche Zeitung on the Covid-19-discourse. For this purpose, a deductive method is applied. The corpus is analysed for the keywords, -terms, and Ngrams to identify the main discourse strands. The thematic domains were determined according to the discourse strands, based on which, idioms that could potentially occur in the analysed discourse were defined onomasiologically and extracted from the phraseological dictionary. Subsequently, the extracted units were searched for in the corpus. In the final step, the relative frequency of the idioms in the studied discourse was compared with their relative frequency in reference corpora. The chapters on the methodology and results of the study are preceded by theoretical reflections on discourse, phraseology, and attempts to combine the assumptions of these two disciplines. The study reveals that the methodology used allows for the identification of phrases that are typical of a particular discourse. First, some of the phraseological units studied actually occurred in the discourse. Second, most of the idioms with the highest frequency in the studied discourse occurred more frequently in the Covid-19-corpus than in the reference corpora. However, the method has some limitations in terms of the time required and the number of results obtained. The qualitative analysis of the idiom with the highest increase in frequency shows that it is indeed used because of the main topic of the discourse, and thus can be considered both typical and as an indicator of this discourse.