Large Text Corpora Research Articles

Building the Science News Agenda: The Permeability of Science Journalism to Public Relations

The current study examines the influence of press releases about scientific studies in terms of their impact on news coverage. Using an innovative approach that allowed for analysis of a large corpus of text and calculation of similarity scores, we were able to trace the uptake of press release materials into news media articles. In some cases, up to 65% of sentences in science news articles reflected high similarity to press release material—a potent indication of how powerful information subsidies can be. While our results contain some good news for public relations practitioners, they also carry a warning for consumers of journalism and for the public science agenda, which may be left vulnerable to bad actors undermining the trust that the public, and journalists, have in science. News organizations that had a history of producing award-winning science journalism were much less likely to draw on press release materials, indicating the importance of topic area expertise in producing independent science journalism.

Semantic and phonetic similarity of verbal fluency responses in early-stage psychosis

Linguistic abnormalities can emerge early in the course of psychotic illness. Computational tools that quantify similarity of responses in standardized language-based tasks such as the verbal fluency test could efficiently characterize the nature and functional correlates of these disturbances. Participants with early-stage psychosis (n=20) and demographically matched controls without a psychiatric diagnosis (n=20) performed category and letter verbal fluency. Semantic similarity was measured via predicted context co-occurrence in a large text corpus using Word2Vec. Phonetic similarity was measured via edit distance using the VFClust tool. Responses were designated as clusters (related items) or switches (transitions to less related items) using similarity-based thresholds. Results revealed that participants with early-stage psychosis compared to controls had lower fluency scores, lower cluster-related semantic similarity, and fewer switches; mean cluster size and phonetic similarity did not differ by group. Lower fluency semantic similarity was correlated with greater speech disorganization (Communication Disturbances Index), although more strongly in controls, and correlated with poorer social functioning (Global Functioning: Social), primarily in the psychosis group. Findings suggest that search for semantically related words may be impaired soon after psychosis onset. Future work is warranted to investigate the impact of language disturbances on social functioning over the course of psychotic illness.

A knowledge-enriched ensemble method for word embedding and multi-sense embedding

Representing words as embeddings has been proven to be successful in improving the performance in many natural language processing tasks. Different from the traditional methods that learn the embeddings from large text corpora, ensemble methods have been proposed to leverage the merits of pre-trained word embeddings as well as external semantic sources. In this paper, we propose a knowledge-enriched ensemble method to combine information from both knowledge graphs and pre-trained word embeddings. Specifically, we propose an attention network to retrofit the semantic information in the lexical knowledge graph into the pre-trained word embeddings. In addition, we further extend our method to contextual word embeddings and multi-sense embeddings. Extensive experiments demonstrate that the proposed word embeddings outperform the state-of-the-art models in word analogy, word similarity and several downstream tasks. The proposed word sense embeddings outperform the state-of-the-art models in word similarity and word sense induction tasks.

СУЖДЕНИЯ О ВОЗМОЖНОМ И О ВЕРОЯТНОМ В РАЗЛИЧНЫХ КУЛЬТУРАХ

Ways of presenting opinions depend on mental cultures which include i.a. styles of forming judgments on possibility and on probability of events and of states of affairs. Research on a large multilingual corpus of texts in several West-European languages and in Russian shows that the possibility statements are used more than twice as often as the probability statements. The term ‘possibility’ in Latin and in modern languages denotes a physicalist attitude towards states of affairs. This term was coined much later than the term ‘probability’, originally connected to the human aspects of evaluation. The term ‘probabilis’ itself in Latin was a cognate of ‘probare’, which meant ‘approving’ and/or ‘controlling’ events. Additionally, in modern Romance languages, judgments of doubt and hope, i. e. sentences conveying speaker’s distancing from alien and non-actual opinions, usually contain verbs in a special ‘conjunctive mood’. Creating alternative ‘possible worlds’ as a figure of speech for ‘conjecture’ originated and was extensively used in writings by Leibniz in French, who used this figure in accordance with French grammar. The same ideas formulated in Russian or in German lack the ‘subjunctive’ mood, whereas it is obligatory in French.

Embedding Conquest: Naturalizing Muslim Rule in the Early Islamic Empire (Project Report)

The ERC-funded research project »Embedding Conquest«, based at Leiden University, studies the mechanics by which early Islamic rule took root in a tapestry of diverse territories and different social contexts. Based on a large corpus of documentary and literary texts, especially letters, the research team has identified elements of political, territorial, and institutional cohesion stemming from interpersonal ties of loyalty and dependency binding individuals to each other and to the state. This contribution presents the project’s main scope and achievements. Additionally, it focuses on multilingualism, highlighting the participation of people from different religious groups in shaping early Islamic rule, in line with this special issue’s focus on collaboration among Muslims, Jews, and Christians as colleagues in the early Islamic period.

Domain-specific NLP system to support learning path and curriculum design at tech universities

The tech sector has been growing at a rapid speed, demanding a higher level of expertise from its labor force. New skills and programming languages are introduced and required by the industry every day, while the university courses are not updated adequately. Finding the high-demand skills and relevant courses to study has become essential to both students and faculty members at tech universities, which leads to a growing research interest in building an intelligence system to support decision making. Leveraging recent development in Natural Language Processing, we built an NLP-based course recommendation system specifically for the computer science (CS) and information technology (IT) fields. In particular, we built (1) a Named Entity Recognition (CSIT-NER) model to extract tech-related skills and entities, then used these skills to build (2) a personalized multi-level course recommendation system using a hybrid model (hybrid CSIT-CRS). Our CSIT-NER model, trained and fine-tuned on a large corpus of text extracted from StackOverflow and GitHub, can accurately extract the relevant skills and entities, outperforming state-of-the-art models across all evaluation metrics. Our hybrid CSIT-CRS can provide recommendations on multiple individualized levels of university courses, career paths with job listings, and industry-required with suitable online courses. The whole system received good ratings and feedback from users from our survey with 201 volunteers who are students and faculty members of tech universities in Australia and Vietnam. This research is beneficial to students, faculty members, universities in CS/IT higher education sector, and stakeholders in tech-related industries.

Sentiment Analysis System using Hybrid Word Embeddings with Convolutional Recurrent Neural Network

There have been wide ranges of innovations in sentiment analysis in recent past, with most effective ones involving use of various word embeddings methods for analysis of sentiments. GloVe and Word2Vec are acclaimed to be two most frequently used. A common problem with simple pre-trained embedding methods is that these ignore information related to sentiments of input texts and further depend on large text corpus for training purpose and generation of relevant vectors which is hindrance to researches involving smaller sized corpuses. The aim of proposed study is to propose a novel methodology for sentiment analysis that uses hybrid embeddings with a target to enhance features of available pre-trained embedding. Proposed hybrid embeddings use Part of Speech (POS) tagging and word2position vector over fastText with varied assortments of attached vectors to the pre-trained embedding vectors. The resultant form of hybrid embeddings is fed to our ensemble network-Convolutional Recurrent Neural Network (CRNN). The methodology has been tested for accuracy via different Ensemble models of deep learning and standard sentiment dataset with accuracy value of 90.21 using Movie Review (MVR) Dataset V2. Results show that proposed methodology is effective for sentiment analysis and is capable of incorporating even more linguistic knowledge-based techniques to further improve results of sentiment analysis.

Quantifying graphemic variation via large text corpora

Abstract The use of some basic computer science concepts could expand the possibilities of (manual) graphematic text corpus analysis. With these it can be shown that graphematic variation decreases constantly in printed German texts from 1600 to 1900. While the variability is continuously lesser on a text-internal level, it decreases faster for the whole available writing system of individual decades. But which changes took place exactly? Which types of variation went away more quickly, which ones persisted? How do we deal with large amounts of data which cannot be processed manually anymore? Which aspects are of special importance or go missing while working with a large textual base? The use of a measurement called entropy quantifies the variability of the spellings of a given word form, lemma, text or subcorpus, with few restrictions but also less details in the results. The difference between two spellings can be measured via Damerau-Levenshtein distance. To a certain degree, automated data handling can also determine the exact changes that took place. Afterwards, these differences can be counted and ranked. As data source the German Text Archive of the Berlin-Brandenburg Academy of Sciences and Humanities is used. It offers for example orthographic normalization – which is extremely useful –, preprocessing of parts of speech and lemmatization. As opposed to many other approaches the establishment of today’s normed spellings is not seen as the aim of the developments and is therefore not the focus of the research. Instead, the differences between individual spellings are of interest. Afterwards intra- and extralinguistic factors which caused these developments should be determined. These methodological findings could subsequently be used for improving research methods in other graphematic fields of interest, e. g. computer-mediated communication.

Tracing Long-term Value Change in (Energy) Technologies: Opportunities of Probabilistic Topic Models Using Large Data Sets

We propose a new approach for tracing value change. Value change may lead to a mismatch between current value priorities in society and the values for which technologies were designed in the past, such as energy technologies based on fossil fuels, which were developed when sustainability was not considered a very important value. Better anticipating value change is essential to avoid a lack of social acceptance and moral acceptability of technologies. While value change can be studied historically and qualitatively, we propose a more quantitative approach that uses large text corpora. It uses probabilistic topic models, which allow us to trace (new) values that are (still) latent. We demonstrate the approach for five types of value change in technology. Our approach is useful for testing hypotheses about value change, such as verifying whether value change has occurred and identifying patterns of value change. The approach can be used to trace value change for various technologies and text corpora, including scientific articles, newspaper articles, and policy documents.

Между «своим» и «чужим»: языковые и художественные реалии, проблемы, перспективы литератур народов Крайнего Севера и Дальнего Востока

The article discusses linguistic and artistic realities, problems and perspectives of the literatures of the indigenous peoples of the North and the Far East. The author fixes the widespread writing of works in Russian, comprehends the reasons for the linguistic transition, raises the question of the theoretical and substantive expansion of the concept of “national literature”, emphasizes the importance of the individual author's ethnocultural component. The author considers as a landmark projection of the development of “minority” national literatures the method of including elements of the “native” language in a Russian-language poem, an appeal to the topic “native language”, the experience of accidentally recognizing one's own in the sounds of a genetically non-native language. Special attention is paid to the problem of literary translation into Russian, some scenarios of distortion of texts in translations and reduction precedents of important mythological contexts are presented. The conclusions obtained during the analysis of a large corpus of poetic texts from the Nenets, Dolgan, Even, Chukchi, Koryak, Nanai literatures can be of significant analytical interest in a comparative aspect.

Moving Past Metadata: Improving Digital Libraries with Content‐Based Methods

AbstractThe growth of text mining and corpus analytic scholarship over large digital libraries brings to light the issues created by text duplication and variation within collections that are not adequately addressed in metadata practices. The SaDDL project was a study examining text duplication and similarity in massive digital library collections. Using the HathiTrust Digital Library, this project aims to reduce the bias that duplication gives rise to. We present the content‐based methods of the project, employing a convolutional neural network classification approach, as well as SaDDL's outcomes. The datasets provided by the project will assist in improving cataloging practice and aid scholars in using large text corpora in research.

The analytic‐continental divide in philosophical practice: An empirical study

AbstractPhilosophy is often divided into two traditions: analytic and continental philosophy. Characterizing the analytic‐continental divide, however, is no easy task. Some philosophers explain the divide in terms of the place of argument in these traditions. This raises the following questions: Is analytic philosophy rife with arguments while continental philosophy is devoid of arguments? Or can different types of arguments be found in analytic and continental philosophy? This paper presents the results of an empirical study of a large corpus of philosophical texts mined from the JSTOR database (n = 53,260) designed to find patterns of argumentation by type. Overall, the results suggest that there are no significant differences between the types of arguments advanced in analytic and continental philosophy journal articles. The findings, therefore, provide no empirical support to the hypothesis that the divide between analytic and continental philosophy has to do with the place of argument in these traditions.

The cultural dynamics of concept creep.

Emerging methods for studying cultural dynamics allow researchers to investigate cultural change with newfound rigor. One change that has recently attracted the attention of social commentators is "concept creep," the semantic inflation of harm-related concepts such as trauma, bullying, and prejudice. In theory, concept creep is driven distally by several recent cultural and societal trends, but psychology also plays a proximal role in developing and disseminating expansionary concepts of harm. However, there have been few systematic attempts to document concept creep and none to explore factors that influence it. The present work reviews concept creep from the perspective of cultural dynamics and lays out a conceptual framework for exploring processes implicated in it. Illustrative analyses are presented that apply computational linguistic methods to very large text corpora, including a new corpus of psychology article abstracts. They demonstrate that harm has risen steeply in prominence both in psychology and in the wider culture in recent decades, and that harm-related concepts have inflated their meanings over this period. The analyses also provide evidence of dynamic relationships between the prominence and semantic breadth of harm-related concepts, and between psychology and the culture at large. Implications are drawn for theory and research on concept creep. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

구어체 적응 사전 학습을 통한 한국어 감정 분류 성능 향상

Language models (LMs) pretrained on a large text corpus and fine-tuned on a task data have a remarkable performance for document classification task. Recently, an adaptive pretraining method that re-pretrains the pretrained LMs using an additional dataset in the same domain with the given task to make up the domain discrepancy has reported significant performance improvements. However, current adaptive pretraining methods only focus on the domain gap between pretraining data and fine-tuning data. The writing style is also different because the pretraining data, e.g., Wikipedia, is written in a literary style, but the task data, e.g., customer review, is usually written in a colloquial style. In this work, we propose a colloquial-adaptive pretraining method that re-pretrains the pretrained LM with informal sentences to generalize the LM to colloquial style. We verify the proposed method based on multi-emotion classification datasets. The experimental results show that the proposed method yields improved classification performance on both low- and high-resource data.

Creating Welsh Language Word Embeddings

Word embeddings are representations of words in a vector space that models semantic relationships between words by means of distance and direction. In this study, we adapted two existing methods, word2vec and fastText, to automatically learn Welsh word embeddings taking into account syntactic and morphological idiosyncrasies of this language. These methods exploit the principles of distributional semantics and, therefore, require a large corpus to be trained on. However, Welsh is a minoritised language, hence significantly less Welsh language data are publicly available in comparison to English. Consequently, assembling a sufficiently large text corpus is not a straightforward endeavour. Nonetheless, we compiled a corpus of 92,963,671 words from 11 sources, which represents the largest corpus of Welsh. The relative complexity of Welsh punctuation made the tokenisation of this corpus relatively challenging as punctuation could not be used for boundary detection. We considered several tokenisation methods including one designed specifically for Welsh. To account for rich inflection, we used a method for learning word embeddings that is based on subwords and, therefore, can more effectively relate different surface forms during the training phase. We conducted both qualitative and quantitative evaluation of the resulting word embeddings, which outperformed previously described word embeddings in Welsh as part of larger study including 157 languages. Our study was the first to focus specifically on Welsh word embeddings.

Performing Multilingual Analysis With Linguistic Inquiry and Word Count 2015 (LIWC2015). An Equivalence Study of Four Languages.

Today, there is a range of computer-aided techniques to convert text into data. However, they convey not only strengths but also vulnerabilities compared to traditional content analysis. One of the challenges that have gained increasing attention is performing automatic language analysis to make sound inferences in a multilingual assessment setting. The current study is the first to test the equivalence of multiple versions of one of the most appealing and widely used lexicon-based tools worldwide, Linguistic Inquiry and Word Count 2015 (LIWC2015). For this purpose, we employed supervised learning in a classification problem and computed Pearson's correlations and intraclass correlation coefficients on a large corpus of parallel texts in English, Dutch, Brazilian Portuguese, and Romanian. Our findings suggested that LIWC2015 is a valuable tool for multilingual analysis, but within-language standardization is needed when the aim is to analyze texts sourced from different languages.

Expert-Informed Topic Models for Document Set Discovery

ABSTRACT The first step in many text-as-data studies is to find documents that address a specific topic within a larger document set. Researchers often rely on simple keyword searches to do this, even though this may introduce considerable selection bias. Such bias may be even greater when researchers lack the domain knowledge required to make informed search decisions, for example, in cross-national research or research on unfamiliar social contexts. We propose expert-informed topic modeling (EITM) as a hybrid approach to tackle this problem. EITM combines the validity of external domain knowledge captured through expert surveys with probabilistic topic models to help researchers identify subsets of documents that cover initially unknown domain-specific topics, such as specific events and debates, that belong to a researcher-defined master topic. EITM is a flexible and efficient approach to the thematic selection of documents from large text corpora for further study. We benchmark and validate the method by discovering blog posts that address the public role of religion within large corpora of Australian, Swiss, and Turkish blog posts and provide researchers with a complete workflow to guide the application of EITM in their own work.

An Improved Vulnerability Exploitation Prediction Model with Novel Cost Function and Custom Trained Word Vector Embedding.

Successful cyber-attacks are caused by the exploitation of some vulnerabilities in the software and/or hardware that exist in systems deployed in premises or the cloud. Although hundreds of vulnerabilities are discovered every year, only a small fraction of them actually become exploited, thereby there exists a severe class imbalance between the number of exploited and non-exploited vulnerabilities. The open source national vulnerability database, the largest repository to index and maintain all known vulnerabilities, assigns a unique identifier to each vulnerability. Each registered vulnerability also gets a severity score based on the impact it might inflict upon if compromised. Recent research works showed that the cvss score is not the only factor to select a vulnerability for exploitation, and other attributes in the national vulnerability database can be effectively utilized as predictive feature to predict the most exploitable vulnerabilities. Since cybersecurity management is highly resource savvy, organizations such as cloud systems will benefit when the most likely exploitable vulnerabilities that exist in their system software or hardware can be predicted with as much accuracy and reliability as possible, to best utilize the available resources to fix those first. Various existing research works have developed vulnerability exploitation prediction models by addressing the existing class imbalance based on algorithmic and artificial data resampling techniques but still suffer greatly from the overfitting problem to the major class rendering them practically unreliable. In this research, we have designed a novel cost function feature to address the existing class imbalance. We also have utilized the available large text corpus in the extracted dataset to develop a custom-trained word vector that can better capture the context of the local text data for utilization as an embedded layer in neural networks. Our developed vulnerability exploitation prediction models powered by a novel cost function and custom-trained word vector have achieved very high overall performance metrics for accuracy, precision, recall, F1-Score and AUC score with values of 0.92, 0.89, 0.98, 0.94 and 0.97, respectively, thereby outperforming any existing models while successfully overcoming the existing overfitting problem for class imbalance.

Identifying Incident Causal Factors to Improve Aviation Transportation Safety: Proposing a Deep Learning Approach

Aviation is a complicated transportation system, and safety is of paramount importance because aircraft failure often involves casualties. Prevention is clearly the best strategy for aviation transportation safety. Learning from past incident data to prevent potential accidents from happening has proved to be a successful approach. To prevent potential safety hazards and make effective prevention plans, aviation safety experts identify primary and contributing factors from incident reports. However, safety experts’ review processes have become prohibitively expensive nowadays. The number of incident reports is increasing rapidly due to the acceleration of advances in information technologies and the growth of the commercial and private aviation transportation industries. Consequently, advanced text mining algorithms should be applied to help aviation safety experts facilitate the process of incident data extraction. This paper focuses on constructing deep-learning-based models to identify causal factors from incident reports. First, we prepare the data sets used for training, validation, and testing with approximately 200,000 qualified incident reports from the Aviation Safety Reporting System (ASRS). Then, we take an open-source natural language model, which is well trained with a large corpus of Wikipedia texts, as the baseline and fine-tune it with the texts in incident reports to make it more suited to our specific research task. Finally, we build and train an attention-based long short-term memory (LSTM) model to identify primary and contributing factors in each incident report. The solution we propose has multilabel capability and is automated and customizable, and it is more accurate and adaptable than traditional machine learning methods in extant research. This novel application of deep learning algorithms to the incident reporting system can efficiently improve aviation safety.

HopRetriever: Retrieve Hops over Wikipedia to Answer Complex Questions

Collecting supporting evidence from large corpora of text (e.g., Wikipedia) is of great challenge for open-domain Question Answering (QA). Especially, for multi-hop open-domain QA, scattered evidence pieces are required to be gathered together to support the answer extraction. In this paper, we propose a new retrieval target, hop, to collect the hidden reasoning evidence from Wikipedia for complex question answering. Specifically, the hop in this paper is defined as the combination of a hyperlink and the corresponding outbound link document. The hyperlink is encoded as the mention embedding which models the structured knowledge of how the outbound link entity is mentioned in the textual context, and the corresponding outbound link document is encoded as the document embedding representing the unstructured knowledge within it. Accordingly, we build HopRetriever which retrieves hops over Wikipedia to answer complex questions. Experiments on the HotpotQA dataset demonstrate that HopRetriever outperforms previously published evidence retrieval methods by large margins. Moreover, our approach also yields quantifiable interpretations of the evidence collection process.