Diachronic Word Research Articles

Identifying Semantic Shift Reasons of Domain Entities Using Diachronic Embedding

ABSTRACTIn academic literature, the semantic evolution of scientific terms reflects the development of domain knowledge. While there have been studies using diachronic word embeddings to reveal cultural and social shifts, research on the semantic evolution of scientific terms remains limited. This paper proposes a method utilizing diachronic semantic vectors to elucidate the semantic evolution of scientific entities. Firstly, diachronic semantic vectors of entities are trained using incremental learning to compute the semantic shift (ΔA) of a given entity A between periods t1 and t2. Secondly, entities similar to ΔA in periods t1 and t2 are defined as leading reasons and accompanying reasons, respectively. The experiment result shows that the leading reasons can capture the shifts of entities in the nascent stage, and the accompanying reasons can capture the shifts of words in the mature stage. The proposed method offers a better quantitative insight into the details and reasons behind domain knowledge evolution.

Sense through time: diachronic word sense annotations for word sense induction and Lexical Semantic Change Detection

AbstractThere has been extensive work on human word sense annotation, i.e., manually labeling word uses in natural texts according to their senses. Such labels were primarily created for the tasks of Word Sense Disambiguation (WSD) and Word Sense Induction (WSI). However, almost all datasets annotated with word senses are synchronic datasets, i.e., contain texts created in a relatively short period of time and often do not provide the creation date of the texts. This ignores possible applications in diachronic-historic settings, where the aim is to induce or disambiguate historical word senses or changes in senses across time. To facilitate investigations into historical WSD and WSI and to establish connections with the task of Lexical Semantic Change Detection (LSCD), there is a crucial need for historical word sense-annotated data. Hence, we created a new reliable diachronic WSD/WSI dataset ‘DWUG DE Sense’. We describe the preparation and annotation and analyze central statistics. We then describe a thorough evaluation of different prediction systems for jointly solving both WSI and LSCD tasks. All our systems are based on a state-of-the-art architecture that combines Word-in-Context models and graph clustering techniques with different hyperparameter settings. Our findings reveal that using the WSI task as optimization criterion yields better results for both tasks even when the LSCD task is the focal point of optimization. This underscores that although both tasks are related, WSI seems to be more general and able to incorporate the LSCD task.

Spatiotemporal knowledge graph completion via diachronic and transregional word embedding

Knowledge Graph Completion (KGC) is an essential application in the field of knowledge graphs (KGs) that attempts to fill in the missing information in the process of KG modelling. With the popularity of temporal knowledge graphs (TKGs), a wide range of techniques based on temporal knowledge graph completion (TKGC) have appeared, solving the issue of real-world knowledge with temporal properties. However, there is little study on KGC with spatiotemporal attributes, some real-world data include both spatial and temporal attributes. Effectively handling the completion of missing entities or predicates in spatiotemporal knowledge graphs (STKGs) is an important challenge. Our study fills the gap in knowledge completion techniques in the field of STKG. We present a model for completion based on the well-known tensor factorization canonical polyadic (CP) decomposition. It introduces temporal and spatial attributes into the decomposition vector to achieve entity and link predictions. We name it diachronic and transregional word embedding (DT-WE), which includes two different modules: the embedding framework and the scoring module. Firstly, send the vectors to the embedding framework to get the new vector representation, then, we send it to the scoring module to compute, and finally, the resulting values are added together to compute the prediction probability. We conducted extensive experiments on three real-world STKGs: YAGO10K, Wikidata40K and Opensky. The results indicate that the newly introduced spatiotemporal attributes not only improve accuracy in predicting entities and predicates compared to temporal models but also achieve state-of-the-art performance with lower spatial complexity.

Analysis of the Semantic Shift in Diachronic Word Embeddings for Spanish Before and After COVID-19

Words can shift their meaning across time. This study shows the results obtained by the exploratory analysis of the semantic shifting on Spanish vocabulary using Diachronic Word Embeddings. Diachronic data consists of a 2018 Spanish corpus, before the COVID-19 outbreak, and a second corpus with documents from 2021. This paper addresses the construction of the diachronic Spanish word embeddings model, as well as the results obtained by the analysis using a non-supervised distance vector technique. The results allowed us to identify topics with the most semantic shift between those periods.

Testing of statistical significance of semantic changes detected by diachronic word embedding

In recent years, methods based on word embedding models have been widely used for solving problems of semantic change estimation. The models are trained on text corpora of various years. Semantic change is detected by analyzing changes in distance between words using vector space alignment or by analyzing changes in a set of words that are most similar in meaning to a target word. Testing for statistical significance of the detected effects has not been detailly discussed in previous studies. This paper focuses on the problem of testing statistical significance of semantic change. Besides, we consider the problem of finding a confidence interval of estimates of semantic distance between words. We allow for the influence of two random factors. The first one is associated with the use of random initial conditions and stochastic optimization when training the model, the second one results from a random selection of texts for a training corpus. The proposed approach is based on the use of resampling of a training set of texts. The proposed method is tested on the COHA corpus.

A data-driven approach to studying changing vocabularies in historical newspaper collections

Abstract Nation and nationhood are among the most frequently studied concepts in the field of intellectual history. At the same time, the word ‘nation’ and its historical usage are very vague. The aim in this article was to develop a data-driven method using dependency parsing and neural word embeddings to clarify some of the vagueness in the evolution of this concept. To this end, we propose the following two-step method. First, using linguistic processing, we create a large set of words pertaining to the topic of nation. Second, we train diachronic word embeddings and use them to quantify the strength of the semantic similarity between these words and thereby create meaningful clusters, which are then aligned diachronically. To illustrate the robustness of the study across languages, time spans, as well as large datasets, we apply it to the entirety of five historical newspaper archives in Dutch, Swedish, Finnish, and English. To our knowledge, thus far there have been no large-scale comparative studies of this kind that purport to grasp long-term developments in as many as four different languages in a data-driven way. A particular strength of the method we describe in this article is that, by design, it is not limited to the study of nationhood, but rather expands beyond it to other research questions and is reusable in different contexts.

DUKweb, diachronic word representations from the UK Web Archive corpus

Lexical semantic change (detecting shifts in the meaning and usage of words) is an important task for social and cultural studies as well as for Natural Language Processing applications. Diachronic word embeddings (time-sensitive vector representations of words that preserve their meaning) have become the standard resource for this task. However, given the significant computational resources needed for their generation, very few resources exist that make diachronic word embeddings available to the scientific community. In this paper we present DUKweb, a set of large-scale resources designed for the diachronic analysis of contemporary English. DUKweb was created from the JISC UK Web Domain Dataset (1996–2013), a very large archive which collects resources from the Internet Archive that were hosted on domains ending in ‘.uk’. DUKweb consists of a series word co-occurrence matrices and two types of word embeddings for each year in the JISC UK Web Domain dataset. We show the reuse potential of DUKweb and its quality standards via a case study on word meaning change detection.

Detection of emerging drugs involved in overdose via diachronic word embeddings of substances discussed on social media

Substances involved in overdose deaths have shifted over time and continue to undergo transition. Early detection of emerging drugs involved in overdose is a major challenge for traditional public health data systems. While novel social media data have shown promise, there is a continued need for robust natural language processing approaches that can identify emerging substances. Consequently, we developed a new metric, the relative similarity ratio, based on diachronic word embeddings to measure movement in the semantic proximity of individual substance words to ‘overdose’ over time. Our analysis of 64,420,376 drug-related posts made between January 2011 and December 2018 on Reddit, the largest online forum site, reveals that this approach successfully identified fentanyl, the most significant emerging substance in the overdose epidemic, >1 year earlier than traditional public health data systems. Use of diachronic word embeddings may enable improved identification of emerging substances involved in drug overdose, thereby improving the timeliness of prevention and treatment activities.

A Collection of Swedish Diachronic Word Embedding Models Trained on Historical Newspaper Data

This paper describes the creation of several word embedding models based on a large collection of diachronic Swedish newspaper material available through Språkbanken Text, the Swedish language bank. This data was produced in the context of Språkbanken Text’s continued mission to collaborate with humanities and natural language processing (NLP) researchers and to provide freely available language resources, for the development of state-of-the-art NLP methods and tools.

Abolitionist Networks: Modeling Language Change in Nineteenth-Century Activist Newspapers

The abolitionist movement of the nineteenth-century United States remains among the most significant social and political movements in US history. Abolitionist newspapers played a crucial role in spreading information and shaping public opinion around a range of issues relating to the abolition of slavery. These newspapers also serve as a primary source of information about the movement for scholars today, resulting in powerful new accounts of the movement and its leaders. This paper supplements recent qualitative work on the role of women in abolition’s vanguard, as well as the role of the Black press, with a quantitative text modeling approach. Using diachronic word embeddings, we identify which newspapers tended to lead lexical semantic innovations — the introduction of new usages of specific words — and which newspapers tended to follow. We then aggregate the evidence across hundreds of changes into a weighted network with the newspapers as nodes; directed edge weights represent the frequency with which each newspaper led the other in the adoption of a lexical semantic change. Analysis of this network reveals pathways of lexical semantic influence, distinguishing leaders from followers, as well as others who stood apart from the semantic changes that swept through this period. More specifically, we find that two newspapers edited by women – The Provincial Freeman and The Lily – led a large number of semantic changes in our corpus, lending additional credence to the argument that a multiracial coalition of women led the abolitionist movement in terms of both thought and action. It also contributes additional complexity to the scholarship that has sought to tease apart the relation of the abolitionist movement to the women’s suffrage movement, and the vexed racial politics that characterized their relation.

The expansion of isms, 1820-1917: Data-driven analysis of political language in digitized newspaper collections

Words with the suffix-ism are reductionist terms that help us navigate complex social issues by using a simple one-word label for them. On the one hand they are often associated with political ideologies, but on the other they are present in many other domains of language, especially culture, science, and religion. This has not always been the case. This paper studies isms in a historical record of digitized newspapers from 1820 to 1917 published in Finland to find out how the language of isms developed historically. We use diachronic word embeddings and affinity propagation clustering to trace how new isms entered the lexicon and how they relate to one another over time. We are able to show how they became more common and entered more and more domains. Still, the uses of isms as traditions for political action and thinking stand out in our analysis.

A collection of Swedish diachronic word embedding models trained on historical newspaper data

A collection of Swedish diachronic word embedding models trained on historical newspaper data

A collection of Swedish diachronic word embedding models trained on historical newspaper data

A collection of Swedish diachronic word embedding models trained on historical newspaper data

Follow the leader: Documents on the leading edge of semantic change get more citations

AbstractDiachronic word embeddings—vector representations of words over time—offer remarkable insights into the evolution of language and provide a tool for quantifying sociocultural change from text documents. Prior work has used such embeddings to identify shifts in the meaning of individual words. However, simply knowing that a word has changed in meaning is insufficient to identify the instances of word usage that convey the historical meaning or the newer meaning. In this study, we link diachronic word embeddings to documents, by situating those documents as leaders or laggards with respect to ongoing semantic changes. Specifically, we propose a novel method to quantify the degree of semantic progressiveness in each word usage, and then show how these usages can be aggregated to obtain scores for each document. We analyze two large collections of documents, representing legal opinions and scientific articles. Documents that are scored as semantically progressive receive a larger number of citations, indicating that they are especially influential. Our work thus provides a new technique for identifying lexical semantic leaders and demonstrates a new link between progressive use of language and influence in a citation network.

Patterns and niches in diachronic word formation: the fate of the suffix -men from Latin to Romance

The Latin suffix -men of words like foramen, nutrimen, albumen, etc. has left traces in all Romance idioms, but the outcome has been quite different in each of them. The present paper describes in some detail the distribution and the meaning of the successors of -men in each of the Romance idioms and tries to uncover the mechanisms behind the fragmentation process. Some of the changes already took place in Latin, while others occurred in Romance times. It is argued that in the absence of lexical continuities between Latin and Romance a close attention to word-formation patterns and semantic niches, as well as a close comparison of similar developments in Romance idioms can be helpful in reconstruction.

A state-of-the-art of semantic change computation

AbstractThis paper reviews the state-of-the-art of one emergent field in computational linguistics—semantic change computation. It summarizes the literature by proposing a framework that identifies five components in the field: diachronic corpus, diachronic word sense characterization, change modelling, evaluation and data visualization. Despite its potentials, the review shows that current studies are mainly focused on testifying hypotheses of semantic change from theoretical linguistics and that several core issues remain to be tackled: the need of diachronic corpora for languages other than English, the comparison and development of approaches to diachronic word sense characterization and change modelling, the need of comprehensive evaluation data and further exploration of data visualization techniques for hypothesis justification.

How does language change as a lexical network? An investigation based on written Chinese word co-occurrence networks.

Language is a complex adaptive system, but how does it change? For investigating this process, four diachronic Chinese word co-occurrence networks have been built based on texts that were written during the last 2,000 years. By comparing the network indicators that are associated with the hierarchical features in language networks, we learn that the hierarchy of Chinese lexical networks has indeed evolved over time at three different levels. The connections of words at the micro level are continually weakening; the number of words in the meso-level communities has increased significantly; and the network is expanding at the macro level. This means that more and more words tend to be connected to medium-central words and form different communities. Meanwhile, fewer high-central words link these communities into a highly efficient small-world network. Understanding this process may be crucial for understanding the increasing structural complexity of the language system.