Semantic Drift Research Articles

Coming To: Consciousness and Natality in Early Modern England

Coming To: Consciousness and Natality in Early Modern England

Multi-Hop Question Generation Using Hierarchical Encoding-Decoding and Context Switch Mechanism.

Neural auto-regressive sequence-to-sequence models have been dominant in text generation tasks, especially the question generation task. However, neural generation models suffer from the global and local semantic semantic drift problems. Hence, we propose the hierarchical encoding–decoding mechanism that aims at encoding rich structure information of the input passages and reducing the variance in the decoding phase. In the encoder, we hierarchically encode the input passages according to its structure at four granularity-levels: [word, chunk, sentence, document]-level. Second, we progressively select the context vector from the document-level representations to the word-level representations at each decoding time step. At each time-step in the decoding phase, we progressively select the context vector from the document-level representations to word-level. We also propose the context switch mechanism that enables the decoder to use the context vector from the last step when generating the current word at each time-step.It provides a means of improving the stability of the text generation process during the decoding phase when generating a set of consecutive words. Additionally, we inject syntactic parsing knowledge to enrich the word representations. Experimental results show that our proposed model substantially improves the performance and outperforms previous baselines according to both automatic and human evaluation. Besides, we implement a deep and comprehensive analysis of generated questions based on their types.

A semantic approach to post-retrieval query performance prediction

The importance of query performance prediction has been widely acknowledged in the literature, especially for query expansion, refinement, and interpolating different retrieval approaches. This paper proposes a novel semantics-based query performance prediction approach based on estimating semantic similarities between queries and documents. We introduce three post-retrieval predictors, namely (1) semantic distinction, (2) semantic query drift, and (3) semantic cohesion based on (1) the semantic similarity of a query to the top-ranked documents compared to the whole collection, (2) the estimation of non-query related aspects of the retrieved documents using semantic measures, and (3) the semantic cohesion of the retrieved documents. We assume that queries and documents are modeled as sets of entities from a knowledge graph, e.g., DBPedia concepts, instead of bags of words. With this assumption, semantic similarities between two texts are measured based on the relatedness between entities, which are learned from the contextual information represented in the knowledge graph. We empirically illustrate these predictors’ effectiveness, especially when term-based measures fail to quantify query performance prediction hypotheses correctly. We report our findings on the proposed predictors’ performance and their interpolation on three standard collections, namely ClueWeb09-B, ClueWeb12-B, and Robust04. We show that the proposed predictors are effective across different datasets in terms of Pearson and Kendall correlation coefficients between the predicted performance and the average precision measured by relevance judgments.

Promoting a standardized description of fossil tracheidoxyls

The paleoxylology of coniferous woods has progressed considerably since Kräusel's last (incomplete) synthesis in 1949. Nomenclatural practice has slowly purged itself of the diagnostic use of etymology. The use of certain new words or phrases has become essential, while other words have been subject to significant semantic drift or unjustified omission. The interest of secondary characters was highlighted. Ways of describing have also evolved, sometimes in divergent ways. The list of characters (and states) published by the International Association of Wood Anatomists in 2004 represents a fundamental step towards a standardization of practices. However, it does not take into account some of the particularities of palaeoxylology. Here we propose: (1) a plea that etymology should definitely no longer be considered diagnostic; (2) a standardized format for the description of fossil tracheidoxyls; (3) an illustrated glossary of certain terms specific to paleoxylology.

Bayesian compression for dynamically expandable networks

This paper develops Bayesian Compression for Dynamically Expandable Network (BCDEN), which can learn a compact model structure with preserving the accuracy in a continual learning scenarios. Dynamically Expandable Network (DEN) is efficiently trained by performing selective retraining, dynamically expands network capacity with only the necessary number of units, and effectively prevents semantic drift by duplicating and timestamping units in an online manner. Overcoming conventional DEN only giving point estimates, we providing the Bayesian inference under the principle framework. We validate our BCDEN on multiple public datasets under continual learning setting, on which it can outperform existing continual learning methods on a variety of tasks, and with the state-of-the-art compression results, while still maintaining comparable performance.

Effective Slot Filling via Weakly-Supervised Dual-Model Learning

Slot filling is a challenging task in Spoken Language Understanding (SLU). Supervised methods usually require large amounts of annotation to maintain desirable performance. A solution to relieve the heavy dependency on labeled data is to employ bootstrapping, which leverages unlabeled data. However, bootstrapping is known to suffer from semantic drift. We argue that semantic drift can be tackled by exploiting the correlation between slot values (phrases) and their respective types. By using some particular weakly labeled data, namely the plain phrases included in sentences, we propose a weakly-supervised slot filling approach. Our approach trains two models, namely a classifier and a tagger, which can effectively learn from each other on the weakly labeled data. The experimental results demonstrate that our approach achieves better results than standard baselines on multiple datasets, especially in the low-resource setting.

Covid-19 knowledge graphs in Health Communication and Information

This commentary discusses recent developments in ‘knowledge graph’ technology over the course of the Covid-19 pandemic. Recently experiencing a surge in popularity, knowledge graphs are technologies that assist with data integration through structured metadata modeling. Researchers tag and collate vast amounts of diverse data using knowledge graphs, yet problems related to semantic drift and more salient issues related to the political economy of information and communication technologies persist. Researchers should anticipate that the semantics of these Covid-19 knowledge graphs can change over time. Equally important, researchers should also consider all stakeholders involved, including those stakeholders that might be excluded.

Quantifying people's experience during flood events with implications for hazard risk communication.

Semantic drift is a well-known concept in distributional semantics, which is used to demonstrate gradual, long-term changes in meanings and sentiments of words and is largely detectable by studying the composition of large corpora. In our previous work, which used ontological relationships between words and phrases, we established that certain kinds of semantic micro-changes can be found in social media emerging around natural hazard events, such as floods. Our previous results confirmed that semantic drift in social media can be used to for early detection of floods and to increase the volume of ‘useful’ geo-referenced data for event monitoring. In this work we use deep learning in order to determine whether images associated with ‘semantically drifted’ social media tags reflect changes in crowd navigation strategies during floods. Our results show that alternative tags can be used to differentiate naïve and experienced crowds witnessing flooding of various degrees of severity.

Semantic Drift in Multilingual Representations

Multilingual representations have mostly been evaluated based on their performance on specific tasks. In this article, we look beyond engineering goals and analyze the relations between languages in computational representations. We introduce a methodology for comparing languages based on their organization of semantic concepts. We propose to conduct an adapted version of representational similarity analysis of a selected set of concepts in computational multilingual representations. Using this analysis method, we can reconstruct a phylogenetic tree that closely resembles those assumed by linguistic experts. These results indicate that multilingual distributional representations that are only trained on monolingual text and bilingual dictionaries preserve relations between languages without the need for any etymological information. In addition, we propose a measure to identify semantic drift between language families. We perform experiments on word-based and sentence-based multilingual models and provide both quantitative results and qualitative examples. Analyses of semantic drift in multilingual representations can serve two purposes: They can indicate unwanted characteristics of the computational models and they provide a quantitative means to study linguistic phenomena across languages.

Dual-View Semantic Inference Network for image-text matching

Recently, image-text matching based on local region-word semantic alignment has attracted considerable research attention. The fine-grained interplay can be achieved by aggregating the similarities of the region-word pairs. However, the similarities of aligned region-word pairs are treated equally in most cross-modal matching literatures, without considering their respective importance. Moreover, the local alignment methods are prone to bring about a global semantic drift due to the ignorance of thematic considerations for the image-text pairs. In this paper, a novel Dual-View Semantic Inference (DVSI) network is proposed to leverage both local and global semantic matching in a holistic deep framework. For the local view, a region enhancement module is proposed to mine the priorities for different regions in the image, which provides differentiate abilities to discover the latent region-word relationships. For the global view, the overall semantics of image is summarized for global semantic matching to avoid global semantic drift. The two views are unified together for final image-text matching. Extensive experiments conducted on MSCOCO and Flicr30K demonstrate the effectiveness of the proposed DVSI.

A brief history of risk

Despite increasing life expectancy and high levels of welfare, health care, and public safety in most post-industrial countries, the public discourse often revolves around perceived threats. Terrorism, global pandemics, and environmental catastrophes are just a few of the risks that dominate media coverage. Is this public discourse on risk disconnected from reality? To examine this issue, we analyzed the dynamics of the risk discourse in two natural language text corpora. Specifically, we tracked latent semantic patterns over a period of 150 years to address four questions: First, we examined how the frequency of the word risk has changed over historical time. Is the construct of risk playing an ever-increasing role in the public discourse, as the sociological notion of a ‘risk society’ suggests? Second, we investigated how the sentiments for the words co-occurring with risk have changed. Are the connotations of risk becoming increasingly ominous? Third, how has the meaning of risk changed relative to close associates such as danger and hazard? Is risk more subject to semantic change? Finally, we decompose the construct of risk into the specific topics with which it has been associated and track those topics over historical time. This brief history of the semantics of risk reveals new and surprising insights—a fourfold increase in frequency, increasingly negative sentiment, a semantic drift toward forecasting and prevention, and a shift away from war toward chronic disease—reflecting the conceptual evolution of risk in the archeological records of public discourse.

End-to-End Bootstrapping Neural Network for Entity Set Expansion

Bootstrapping for entity set expansion (ESE) has long been modeled as a multi-step pipelined process. Such a paradigm, unfortunately, often suffers from two main challenges: 1) the entities are expanded in multiple separate steps, which tends to introduce noisy entities and results in the semantic drift problem; 2) it is hard to exploit the high-order entity-pattern relations for entity set expansion. In this paper, we propose an end-to-end bootstrapping neural network for entity set expansion, named BootstrapNet, which models the bootstrapping in an encoder-decoder architecture. In the encoding stage, a graph attention network is used to capture both the first- and the high-order relations between entities and patterns, and encode useful information into their representations. In the decoding stage, the entities are sequentially expanded through a recurrent neural network, which outputs entities at each stage, and its hidden state vectors, representing the target category, are updated at each expansion step. Experimental results demonstrate substantial improvement of our model over previous ESE approaches.

Evolving graphs with semantic neutral drift

We introduce the concept of Semantic Neutral Drift (SND) for genetic programming (GP), where we exploit equivalence laws to design semantics preserving mutations guaranteed to preserve individuals’ fitness scores. A number of digital circuit benchmark problems have been implemented with rule-based graph programs and empirically evaluated, demonstrating quantitative improvements in evolutionary performance. Analysis reveals that the benefits of the designed SND reside in more complex processes than simple growth of individuals, and that there are circumstances where it is beneficial to choose otherwise detrimental parameters for a GP system if that facilitates the inclusion of SND.

Efficient Computational linguistics Framework for Concept Drift Detection

Semantic drift is a common problem in iterative information extraction. Unsupervised bagging and incorporated distributional similarity is used to reduce the difficulty of semantic drift in iterative bootstrapping algorithms, particularly when extracting large semantic lexicons. Compared to previous approaches which usually incur substantial loss in recall, DP-based cleaning method can effectively clean a large proportion of semantic drift errors while keeping a high recall.

‘How well you read me, you witch!’: semantic drift of ‘witch’ and the choice of Jane Eyre

One signifier whose meaning has drifted since Jane Eyre’s time is ‘witch’, a word Rochester applies to Jane during his courtship. Witch was once a label of considerable darkness, casting shadows of terror over the original audience of Jane Eyre, whereas for modern readers it is less threatening. Through a feminist and new historicist lens, ‘witch’ deconstructs to reveal vital aspects about the relationship between Jane Eyre and Edward Rochester, and provides clues to her eventual choice of Rochester over St John. Charlotte Brontë’s juvenilia (specifically ‘The Fairy Gift’), Sir Walter Scott’s novels and his Letters on Demonology and Witchcraft, and the witch-hunters’ manual, Malleus Maleficarum, are all mined for clues to discern a meaning lost to modern readers by the evolution of English and the march of time.

The Macroscope: A tool for examining the historical structure of language

The recent rise in digitized historical text has made it possible to quantitatively study our psychological past. This involves understanding changes in what words meant, how words were used, and how these changes may have responded to changes in the environment, such as in healthcare, wealth disparity, and war. Here we make available a tool, the Macroscope, for studying historical changes in language over the last two centuries. The Macroscope uses over 155 billion words of historical text, which will grow as we include new historical corpora, and derives word properties from frequency-of-usage and co-occurrence patterns over time. Using co-occurrence patterns, the Macroscope can track changes in semantics, allowing researchers to identify semantically stable and unstable words in historical text and providing quantitative information about changes in a word’s valence, arousal, and concreteness, as well as information about new properties, such as semantic drift. The Macroscope provides information about both the local and global properties of words, as well as information about how these properties change over time, allowing researchers to visualize and download data in order to make inferences about historical psychology. Although quantitative historical psychology represents a largely new field of study, we see this work as complementing a wealth of other historical investigations, offering new insights and new approaches to understanding existing theory. The Macroscope is available online at http://www.macroscope.tech.

Comprehensive Review on Effectual Information Retrieval of Semantic Drift using Deep Neural Network

Semantic drift is a common problem in iterative information extraction. Unsupervised bagging and incorporated distributional similarity is used to reduce the difficulty of semantic drift in iterative bootstrapping algorithms, particularly when extracting large semantic lexicons. In this research work, a method to minimize semantic drift by identifying the (Drifting Points) DPs and removing the effect introduced by the DPs is proposed. Previous methods for identifying drifting errors can be roughly divided into two categories: (1) multi-class based, and (2) single-class based, according to the settings of Information Extraction systems that adopt them. Compared to previous approaches which usually incur substantial loss in recall, DP-based cleaning method can effectively clean a large proportion of semantic drift errors while keeping a high recall.

An automatic scoring method for Chinese-English spoken translation based on attention LSTM

In this paper, we propose an automatic scoring method for Chinese-English spoken translation based on attention LSTM. We select semantic keywords, sentence drift and spoken fluency as the main parameters of scoring. In order to improve the accuracy of keyword scoring, this paper uses synonym discr

SemaDrift: A hybrid method and visual tools to measure semantic drift in ontologies

Semantic drift is an active field of research, aiming to identify and measure changes in ontologies across versions in time, closely related to several fields such as ontology evolution and versioning. However, practical and widely adopted methods for measuring semantic drift are mostly either not directly applied to Semantic Web formalisms or largely dependent on specific models and domains. This paper presents a novel hybrid method, which combines existing identity-based and morphing-based approaches, and, in turn, introduces further insights such as hybrid chains of concepts across ontology versions and concept stability ranking. Furthermore, it introduces the SemaDrift Application Suite, which integrates both new and existing methods, including structural and text similarity measures, provided as open-source. The applications offer a graphical user interface to calculate and visually explore semantic drift metrics, in order to support and promote their use to Semantic Web and domain experts alike. Specifically, the SemaDrift plugin for the popular Protègè platform is intended for ontology engineers, while SemaDriftFx allows a wider audience of users to visually investigate drift either as numeric output or visually through graphs. Two use case scenarios demonstrate the applicability and usefulness of the methods and tools in the domain of digital preservation and Web Services, with insights previously hard to obtain. Standard scale and concrete end-user evaluation tasks were used to acquire positive preliminary feedback on the usability of the SemaDriftFx tool.

Diagnosing and Minimizing Semantic Drift in Iterative Bootstrapping Extraction

Semantic drift is a common problem in iterative information extraction. Previous approaches for minimizing semantic drift may incur substantial loss in recall. We observe that most semantic drifts are introduced by a small number of questionable extractions in the earlier rounds of iterations. These extractions subsequently introduce a large number of questionable results, which lead to the semantic drift phenomenon. We call these questionable extractions Drifting Points (DPs). If erroneous extractions are the “symptoms” of semantic drift, then DPs are the “causes” of semantic drift. In this paper, we propose a method to minimize semantic drift by identifying the DPs and removing the effect introduced by the DPs. We use isA (concept-instance) extraction as an example to describe our approach in cleaning information extraction errors caused by semantic drift, but we perform experiments on different relation extraction processes on three large real data extraction collections. The experimental results show that our DP cleaning method enables us to clean around 90 percent incorrect instances or patterns with about 90 percent precision, which outperforms the previous approaches we compare with.