Abstract
This paper describes SChME (Semantic Change Detection with Model Ensemble), a method used in SemEval-2020 Task 1 on unsupervised detection of lexical semantic change. SChME uses a model ensemble combining signals distributional models (word embeddings) and word frequency where each model casts a vote indicating the probability that a word suffered semantic change according to that feature. More specifically, we combine cosine distance of word vectors combined with a neighborhood-based metric we named Mapped Neighborhood Distance (MAP), and a word frequency differential metric as input signals to our model. Additionally, we explore alignment-based methods to investigate the importance of the landmarks used in this process. Our results show evidence that the number of landmarks used for alignment has a direct impact on the predictive performance of the model. Moreover, we show that languages that suffer less semantic change tend to benefit from using a large number of landmarks, whereas languages with more semantic change benefit from a more careful choice of landmark number for alignment.
Highlights
The problem of detecting Lexical Semantic Change (LSC) consists of measuring and identifying change in word sense across time, such as in the study of language evolution, or across domains, such as determining discrepancies in word usage over specific communities (Schlechtweg et al, 2019)
In this paper we describe a novel model ensemble method based on different features that we can extract from the text using distribution models and word frequency
We align the learned word vectors via orthogonal procrustes (OP) using the intersecting vocabulary as landmarks
Summary
The problem of detecting Lexical Semantic Change (LSC) consists of measuring and identifying change in word sense across time, such as in the study of language evolution, or across domains, such as determining discrepancies in word usage over specific communities (Schlechtweg et al, 2019). The vast majority of the related work in the literature pursue this problem from an unsupervised perspective, that is, detecting semantic change without having prior knowledge of “truth” The importance of such task is manifold: to humans, it can be a powerful tool for studying language change and its cultural implications; to machines, it can be used to improve language models in downstream tasks such as unsupervised word translation, and fine-tuning of word embeddings (Joulin et al, 2018; Bojanowski et al, 2019). The code for the model can be obtained at https://github.com/mgruppi/schme
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have