Abstract

This paper describes SChME (Semantic Change Detection with Model Ensemble), a method used in SemEval-2020 Task 1 on unsupervised detection of lexical semantic change. SChME uses a model ensemble combining signals distributional models (word embeddings) and word frequency where each model casts a vote indicating the probability that a word suffered semantic change according to that feature. More specifically, we combine cosine distance of word vectors combined with a neighborhood-based metric we named Mapped Neighborhood Distance (MAP), and a word frequency differential metric as input signals to our model. Additionally, we explore alignment-based methods to investigate the importance of the landmarks used in this process. Our results show evidence that the number of landmarks used for alignment has a direct impact on the predictive performance of the model. Moreover, we show that languages that suffer less semantic change tend to benefit from using a large number of landmarks, whereas languages with more semantic change benefit from a more careful choice of landmark number for alignment.

Highlights

  • The problem of detecting Lexical Semantic Change (LSC) consists of measuring and identifying change in word sense across time, such as in the study of language evolution, or across domains, such as determining discrepancies in word usage over specific communities (Schlechtweg et al, 2019)

  • In this paper we describe a novel model ensemble method based on different features that we can extract from the text using distribution models and word frequency

  • We align the learned word vectors via orthogonal procrustes (OP) using the intersecting vocabulary as landmarks

Read more

Summary

Introduction

The problem of detecting Lexical Semantic Change (LSC) consists of measuring and identifying change in word sense across time, such as in the study of language evolution, or across domains, such as determining discrepancies in word usage over specific communities (Schlechtweg et al, 2019). The vast majority of the related work in the literature pursue this problem from an unsupervised perspective, that is, detecting semantic change without having prior knowledge of “truth” The importance of such task is manifold: to humans, it can be a powerful tool for studying language change and its cultural implications; to machines, it can be used to improve language models in downstream tasks such as unsupervised word translation, and fine-tuning of word embeddings (Joulin et al, 2018; Bojanowski et al, 2019). The code for the model can be obtained at https://github.com/mgruppi/schme

Related Work
Model Overview and Data
Word Representations
Distance Measures
Model Ensemble
Evaluation
Post-Evaluation
Landmarks Are Important
Conclusions
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call