Knowledge-enhanced temporal word embedding for diachronic semantic change estimation

Abstract

Historical, social and linguistic factors cause semantic changes that can narrow, broaden or completely alter the meanings of words. Frequency, syntactic and semantic variations are to be studied to examine such changes. Syntactic changes cannot be observed in many cases, if words have no POS variation. Context and connotation contribute more to semantic alteration. In addition, words have similar or related meanings in certain contexts, and the context is considered with diverse features such as co-occurrence and word association. The semantic change is generally related to the variation in n-gram context with a maximum of 5 g. However, distant context terms also play a prominent role in semantic change. There is also a link between the type of change and the use of lexical relations. This paper builds a knowledge-enhanced temporal word embedding model that utilizes ‘word-centric dependency relations’ for capturing context words irrespective of their n-gram position and ‘syntactic patterns for lexical relations’ for determining the type of semantic change. The joint learning of contexts with both dependency and lexical relations from diachronic corpora is performed to obtain temporal word embedding vectors. The proposed model outperforms other n-gram-based approaches when evaluated with standard diachronic corpora.