Transitivity of transformation matrices to bridge word vector spaces over 1000 years

Abstract

We proposed a synonym search method to solve $$(A,B)\sim (C,D)$$ problem over time with a query by an example in a known domain for information in an unknown domain. It seems a natural relation that “Bush in the 2000s” is similar to “Reagan in the 1980s” because “Bush” and “Reagan” are the president of the USA in these decades. The abstraction is “A in B ” which is similar to “C in D.” We solve the $$(A,B)\sim (C,D)$$ problem over time by using transformation matrix word vectors over time in the Skip-gram model. The example of the $$(A,B)\sim (C,D)$$ problem is as below. For instance, the sentence “Bush in the 2000s” is similar to “X in the 1980s” is given. We search for the appropriate entity to X of the sentence. Therefore, we focus on the transitivity between the transformation matrix. Our approach is to convert the vector representation of “A ” in the model of the word embedding model of the “B ” to the vector representation of “X ” in the model of the word embedding model of the “D ” by getting the transformation matrix between word embedding models. We discuss the parameters of previous work and improve choosing words to make transformation matrix using co-occurrence cluster. Our aim is to search for synonyms in which there are more than the 1000 years of separation. However, there are a few common stable meaning words between the “2000s” and the “1000s.” Therefore, in the situation, there is difficulty to use co-occurrence cluster because the clusters of common words are more than 100,000. That is why, we use the transitive relation (2000s, X) $$\sim $$ (1500s, Y) , (1500s, Y) $$\sim $$ (1000s, Z) $$\Rightarrow $$ (2000s, X) $$\sim $$ (1000s, Z) (in the abstruction, the transitivity is $$x\sim y, y\sim z\Rightarrow x\sim z$$ ) to solve the $$(A,B)\sim (C,D)$$ problem over the 1000 years of separation. We had experiments as the demonstration to solve the $$(A,B)\sim (C,D)$$ problem and evaluate nDCG and MRR.