A New Approach for Explicit Distributional Semantic Vector Representation Based on Dependency Distance and Bias Elimination

Abstract

We consider the problem of the semantic representation of words based on explicit basis word vectors. We focus on explicit semantic vectors in which each basis vector is equivalent to a unique word. In this paper, we propose a new approach to represent semantic vectors explicitly. The approach consists of three steps: First, we introduce two new factors called word similarity (WS) and the number of zeroes (NZ) for choosing informative context words. Second, we propose a dependency distance method by incorporating the syntactic relations of the words in the sentences. This means that the method uses not only semantic information but also syntactic one. Third, we propose two shifting methods based on WF and WS factors to eliminate the bias of the PPMI matrix. In this work, we use the ukWaC corpus to select context words and compute co-occurrence matrices. Then, we examine the results on MEN, RG-65, SimLex-999, and WordSim353 test sets. We report the performance in comparison with a Baseline in which, the word frequency has been used. The Baseline method uses a window to fix the influence of the neighboring words. Our approach improves the Spearman correlation coefficient for MEN, RG-65, SimLex-999, and WordSim353 datasets by 3.87%, 10.25%, 5.04%, and 5.24%, respectively. So, the experiment results show that the proposed context words selection, incorporating dependency distance, and using the WF-based shifting method significantly improve the performance.