Abstract
Categorical compositional distributional models unify compositional formal semantic models and distributional models by composing phrases with tensor-based methods from vector representations. For the tensor-based compositions, Milajevs et al. (2014) showed that word vectors obtained from the continuous bag-of-words (CBOW) model are competitive with those from co-occurrence based models. However, because word vectors from the CBOW model are trained assuming additive interactions between context words, the word composition used for the training mismatches to the tensor-based methods used for evaluating the actual compositions including pointwise multiplication and tensor product of context vectors. In this work, we show whether the word embeddings from extended CBOW models using multiplication or tensor product between context words, reflecting the actual composition methods, can show better performance than those from the baseline CBOW model in actual tasks of compositions with multiplication or tensor-based methods.
Highlights
In recent years, there has been a surge of interest in using word vectors for modeling semantics. Mikolov et al (2013a,b) introduced word2vec that includes the continuous bag-of-words (CBOW) model and the skip-gram model.1 These models have been most widely used for generating word vectors to be used for word related tasks because of1https://code.google.com/p/word2vec their efficient but still effective architectures
Milajevs et al (2014) showed that the word vectors generated from the CBOW model are competitive with those from co-occurrence based models for both simple arithmetic compositions and tensorbased compositions for categorical compositional distributional models (Coecke et al, 2010)
For four datasets, evaluating different types of compositions, we show that those extensions of the CBOW model improve the performance of the actual composition tasks with multiplication or tensor product operations
Summary
There has been a surge of interest in using word vectors for modeling semantics. Mikolov et al (2013a,b) introduced word2vec that includes the continuous bag-of-words (CBOW) model and the skip-gram model. These models have been most widely used for generating word vectors to be used for word related tasks because of. Mikolov et al (2013a,b) introduced word2vec that includes the continuous bag-of-words (CBOW) model and the skip-gram model.1 These models have been most widely used for generating word vectors to be used for word related tasks because of. Most tensor-based compositions use point-wise multiplication or tensor product as composition operators This means that there is a mismatch between the composition method used for the training of the underlying word vectors and the actual composition methods we evaluate. We introduce extensions of the CBOW model with multiplicative interactions between word projections to obtain word embeddings more suitable for the tensor-based compositions. For four datasets, evaluating different types of compositions, we show that those extensions of the CBOW model improve the performance of the actual composition tasks with multiplication or tensor product operations
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
