Abstract
Composition models of distributional semantics are used to construct phrase representations from the representations of their words. Composition models are typically situated on two ends of a spectrum. They either have a small number of parameters but compose all phrases in the same way, or they perform word-specific compositions at the cost of a far larger number of parameters. In this paper we propose transformation weighting (TransWeight), a composition model that consistently outperforms existing models on nominal compounds, adjective-noun phrases, and adverb-adjective phrases in English, German, and Dutch. TransWeight drastically reduces the number of parameters needed compared with the best model in the literature by composing similar words in the same way.
Highlights
The phrases black car and purple car are very similar—except for their frequency
A composition model is a function f that combines the vectors of individual words, for example, u for black and v for car into a phrase representation p. p is the result of applying the composition function f to the word vectors u, v: p = f (u, v)
It performs on par with the TransWeightmat variation, the latter has a larger number of parameters (20,200 in our setup)
Summary
The phrases black car and purple car are very similar—except for their frequency. Distributional word representations derived from large, unannotated corpora (Collobert et al, 2011; Mikolov et al, 2013; Pennington et al, 2014) capture information about individual words like purple and car and are able to express, in vector space, different types of word similarity (the similarity between color adjectives like black and purple, between car and truck, etc.). In particular representing low-frequency phrases like purple car, is a task for composition models of distributional semantics. A composition model is a function f that combines the vectors of individual words, for example, u for black and v for car into a phrase representation p.
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: Transactions of the Association for Computational Linguistics
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
