Adversarial Context-Aware Representation Learning of Multiword Expressions

Abstract

Word composition is a promising method to learn the representations of long text. Unfortunately, the representations of non-compositional multiword expressions (e.g., go banana) can not be inferred by word composition. Most current methods regard a multiword expression as a single word, and learn its representation in the same way as word representations. However, many multiword expressions are ambiguous, that they express distinct meanings (literal or idiomatic) in different contexts. To resolve this problem, this paper proposes an adversarial context-aware representation learning method for multiword expressions, which generates representations based on the contexts of their occurrences. An adversarial training framework is introduced for further enhancing the representation learning method. The experimental results verify the beneficial of sense disambiguation of multiword expression for representations learning, and the proposed method achieved competitive performances on both the idiom token classification and compositionality prediction tasks.