Contextual Adversarial Representation Learning for Multiword Expressions

Abstract

Word composition is a promising method to learn the representations of long text. Unfortunately, word composition falls short when inferring representations for non-compositional multiword expressions, such as “go banana.” Presently, many methods treat multiword expressions as single words and learn their representations in a manner similar to individual word representations. However, numerous multiword expressions exhibit ambiguity, expressing distinct meanings, whether literal or idiomatic, depending on the context. In response to these challenges, our paper proposes an adversarial context-aware representation learning method for multiword expressions, which generates representations based on the contexts of their occurrences. An adversarial training framework is introduced to further enhance the representation learning method. Experimental results confirm the benefits of sense disambiguation for multiword expressions in representation learning. Moreover, our proposed method demonstrates competitive performance on idiom token classification and compositionality prediction tasks.