Abstract

Recent developments in deep learning have prompted a surge of interest in the application of multitask and transfer learning to NLP problems. In this study, we explore for the first time, the application of transfer learning (TRL) and multitask learning (MTL) to the identification of Multiword Expressions (MWEs). For MTL, we exploit the shared syntactic information between MWE and dependency parsing models to jointly train a single model on both tasks. We specifically predict two types of labels: MWE and dependency parse. Our neural MTL architecture utilises the supervision of dependency parsing in lower layers and predicts MWE tags in upper layers. In the TRL scenario, we overcome the scarcity of data by learning a model on a larger MWE dataset and transferring the knowledge to a resource-poor setting in another language. In both scenarios, the resulting models achieved higher performance compared to standard neural approaches.

Highlights

  • Multiword Expressions (MWEs) are combinations of two or more lexical components that form non/semi-compositional meaning units

  • The contributions of this work are: 1) we propose a neural model that improves MWE identification by jointly learning MWE and dependency parse labels; 2) We show that MWE identification models, when multitasked with dependency parsing, outperform the models which naively add dependency parse information as additional features; 3) we propose, to the best of our knowledge for the first time, a cross-lingual transfer learning method for processing MWEs, making a contribution towards the study of low-resource languages

  • We experiment with the multilingual dataset from the PARSEME project (Savary et al, 2018) which was made available for the shared task on identification of verbal MWEs (Ramisch et al, 2018)

Read more

Summary

Introduction

Multiword Expressions (MWEs) are combinations of two or more lexical components that form non/semi-compositional meaning units. In order to deal with data scarcity in the English dataset, in another setting we train our model on a language with a larger data and transfer the learned knowledge for predicting MWE tags in English. In this study we build upon recent neural network systems that have proved to be successful in representing syntactic and semantic features of text and design novel multitask and transfer learning architectures for MWE identification. The contributions of this work are: 1) we propose a neural model that improves MWE identification by jointly learning MWE and dependency parse labels; 2) We show that MWE identification models, when multitasked with dependency parsing, outperform the models which naively add dependency parse information as additional features; 3) we propose, to the best of our knowledge for the first time, a cross-lingual transfer learning method for processing MWEs, making a contribution towards the study of low-resource languages

Related Work
Methodology
Multitask Learning
Transfer Learning
Experiments
Evaluation
Results
The Effect of Learning Rate in TRL
Conclusions and Future Work
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call