Multi-Task Learning for Semantic Relatedness and Textual Entailment

Dan Moldovan,Linrui Zhang

doi:10.4236/jsea.2019.126012

Abstract

Recently, several deep learning models have been successfully proposed and have been applied to solve different Natural Language Processing (NLP) tasks. However, these models solve the problem based on single-task supervised learning and do not consider the correlation between the tasks. Based on this observation, in this paper, we implemented a multi-task learning model to joint learn two related NLP tasks simultaneously and conducted experiments to evaluate if learning these tasks jointly can improve the system performance compared with learning them individually. In addition, a comparison of our model with the state-of-the-art learning models, including multi-task learning, transfer learning, unsupervised learning and feature based traditional machine learning models is presented. This paper aims to 1) show the advantage of multi-task learning over single-task learning in training related NLP tasks, 2) illustrate the influence of various encoding structures to the proposed single- and multi-task learning models, and 3) compare the performance between multi-task learning and other learning models in literature on textual entailment task and semantic relatedness task.

Highlights

Traditional Deep Learning models typically care about optimizing a single metric
This paper aims to 1) show the advantage of multi-task learning over single-task learning in training related NLP tasks, 2) illustrate the influence of various encoding structures to the proposed single- and multi-task learning models, and 3) compare the performance between multi-task learning and other learning models in literature on textual entailment task and semantic relatedness task
We will analyze the results of our experiments, including the comparisons 1) between the proposed single- and multi-task learning models on the given tasks, 2) among various encoding methods of the proposed Recurrent Neural Network (RNN) and Convolutional Neural Network (CNN) models, and 3) between our multi-task learning model and other state-of-the learning models in literature

Summary

Introduction

Traditional Deep Learning models typically care about optimizing a single metric. We generally train a model for a specific task and fine-tune the model until the system researches to the best performance [1]. A major problem with this single task learning technique is the data insufficient issue, i.e. a model requires a large number of training samples to achieve a satisfied accuracy. Multi-task learning has provided a good solution to solve this issue.

Objectives

Results

Conclusion