Neural Paraphrase Generation using Transfer Learning

Florin Brad,Traian Rebedea

doi:10.18653/v1/w17-3542

Abstract

Progress in statistical paraphrase generation has been hindered for a long time by the lack of large monolingual parallel corpora. In this paper, we adapt the neural machine translation approach to paraphrase generation and perform transfer learning from the closely related task of entailment generation. We evaluate the model on the Microsoft Research Paraphrase (MSRP) corpus and show that the model is able to generate sentences that capture part of the original meaning, but fails to pick up on important words or to show large lexical variation.

Highlights

Paraphrase generation is the problem of restating a given sentence such that its overall meaning is preserved
Paraphrase generation has been previously treated as a monolingual machine translation (MT) problem (Quirk et al, 2004; Finch et al, 2004)
Neural Machine Translation (NMT) has revived interest in statistical machine translation through the use of sequence-to-sequence (SEQ2SEQ) models that learn to maximize the probability of a sentence in a target language, given a sentence in a source language (Cho et al, 2014; Sutskever et al, 2014)

Summary

Introduction

Paraphrase generation is the problem of restating a given sentence such that its overall meaning is preserved. The SEQ2SEQ model is composed of an encoder that recurrently consumes the words in the source sentence and a decoder that sequentially predicts words in the target sentence, conditioned on the encoder’s last hidden state and the previously translated words. This model was later improved by using an attention mechanism (Bahdanau et al, 2014) that allowed the decoder to focus on the relevant words from the source sentence

Objectives

Methods

Results

Conclusion