Abstract
Stochastic natural language generation systems that are trained from labelled datasets are often domainspecific in their annotation and in their mapping from semantic input representations to lexical-syntactic outputs. As a result, learnt models fail to generalize across domains, heavily restricting their usability beyond single applications. In this article, we focus on the problem of domain adaptation for natural language generation. We show how linguistic knowledge from a source domain, for which labelled data is available, can be adapted to a target domain by reusing training data across domains. As a key to this, we propose to employ abstract meaning representations as a common semantic representation across domains. We model natural language generation as a long short-term memory recurrent neural network encoderdecoder, in which one recurrent neural network learns a latent representation of a semantic input, and a second recurrent neural network learns to decode it to a sequence of words. We show that the learnt representations can be transferred across domains and can be leveraged effectively to improve training on new unseen domains. Experiments in three different domains and with six datasets demonstrate that the lexical-syntactic constructions learnt in one domain can be transferred to new domains and achieve up to 75-100% of the performance of in-domain training. This is based on objective metrics such as BLEU and semantic error rate and a subjective human rating study. Training a policy from prior knowledge from a different domain is consistently better than pure in-domain training by up to 10%.
Highlights
N ATURAL language generation (NLG) is the task of finding a natural language description for a non-linguistic input representation, such as a database entry, dialogue act or other non-linguistic object
To circumvent the problem of incompatible input representations, we propose in this article that using abstract meaning representations (AMRs) as a common representation language across domains has the advantage that data from one domain can be reused in another domain without problems
We model our natural language generator as a Long Short-Term Memory (LSTM) encoderdecoder, in which two LSTMs are jointly trained to learn a probability distribution that conditions a sequence of words on a sequence of semantic symbols
Summary
N ATURAL language generation (NLG) is the task of finding a natural language description for a non-linguistic input representation, such as a database entry, dialogue act or other non-linguistic object. Dusek and Jurcicek [35] showed how a sequence-to-sequence model for NLG can generate outputs from unaligned training data and outperform previous work [5] that relied on aligned semantic inputs and lexical-syntactic outputs. Given the specificity of input representations in Wen et al [12], it is not clear that the model is general enough to learn basic linguistic patterns that can be transferred across semantically more distant domains as was shown for work in other areas of NLP. We present a novel approach to domain adaptation for NLG that represents data across domains in a common input representation In this way, we are able to learn basic linguistic patterns from multiple domains and reproduce the positive effects of multi-domain training reported for other areas of NLP. The size of input/output sequences x and y was the sum of unique vocabulary items and semantic symbols in each domain (after delexicalization) and we use 50 hidden nodes
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.