Abstract
Modern NLP defines the task of style transfer as modifying the style of a given sentence without appreciably changing its semantics, which implies that the outputs of style transfer systems should be paraphrases of their inputs. However, many existing systems purportedly designed for style transfer inherently warp the input's meaning through attribute transfer, which changes semantic properties such as sentiment. In this paper, we reformulate unsupervised style transfer as a paraphrase generation problem, and present a simple methodology based on fine-tuning pretrained language models on automatically generated paraphrase data. Despite its simplicity, our method significantly outperforms state-of-the-art style transfer systems on both human and automatic evaluations. We also survey 23 style transfer papers and discover that existing automatic metrics can be easily gamed and propose fixed variants. Finally, we pivot to a more real-world style transfer setting by collecting a large dataset of 15M sentences in 11 diverse styles, which we use for an in-depth analysis of our system.
Highlights
The task of style transfer on text data involves changing the style of a given sentence while preserving its semantics.1 Recent work in this area conflates style transfer with the related task of attribute transfer (Subramanian et al, 2019; He et al, 2020), in which modifications to attributespecific content words warp both stylistic and semantic properties of a sentence (Preotiuc-Pietro et al, 2016)
Our contributions are: (1) a simple approach to perform lexically and syntactically diverse paraphrasing with pretrained language models; (2) a simple unsupervised style transfer method that models semantic preservation with our paraphraser and significantly outperforms prior work; (3) a critique of existing style transfer evaluation based on a naıve baseline that performs on par with prior work on poorly designed metrics; (4) a new benchmark dataset that contains 15M sentences from 11 diverse styles
We focus exclusively on semantics-preserving style transfer tasks, which means that we do not evaluate on attribute transfer datasets such as sentiment, gender, and political transfer
Summary
The task of style transfer on text data involves changing the style of a given sentence while preserving its semantics. Recent work in this area conflates style transfer with the related task of attribute transfer (Subramanian et al, 2019; He et al, 2020), in which modifications to attributespecific content words (e.g., those that carry sentiment) warp both stylistic and semantic properties of a sentence (Preotiuc-Pietro et al, 2016). The task of style transfer on text data involves changing the style of a given sentence while preserving its semantics.. Our unsupervised method (Style Transfer via Paraphrasing, or STRAP) requires no parallel data between different styles and proceeds in three simple stages: 1. Create pseudo-parallel data by feeding sentences from different styles through a diverse paraphrase model (Figure 1, left). Our contributions are: (1) a simple approach to perform lexically and syntactically diverse paraphrasing with pretrained language models; (2) a simple unsupervised style transfer method that models semantic preservation with our paraphraser and significantly outperforms prior work; (3) a critique of existing style transfer evaluation based on a naıve baseline that performs on par with prior work on poorly designed metrics; (4) a new benchmark dataset that contains 15M sentences from 11 diverse styles
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
