Token-level disentanglement for unsupervised text style transfer

Abstract

Text style transfer is the task of altering the style of a source text to a desired style while preserving the style-independent content. A common approach involves disentangling a given sentence into a style-agnostic content representation within the latent space and then generating the text in the desired style, guided by a separate style embedding. However, previous methods have a limitation in that they assume the input sentence is encoded by a fix-sized latent vector at the sentence level, making it challenging to capture rich semantic information at the token level, especially when dealing with longer texts. Consequently, this leads to suboptimal preservation of non-stylistic semantic content. In this paper, we address this challenge, and propose TED, a fine-grained model for disentangling content and style representation at the token level to enhance content preservation. Specifically, TED use tf–idfs to estimate the pivot tokens for different styles and incorporates multi-task and adversarial objectives to disentangle the content and style information of each token within the latent space. Experimental results on two popular text style transfer datasets show that our proposed model significantly outperforms state-of-the-art baselines, particularly in terms of content preservation. Moreover, the quantitative and qualitative experiments demonstrate the effectiveness of our model in achieving token-level disentanglement within the latent space.