Image inspired Chinese couplet generation

Abstract

Chinese couplets, as one of the traditional Chinese culture, is the treasure of Chinese civilization and the inheritance of Chinese history. Given a sentence (namely an antecedent clause), people reply with another sentence (namely a subsequent clause) equal in length. Because of the complexity of the semantic and grammatical rules of couplet, it is not easy to create a suitable couplet that meets the requirements of sentence pattern, context, and flatness. With the development of neural models and natural language processing, automatic generation of Chinese couplets has drawn significant attention due to its artistic and cultural value, most of these works mainly focus on generating couplet by given text information, while visual inspirations for couplet generation have been rarely explored. In this paper, we design a Chinese couplet generation model based on NIC (Neural Image Caption), which can compose a piece of couplet suitable to the artistic conception in an image. At first, we use the improved VGG16 model to predict the input image. The content of the image can be automatically recognized and the corresponding description are generated and translated into Chinese keywords. Then, the encoder-decoder framework is used repeatedly to process these keywords, and finally the couplet can be generated. Moreover, to satisfy special characteristics of couplets, we incorporate the attention mechanism into the encoding-decoding process, which greatly improves the accuracy of couplets generated automatically.