Research and Implementation of Chinese Couplet Generation System With Attention-Based Transformer Mechanism

Abstract

Couplet is a unique art form in Chinese traditional culture. The development of deep neural network (DNN) technology makes it possible for computers to automatically generate couplets. Especially, Transformer is a DNN-based “Encoder–Decoder” framework, and widely used in natural language processing (NLP). However, the existed Transformer mechanism cannot fully exploit the essential linguistic knowledge in Chinese, including the special format and requirements of Chinese couplets. Therefore, this article adapts the Transformer mechanism to generate meaningful Chinese couplets. Specifically, the contributions of our work are threefold. First, considering the fact that the words in the corresponding positions of the antecedent clause and the subsequent clause in a Chinese couplet always have same part-of-speech (pos, i.e., word class), pos information is intentionally added into the Transformer to improve the accuracy of the conceived couplet. Second, to deal with the large number of unregistered and low-frequency words in Chinese couplet, a specific unregistered/low-frequency word processing mechanism (UWP) is designed and combined with the Transformer model. Third, to further improve the coherence of couplets, we incorporate the polish mechanisms (PMs) into Transformer model. In terms of three evaluation criteria including bilingual evaluation understudy (BLEU), perplexity, and human evaluation, the experimental results demonstrate the effectiveness of our designed Chinese couplet generation system.