Dual Learning of the Generator and Recognizer for Chinese Characters

Abstract

Recently, deep learning based approaches become increasingly popular for both generation and recognition of Chinese characters. In this study, we propose a dual learning framework of offline Chinese character generator (G) and recognizer (R) based on deep models. The motivation is that G and R can well collaborate to improve the performance for both. On one hand, the learning of G, aiming at the font style transfer, is enhanced via the regression loss defined on the intermediate layers of R and the output layer of G rather than only the output layer of G. On the other hand, the learning of R is boosted by using the augmented data from the generator G to improve the recognition of character classes with unseen font style. Tested on the dataset of printed Chinese characters with a vocabulary of 3755, the proposed approach can significantly improve both the recognition performance of R and the generation performance of G with high-resolution details for the adaptation of the unseen font styles.