User-Guided Chinese Painting Completion–A Generative Adversarial Network Approach

Jieting Xue,Yi Liu,Jingtao Guo

doi:10.1109/access.2020.3029084

Abstract

Image completion models based on deep neural networks have been a research hot spot in computer vision. However, most of the previous methods focus on natural images, such as faces and landscapes. In this paper, we propose a novel image completion model for a special set of artificial ancient Chinese paintings to address this limitation. Specifically, we integrate three complements: the Wasserstein Generative Adversarial Networks (WGAN), Perceptual loss, and Mean Squared Error (MSE) to train the model robustly. We propose a unique generator which can not only pay more attention to complete the details of ancient Chinese paintings but also can provide the synthesized lines to help artists to analyze paintings conveniently. Additionally, we also allow a user to supply a structure hint to guide our model to complete Chinese paintings according to his/her preference. Extensive experiments firmly demonstrate the effectiveness of our approach to complete ancient Chinese paintings and remove abnormal color blocks from them.

Highlights

Chinese painting is an essential core of Chinese culture
We propose a user-guided supervised model based on Wasserstein Generative Adversarial Networks (WGAN) to complete artificial ancient Chinese paintings with random irregular holes
We propose a novel deep learning model based on WGAN to complete artificial ancient Chinese paintings with irregular holes

Summary

Introduction

Chinese painting is an essential core of Chinese culture. Compared with European paintings, Chinese paintings pay more attention to profound connotations, they are not gorgeous and multicolored. There are many world-renowned ancient Chinese paintings, such as the Along the River During the Qingming Festival (Fig. 1a), Spring Morning in the Han Palace (Fig. 1d), etc. Many paintings have been damaged during the long history. Traditional physical repair methods may cause secondary damages, and it is difficult for us to fix it correctly. Existing image processing software, such as Photoshop, may be helpful in this perspective, these methods cannot fix missing contents of paintings intelligently as artists, and the labor costs are usually expensive. Thereby, it is necessary to develop an image completion approach to fix these paintings based on a deep understanding of semantics. With the development of smart convolution neural networks, we can automatically restore the missing part of the paintings. This new approach reduces the repair cost significantly.

Methods

Results

Conclusion