The Style Transfer of Photos and Landscape Paintings Based on CycleGAN Combined with Nested Edge Detection Module

Abstract

Chinese landscape paintings, especially Chinese landscape ink paintings have always been the treasure of the Chinese, or even the world culture. Due to the advancement of the science and knowledge, there are always interests for finding ways to produce Chinese landscape paintings through the way of the technology. In this paper, based on the original CycleGAN model, we are trying to transfer a real-world photograph to a typical Chinese landscape painting. To be specific, the brushwork in Chinese ink paintings contains enormous numbers of distinct and technical brush strokes which makes it extremely hard for CycleGAN to recognize and transfer Chinese ink painting. We improved the brushstroke effect of Chinese ink painting still by incorporating the Holistically Nested Edge Detection (HED) method into the CycleGAN model. The HED module puts the source image and the corresponding generated image into the convolution layer of five stages to generate their respective edge maps. The balanced cross-entropy loss calculated with edge maps is added to the total loss of CycleGAN to train the generator which is confronted with discriminator to improve output results. The addition of HED enables the extraction of edge features from the images, preserving the structural information and enhancing the accuracy of the style transfer, so the detailed of the Chinese ink painting can be transferred better in this sense. Experimental process with model building, training, validating, and predicting concludes that, complementing HED method into the original CycleGAN models well preserved distinct features in the conversion processes from real-world photographs to traditional Chinese landscape paintings.