Колоризация эскизов на основе генеративно-состязательных нейронных сетей

Abstract

Generative adversarial neural networks (GAN) successfully perform automatic colorization of character sketches. Such models can be further improved in such aspects as increasing the throughput, improving the colorization quality, and reducing the model size. Steps aimed at improving the colorization quality are taken. Known solutions are reviewed, and an initial GAN model with eight blocks in the generator encoder and decoder is developed and trained based on the review results. The second GAN model is obtained on the basis of the first one by including residual blocks in the generator encoder blocks with simultaneously using the attention blocks and residual blocks in the generator decoder. The third GAN model has been developed based on the second one: one block is added to the generator encoder and decoder. All models, including the initial and modified ones, have been trained on the same data set. The models have been trained either with or without using additional information about the image color (the color palette of the reference image or color hint labels). The trained models are evaluated with respect to the quality of the images they generate (colored sketches), determined by the Frechet Inception Distance metric. All modified GAN models generate images with quality superior to that of the initial model.