Method for automatic cartoon colorization

Abstract

Colorization task consists of acquiring a full-color RGB image from grayscale image or a sketch. Article is concerned with the task of colorizing grayscale cartoon images and image sequences using neural networks. Efficiency of an existing prototype algorithm is reviewed with different modifications, as well as different combinations of loss functions. A new neural network loss function is proposed. It is based on a hypothesis that specifics of cartoons, such as clear object boundaries and color consistency within those boundaries can be used to improve colorization quality. Proposed loss function uses segmentation of cartoon images in the bilateral space, and minimizes difference between closest found segments and inside each segment, thus bringing closer predicted colors within the segment and between neighboring segments. Quantitative and qualitative experiments are conducted on efficiency as well as generalization ability of modified prototype algorithm with proposed loss function. Quantitative experiments consisted of measuring PSNR, LPIPS, MSE in Lab color space and CC, while qualitative focused on comparing temporal consistency, quality of colorization and quality of generalization.