Automatic colorization of digital halftone images using neural networks

Abstract

The problem of automatic colorization of monochrome images is considered. methods of colorizing are used in film industry to restore chromaticity of old movies and photographic materials, in computer vision problems, in medical images processing etc. Modern techniques of colorization contain many manual operations, take a lot of time and are expensive. Many methods of colorization require human participation to correctly determine colors, since there is no one-to-one accordance between grayscale and color. In this paper we discuss method for fully automatic colorization of monochrome images using a convolutional neural network. This approach has reduced using of manual operations to minimum. Structure of the neural network for coloration based on the VGG16 model is considered in the paper. Types of layers that are appropriate for solving the problem of colorization are determined and analyzed. Proposed structure consists of 13 convolutional layers and three upsampling layers. The subsample layers are replaced with the necessary zero addition with a step of 2x2. All layers’ filters have 3x3 size. The activation function of all convolutional layers is ReLU and hyperbolic tangent of the last layer. The presented model is implemented in a software system for automatic image colorization. The software system includes two parts. The first part implements construction and training of the neural network. The second part uses obtained neural network to generate colorized images from grayscale images. Network training was carried out on a sample of Caltech-256, which contains 256 categories of objects. After training the system was tested on series of grayscale images. Testing showed that the system performs enough plausible colorization of certain types objects. Acceptable results were obtained in the colorization of images of nature, ordinary animals, portrait photos. In unsuccessful cases objects were painted in brown shades. Unsuccessful results were obtained for images that contained only parts of objects or these objects were represented in the training sample in too different colors.