Grayscale Image Colorization Methods: Overview and Evaluation

Abstract

Colorization is a process of converting grayscale images into visually acceptable color images. The main goal is to convince the viewer of the authenticity of the result. Grayscale images that need to be colorized are, in most cases, images with natural scenes. Over the last 20 years a wide range of colorization methods has been developed – from algorithmically simple, yet time- and energy-consuming because of unavoidable human intervention to more complicated, but simultaneously more automated methods. Automatic conversion has become a challenging area that combines machine learning and deep learning with art. This paper presents an overview and evaluation of grayscale image colorization methods and techniques applied to natural images. The paper provides a classification of existing colorization methods, explains the principles on which they are based, and highlights their advantages and disadvantages. Special attention is paid to deep learning methods. Relevant methods are compared in terms of image quality and processing time. Different metrics for color image quality assessment are used. Measuring the perceived quality of a color image is challenging due to the complexity of the human visual system. Multiple metrics used to evaluate colorization methods provide results by determining the difference between the predicted color value and the ground truth, which in several cases is not in coherence with image plausibility. The results show that user-guided neural networks are the most promising category for colorization because they successfully combine human intervention and neural network automation.