Automatic Colorization Research Articles

Global Distribution Adjustment and Nonlinear Feature Transformation for Automatic Colorization

Automatic colorization is generally classified into two groups: propagation-based methods and reference-based methods. In reference-based automatic colorization methods, color image(s) are used as reference(s) to reconstruct original color of a gray target image. The most important task here is to find the best matching pairs for all pixels between reference and target images in order to transfer color information from reference to target pixels. A lot of attractive local feature-based image matching methods have already been developed for the last two decades. Unfortunately, as far as we know, there are no optimal matching methods for automatic colorization because the requirements for pixel matching in automatic colorization are wholly different from those for traditional image matching. To design an efficient matching algorithm for automatic colorization, clustering pixel with low computational cost and generating descriptive feature vector are the most important challenges to be solved. In this paper, we present a novel method to address these two problems. In particular, our work concentrates on solving the second problem (designing a descriptive feature vector); namely, we will discuss how to learn a descriptive texture feature using scaled sparse texture feature combining with a nonlinear transformation to construct an optimal feature descriptor. Our experimental results show our proposed method outperforms the state-of-the-art methods in terms of robustness for color reconstruction for automatic colorization applications.

Overview of Scribbled-Based Colorization

Hand-coloring is very useful when people draw pictures or produce movies, but it is also very time-consuming and expensive. Therefore, with the development of segmentation algorithms and deep learning, a lot of computer-assisted colorization algorithms have been invented, including user-guided colorization, semi-automatic colorization and automatic colorization. In this survey paper, we will only focus on scribble-based colorization in user-guided colorization and review the development on it. We will discuss various principles and methods among different kinds of scribble-based colorization algorithms. We will also compare visual result and performance of algorithms among colorization on grayscale image, colorization on sketch and colorization with networks to conclude their advantages and disadvantages.

Automatic Colorization with Improved Spatial Coherence and Boundary Localization

Grayscale image colorization is an important computer graphics problem with a variety of applications. Recent fully automatic colorization methods have made impressive progress by formulating image colorization as a pixel-wise prediction task and utilizing deep convolutional neural networks. Though tremendous improvements have been made, the result of automatic colorization is still far from perfect. Specifically, there still exist common pitfalls in maintaining color consistency in homogeneous regions as well as precisely distinguishing colors near region boundaries. To tackle these problems, we propose a novel fully automatic colorization pipeline which involves a boundary-guided CRF (conditional random field) and a CNN-based color transform as post-processing steps. In addition, as there usually exist multiple plausible colorization proposals for a single image, automatic evaluation for different colorization methods remains a challenging task. We further introduce two novel automatic evaluation schemes to efficiently assess colorization quality in terms of spatial coherence and localization. Comprehensive experiments demonstrate great quality improvement in results of our proposed colorization method under multiple evaluation metrics.

Deep Manga Colorization with Color Style Extraction by Conditional Adversarially Learned Inference

Many comic books are now published as digital books, which easily provide colored contents compared to physical books. The motivation of automatic colorization of comic books now arises. Previous studies colorize sketches with spatial color annotations or no clues at all. They are expected to reduce workloads of comic artists but still require spatial color annotations in order to produce desirable colorizations. This study introduces color style information and combines it with conditional adversarially learned inference. Experimental results demonstrate that objects in manga are painted with colors depending on color style information and that color style information extracted from another colored image to paint an object with desired color.

MixtureTree annotator: a program for automatic colorization and visual annotation of MixtureTree.

The MixtureTree Annotator, written in JAVA, allows the user to automatically color any phylogenetic tree in Newick format generated from any phylogeny reconstruction program and output the Nexus file. By providing the ability to automatically color the tree by sequence name, the MixtureTree Annotator provides a unique advantage over any other programs which perform a similar function. In addition, the MixtureTree Annotator is the only package that can efficiently annotate the output produced by MixtureTree with mutation information and coalescent time information. In order to visualize the resulting output file, a modified version of FigTree is used. Certain popular methods, which lack good built-in visualization tools, for example, MEGA, Mesquite, PHY-FI, TreeView, treeGraph and Geneious, may give results with human errors due to either manually adding colors to each node or with other limitations, for example only using color based on a number, such as branch length, or by taxonomy. In addition to allowing the user to automatically color any given Newick tree by sequence name, the MixtureTree Annotator is the only method that allows the user to automatically annotate the resulting tree created by the MixtureTree program. The MixtureTree Annotator is fast and easy-to-use, while still allowing the user full control over the coloring and annotating process.

Affective Image Colorization

Colorization of gray-scale images has attracted many attentions for a long time. An important role of image color is the conveyer of emotions (through color themes). The colorization with an undesired color theme is less useful, even it is semantically correct. However this has been rarely considered. Automatic colorization respecting both the semantics and the emotions is undoubtedly a challenge. In this paper, we propose a complete system for affective image colorization. We only need the user to assist object segmentation along with text labels and an affective word. First, the text labels along with other object characters are jointly used to filter the internet images to give each object a set of semantically correct reference images. Second, we select a set of color themes according to the affective word based on art theories. With these themes, a generic algorithm is used to select the best reference for each object, balancing various requirements. Finally, we propose a hybrid texture synthesis approach for colorization. To the best of our knowledge, it is the first system which is able to efficiently colorize a gray-scale image semantically by an emotionally controllable fashion. Our experiments show the effectiveness of our system, especially the benefit compared with the previous Markov random field (MRF) based method.

Automatic black and white film colorization using texture features and artificial neural networks

This study represents an innovative automatic method for black and white films colorization using texture features and a multilayer perceptron artificial neural network. In the proposed method, efforts are made to remove human interference in the process of colorization and replace it with an artificial neural network (ANN) which is trained using the features of the reference frame. Later, this network is employed for automatic colorization of the remained black and white frames. The reference frames of the black and white film are manually colored. Using a Gabor filter bank, texture features of all the pixels of the reference frame are extracted and used as the input feature vector of the ANN, while the output will be the color vector of the corresponding pixel. Finally, the next frames’ feature vectors are fed respectively to the trained neural network, and color vectors of those frames are the output. Applying AVI videos and using various color spaces, a series of experiments are conducted to evaluate the proposed colorization process. This method needs considerable time to provide a reasonable output, given rapidly changing scenes. Fortunately however, due to the high correlation between consecutive frames in typical video footage, the overall performance is promising regarding both visual appearance and the calculated MSE error. Apart from the application, we also aim to show the importance of the low level features in a mainly high level process, and the mapping ability of a neural network.

A New Colorization Algorithm Based on Dynamic Neighbor-Relevant Chromatic Channel Information

This paper is concerned with the problem of automatic colorization of grayscale images via color transfer. Some deficiencies of Welsh's colorization algorithm are pointed out and analyzed. Then, a new colorization algorithm is proposed, which uses a new similarity function and a new search strategy for pixel matching. The new function incorporates both the luminance information and chromatic values of previously colorized neighboring pixels. The intrinsic space clustering property of an image is taken into account in the strategy by using Hilbert filling curve scanning order. Experimental results showed that the proposed algorithm outperforms Welsh's algorithm in both colorization effect and computational complexity. Moreover, the proposed similarity function and search strategy can easily be embedded into other colorization algorithms to improve their performance.

Gradient-based value mapping for pseudocolor images

We develop a method for automatic colorization of im- ages (or two-dimensional fields) in order to visualize pixel values and their local differences. In many applications, local differences in pixel values are as important as their values. For example, in topog- raphy, both elevation and slope often must be considered. Gradient- based value mapping (GBVM) is a technique for colorizing pixels based on value (e.g., intensity or elevation) and gradient (e.g., local differences or slope). The method maps pixel values to a color scale (either gray-scale or pseudocolor) in a manner that emphasizes gra- dients in the image while maintaining ordinal relationships of values. GBVM is especially useful for high-precision data, in which the num- ber of possible values is large. Colorization with GBVM is demon- strated with data from comprehensive two-dimensional gas chroma- tography (GCxGC), using both gray-scale and pseudocolor to visualize both small and large peaks, and with data from the Global Land One-Kilometer Base Elevation (GLOBE) Project, using gray- scale to visualize features that are not visible in images produced with popular value-mapping algorithms. © 2007 SPIE and

A Novel Picture Coding Using Colorization Technique

Colorization is a computerized process of adding color to a black and white print, movie and TV program. The authors have proposed automatic colorization algorithms by giving a partial color to a monochrome image. This paper focuses on the colorization process which can produce a color image from a monochrome image with a small number of color pixels, and proposes a novel color image coding algorithm based on the colorization technique. At first, luminance component is separated from an input color image. Next, selected color seeds from the original color image are sown on the luminance image domain automatically and the monochrome image is colorized. The sowing process is continued until the colorized image satisfies the desired quality. Finally, both orthogonal transform-coded luminance component and the set of color seeds are transmitted as coded data. The decoding can be performed by tracing the same colorization process. We confirmed that the colorization technique is effective to image coding through the experiments.