Grayscale Image Colorization Research Articles

Research on Image Colorization Based on Deep Learning

Abstract Gray scale picture colorization, a research hotspot in the domain of computer vision, seeks to assign reasonable colours to every single pixel of a grayscale picture, enriching the visual information of the coloured picture. Neural networks take grayscale images as input and output colorized images. From a psychological perspective, colours can provide observers with a more pleasant perceptual experience. Old photos can also be coloured, which are important documentary resources for recording real history and restoring the social landscape of the time. Additionally, due to the limitations of imaging mechanisms, most medical images are grayscale. Therefore, research on grayscale image colorization is a task of significant importance. However, existing automatic image colorization systems have issues such as “blurry boundaries”, “colour overflow”, “inappropriate colour choices” and “incorrect colour regions”. This paper proposes a model combining adversarial neural networks and U-Net networks, optimizing them by incorporating global feature fusion modules, Pathway emphasis mechanisms and spatial focus mechanisms. The idea of Markov discriminator was added to the discriminator, and the loss function was optimized. The ultimate investigative conclusions demonstrate that the PSNR and SSIM of the suggested system are 24.45 and 0.941, accordingly, in relation to the preceding protocol, which are 3% and 5% higher than the previous algorithm, and the example has been ameliorated to some degree. And the loss function is modified, and the final algorithm can improve the effect of picture colouring, which encompasses a broad spectrum of application prospects.

Boosting the Performance of Deep Ear Recognition Systems Using Generative Adversarial Networks and Mean Class Activation Maps

Ear recognition is a complex research domain within biometrics, aiming to identify individuals using their ears in uncontrolled conditions. Despite the exceptional performance of convolutional neural networks (CNNs) in various applications, the efficacy of deep ear recognition systems is nascent. This paper proposes a two-step ear recognition approach. The initial step employs deep convolutional generative adversarial networks (DCGANs) to enhance ear images. This involves the colorization of grayscale images and the enhancement of dark shades, addressing visual imperfections. Subsequently, a feature extraction and classification technique, referred to as Mean-CAM-CNN, is introduced. This technique leverages mean-class activation maps in conjunction with CNNs. The Mean-CAM approach directs the CNN to focus specifically on relevant information, extracting and assessing only significant regions within the entire image. The process involves the implementation of a mask to selectively crop the pertinent area of the image. The cropped region is then utilized to train a CNN for discriminative classification. Extensive evaluations were conducted using two ear recognition datasets: mathematical analysis of images (MAI) and annotated web ears (AWEs). The experimental results indicate that the proposed approach shows notable improvements and competitive performance: the Rank-1 recognition rates are 100.00% and 76.25% for MAI and AWE datasets, respectively.

Colorization of gray scale images based on convolutional block attention and Pix2Pix network

Image colorization is a key research direction in the field of computer vision, which aims to predict the color information of grayscale images and convert them into color images. Traditional image colorization methods need to provide a large number of color points to mark each object in the image. As a result of the rapid development of convolutional neural networks, image colorization methods have also improved, with deep learning methods benefiting the most. However, due to the lack of feature quality, there are still many challenges in practical applications, such as boundary blurring. In this paper, we propose a gray image colorization method based on the Convolutional Block Attention Module (CBAM) and Pix2Pix network. Specifically, we add the CBAM module to the U-net generator network, which helps adaptively capture sensitive features, thereby enhancing the quality of features and image coloring. The effectiveness of the proposed model is verified by experiments.

Thermal infrared image coloring method and evaluation method based on edge consistency

Thermal imaging cameras have broad application prospects due to the characteristics of their shooting principles. With the development of deep learning technology, thermal infrared grayscale images can be transformed into RGB color images through a grayscale image colorization model, which provides more application possibilities for thermal infrared grayscale images. However, thermal infrared grayscale images are more difficult to color than visible and near-infrared grayscale images due to their lack of pixel-level matching visible light color map data. Therefore, we propose an unsupervised learning scheme based on CycleGAN, which reduces the loss of effective edge information and suppresses the generation of abnormal edge information during the colorization process by increasing the self-supervision of edge information in the cycle process. In addition, we propose a new edge similarity indicator Edge Consistency Index Measure (ECIM), which evaluates the quality of the coloring results from the perspective of edge consistency before and after colorization. By comparing with the existing methods, we show that our method is better than the existing methods in coloring effect, and the proposed ECIM can well describe the consistency of the edges before and after colorization.

Image Colorization using Convolutional Neural Network

With its advent and extension in many areas, technology has become crucially integrated with human lifestyle and activities. Various branches of technology have found ways to progress and they serve imperatively in various fields of development, entertainment and utility. Artificial Intelligence and domains within it encompass a huge portion of technological applications, and deep learning being a subset of AI can be used for many such implementations. Image colourisation is one such domain which can utilize the deep learning capabilities of a machine to effectively produce results. Colorization of gray-scale images is generally carried out using photo editing software and is a tedious and expensive job.With the collaboration of different pre-trained models in a baseline CNN model,, we analyzed the one with the highest accuracy and the least loss,comparing it to the others.Based on evaluation metrics such as root mean square error, loss, and accuracy, we determined that the best model is the one incorporating EFFicientB0. Hence, our project makes this process automated and accurate utilizing the convolutional neural architecture combined with EfficientNetB0. In this project, we have realized and implemented several compositions of the baseline CNN model with pre-trained models to deduce the best version.

Deep Learning-Based Machine Color Emotion Generation

This paper investigates generating machine color emotion through deep learning. The grayscale image colorization model's training process resembles human memory color. Sixty images were recolored and quality evaluated to explore machine generated color impressions. Six experimental samples were recolored under D65, A, CWF, and TL84 light sources. Changes in lightness, chroma, and hue angle compared the original and colorized images, exploring light source effects on machine color perception. Analyzing differences in coloring results within the CIEL* a* b* color space for pixels with equal grayscale verified machine color emotion generation. Results show the machine learns to form color impressions from samples. Different light source color temperatures impact color prediction accuracy. The machine accurately colors images based on semantic context, demonstrating spontaneous color emotion generation through deep learning. This research positively contributes to the development of intelligent devices with color emotion.

A Deep Learning-based Approach for Colorization of Grayscale Images and Videos

Colorization is the process of converting grayscale photos into colorful ones that are more visually appealing. Previously, a wide range of colorization techniques has been developed, which require the involvement of the human brain which consumes a lot of time and energy. In today’s world, there are many procedures that will automatically convert the grayscale image to a color image. Most of the conversion techniques incorporate elements of deep learning, machine learning, and art. This study gives a novel technique for coloring grayscale images that makes use of GAN and U-Net model characteristics. By using this technique, the model is able to learn how to colorize images from a trained U- Net. Additionally, the Fusion layer is used to combine the global priors for each class with the local information finds for each class, which are based on small image patches. This produces colorization outcomes that are more attractive on a visual level. Finally, the results of the method were obtained by doing an evaluation based on user research and comparing it to the state-of-the-art

Texture-aware gray-scale image colorization using a bistream generative adversarial network with multi scale attention structure

Various methods based on deep neural networks have been proposed to generate color images from gray-scale images, meanwhile, Generative adversarial networks (GANs) are also gradually applied to image colorization. However, the existing methods are texture-unaware, resulting in dullish color and color bleeding artifacts in the output images. This paper attempt to integrate a novel texture-aware bistream GAN into the conventional encoder–decoder structure for image colorization. In this study, the proposed bistream feature extraction module (BSFEM) and the feature boosting module (FBM), extract the global and local features from two parallel encoders and fuse them via a novel hybrid attention structure, this novel structure could emphasize the importance of certain channels and locations of features that may potentially benefit image colorization. In addition, the texture colors can be better recovered though the proposed multi-scale feature attention module (MSFAM). The quantitative experiments demonstrate that, compared to the state-of-the-art approaches, the proposed method has improved the PSNR and SSIM metrics by 18% and 8% respectively. Moreover, the qualitative results show that this method is capable of producing visually pleasant color images especially in terms of recovering texture details and eliminating color bleeding along the edges. The source code and data are available online at https://github.com/JarryZang/Image-Colorization-.

Image Colourization using Clustering Technique

Colorization of grayscale images has become a more researched area in the recent years, thanks to the advent of K-mean clustering networks. We attempt to apply this concept to colorization of real images obtained from video sequences. Previous similar research focused mainly colorization of natural images, while colorization of real is traditionally done by leveraging manual scribble methods. Our proposed method is a fully automated process. To implement it, we propose and compare two distinct K-mean clustering architectures trained under various loss functions. We aim to compare each variant based on results obtained as individual images.

Invertible Color-to-Grayscale Conversion Using Lossy Compression and High-Capacity Data Hiding

Invertible color-to-grayscale conversion is a research issue in grayscale image colorization, which is a complex problem resulting from information loss. This paper presents an innovative invertible color-to-grayscale conversion idea by independently compressing the chromaticity plane and hiding it into the corresponding luminance plane as a watermark. Since the chromaticity and luminance planes are orthogonal in the proposed method, they can be processed efficiently without mutual influence. By using an efficient lossy compression operation, we can save more chromatic information. By using two high-capacity data hiding techniques (reversible watermarking (RW) and least significant bit (LSB) substitution), we can embed the compressed chromatic information into the luminance plane perfectly. For the purpose of integrality authentication, the luminance plane is hashed as part of the embedded information before the embedding. Experimental results have shown that higher quality of reconstructed color images can be achieved by using RW, but the quality of the synthesized grayscale drops sharply. By using LSB substitution, we can obtain high-quality synthesized grayscale and reconstructed color images simultaneously. Furthermore, we have compared the proposed LSB-based scheme with several recently reported state-of-the-art methods to validate the superiority of the proposed approach.

Colorizing the Past: Deep Learning for the Automatic Colorization of Historical Aerial Images.

The colorization of grayscale images can, nowadays, take advantage of recent progress and the automation of deep-learning techniques. From the media industry to medical or geospatial applications, image colorization is an attractive and investigated image processing practice, and it is also helpful for revitalizing historical photographs. After exploring some of the existing fully automatic learning methods, the article presents a new neural network architecture, Hyper-U-NET, which combines a U-NET-like architecture and HyperConnections to handle the colorization of historical black and white aerial images. The training dataset (about 10,000 colored aerial image patches) and the realized neural network are available on our GitHub page to boost further research investigations in this field.

Adversarial Image Colorization Method Based on Semantic Optimization and Edge Preservation

As a medium for transmitting visual information, image is a direct reflection of the objective existence of the natural world. Grayscale images lack more visual information than color images. Therefore, it is of great significance to study the colorization of grayscale images. At present, the problems of semantic ambiguity, boundary overflow and lack of color saturation exist in both traditional and deep learning methods. To solve the above problems, an adversarial image colorization method based on semantic optimization and edge preservation is proposed. By improving generative and discriminative networks and designing loss functions, deeper semantic information and sharper edges of images can be learned by our network. Our experiments are carried out on the public datasets Place365 and ImageNet. The experimental results show that the method in this paper can reduce the color anomaly caused by semantic ambiguity, suppress the color blooming in the image boundary area and improve the saturation of the image. Our work achieves competitive results on objective indicators of peak signal-to-noise ratio (PSNR), structural similarity (SSIM) and learned perceptual image patch similarity (LPIPS), with values of 30.903 dB, 0.956 and 0.147 on Place365 and 30.545 dB, 0.946 and 0.150 on ImageNet, which proves that this method can effectively colorize grayscale images.

Study on the Application of Visual Communication Design in APP Interface Design in the Context of Deep Learning.

Visual communication concepts enable linguistics or semiotics to the teaching of visual communication designs, creating graphic designs into an innovative and scientific discipline. The use of storyline techniques in visual communication not only inspires the imagination of designer but also arouses the visual memory of the audience. Besides, improving cultural heritage such as historical images is important to protect cultural diversity. Recently, the developments of deep learning (DL) and computer vision (CV) approaches make it possible for the automatic colorization of grayscale images into color images. Also, the usage of visual communication design in APP interface design has increased. With this motivation, this work introduces the enhanced deep learning-based automated historical image colorization (EDL-AHIC) technique for wireless network-enabled visual communication. The proposed EDL-AHIC technique intends to effectually convert the grayscale images into color images. The presented EDL-AHIC technique extracts the local as well as global features. For global feature extraction, the enhanced capsule network (ECN) model is applied. Finally, the fusion layer and decoding unit are employed to determine the output, i.e., chrominance component of the input image. A comprehensive experimental validation process is performed to ensure the betterment of the EDL-AHIC technique. The comparison study reported the supremacy of the EDL-AHIC technique over the other recent methods.

A Novel Brain Tumor Detection and Coloring Technique from 2D MRI Images

The early automated identification of brain tumors is a difficult task in MRI images. For a long time, continuous research efforts have floated a new idea of replacing different grayscale anatomic regions of diagnostic images with appropriate colors that could overcome the problems being faced by radiologists. The colorization of grayscale images is challenging for enhancing various regions’ contrasts by transforming grayscale images into high-contrast color images. This study investigates standard solutions in discriminating between normal and abnormal regions by assigning colors to grayscale human brain MR images to differentiate different kinds of tissues. The proposed approach is influenced by connected component and index-based colorization methods for applying colors to different regions and abnormal areas. It is an automated approach that varies its inputs using luminance and pixel matrix values and provides the possible outcome. After segmentation, a specific algorithm is devised to colorize the region-of-interest (ROI) areas, which distinguishes and applies colors to differentiate the regions. Results show that implementing the watershed-based area segmentation method and ROI selection method based on the morphological operation helps identify tissues during processing. Moreover, the colorization approach based on luminance and pixel matrix after segmentation and ROI selection is beneficial due to better PSNR and SSIM values and visible contrast improvement. Our proposed algorithm works with less processing overhead and uses less time than those of the industry’s previously used color transfer method.

Learn to Be Clear and Colorful: An End-to-End Network for Panchromatic Image Enhancement

Benefiting from the high coverage and re-visiting frequency, the satellite imagery is an ideal data for large-scale, real-time earth observation. However, due to the limited resolution and chromatic information, the satellite images, especially the panchromatic images, are not capable of being used for accurate earth observations, such as road extraction, vehicle detection, and building segmentation. In this research, we propose a cascaded fully convolutional network (CFCN) consists of a residual dense super-resolution network (RDSRN) for grayscale image super-resolution (SR) and a residual deconvolution colorization network (RDCN) for grayscale image colorization. The unique architecture can simultaneously learn texture detail and color information from aerial images and then transfer to enhance panchromatic images. Furthermore, we also introduce an indirect evaluation metric, learned extraction similarity (LES), to estimate the image quality of the generated image in the absence of the ground truth. Experiments on a multispectral image dataset demonstrate that panchromatic images enhanced by the proposed CFCN are with both texture and color fidelity as compared to aerial image. For pre-trained U-Net, compared to the performance on raw panchromatic images, the CFCN enhanced images increase 12.8% of LES of overall accuracy (96.4% versus 83.6%).

Fine‐grained semantic ethnic costume high‐resolution image colorization with conditional GAN

Grayscale image colorization, especially for ethnic costume images, is highly challenging due to its rich and complex color features. The existing image colorization methods usually take the costume image as a whole in practical applications that lead to the ignorance of the semantic information of different parts of the costume. It is known that each part's color distribution of the ethnic costume is different. So, the color mapping of other parts is also diverse, which is determined by distinctive ethnic characteristics. This study introduces fine-grained level semantic information and proposes a high-resolution image colorization model for ethnic costumes targeting enhancement, inspired by semantic-level colorization. The semantic information of different regions of ethnic costumes has a significant impact on the performance of the coloring task. Using Pix2PixHD as the backbone network, we create a new network architecture that maintains color distribution correspondence and spatial consistency of costume images using fine-grained semantic information. In our network, we take the splice result of fine-grained semantic for ethnic costume and grayscale image as the conditions and then feed them into the generative adversarial networks. We also discuss and analyze the influences of the grayscale channel and fine-grained semantics on discriminator. Extensive experiments demonstrate that our method performs well compared with other state-of-the-art automatic colorization methods.

Image Colorization Using Autoencoders

Abstract: Modification of art may be viewed as enhancement or vandalization. Even though for a long time many were opposed to the idea of colorizing images, they now have finally viewed it for what it is - an enhancement of the art form. Grayscale image colorization has since been a long-standing artistic division. It has been used to revive or modify images taken prior to the invention of colour photography. This paper explores one method to reinvigorate grayscale images by colorizing them. We propose the use of deep learning, specifically the use of convolution neural networks. The obtained results show the ability of our model to realistically colorize grayscale images. Keywords: Deep Learning, Convolutional Neural Network, Image Colorization, Autoencoders.

Automatic Grayscale Image Colorization using a Deep Hybrid Model

Image colorization is an interesting yet challenging task due to the descriptive nature of getting a natural-looking color image from any grayscale image. To tackle this challenge and also have a fully automatic procedure, we propose a Convolutional Neural Network (CNN)-based model to benefit from the impressive ability of CNN in the image processing tasks. To this end, we propose a deep-based model for automatic grayscale image colorization. Harnessing from convolutional-based pre-trained models, we fuse three pre-trained models, VGG16, ResNet50, and Inception-v2, to improve the model performance. The average of three model outputs is used to obtain more rich features in the model. The fused features are fed to an encoder-decoder network to obtain a color image from a grayscale input image. We perform a step-by-step analysis of different pre-trained models and fusion methodologies to include a more accurate combination of these models in the proposed model. Results on LFW and ImageNet datasets confirm the effectiveness of our model compared to state-of-the-art alternatives in the field.

Color-UNet++: A resolution for colorization of grayscale images using improved UNet++

Colorization is the computer-assisted application of color to a gray scale image, which presents two problems to modern deep learning-based approaches. One is to provide colorization models with both high expressibility and strong learning ability, as current models have difficulty both excelling at coloring and being easy to train. The other is to return a picture without uneven overlap. This paper proposes a deep convolutional network framework called Color-UNet++ for the end-to-end solution of these colorization problems. Color-UNet++ is adjusted to settle gradient dispersion and explosion by capturing more transfer and intermediate results during backpropagation. We adjust the de-convolution structure to solve the problem of uneven overlap. We design the model in YUV instead of RGB color space, with an objective function that is appropriate to the coloring problem and can capture a wide range of colors. A large number of experimental results on LFW and LSUN datasets confirm the method’s superiority.

A DenseUnet generative adversarial network for near-infrared face image colorization

A DenseUnet Generative Adversarial Network (GAN) is proposed to colorize near-infrared (NIR) face images. The GAN generator incorporates the advantages of both DenseNet and Unet structures. The DenseNet extracts facial features effectively by increasing the network depth, and the Unet keeps important facial details through skip-connection. The generator further integrates an optimized facial loss function designed by considering pixel loss, color loss and feature loss. The proposed network is evaluated against state-of-the-art grayscale image colorization GANs over two separate datasets. Both qualitative and quantitative comparisons demonstrate that with improved network structures and loss constrains the DenseUnet GAN can colorize NIR face images with natural color, minimal face shape distortion, and rich facial details.