Image Colorization Research Articles

DBSF-Net: Infrared Image Colorization Based on the Generative Adversarial Model with Dual-Branch Feature Extraction and Spatial-Frequency-Domain Discrimination

Thermal infrared cameras can image stably in complex scenes such as night, rain, snow, and dense fog. Still, humans are more sensitive to visual colors, so there is an urgent need to convert infrared images into color images in areas such as assisted driving. This paper studies a colorization method for infrared images based on a generative adversarial model. The proposed dual-branch feature extraction network ensures the stability of the content and structure of the generated visible light image; the proposed discrimination strategy combining spatial and frequency domain hybrid constraints effectively improves the problem of undersaturated coloring and the loss of texture details in the edge area of the generated visible light image. The comparative experiment of the public infrared visible light paired data set shows that the algorithm proposed in this paper has achieved the best performance in maintaining the consistency of the content structure of the generated image, restoring the image color distribution, and restoring the image texture details.

Image colorization: A survey and dataset

Image colorization estimates RGB colors for grayscale images or video frames to improve their aesthetic and perceptual quality. Over the last decade, deep learning techniques for image colorization have significantly progressed, necessitating a systematic survey and benchmarking of these techniques. This article presents a comprehensive survey of recent state-of-the-art deep learning-based image colorization techniques, describing their fundamental block architectures, inputs, optimizers, loss functions, training protocols, training data, etc. It categorizes the existing colorization techniques into seven classes and discusses important factors governing their performance, such as benchmark datasets and evaluation metrics. We highlight the limitations of existing datasets and introduce a new dataset specific to colorization. We perform an extensive experimental evaluation of existing image colorization methods using both existing datasets and our proposed one. Finally, we discuss the limitations of existing methods and recommend possible solutions and future research directions for this rapidly evolving topic of deep image colorization. The dataset and codes for evaluation are publicly available at https://github.com/saeed-anwar/ColorSurvey.

Color and Attention for U : Modified Multi Attention U-Net for a Better Image Colorization

Image colorization is a tedious task that requires creativity and understanding of the image context and semantic information. Many models have been made by harnessing various deep learning architectures to learn the plausible colorization. With the rapid discovery of new architecture and image generation techniques, more powerful options can be explored and improved for image colorization tasks. This research explores a new architecture to colorize an image by using pre-trained embeddings on U-Net combined with several attention modules across the model. Using embeddings from a pre-trained classifier provides a high-level feature extraction from the image. Conversely, multi-attention gives a little taste of image segmentation so that the model can distinguish objects in the image and further enhance the additional information given by the pre-trained embeddings. Adversarial training is also utilized as a normalization to make the generated image more realistic. This research preferred Parch GAN over base GAN as the discriminator model to ensure that the colorization has a consistent quality across all patches. The study shows that this U-Net modification can improve the generated image quality compared to a normal U-Net. The proposed architecture reaches an FID of 48.6253, SSIM of 0.8568, and PSNR of 19.7831 by only training it for 25 epochs; hence, this research offers another view of image colorization by using modules that are often used for image segmentation tasks.

APE-GAN: A colorization method for focal areas of infrared images guided by an improved attention mask mechanism

Due to their minimal susceptibility to environmental changes, infrared images are widely applicable across various fields, particularly in the realm of traffic. Nonetheless, a common drawback of infrared images lies in their limited chroma and detail information, posing challenges for clear information retrieval. While extensive research has been conducted on colorizing infrared images in recent years, existing methods primarily focus on overall translation without adequately addressing the foreground area containing crucial details. To address this issue, we propose a novel approach that distinguishes and colors the foreground content with important information and the background content with less significant details separately before fusing them into a colored image. Consequently, we introduce an enhanced generative adversarial network based on Attention mask to meticulously translate the foreground content containing vital information more comprehensively. Furthermore, we have carefully designed a new composite loss function to optimize high-level detail generation and improve image colorization at a finer granularity. Detailed testing on IRVI datasets validates the effectiveness of our proposed method in solving the problem of infrared image coloring.

Quantum deep generative prior with programmable quantum circuits

Exploiting the utility of near-term quantum devices is a long-standing challenge whereas hybrid quantum machine learning emerges as a promising candidate. Here we propose a quantum-enhanced deep generative algorithm based on programmable quantum circuit-induced quantum latent codes. To validate the effectiveness of the algorithm, we conduct optical ghost imaging experiments, collecting dataset under varying physical sampling rates. Leveraging a physically enhanced loss function and a pretrained neural network, the quantum algorithm exhibits superior reconstruction performance compared to conventional algorithms. We observe the learnable quantum latent space enhances the out-of-distribution generalization capability of the pretrained model. In the context of computer vision problems, numerical results indicate that quantum latent space can increase the generation diversity of the pretrained model. Furthermore, the algorithm outperforms classical counterparts, particularly in image inpainting and colorization, by a significant margin. Our study demonstrates the utility of hybrid quantum-classical algorithms in enhancing generalization capability, highlighting the potential of near-term quantum devices in large-scale generative artificial intelligence.

Abstract We016: Generative Artificial Intelligence for hPSC-derived Cardiac Organoid Florescence Generation

Background: Human pluripotent stem cell (hPSC)-derived cardiac organoids (COs) are extensively applied in modeling heart development and disease. COs are daily monitored by bright-field microscopic imaging for tracking organoid differentiation, morphology, and size, however, it does not provide important information on cell distribution and network in COs. Then, fluorescence microscopic imaging is required with extra steps of experiments when multiple hPSC lines are required. Recently, generative artificial intelligence (AI) has been increasingly applied for image colorization, which extends the possibility of colorizing COs with fluorescence from phase/contrast images. While the phase/contrast image is routinely applied and conveniently used in every biomedical lab daily, is it possible to generate accurate and useful fluorescence information or image colorization from the phase/contrast images of cardiac organoids under the same differentiation protocol is crucial for a broader characterization and analysis of hPSC-derived cardiac organoids and further applications? Hypothesis: We hypothesize that generative AI will generate accurate fluorescence images from phase/contrast images of COs. Approach: 1300 images from COs (differentiated from one reporter hPSC line) were used as the training dataset for the Pix2Pix Conditional Generative Adversarial Networks (GANs) model ( Fig.1a ) with the Convolution Block Attention Module (CBAM) in the U-Net generator ( Fig.1b ). Three models were designed: U-Net (Model 1), U-Net generator with CBAM (Model 2), and U-Net with CBAM and Generator Iteration (Model 3). Three different metrics were included: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Weighted Patch Histogram to evaluate the prediction outcomes. Results: The predicted fluorescence images from phase/contrast images of different COs are very similar to the ground truths ( Fig.1c ). The similarities evaluated by all three metrics ( Fig.1d ) have the scores of PSNR over 30, SSIM over 0.92, and a Weighted Patch Histogram score over 0.75, which are the most accurate and similar to the ground truth. Conclusions: A generative AI model was established to colorize phase images of COs using cGANs and CBAM, which capture intricate fluorescence details to support the biomedical research of COs with enhanced interpretability and analysis of cardiac organoid morphology and structure in biomedical research and applications.

Black and White Image Colorization with Deep Learning

Abstract: The project "Black and White Image Colorization with Deep Learning" develops a system to automatically infuse colors into grayscale images through cutting-edge deep learning approaches. Utilizing convolutional neural networks (CNNs) and generative adversarial networks (GANs), the system aims to deliver both realistic and visually pleasing colorization outcomes. This process includes training the deep learning models on diverse image datasets, optimizing the model parameters, and using advanced loss functions to ensure precise color representation.This project addresses the complex challenges associated with adding colors to monochrome images and also seeks to improve user experience by introducing customizable options such as style modification and color tweaks. By merging deep learning with computer vision techniques, this initiative is poised to contribute significantly to various image processing fields, especially in the restoration of historical images and the enhancement of visual content production. With thorough experimentation and ongoing refinements, the project aspires to establish a reliable and easy-to-use system that advances the state of image colorization technology in the modern digital age..

Transforming Color: A Novel Image Colorization Method

This paper introduces a novel method for image colorization that utilizes a color transformer and generative adversarial networks (GANs) to address the challenge of generating visually appealing colorized images. Conventional approaches often struggle with capturing long-range dependencies and producing realistic colorizations. The proposed method integrates a transformer architecture to capture global information and a GAN framework to improve visual quality. In this study, a color encoder that utilizes a random normal distribution to generate color features is applied. These features are then integrated with grayscale image features to enhance the overall representation of the images. Our method demonstrates superior performance compared with existing approaches by utilizing the capacity of the transformer, which can capture long-range dependencies and generate a realistic colorization of the GAN. Experimental results show that the proposed network significantly outperforms other state-of-the-art colorization techniques, highlighting its potential for image colorization. This research opens new possibilities for precise and visually compelling image colorization in domains such as digital restoration and historical image analysis.

Interactive image colorization method via dual local self-attention

The interactive colorization allows users to add a desired color to any location in a grayscale image to obtain the desired color image. While transformers can capture larger receptive fields compared to convolutional neural networks, their computational complexity in image colorization is too high. As a solution, this paper introduces a novel image colorization framework for processing image colorization assignments. In our framework, the dual local self-attention mechanism model is formulated as a solution for reducing the computational complexity associated with global self-attention mechanisms, where the improvement is to combine shift window local self-attention and feature space local self-attention. Both local spatial connections and distant relations are captured to enhance the local quality for the reconstruction. Prior to computing feature space local self-attention, a brightness similarity metric is introduced to cluster the interested area to different local regions. Aiming for real-time inference, the DUpsampling is adopted for signal reconstruction. Specifically, the lightweight convolutional layer is employed before the DUpsampling to mitigate artifacts. Our experimental results on multiple datasets demonstrate that our proposed method outperforms existing interactive colorization methods while having lower model parameters and computational complexity than other Transformer-based models.

Interactive Deep Image Colorization of Quality

Deep Image Colonization is a pioneering project aimed to revolutionizing the field of automated image colorization, particularly focusing on enhancing grayscale photographs' visual appeal and historical significance. Leveraging advanced deep learning models like VGG16 and UNET GAN, the project seeks to accurately and faithfully Add images in black and white some color. Through meticulous evaluation and comparison of different colorization algorithms, including real-time display of results and batch processing capabilities, the project strives to provide users with a seamless and intuitive experience. Beyond aesthetic enhancement, the project explores the implications of automated image colorization in various domains, from historical image restoration to creative visual storytelling. By evaluating colorization accuracy and refining models for real-world usage, the project aims to contribute to the advancement of image processing technologies. Ultimately, "Interactive Deep Image Colonization of Quality" endeavour to fill the void left by the past and the present, providing monochromatic imagery through vibrant hues and precision colorization techniques.

Shadow-aware image colorization

Significant advancements have been made in colorization in recent years, especially with the introduction of deep learning technology. However, challenges remain in accurately colorizing images under certain lighting conditions, such as shadow. Shadows often cause distortions and inaccuracies in object recognition and visual data interpretation, impacting the reliability and effectiveness of colorization techniques. These problems often lead to unsaturated colors in shadowed images and incorrect colorization of shadows as objects. Our research proposes the first shadow-aware image colorization method, addressing two key challenges that previous studies have overlooked: integrating shadow information with general semantic understanding and preserving saturated colors while accurately colorizing shadow areas. To tackle these challenges, we develop a dual-branch shadow-aware colorization network. Additionally, we introduce our shadow-aware block, an innovative mechanism that seamlessly integrates shadow-specific information into the colorization process, distinguishing between shadow and non-shadow areas. This research significantly improves the accuracy and realism of image colorization, particularly in shadow scenarios, thereby enhancing the practical application of colorization in real-world scenarios.

THE IMPACT OF MOVIE COLORIZATION BY ARTIFICIAL INTELLIGENCE ON CINEMATIC SYMBOLISM: A CASE STUDY OF SATYAJIT RAY’S ‘PATHER PANCHALI’

The objective of image colorization is to add color to a monochrome input picture to generate a colorful outcome, which is a classic and essential issue in visual effects. In cinema, color has a significant role in on various levels. But it was not the fundamental component of cinema. Although color was added in film later, but for a long time people were used to watching movies on their black and white television set.This study depends on the effort of a Bengali professor based in the United States who used artificial intelligence to colorize Satyajit Ray’s ‘Pather Panchali’ as a quarantine experiment. This research analyzes the use of colorization in ‘Pather Panchali’ to determine whether the colors work well to emphasize the film's symbolic meaning. It also examines whether there is still a place for sentimentality after colorization and how well AI works at coloring in black and white movies.The research objectives include evaluating the role that colorization plays in bringing the film's meaning to light with regard to certain situations, examining among specialists the nostalgia/emotions linked with classic black and white films, and gauging experts reaction to the practice of using AI to colorize previously black-and-white movies. This study uses in-depth interviews as a qualitative research approach for gathering expert opinions.This study concludes that the cinematic symbolism of the black and white version of ‘Pather Panchali’ is lost in the colorized version. Expert interviews revealed the complex connection between colorization and the film's visual style. The findings emphasize the need for careful consideration and preservation of the original black-and-white format, while also recognizing the advancements and limitations of artificial intelligence in the colorization process.

Nighttime Thermal Infrared Image Colorization With Feedback-Based Object Appearance Learning

Nighttime Thermal Infrared Image Colorization With Feedback-Based Object Appearance Learning

Image Colorization

Abstract: Image colorization is a complex and challenging task in computer vision, with numerous applications in art, entertainment, restoration, and more. This project aims to develop an automated image colorization system leveraging the power of deep learning techniques. The primary objective is to train a deep neural network model capable of accurately and semantically colorizing grayscale images, reproducing natural and visually appealing colour distributions. Our approach utilizes Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs) to learn the intricate relationships between grayscale images and their corresponding colour versions. The project involves the following key steps: Data Collection and Preprocessing, Model architecture, Training, Evaluation, Application. The project's outcome is expected to provide a powerful and versatile tool for automating image colorization tasks, offering high-quality results while preserving the artistic intent of the original images. The fusion of deep learning and computer vision techniques in this project exemplifies the potential for artificial intelligence to revolutionize image processing and creative industries.

Thermal image visualization using multi-discriminator CycleGAN with unpaired thermal–visible image training set

Thermal imaging utilizes the differences in thermal radiation within a scene to capture images, effectively compensating for the limited imaging capabilities of visible-light cameras in low-light environments. In order to further enhance thermal imaging, we propose a novel approach that transforms the visualization of thermal images into an image conversion task. Our method, named TIVNet (Thermal Images Visualization Network), is based on a multi-discriminator cycle-consistent adversarial network (CycleGAN) and operates in the YUV color space to accurately colorize and represent thermal infrared images. The utilization of color and texture discriminators in our framework ensures that the converted images exhibit appropriate color styles and capture fine details, respectively. In the generator, we employed a Dense Residual Module (DRM) to facilitate the reuse of features and enhance the flow of information. By utilizing the DRM, we aim to enhance the overall performance and efficiency of the generator network. Additionally, we incorporated the Dual-channel Self-Attention Module(DCSA) to enhance the feature representation. By leveraging self-attention, the generator can effectively learn and exploit the inherent relationships between different regions, leading to improved feature representation and, consequently, more accurate colorization and representation of thermal infrared images. Notably, our proposed method leverages unsupervised learning, eliminating the need for a fully paired thermal infrared-visible image dataset. Through extensive experimentation on publicly available datasets as well as our own, we demonstrate that our approach outperforms state-of-the-art methods in thermal image visualization. Furthermore, the converted images produced by TIVNet exhibit a high level of realism, thereby enhancing the visual impact of thermal imaging. This improvement is particularly beneficial for human perception and subsequent target recognition tasks. Our findings suggest that the application of our method yields superior results both quantitatively and qualitatively, showcasing its potential for advancing thermal imaging capabilities.

FAColorGAN: a dual-branch generative adversarial network for near-infrared image colorization

FAColorGAN: a dual-branch generative adversarial network for near-infrared image colorization

A Short Analysis of BigColor for Image Colorization

A Short Analysis of BigColor for Image Colorization

A Brief Analysis of iColoriT for Interactive Image Colorization

A Brief Analysis of iColoriT for Interactive Image Colorization

Boosting the Performance of Deep Ear Recognition Systems Using Generative Adversarial Networks and Mean Class Activation Maps

Ear recognition is a complex research domain within biometrics, aiming to identify individuals using their ears in uncontrolled conditions. Despite the exceptional performance of convolutional neural networks (CNNs) in various applications, the efficacy of deep ear recognition systems is nascent. This paper proposes a two-step ear recognition approach. The initial step employs deep convolutional generative adversarial networks (DCGANs) to enhance ear images. This involves the colorization of grayscale images and the enhancement of dark shades, addressing visual imperfections. Subsequently, a feature extraction and classification technique, referred to as Mean-CAM-CNN, is introduced. This technique leverages mean-class activation maps in conjunction with CNNs. The Mean-CAM approach directs the CNN to focus specifically on relevant information, extracting and assessing only significant regions within the entire image. The process involves the implementation of a mask to selectively crop the pertinent area of the image. The cropped region is then utilized to train a CNN for discriminative classification. Extensive evaluations were conducted using two ear recognition datasets: mathematical analysis of images (MAI) and annotated web ears (AWEs). The experimental results indicate that the proposed approach shows notable improvements and competitive performance: the Rank-1 recognition rates are 100.00% and 76.25% for MAI and AWE datasets, respectively.

Principal Uncertainty Quantification With Spatial Correlation for Image Restoration Problems.

Uncertainty quantification for inverse problems in imaging has drawn much attention lately. Existing approaches towards this task define uncertainty regions based on probable values per pixel, while ignoring spatial correlations within the image, resulting in an exaggerated volume of uncertainty. In this paper, we propose PUQ (Principal Uncertainty Quantification) - a novel definition and corresponding analysis of uncertainty regions that takes into account spatial relationships within the image, thus providing reduced volume regions. Using recent advancements in generative models, we derive uncertainty intervals around principal components of the empirical posterior distribution, forming an ambiguity region that guarantees the inclusion of true unseen values with a user-defined confidence probability. To improve computational efficiency and interpretability, we also guarantee the recovery of true unseen values using only a few principal directions, resulting in more informative uncertainty regions. Our approach is verified through experiments on image colorization, super-resolution, and inpainting; its effectiveness is shown through comparison to baseline methods, demonstrating significantly tighter uncertainty regions.