Natural Color Images Research Articles

Automatic Marker Generation to Similarity Based Region Merging Algorithm using Edge Information

One of the most important tasks in computer vision is image segmentation. Several interactive strategies are utilized for picture segmentation because automated techniques are difficult for this kind of work. The outcome of interactive methods is mostly determined by user input. Obtaining excellent interactions for huge datasets is challenging. However, automated picture segmentation is starting to play a significant role in image analysis and computer vision. Effective and efficient segmentation outcomes are obtained using the interacting region merging method suggested by Maximal Similarity based region merging algorithm. Limitation of MSBRM is it requires some efforts on the part of users and is yet not a fully-automatic approach. Here, we suggest a completely fresh unsupervised image segmentation method that combines edge information with MSBRM. We propose an integrated framework to generate object markers for similarity based region algorithm using edge information. Long edges give rough distribution of objects in image. After retrieving edges using phase congruency, edge processing operations are employed to remove small edges and to group color similar long boundaries. Centroids of long boundaries are used as object markers to the MSBRM algorithm. The generation of object markers is done using edge segment grouping. These object markers guide the region merging process. The proposed method shows its effectiveness in segmenting natural real world color images.

Graph Based Cross-Channel Transform for Color Image Compression

Adaptive transform coding is gaining more and more attention for better mining of image content over fixed transforms such as discrete cosine transform (DCT). As a special case, graph transform learning establishes a novel paradigm for the graph-based transforms. However, there still exists a challenge for graph transform learning-based image codecs design on natural image compression, and graph representation cannot describe regular image samples well over graph-structured data. Therefore, in this article, we propose a cross-channel graph-based transform (CCGBT) for natural color image compression. We observe that neighboring pixels having similar intensities should have similar values in the chroma channels, which means that the prominent structure of the luminance channel is related to the contours of the chrominance channels. A collaborative design of the learned graphs and their corresponding distinctive transforms lies in the assumption that a sufficiently small block can be considered smooth, meanwhile, guaranteeing the compression of the luma and chroma signals at the cost of a small overhead for coding the description of the designed luma graph. In addition, a color image compression framework based on the CCGBT is designed for comparing DCT on the classic JPEG codec. The proposed method benefits from its flexible transform block design on arbitrary sizes to exploit image content better than the fixed transform. The experimental results show that the unified graph-based transform outperforms conventional DCT, while close to discrete wavelet transform on JPEG2000 at high bit-rates.

Learning-based adaptive under-sampling for Fourier single-pixel imaging.

In this Letter, we present a learning-based method for efficient Fourier single-pixel imaging (FSI). Based on the auto-encoder, the proposed adaptive under-sampling technique (AuSamNet) manages to optimize a sampling mask and a deep neural network at the same time to achieve both under-sampling of the object image's Fourier spectrum and high-quality reconstruction from the under-sampled measurements. It is thus helpful in determining the best encoding and decoding scheme for FSI. Simulation and experiments demonstrate that AuSamNet can reconstruct high-quality natural color images even when the sampling ratio is as low as 7.5%. The proposed adaptive under-sampling strategy can be used for other computational imaging modalities, such as tomography and ptychography. We have released our source code.

Adversarial Interactive Cartoon Sketch Colourization with Texture Constraint and Auxiliary Auto‐Encoder

AbstractColouring cartoon sketches can help children develop their intellect and inspire their artistic creativity. Unlike photo colourization or anime line art colourization, cartoon sketch colourization is challenging due to the scarcity of texture information and the irregularity of the line structure, which is mainly reflected in the phenomenon of colour‐bleeding artifacts in generated images. We propose a colourization approach for cartoon sketches, which takes both sketches and colour hints as inputs to produce impressive images. To solve the problem of colour‐bleeding artifacts, we propose a multi‐discriminator colourization framework that introduces a texture discriminator in the conditional generative adversarial network (cGAN). Then we combined this framework with a pre‐trained auxiliary auto‐encoder, where an auxiliary feature loss is designed to further improve colour quality, and a condition input is introduced to increase the generalization ability over hand‐drawn sketches. We present both quantitative and qualitative evaluations, which prove the effectiveness of our proposed method. We test our method on sketches of varying complexity and structure, then build an interactive programme based on our model for user study. Experimental results demonstrate that the method generates natural and consistent colour images in real time from sketches drawn by non‐professionals.

Color Image Visual Secret Sharing with Expressive Shares using Color to Gray & Back and Cosine Transform

Color Visual Secret Sharing (VSS) is an essential form of VSS. It is so because nowadays, most people like to share visual data as a color image. There are color VSS schemes capable of dealing with halftone color images or color images with selected colors, and some dealing with natural color images, which generate low quality of recovered secret. The proposed scheme deals with a color image in the RGB domain and generates gray shares for color images using color to gray and back through compression. These shares are encrypted into an innocent-looking gray cover image using a Discrete Cosine Transform (DCT) to make meaningful shares. Reconstruct a high-quality color image through the gray shares extracted from an innocent-looking gray cover image. Thus, using lower bandwidth for transmission and less storage.

Texture Inpainting for Photogrammetric Models

AbstractWe devise a technique designed to remove the texturing artefacts that are typical of 3D models representing real‐world objects, acquired by photogrammetric techniques. Our technique leverages the recent advancements in inpainting of natural colour images, adapting them to the specific context. A neural network, modified and trained for our purposes, replaces the texture areas containing the defects, substituting them with new plausible patches of texels, reconstructed from the surrounding surface texture. We train and apply the network model on locally reparametrized texture patches, so to provide an input that simplifies the learning process, because it avoids any texture seams, unused texture areas, background, depth jumps and so on. We automatically extract appropriate training data from real‐world datasets. We show two applications of the resulting method: one, as a fully automatic tool, addressing all problems that can be detected by analysing the UV‐map of the input model; and another, as an interactive semi‐automatic tool, presented to the user as a 3D ‘fixing’ brush that has the effect of removing artefacts from any zone the users paints on. We demonstrate our method on a variety of real‐world inputs and provide a reference usable implementation.

Recaptured Screen Image Demoiréing in Raw Domain

Capturing screen content by smart-phone cameras has become a daily routine to record or share instant information from display screens for convenience. However, these recaptured screen images are often degraded by moir&#x00E9; patterns and usually present color cast against the original screen source. We observe that performing demoir&#x00E9;ing in raw domain before feeding into the image signal processor (ISP) is more effective than demoir&#x00E9;ing in the sRGB domain as done in recent demoir&#x00E9;ing works. In this paper, we investigate the demoir&#x00E9;ing of raw images through a class-specific learning approach. To this end, we build the first well-aligned raw moir&#x00E9; image dataset by pixel-wise alignment between the recaptured images and source ones. Noting that document images occupy a large portion of screen contents and have different properties from generic images, we propose a class-specific learning strategy for textual images and natural color images. In addition, to deal with moir&#x00E9; patterns with various scales, a multi-scale encoder with multi-level feature fusion is proposed. The shared encoder enables us to extract rich representations for the two kinds of contents and the class-specific decoders benefit the specific content reconstruction by focusing on targeted representations. Experiment results demonstrate that our method achieves state-of-the-art demoir&#x00E9;ing performance. We have released the code and dataset in <uri>https://github.com/tju-chengyijia/RDNet</uri>.

A colorization-based anisotropic variational model for vector-valued image compression

&lt;p style='text-indent:20px;'&gt;Image compression is an important technology in digital image processing. In this paper, a novel colorization-based codec for vector-valued images is proposed. In compression, we first define the concept of "structure image", which contains rich geometric structure information of the vector-valued image. Then, to extract representative pixels from the original vector-valued image, a "one-iteration method" is proposed. It can tremendously improve compression efficiency. In decompression, starting from colorizing the structure image, an anisotropic variational model is proposed. The existence and uniqueness of minimizers for the proposed variational model are established. Besides, we develop a fast and efficient algorithm for solving the model numerically by employing the scaled form of the alternating direction method of multipliers (ADMM). Numerical experiments on natural color images demonstrate that the proposed method outperforms the state-of-art colorization-based image compression method. Compared with the transform-based approaches, experiments on satellite multispectral images illustrate that the proposed method is superior to the JPEG and JPEG2000 standards.&lt;/p&gt;

Effect of Hydrogen Enhancement on Natural Flame Luminosity of Tri-Fuel Combustion in an Optical Engine

A novel combustion mode, namely tri-fuel (TF) combustion using a diesel pilot to ignite the premixed methane–hydrogen–air (CH4–H2–air) mixtures, was experimentally investigated under various H2 fractions (0%, 10%, 20%, 40%, 60%) and ultra-lean conditions (equivalence ratio of φ= 0.5). The overarching objective is to evaluate the effect of H2 fraction on flame characteristics and engine performance. To visualize the effect of H2 fraction on the combustion process and flame characteristics, a high-speed color camera (Photron SA-Z) was employed for natural flame luminosity (NFL) imaging to visualize the instantaneous TF combustion process. The engine performance, flame characteristics, and flame stability are characterized based on cylinder pressure and color natural flame images. Both pressure-based and optical imaging-based analyses indicate that adding H2 into the CH4–air mixture can dramatically improve engine performance, such as combustion efficiency, flame speed, and flame stability. The visualization results of NFL show that the addition of H2 promotes the high-temperature reaction, which exhibits a brighter bluish flame during the start of combustion and main combustion, however, a brighter orangish flame during the end of combustion. Since the combustion is ultra-lean, increasing the H2 concentration in the CH4–air mixture dramatically improves the flame propagation, which might reduce the CH4 slip. However, higher H2 concentration in the CH4–air mixture might lead to a high-temperature reaction that sequentially promotes soot emissions, which emit a bright yellowish flame.

Deep unsupervised endoscopic image enhancement based on multi-image fusion

Background and objectiveA deep unsupervised endoscopic image enhancement method is proposed based on multi-image fusion to achieve high quality endoscope images from poorly illuminated, low contrast and color deviated images through an unsupervised mapping and deep learning network without the need for ground truth. MethodsFirstly, three image enhancement methods are used to process original endoscopic images to obtain three derived images, which are then transformed into HSI color space. Secondly, a deep unsupervised multi-image fusion network (DerivedFuse) is proposed to extract and fuse features of the derived images accurately by utilizing a new no-reference quality metric as loss function. I-channel images of the three derived images are inputted into the DerivedFuse network to enhance the intensity component of the original image. Finally, a saturation adjustment function is proposed to adaptive adjusting the saturation component of HSI color space to enrich the color information of the original input image. ResultsThree evaluation metrics: Entropy, Contrast Improvement Index (CII) and Average Gradient (AG) are used to evaluate the performance of the proposed method. The results are compared with that of fourteen state-of-the-art algorithms. Experiments on endoscopic image enhancement show that the Entropy value of our method is 3.27% higher than the optimal entropy value of comparison algorithms. The CII of our proposed method is 6.19% higher than that of comparison algorithms. The AG of our method is 7.83% higher than the optimal AG of comparison algorithms. ConclusionsThe proposed deep unsupervised multi-image fusion method can obtain image information details, enhance endoscopic images with high contrast, rich and natural color information, visual and image quality. Sixteen doctors and medical students have given their assessments on the proposed method for assisting clinical diagnoses.

Efficient color image retrieval method using deep stacked sparse autoencoder

The recent advancement in deep learning-based approaches vastly outperforms the traditional image descriptors. Deep learning models, such as residual networks (ResNet), are well known for finding salient features. Although effective, high-level description often has a high dimensionality that increases computational overhead. The autoencoders find the useful approximation of the input data without losing critical information. Considering this, we propose a content-based image retrieval system for natural color images using a deep stacked sparse autoencoder (DSSA). The DSSA model learns latent features in an unsupervised way from the high-level description obtained using ResNet. The DSSA model achieves a nearly 50% reduction in size compared with the full-length features for the simple distance-based retrieval approach while increasing accuracy. The image retrieval efficacy of the learned latent features is also evaluated for two classifier-based methods using a Softmax classifier. Further, this study investigates the impact of unsupervised feature learning on retrieval using three benchmark natural color image databases of varying complexities, viz., Corel-1K, Corel-10K, and Canadian Institute for Advanced Research (CIFAR)-10. The latent features learned by the DSSA model with the fuzzy class membership-based retrieval method achieve promising improvements and yield a highly competitive retrieval performance with the large-size CIFAR-10 database.

Perceptually Optimal Color Representation of Fully Polarimetric SAR Imagery.

The four bands of fully polarimetric SAR data convey scattering characteristics of the Earth’s background, but perceptually are not very easy for an observer to use. In this work, the four different channels of fully polarimetric SAR images, namely HH, HV, VH, and VV, are combined so that a color image of the Earth’s background is derived that is perceptually excellent for the human eye and at the same time provides accurate information regarding the scattering mechanisms in each pixel. Most of the elementary scattering mechanisms are related to specific color and land cover types. The innovative nature of the proposed approach is due to the two different consecutive coloring procedures. The first one is a fusion procedure that moves all the information contained in the four polarimetric channels into three derived RGB bands. This is achieved by means of Cholesky decomposition and brings to the RGB output the correlation properties of a natural color image. The second procedure moves the color information of the RGB image to the CIELab color space, which is perceptually uniform. The color information is then evenly distributed by means of color equalization in the CIELab color space. After that, the inverse procedure to obtain the final RGB image is performed. These two procedures bring the PolSAR information regarding the scattering mechanisms on the Earth’s surface onto a meaningful color image, the appearance of which is close to Google Earth maps. Simultaneously, they give better color correspondence to various land cover types compared with existing SAR color representation methods.

Deep-Learning-Based Multispectral Image Reconstruction from Single Natural Color RGB Image—Enhancing UAV-Based Phenotyping

Multispectral images (MSIs) are valuable for precision agriculture due to the extra spectral information acquired compared to natural color RGB (ncRGB) images. In this paper, we thus aim to generate high spatial MSIs through a robust, deep-learning-based reconstruction method using ncRGB images. Using the data from the agronomic research trial for maize and breeding research trial for rice, we first reproduced ncRGB images from MSIs through a rendering model, Model-True to natural color image (Model-TN), which was built using a benchmark hyperspectral image dataset. Subsequently, an MSI reconstruction model, Model-Natural color to Multispectral image (Model-NM), was trained based on prepared ncRGB (ncRGB-Con) images and MSI pairs, ensuring the model can use widely available ncRGB images as input. The integrated loss function of mean relative absolute error (MRAEloss) and spectral information divergence (SIDloss) were most effective during the building of both models, while models using the MRAEloss function were more robust towards variability between growing seasons and species. The reliability of the reconstructed MSIs was demonstrated by high coefficients of determination compared to ground truth values, using the Normalized Difference Vegetation Index (NDVI) as an example. The advantages of using “reconstructed” NDVI over Triangular Greenness Index (TGI), as calculated directly from RGB images, were illustrated by their higher capabilities in differentiating three levels of irrigation treatments on maize plants. This study emphasizes that the performance of MSI reconstruction models could benefit from an optimized loss function and the intermediate step of ncRGB image preparation. The ability of the developed models to reconstruct high-quality MSIs from low-cost ncRGB images will, in particular, promote the application for plant phenotyping in precision agriculture.

Unsupervised Low-Light Image Enhancement by Extracting Structural Similarity and Color Consistency

A novel structure-aware unsupervised network is proposed to deal with low-light image enhancement issues based on the inspiration of Retinex theory and self-supervised perceptual loss. It comprises four main components, namely the original structural similarity module, the novel color consistency module, the attentional enhancement module, and the naturalness discriminator module. Specially for the structural similarity module, an embedded structural feature extractor (SFE) model capable of generating structure correspondence is well designed and pre-trained by employing the contrastive learning technique, and a multi-scale structural similarity distance is introduced to optimize the SFE network. Besides, a self-supervised color consistency module is established by using a degraded estimation algorithm for recovering the missing colors. The whole enhancement framework operates in unsupervised manners and finally obtains the best naturalness image quality evaluator metric. Experimental results demonstrate that the proposed unsupervised network is able to recover natural structure and color images more effectively, which would also help to enlarge the practical application without collecting paired datasets in advance.

Non-Local Robust Quaternion Matrix Completion for Large-Scale Color Image and Video Inpainting.

The image nonlocal self-similarity (NSS) prior refers to the fact that a local patch often has many nonlocal similar patches to it across the image and has been widely applied in many recently proposed machining learning algorithms for image processing. However, there is no theoretical analysis on its working principle in the literature. In this paper, we discover a potential causality between NSS and low-rank property of color images, which is also available to grey images. A new patch group based NSS prior scheme is proposed to learn explicit NSS models of natural color images. The numerical low-rank property of patched matrices is also rigorously proved. The NSS-based QMC algorithm computes an optimal low-rank approximation to the high-rank color image, resulting in high PSNR and SSIM measures and particularly the better visual quality. A new tensor NSS-based QMC method is also presented to solve the color video inpainting problem based on quaternion tensor representation. The numerical experiments on color images and videos indicate the advantages of NSS-based QMC over the state-of-the-art methods.

Multi-ColorGAN: Few-shot vehicle recoloring via memory-augmented networks

Despite the notable successes of Generative adversarial networks (GANs) achieved to date, applying them to real-world problems still poses significant challenges. In real traffic surveillance scenarios, for the task of generating images of multiple color of truck heads and cars without changing textures and license plates, conditional image generation hardly manipulate the generated images by the color attribute. Image style transfer methods inevitably produce color smearing. Even state-of-the-art methods of disentangled representation learning (e.g. MixNMatch) cannot disentangle colors individually, ensuring that irrelevant factors, such as texture remain the same. To solve this problem, we present an approach called Multi-ColorGAN based on memory-augmented networks for multi-color real vehicle coloring/generation with limited data. In particular, our model could filter out unwanted color changes in specific areas with a simple but effective method called Fusion Module, and generate more natural color images. Experiments on three vehicle image benchmarks and a new truck image dataset are conducted to evaluate the proposed Multi-ColorGAN compared to state-of-the-art.

Detection of GAN-Synthesized Image Based on Discrete Wavelet Transform

Recently, generative adversarial networks (GANs) and its variants have shown impressive ability in image synthesis. The synthesized fake images spread widely on the Internet, and it is challenging for Internet users to identify the authenticity, which poses huge security risk to the society. However, compared with the powerful image synthesis technology, the detection of GAN-synthesized images is still in its infancy and face a variety of challenges. In this study, a method named fake images discriminator (FID) is proposed, which detects that GAN-synthesized fake images use the strong spectral correlation in the imaging process of natural color images. The proposed method first converts the color image into three color components of R, G, and B. Discrete wavelet transform (DWT) is then applied to RGB components separately. Finally, the correlation coefficient between the subband images is used as a feature vector for authenticity classification. Experimental results show that the proposed FID method achieves impressive effectiveness on the StyleGAN2-synthesized faces and multitype fake images synthesized with the state-of-the-art GANs. Also, the FID method exhibits good robustness against the four common perturbation attacks.

Color Texture Image Complexity—EEG-Sensed Human Brain Perception vs. Computed Measures

In practical applications, such as patient brain signals monitoring, a non-invasive recording system with fewer channels for an easy setup and a wireless connection for remotely monitor physiological signals will be beneficial. In this paper, we investigate the feasibility of using such a system in a visual perception scenario. We investigate the complexity perception of color natural and synthetic fractal texture images, by studying the correlations between four types of data: image complexity that is expressed by computed color entropy and color fractal dimension, human subjective evaluation by scoring, and the measured brain EEG responses via Event-Related Potentials. We report on the considerable correlation experimentally observed between the recorded EEG signals and image complexity while considering three complexity levels, as well on the use of an EEG wireless system with few channels for practical applications, with the corresponding electrodes placement in accordance with the type of neural activity recorded.

A fully-automatic image colorization scheme using improved CycleGAN with skip connections

Image colorization is the process of assigning different RGB values to each pixel of a given grayscale image to obtain the corresponding colorized image. In this work, we propose a new automatic image colorization method based on the modified cycle-consistent generative adversarial network (CycleGAN). This method can generate a natural color image with only one given gray image without reference image or manual interaction. In the proposed method, we first modify the original network structure by combining a u-shaped network with a skip connection to improve the ability of feature representation in image colorization. Meanwhile, we design a compounded loss function to measure the errors between the ground-truth image and the predicted result to improve the authenticity and naturalness of the colorized image; further, we also add the detail loss function to ensure that the details of the generated color and grayscale images are substantially similar. Finally, the performance of the proposed model is verified on different datasets. Experiments show that our method can generate more realistic color images when compared to other methods.

Superpixels extraction by an Intuitionistic fuzzy clustering algorithm

A scheme to develop the image over-segmentation task is introduced in this paper, it considers the pixels of an image as intuitive fuzzy sets and develops an intuitionistic clustering process of them. In this regard, the main contribution is to provide a method for extracting superpixels with greater adherence to the edges of the regions. Experimental tests were developed considering biomedical grayscale and natural color images. The robustness and effectiveness of this proposal was verified by quantitative and qualitative results.