Color Image Reconstruction Research Articles

Deep learning-based color holographic microscopy.

We report a framework based on a generative adversarial network that performs high-fidelity color image reconstruction using a single hologram of a sample that is illuminated simultaneously by light at three different wavelengths. The trained network learns to eliminate missing-phase-related artifacts, and generates an accurate color transformation for the reconstructed image. Our framework is experimentally demonstrated using lung and prostate tissue sections that are labeled with different histological stains. This framework is envisaged to be applicable to point-of-care histopathology and presents a significant improvement in the throughput of coherent microscopy systems given that only a single hologram of the specimen is required for accurate color imaging.

Quaternion weighted spherical Bessel-Fourier moment and its invariant for color image reconstruction and object recognition

Moments with capable of representing global features have been widely used in image analysis, pattern recognition and computer vision applications. However, existing image moment theories are still mainly paid close attention to gray-scale images. In this paper, a new quaternion moment termed as Quaternion weighted Spherical Bessel–Fourier Moment (QSBFM) is proposed to address the issue of color image analysis. Moreover, the relationship between QSBFM and the conventional weighted spherical Bessel–Fourier moment has been established so that the computational cost of QSBFM can be efficiently reduced. Furthermore, the natively geometric invariance of QSBFM is presented and discussed in detail. Experimental results show that the proposed QSBFM outperforms frequently-used quaternion moments including quaternion Bessel–Fourier moment, quaternion Radial-Harmonic-Fourier moment, quaternion Chebyshev-Fourier moment and quaternion orthogonal Fourier–Mellin moment in terms of the color image reconstruction capability and invariant recognition accuracy in noise-free, Salt & Pepper noise and the Gaussian noise conditions.

Three dimensional colorization based image/video reconstruction from white-dominant RGBW pattern images

RGBW (RGB-White) CFAs (Color Filter Arrays) which contain white pixels as well as RGB pixels have been actively proposed, which is due to the higher SNR (Signal to Noise Ratio) characteristics of the white pixels. The higher SNR characteristics of the white pixels results in the reconstruction of color images with higher resolutions than those reconstructed with widely used RGB CFAs, especially in low illumination environments. Recently, we proposed a new RGBW demosaicking method which uses an RGBW CFA where the white pixels cover 75% of the CFA, and which reconstructs the color image from the RGBW pattern image by use of the colorization method. The use of a large number of white pixels, and the use of colorization as an interpolation method made it possible to reconstruct a color image with high SNR values.In this paper, we extend this method to reconstruct video sequences by extending the colorization method to the 3-D domain, i.e., we propose a 3-D color reconstruction method which takes an RGBW pattern video sequence as the input and produces a reconstructed color video sequence. We implement the colorization as a 3-D diffusion instead of solving an overall colorization matrix, since the computation is high for the 3-D domain. The 3-D diffusion flow stops at temporal edges, which prevents the false colorizing of moving objects. Furthermore, the 3-D diffusion flow also eliminates the incoherence between colors of neighboring frames, which eliminates the flickering artifact. One major difficulty in extending to the temporal axis is that the noise in the few color seeds can propagate throughout the 3-D domain resulting in many false colors. This is especially important in the case when the image/video is taken in low illumination, since the energy of the thermal noise becomes relatively large compared to the light energy making the RGBW pattern image/video very noisy. To prevent the propagation of false colors caused by the noise in the color seeds, we use the 3-D stacked white channels as a guidance for denoising, so that the denoising takes into account only the color seeds that belong to the same object region in the 3-D space. The stacked white channels also guide the 3-D diffusion throughout the 3-D space. Experimental results show that the image/video reconstructed by the proposed method has better visual quality than those reconstructed by conventional RGB/RGBW CFA based methods. Therefore, the proposed method can be used in surveillance camera applications to recover important information from noisy video sequences taken in very low illumination environment.

Megapixel photon-counting color imaging using quanta image sensor.

Quanta Image Sensor (QIS) is a single-photon detector designed for extremely low light imaging conditions. Majority of the existing QIS prototypes are monochrome based on single-photon avalanche diodes (SPAD). Passive color imaging has not been demonstrated with single-photon detectors due to the intrinsic difficulty of shrinking the pixel size and increasing the spatial resolution while maintaining acceptable intra-pixel cross-talk. In this paper, we present image reconstruction of the first color QIS with a resolution of 1024 × 1024 pixels, supporting both single-bit and multi-bit photon counting capability. Our color image reconstruction is enabled by a customized joint demosaicing-denoising algorithm, leveraging truncated Poisson statistics andvariance stabilizing transforms. Experimental results of the new sensor and algorithm demonstrate superior color imaging performance for very low-light conditions with a mean exposure of as low as a few photons per pixel in both real and simulated images.

Convolutional sparse coding for demosaicking with panchromatic pixels

Image demosaicking is the problem of reconstructing color images from raw images captured by a digital camera covered by a Color Filter Array (CFA). CFAs allow only part of the incident light to transmit, so they will lose some incident light. Recently, CFAs with panchromatic pixels are used to avoid excessive loss of light. These CFAs’ color is sampled at a sparse set of locations, making demosaicking more challenging. In this paper, we propose a Convolutional Sparse Coding based method for Demosaicking (CSCD) with panchromatic pixels. The CSCD learns a set of filters to decompose training images into color sparse feature maps, then it estimates a set of feature maps of the to-be-reconstructed raw images and gets color images. Convolutional Sparse Coding (CSC) commonly operates on gray images. Naively extending CSC to color images will make the learned filters to be gray, which means that the values of the red, green, and blue channels are almost equal. To address this issue, we add a regularization term to penalize gray filters. We use the Alternating Direction Method of Multipliers (ADMM) to solve the CSCD. We compare the proposed CSCD with the demosaicking methods that are applicable to CFAs with panchromatic pixels. Experimental results validate its advantages in terms of CPSNR and visual quality.

Adaptive compressed sensing of color images based on salient region detection

We propose a novel algorithm for color image compressed sensing (CS). Our method involves the adaptive measurement and reconstruction of color images based on visual saliency detection. First, we divide the image into blocks and transform the RGB channel into the YUV channel. Secondly, we use statistical texture distinctiveness to calculate the saliency of each block and normalize energy, thereby establishing an adaptive measurement rate and measurement matrix. Thirdly, we adaptively measure the Y channel according to block prominence and preserve the information of the UV channel. During reconstruction, we utilize adaptive block measurement rate to re-estimate block saliency and then reconstruct the objective function of the weighted reconstruction model according to the re-estimated block saliency. Finally, we combine the reconstructed Y channel with the reserved UV channel to obtain the final image. Experimental results show that compared with other state-of-the-art approaches, the proposed algorithm can not only provide good subjective visual quality but can also present higher peak signal to noise ratio (PSNR) under the same sampling rate.

Color Inverse Halftoning Method with the Correlation of Multi-Color Components Based on Extreme Learning Machine

Look-up table (LUT) based method is a popular and effective way for inverse halftoning. However, it still has very large development space to improve the reconstructed color image quality for color halftone images, because most of the existing color inverse halftoning methods are the simple extension of LUT methods to each color components separately. To this end, this paper presents a novel color inverse halftoning method by exploiting the correlation of multi-color components. Through considering all existent contone values with the same halftone pattern in three color component tables, we firstly propose a concept of common pattern. Then the extreme learning machine (ELM) is employed to estimate the contone values for nonexistent patterns according to common patterns in color LUT, which can not only improve the fitting precision of nonexistent values but also has fast transformation speed. Experimental results show that the proposed method achieves a better image quality when compared to previously published methods.

반복적 직접 이진 탐색 알고리즘을 이용한 컴퓨터 생성 홀로그래피 기반의 풀 컬러 이미지의 광학적 복원

우리는 컴퓨터 생성 홀로그래피를 기반으로 획득한 풀 컬러 이미지를 광학적으로 복원하는 과정을 제안한다. 풀컬러의 복소 홀로그램을 각각 기록하고 직접 이진 검색 알고리즘을 사용하여 이진 바이너리 홀로그램으로 변환한다. 생성한 이진 홀로그램은 공간 광 변조기를 사용하여 광학적으로 복원하였다. 각 색상에 의한 색수차로 인해 발생하는 객체의 크기 불일치는 이진 바이너리 홀로그램 데이터를 획득 할 때 복원 파라미터를 조정하여 보정하였다. 우리가 알고 있는 한, 이것은 컴퓨터 생성 홀로그래피로 기록된 풀 컬러 홀로그램을 광학 복원한 최초의 연구이다.

Piecewise constant reconstruction of damaged color images

A variational model for reconstruction of damaged color images is studied, in particular in the case where only finitely many colors are admissible for the reconstructed image. An existence result and regularity properties of minimizers are presented.

Improved Two-Dimensional Quaternion Principal Component Analysis

The two-dimensional quaternion principal component analysis (2D-QPCA) is first improved into abstracting the features of quaternion matrix samples in both row and column directions, being the generalization ability, and with the components weighted by corresponding eigenvalues. The main components of the 2D-QPCA in row and column directions are defined as the solution of one optimal problem with orthogonality constraints on two variables. In advance, 2D-QPCA is innovatively armed with the generalization ability by applying the label information (if known) to generate the weight of each class of training set. With such generalization ability, 2D-QPCA may enlarge the variance of all projections of known and unknown samples. Different from the well-known PCA-like methods, the improved 2D-QPCA prefers to the components that have larger variances of projected samples and weight them with larger factors. The improved versions of 2D-QPCA are applied to the color face recognition and image reconstruction. The numerical results based on the real face data sets validate that the newly proposed method performs better than the state-of-the-art ones.

Color image identification and reconstruction using artificial neural networks on multimode fiber images: towards an all-optical design.

The rapid growth of applications that rely on artificial neural network (ANN) concepts gives rise to a staggering increase in the demand for hardware implementations of neural networks. New types of hardware that can support the requirements of high-speed associative computing while maintaining low power consumption are sought, and optical artificial neural networks fit the task well. Inherently, optical artificial neural networks can be faster, support larger bandwidth, and produce less heat than their electronic counterparts. Here we propose the design of an optical ANN-based imaging system that has the ability to self-study image signals from an incoherent light source in different colors. Our design consists of a combination of a multimode fiber and a multi-core optical fiber realizing a neural network. We show that the signals, transmitted through the multimode fiber, can be used for image identification purposes and can also be reconstructed using ANNs with a low number of nodes. An all-optical solution can then be achieved by realizing these networks with the multi-core optical neural network fiber.

Color Image Reconstruction Based On Digital Holography

In this paper, a color image reconstruction algorithm based on digital hologram is proposed. The color image is decomposed into three primary color images, and the corresponding hologram is generated in each channel using the four step phase shift method. Then the image of each channel is reconstructed using the inverse Fourier transform, the reconstructed graph of three channels is superimposed to obtain the final color reconstruction image. The reconstructed image overcomes the zero-order image and the straight-through bright spot generated during Fresnel digital holographic image generation in the coaxial recording mode.

Demosaicking based on channel-correlation adaptive dictionary learning

Image demosaicking is the problem of reconstructing color images from raw images captured using a digital camera with a color filter array. Sparse representation-based demosaicking method achieves superior performance on the commonly used Kodak dataset; however, it performs inferior on the IMAX dataset. We analyze that the factor of the sparse representation-based demosaicking methods that perform inconsistently is channel-correlation, which we define as the mean value of correlation coefficients between the RGB channels. Accordingly, we propose a channel-correlation adaptive dictionary learning-based demosaicking method. Different from the sparse representation-based demosaicking methods that use a fixed dictionary, our method trains a general dictionary on training image patches with various channel-correlations. Then, we learn a function matrix between the general dictionary and channel-correlations. For a raw image patch with an estimated channel-correlation, we compute a dictionary corresponding to its channel-correlation through the function matrix. Finally, we demosaick it with the corresponding dictionary using the sparse representation model. Experiments confirm that the proposed method performs adaptively well on raw images with various channel-correlations.

Single-Pixel Color Imaging Method with a Compressive Sensing Measurement Matrix

Compressive sensing theory has addressed the limitations of traditional methods in the field of information technology, and led to a revolution. On the basis of compressive sensing theory research, this study utilized the exterior determinacy and inherent randomness of chaotic sequences, designed a pseudo-random circulant measurement matrix based on chaotic sequence. Compared with other deterministic measurement matrices, the restoration effect of the designed measurement matrix remarkably improved and showed advantages in hardware and storage. Then, this study developed a single-pixel imaging scheme that could accurately obtain color information. The proposed improved measurement matrix combined with the hardware system could accurately reconstruct color images and had good robustness according to various experimental data.

Coherent Narrow-Band Light Source for Miniature Endoscopes

In this work, we report the successful implementation of a coherent narrow-band light source for miniature endoscopy applications. An RGB laser module that provides much higher luminosity than traditional incoherent white light sources is used for illumination, taking advantages of the laser light's high spatial coherence for efficient light coupling. Notably, the narrow spectral band of the laser light sources also enables spectrally resolved imaging, to distinguish certain biological tissues or components. A monochrome CMOS camera is employed to synchronize with the time lapsed RGB laser module illumination for color image acquisition and reconstruction, which provides better spatial resolution than a color CMOS camera of comparable pixel number, in addition to spectral resolving.

Storage and reconstruction of multiple color images with a phase-only hologram

An optical method for storing and recovering multiple color images is presented in this paper. In the storing process, the red, green, and blue components of a color image are separated and respectively quantized. Then the three quantized components are combined as a gray image. Multiple gray images are modulated with corresponding blazed gratings and superposed in a phase-only hologram. In the recovering process, a spatial filter is used to extract all the gray images from the hologram, and the gray scales of the gray images are then converted to the pre-defined colors. The simulation shows that 206 color images can be stored in a phase-only hologram of a size of 512 by 512 pixels and reconstructed from the hologram with high fidelity.

Color-image reconstruction for two-wavelength digital holography using a generalized phase-shifting approach.

We propose a color-image reconstruction method for two-wavelength digital holography using generalized phase-shifting digital holography (GPSDH). In this method, color interference fringes are captured by a digital camera with a Bayer array color filter, and phase shifting is simultaneously performed for all wavelengths. Color interference fringes are separated into three monochromatic interference fringes using a color-separation method that suppresses the color-filter crosstalk. The object wave is extracted from each monochromatic interference fringe using GPSDH, which prevents problems due to the phase shift wavelength dependence. Image reconstruction is performed using a shifted Fresnel transform-based method, in which the color reconstructed image is obtained by directly superposing the reconstructed images for all wavelengths. We verify the proposed method through optical experiments with a two-wavelength digital holography system. The results show that the dual-color image can be successfully reconstructed without chromatic aberration.

Color holographic system without undesirable light based on area sampling of digital lens

AbstractIn this paper, we demonstrate a color holographic system without undesirable light based on area sampling of a digital lens. This system, which improves the quality of the reconstructed color image, is fundamentally realized by using three spatial light modulators and a rectangle aperture. By altering the sampling area of the digital lens on the spatial light modulator panels, we can control the offset position of the reconstructed image. Thus, the zero‐order diffraction light as the undesirable light and the reconstructed images are separated, and the chromatic aberration can be corrected. In addition, a rectangle aperture is used to eliminate the zero‐order diffraction light. The final reconstructed image with improved quality can be observed, which verifies the effectiveness of our proposed system.

Colorization-Based RGB-White Color Interpolation using Color Filter Array with Randomly Sampled Pattern

Recently, several RGB-White (RGBW) color filter arrays (CFAs) have been proposed, which have extra white (W) pixels in the filter array that are highly sensitive. Due to the high sensitivity, the W pixels have better SNR (Signal to Noise Ratio) characteristics than other color pixels in the filter array, especially, in low light conditions. However, most of the RGBW CFAs are designed so that the acquired RGBW pattern image can be converted into the conventional Bayer pattern image, which is then again converted into the final color image by using conventional demosaicing methods, i.e., color interpolation techniques. In this paper, we propose a new RGBW color filter array based on a totally different color interpolation technique, the colorization algorithm. The colorization algorithm was initially proposed for colorizing a gray image into a color image using a small number of color seeds. Here, we adopt this algorithm as a color interpolation technique, so that the RGBW color filter array can be designed with a very large number of W pixels to make the most of the highly sensitive characteristics of the W channel. The resulting RGBW color filter array has a pattern with a large proportion of W pixels, while the small-numbered RGB pixels are randomly distributed over the array. The colorization algorithm makes it possible to reconstruct the colors from such a small number of RGB values. Due to the large proportion of W pixels, the reconstructed color image has a high SNR value, especially higher than those of conventional CFAs in low light condition. Experimental results show that many important information which are not perceived in color images reconstructed with conventional CFAs are perceived in the images reconstructed with the proposed method.

Color Image Reconstruction by Discrete Orthogonal Moment

Image reconstruction can help to determine how well an image may be characterized by a small finite set of its moments. Also, we can identify the number of descriptors needed to describe an image. In this work, we present a comparative analysis using different set of discrete orthogonal moments in terms of normalized image reconstruction error (NIRE). Color image reconstruction is performed with different color channels and various orders of different discrete orthogonal moments. Finally the results obtained by the reconstruction of three color images with different families of orthogonal moments and an error analysis to compare their capacity of description are presented, also the conclusions obtained from this work are presented.