Color Filter Array Image Research Articles

Color texture analysis using CFA chromatic co-occurrence matrices

Most color cameras are fitted with a single sensor that provides color filter array (CFA) images, in which each pixel is characterized by one of the three color components (either red, green, or blue). To produce a color image, the two missing color components have to be estimated at each pixel of the corresponding CFA image. This process is commonly referred to as demosaicing, and its result as the demosaiced color image.Since demosaicing methods intend to produce “perceptually satisfying” demosaiced color images, they attempt to avoid color artifacts. Because this is often achieved by filtering, demosaicing schemes tend to alter the local texture information that is, however, useful to discriminate texture images. To avoid this issue while exploiting color information for texture classification, it may be relevant to compute texture descriptors directly from CFA images.From chromatic co-occurrence matrices (CCMs) that capture the spatial interaction between color components, we derive new descriptors (CFA CCMs) for CFA texture images. Color textures are then compared by means of the similarity between their CFA CCMs. Experimental results achieved on benchmark color texture databases show the efficiency of this approach for texture classification.

CFA local binary patterns for fast illuminant-invariant color texture classification

This paper focuses on the classification of color textures acquired by single-sensor color cameras under various illuminants. Local binary patterns (LBPs) are robust texture descriptors suited to such conditions. This property is still improved when LBPs are computed from the level ranks. Our main contribution is to avoid the demosaicing step that is classically performed in single-sensor color cameras to estimate color images from raw data. We instead compute rank-based LBPs from the color filter array image, in which each pixel is associated to a single color component. Experimental results achieved on a benchmark color texture database show the effectiveness of the proposed approach for texture classification, and a complexity study highlights its computational efficiency.

Statistical grey level and noise evaluation of Foveon X3 and CFA image sensors

This paper presents a practical, comprehensive and flexible statistical framework to analyze the radiometric response values and their uncertainty effects on image-based sensor systems. The statistical framework is based on experimental techniques and is tested on two single lens reflex (SLR) cameras powered by different image sensors, Foveon X3 and Color Filter Array (CFA). A colorchecker inside an indoor light booth is used as target. Design of Experiments (DoE) for Linear Models (LM) are derived to analyze and characterize the variability of the output grey level signal (digital number) and their uncertainty effects according to influential factors such as scene reflectance, wavelength range and time. The digital numbers are statistically modeled, and their noise and temporal trend are the uncertainty effects analyzed. Experiments are carried out under laboratory conditions to minimize the rest of uncertainty sources that might affect digital numbers. The flexibility of the statistical framework is confirmed to model and characterize the digital numbers, as well as the noise of a single image and the stability (trend and noise) of a temporal sequence of images.

Colour texture classification from colour filter array images using various colour spaces

This study focuses on the classification of colour textures acquired by single-sensor colour cameras. In such cameras, the colour filter array (CFA) makes each photosensor sensitive to only one colour component, and CFA images must be demosaiced to estimate the final colour images. We show that demosaicing is detrimental to the textural information because it affects colour texture descriptors such as chromatic co-occurrence matrices (CCMs). However, it remains desirable to take advantage of the chromatic information for colour texture classification. This information is incompletely defined in CFA images, in which each pixel is associated to a single colour component. It is hence a challenge to extract standard colour texture descriptors from CFA images without demosaicing. We propose to form a pair of quarter-size colour images directly from CFA images without any estimation, then to compute the CCMs of these quarter-size images. This allows us to compare textures by means of their CCM-based similarity in texture classification or retrieval schemes, with still the ability to use different colour spaces. Experimental results achieved on benchmark colour texture databases show the effectiveness of the proposed approach for texture classification, and a complexity study highlights its computational efficiency.

CMOS Image Sensor Performance from Different Order of Interpolation and Noise Reduction

Three noise reduction programs based on Color Filter Array (CFA) image from Complementary Metal-Oxide-Semiconductor (CMOS) image sensor system are proposed. Imaging performance from different order of interpolation and noise reduction is researched. The relevant models are established and three interpolation methods and four noise reduction methods are selected in the paper. The results show that under different imaging sequence, whether from objective evaluation or from the subjective point of view, Linear Filtering noise reduction is the worst, noise reduction on Least Squares Method is optimal. The conclusions have references to digital camera imaging.

Demosaicking of Noisy Bayer-Sampled Color Images With Least-Squares Luma-Chroma Demultiplexing and Noise Level Estimation

This paper adapts the least-squares luma-chroma demultiplexing (LSLCD) demosaicking method to noisy Bayer color filter array (CFA) images. A model is presented for the noise in white-balanced gamma-corrected CFA images. A method to estimate the noise level in each of the red, green, and blue color channels is then developed. Based on the estimated noise parameters, one of a finite set of configurations adapted to a particular level of noise is selected to demosaic the noisy data. The noise-adaptive demosaicking scheme is called LSLCD with noise estimation (LSLCD-NE). Experimental results demonstrate state-of-the-art performance over a wide range of noise levels, with low computational complexity. Many results with several algorithms, noise levels, and images are presented on our companion web site along with software to allow reproduction of our results.

Robust high-high frequency sub-band for demosaicking the inter-channel weak correlated CFA image

A novel approach to get the robust high-high frequency sub-band (HHFSB) of the missing color channel for reconstructing the inter-channel weak correlated color filter array (CFA) image is proposed in this paper. This algorithm can adaptively adopt one of the two distinct schemes to estimate the HHFSB of the missing color channel in each local region according to the correlation coefficient of the local inter-channel HHFSB. The experimental results confirm that the whole HHFSB of the missing color channel achieved from the proposed method is more accurate and closer to the actual one reported from the state-of-the-art approaches.

Lossless compression of HDR color filter array image for the digital camera pipeline

This paper introduces a lossless color filter array (CFA) image compression scheme capable of handling high dynamic range (HDR) representation. The proposed pipeline consists of a series of pre-processing operations followed by a JPEG XR encoding module. A deinterleaving step separates the CFA image to sub-images of a single color channel, and each sub-image is processed by a proposed weighted template matching prediction. The utilized JPEG XR codec allows the compression of HDR data at low computational cost. Extensive experimentation is performed using sample test HDR images to validate performance and the proposed pipeline outperforms existing lossless CFA compression solutions in terms of compression efficiency.

Efficient reversible data hiding for color filter array images

A reversible data hiding algorithm which uses prediction errors in the color difference domain for mosaic images with the Bayer color filter array (CFA) is proposed. Furthermore, the proposed algorithm can be extended to deal with the digital time delay and integration (DTDI) mosaic images and Lukac and Plataniotis (LP) mosaic images. Experimental results on CFA, DTDI, and LP mosaic images demonstrate that the proposed algorithm can achieve high embedding capacity while maintaining good image quality.

Improvement of BM3D Algorithm and Employment to Satellite and CFA Images Denoising

This paper proposes a new procedure in order to improve the performance of block matching and 3-D filtering (BM3D) image denoising algorithm. It is demonstrated that it is possible to achieve a better performance than that of BM3D algorithm in a variety of noise levels. This method changes BM3D algorithm parameter values according to noise level, removes prefiltering, which is used in high noise level; therefore Peak Signal-to-Noise Ratio (PSNR) and visual quality get improved, and BM3D complexities and processing time are reduced. This improved BM3D algorithm is extended and used to denoise satellite and color filter array (CFA) images. Output results show that the performance has upgraded in comparison with current methods of denoising satellite and CFA images. In this regard this algorithm is compared with Adaptive PCA algorithm, that has led to superior performance for denoising CFA images, on the subject of PSNR and visual quality. Also the processing time has decreased significantly.

Novel color demosaicking for noisy color filter array data

Single sensor digital color still/video cameras use color demosaicking to reproduce full color images from color filter array (CFA) data. The quality of interpolated image will be degraded due to the sensor noise introduced during the image capture process. Many conventional demosaicking–denoising solutions adopt the channel-dependent noise model, which may fit the CMOS/CCD image sensor less than signal-dependent noise model. In this paper, the wavelet sub-band decomposition and synthesis are applied to interpolate the CFA data with signal-dependent noise model. The major contributions of this work include: (1) The combination of LMMSE and statistical calculation in wavelet domain are utilized to suppress the signal-dependent noise, which is separated into additive noise and multiplicative noise. (2) In CFA data, it has been verified that the quantitative relationship between the current pixel and the adjacent pixel, which locate in the same edge. Both simulated and real CFA images are employed to compare the proposed algorithm with the state-of-the-art techniques reported in the literature. The experimental results confirm that our method outperforms them both on demosaicking performance and on computational cost, when they process the noisy color filter array data.

RETRACTED ARTICLE: Dense stereovision using mono-CCD color cameras

Most stereo algorithms are based only on an analysis of the luminance information. However, with advances in camera technology, in addition to the fact that color information can robustly improve matching, color stereovision is receiving more and more attention. Color stereovision setups are usually based on single-sensor cameras which provide color filter array (CFA) images. In these images, a single color component is sampled at each pixel rather than the three required components red, green, and blue (RGB). We show that standard demosaicing techniques, which are used to interpolate the missing components, are not well adapted when the resulting color pixels are matched in order to estimate image disparities. In order to avoid this problem while exploiting color information, we propose a new matching system designed for dense stereovision based on pairs of CFA images.

CFA-based motion blur removal using long/short exposure pairs

This paper presents an efficient and effective motion blur removal method based on long and short exposure images. The two images are captured sequentially and motion pixels between the images are then robustly detected, with suppression of noise and prevention of artifacts around object boundaries. Object motion blur is removed and high quality image is obtained by merging the two images with taking into account the detected motion pixels. The proposed method is directly performed on the CFA (Color Filter Array) image which only has one color component per pixel. It has low computational complexity and low memory requirements. The proposed method also achieves a HDR (High Dynamic Range) image at the same time.

PCA-Based Spatially Adaptive Denoising of CFA Images for Single-Sensor Digital Cameras

Single-sensor digital color cameras use a process called color demosiacking to produce full color images from the data captured by a color filter array (CAF). The quality of demosiacked images is degraded due to the sensor noise introduced during the image acquisition process. The conventional solution to combating CFA sensor noise is demosiacking first, followed by a separate denoising processing. This strategy will generate many noise-caused color artifacts in the demosiacking process, which are hard to remove in the denoising process. Few denoising schemes that work directly on the CFA images have been presented because of the difficulties arisen from the red, green and blue interlaced mosaic pattern, yet a well-designed "denoising first and demosiacking later" scheme can have advantages such as less noise-caused color artifacts and cost-effective implementation. This paper presents a principle component analysis (PCA)-based spatially-adaptive denoising algorithm, which works directly on the CFA data using a supporting window to analyze the local image statistics. By exploiting the spatial and spectral correlations existing in the CFA image, the proposed method can effectively suppress noise while preserving color edges and details. Experiments using both simulated and real CFA images indicate that the proposed scheme outperforms many existing approaches, including those sophisticated demosiacking and denoising schemes, in terms of both objective measurement and visual evaluation.

Effective soft-decision demosaicking using directional filtering and embedded artifact refinement

Demosaicking is an interpolation process that transforms a color filter array (CFA) image into a full-color image in a single-sensor imaging pipeline. In all demosaicking techniques, the interpolation of the green components plays a central role in dictating the visual quality of reconstructed images because green light is of maximum sensitivity in the human visual system. Guided by this point, we propose a new soft-decision demosaicking algorithm using directional filtering and embedded artifact refinement. The novelty of this approach is twofold. First, we lift the constraint of the Bayer CFA that results in the absence of diagonal neighboring green color values for directionally recovering diagonal edges. The developed directional interpolation method is fairly robust in dealing with the four edge features, namely, vertical, horizontal, 45-deg diagonal, and 135-deg diagonal. In addition, the proposed embedded refinement scheme provides an efficient way for soft-decision-based algorithms to achieve improved results with fewer computations. We have compared this new approach to six state-of-the-art methods, and it can outstandingly preserve more edge details and handle fine textures well, without requiring a high computational cost.

A Lossless Compression Scheme for Bayer Color Filter Array Images

In most digital cameras, Bayer color filter array (CFA) images are captured and demosaicing is generally carried out before compression. Recently, it was found that compression-first schemes outperform the conventional demosaicing-first schemes in terms of output image quality. An efficient prediction-based lossless compression scheme for Bayer CFA images is proposed in this paper. It exploits a context matching technique to rank the neighboring pixels when predicting a pixel, an adaptive color difference estimation scheme to remove the color spectral redundancy when handling red and blue samples, and an adaptive codeword generation technique to adjust the divisor of Rice code for encoding the prediction residues. Simulation results show that the proposed compression scheme can achieve a better compression performance than conventional lossless CFA image coding schemes.

Low‐complexity hybrid demosaicing for color filter arrays

This paper presents a cost‐effective hybrid demosaicing algorithm for Bayer color filter array images. Unlike many iterative methods that require interpolating each color plane many times, the proposed algorithm requires only three times. Two edge‐sensitive algorithms, one modified and the other new, are proposed to interpolate green pixels along the edge orientation in the color‐difference space. Experimental results show that the proposed algorithm achieves more satisfying image quality than previous demosaicing techniques in terms of subjective and objective measurements, especially at peak signal‐to‐noise ratio.

Adaptive Filtering for Color Filter Array Demosaicking

Most digital still cameras acquire imagery with a color filter array (CFA), sampling only one color value for each pixel and interpolating the other two color values afterwards. The interpolation process is commonly known as demosaicking. In general, a good demosaicking method should preserve the high-frequency information of imagery as much as possible, since such information is essential for image visual quality. We discuss in this paper two key observations for preserving high-frequency information in CFA demosaicking: (1) the high frequencies are similar across three color components, and (2) the high frequencies along the horizontal and vertical axes are essential for image quality. Our frequency analysis of CFA samples indicates that filtering a CFA image can better preserve high frequencies than filtering each color component separately. This motivates us to design an efficient filter for estimating the luminance at green pixels of the CFA image and devise an adaptive filtering approach to estimating the luminance at red and blue pixels. Experimental results on simulated CFA images, as well as raw CFA data, verify that the proposed method outperforms the existing state-of-the-art methods both visually and in terms of peak signal-to-noise ratio, at a notably lower computational cost.

A Low-Complexity Joint Color Demosaicking and Zooming Algorithm for Digital Camera

This paper presents a low complexity joint color demosaicking and digital zooming algorithm for single-sensor digital cameras. The proposed algorithm directly extracts edge information from raw sensor data for interpolation in both demosaicking and zooming to preserve edge features in its output. This allows the extracted information to be exploited consistently in both stages and also efficiently, as no separate extraction process is required in different stages. The proposed algorithm can produce a zoomed full-color image as well as a zoomed Bayer color filter array image with outstanding performance as compared with conventional approaches which generally combine separate color demosaicking and digital zooming schemes.

Practical implementation of LMMSE demosaicing using luminance and chrominance spaces

Most digital color cameras sample only one color at each spatial location, using a single sensor coupled with a color filter array (CFA). An interpolation step called demosaicing (or demosaicking) is required for rendering a color image from the acquired CFA image. Already proposed linear minimum mean square error (LMMSE) demosaicing provides a good tradeoff between quality and computational cost for embedded systems. In this paper we propose a modification of the stacked notation of superpixels, which allows an effective computing of the LMMSE solution from an image database. Moreover, this formalism is used to decompose the CFA sampling into a sum of a luminance estimator and a chrominance projector. This decomposition allows interpreting estimated filters in term of their spatial and chromatic properties and results in a solution with lower computational complexity than other LMMSE approaches for the same quality.