Color Filter Array Research Articles

Joint Color Space GMMs for CFA Demosaicking

We propose a patch-based algorithm for demosaicking a mosaicked color image produced by color filter arrays commonly used in acquiring color images. The proposed algorithm exploits a joint color space Gaussian mixture model (JCS-GMM) prior for jointly characterizing the patches from red, green, and blue channels of a color image. The inter channel correlations captured by the covariance matrices of Gaussian models are exploited to estimate the pixel values missing in the mosaicked image. The proposed JCS-GMM demosaicking algorithm can be seen as the GMM analogue of the Color-KSVD algorithm, which has produced impressive results in color image denoising and demosaicking. We demonstrate that our proposed algorithm achieves superior performance in the case of Kodak and Laurent Condat's databases, and competitive performance in the case of IMAX database, when compared with state-of-the-art demosaicking algorithms.

Hardware-efficient color correlation–adaptive demosaicing with multifiltering

Color demosaicing is a key image processing step aiming to reconstruct the missing pixels from a recorded raw image that has a color filter array (CFA) pattern. The color correlation–based guided filters, such as minimized-Laplacian residual interpolation (MLRI), are known as the state-of-the-art demosaicing techniques. However, in the conventional guided filter-based techniques, the artifacts are generated in areas with low color correlation. Furthermore, a large number of line memories are required in the hardware implementation of a conventional guided filter-based technique because of the large effective field size. To overcome these two problems, we propose a color correlation–adaptive demosaicing algorithm that selectively applies a specific intracolor demosaicing to regions with low color correlation. We also propose an algorithm structure that reduces the effective field size in the vertical direction to reduce the number of line memories, while maintaining the image quality performance during the hardware implementation. The experimental results show that the proposed scheme can reduce the line memory to one-third while showing marginal performance degradation compared to the state-of-the-art MLRI weighted framework in terms of the color peak signal-to-noise ratio for the IMAX datasets.

Sensor interpixel correlation analysis and reduction for color filter array high dynamic range image reconstruction

AbstractHigh dynamic range imaging (HDRI) by bracketing of low dynamic range (LDR) images is demanding, as the sensor is deliberately operated at saturation. This exacerbates any crosstalk, interpixel capacitance, blooming and smear, all causing interpixel correlations (IC) and a deteriorated modulation transfer function (MTF). Established HDRI algorithms exclude saturated pixels, but generally overlook IC. This work presents a calibration method to estimate the affected region from saturated pixels for a color filter array (CFA) sensor, using the native CFA as a matched filter. The method minimizes color crosstalk given a set of candidates for proximity regions, and requires no special setup. Results are shown for a 21‐bit HDR output image with improved color fidelity and reduced noise. The calibration reduces IC in the LDR images and is performed only once for a given sensor. The improvement is applicable to any HDRI algorithm based on CFA image bracketing, irrespective of sensor technology. Generalizations to subsaturated and supersaturated pixels are described, facilitating a suggested irradiance‐exposure dependent point spread function charge repatriation strategy.

Color correction for RGB sensors with dual-band filters for in-cabin imaging applications

Automotive in-cabin monitoring and surveillance imaging applications operate in the visible band during daytime and in the near infra-red band during nighttime. Utilization of the same camera for imaging in both bands is advantageous as it reduces system complexity. This work evaluates four methods for singlecamera dual-band imaging that vary by combinations between color-filter array, i.e., RGB and RGB-IR, camera filter, i.e., infrared-cut filter and dual-band filter, and processing pipeline, i.e., standard RGB and RGB-IR. We conclude that the optimal combination is RGB sensor, dual-band filter, standard color imaging pipeline, and a color correction matrix that is optimized for the visible and near-infrared bands. This method holds the net advantage of system simplicity and sufficient image quality in applications where high color accuracy is of a secondary priority.

Fabrication of Fabry–Perot-cavity-based monolithic full-color filter arrays using a template-confined micro-reflow process

Transmissive color filters based on metal-dielectric-metal (MDM) Fabry–Perot (FP) resonators are of great interest for various applications due to their high stability, environmental friendliness and chemical inertia. However, it is still technically challenging to obtain a compact monolithic MDM-based color filter array (CFA) because of the difficulty in achieving microscale-pixelated stepwise dielectric films with designed thickness. In this work, we proposed a template-confined thermal reflow approach to fabricate monolithic CFA based on silver–polymethyl methacrylate–silver (Ag–PMMA–Ag) sandwich configuration. Via template-confined thermal reflow of periodic PMMA nanogratings with different filling density, the thickness of PMMA dielectric layer in each pixel can be independently controlled, enabling the spectral tunability across the whole visible light region with high transmission efficiency and narrow linewidth. Compared to conventional multiple deposition technique, the approach we proposed is able to control the thickness of dielectric film via a single exposure step and can be potentially applied in high-throughput production by combining nanoimprint lithography.

Study of PCA-Based Adaptive Denoising of CFA Images for Single-Sensor Digital Cameras

To study the Single-sensor digital color cameras use a process called color demosaicking to produce full color images from the data captured by a color filter array (CFA). The quality of demosaicked images is degraded due to the sensor noise introduced during the image acquisition process. The conventional solution to combating CFA sensor noise is demosaicking first, followed by a separate process. This strategy generates many noise-caused color artifacts in the demosaicking process, which are hard to remove in the process. Few schemes that work directly on the CFA images have been presented because of the difficulties arisen from the red, green and blue interlaced mosaic pattern, yet a well designed denoising first and demosaicking later scheme can have advantages such as less noise-caused color artifacts and cost-effective implementation. Already in the technique, first decomposed the noisy CFA image into two parts: the low-pass smooth image and high-pass smooth image by using 2D-Gaussian low-pass filter and denoised the high-pass image only and further combine with lowpass image. Not large but least noise component are present in the low-pass image, which can create more noise in the image when it process further (demosaicking). An adaptive algorithm can be used by using principle component analysis (PCA), which works directly on the high-pass as well as low-pass CFA data. Which can give the better result for the denoising.

An optimised approach to detect the identity of hidden information in grey scale and colour images

Feature-based steganalysis is an emerging trend in the domain of Information Forensics, aims to discover the identity of secret information present in the covert communication by analysing the statistical features of cover/stego image. Due to massive volumes of auditing data as well as complex and dynamic behaviours of steganogram features, optimising those features is an important open problem. This paper focused on optimising the number of features using the proposed quick artificial bee colony (qABC) algorithm. Here we tested for three steganalysers, namely subtractive pixel adjacency matrix (SPAM), phase aware projection model (PHARM) and colour filter array (CFA) for the break our steganographic system (BOSS) 1.01 datasets. The significant improvement in the convergence nature of qABC quickly improves the solution and fine tune the search than their real counterparts. The results reveal that qABC method with support vector machine (SVM) classifier outperforms the non-optimised version concerning classification accuracy and reduced number of feature sets.

Lossless CFA Image Compression Chip Design for Wireless Capsule Endoscopy

This paper presents a hardware-oriented lossless color filter array (CFA) image compression algorithm for very-large-scale integration (VLSI) circuit design. In order to achieve high performance, low complexity and low memory requirement, a novel lossless CFA image compression algorithm based on JPEG-LS is proposed for the VLSI implementation. A previous study showed the usage of a context table with its memory consuming more than 81% of the chip area for a JPEG-LS encoder design. The proposed algorithm implements a JPEG-LS-based lossless image compression algorithm that eliminates the use of the context technique and its memory in order to reduce the chip area while still maintaining its high performance. The proposed algorithm includes a pixel restoration, an adaptive Golomb-Rice parameter prediction and an improved Golomb-Rice coding technique. This paper was realized using a 0.18μ m CMOS process with synthesized gate counts and core area of 4.8 k and 57,625μm 2 , respectively. The synthesized operating frequency of this design reached 200 MHz by using a pipeline scheduling technique. Compared with the previous JPEG-LS-based designs, this paper reduced the gate count to at least 28% and increased the average compression ratio by over 17.15% using the video endoscopy images from the Gastro Gastroenterologist Hospital.

An Improved Algorithm for Digital Image Authentication and Forgery Localization Using Demosaicing Artifacts

This paper focuses on the digital image authentication and forgery localization using demosaicing artifacts. The aim is to build an algorithm allowing a bridge between the color filter array pattern and demosaicing algorithm estimation, and the statistical analysis of demosaicing artifacts in spatial domain to improve the authentication and localization performance. After analyzing the evolution of demosaicing traces in camera acquisition pipeline, a robust feature statistic characterizing demosaiced digital images is first developed on the basis of the noise residue of green channel. Such a feature statistic is less sensitive to the edges problem because only the smooth region of green channel is used in the development. Next, a single normal mixture model is proposed to describe the probability distribution of feature statistics for both original and tampered images. Therefore, normality tests can be used to authenticate automatically digital images. The authentication performance can be further improved by human interpretation of supported graphic tools. Finally, a penalized expectation-maximization algorithm is used to localize forged areas in tampered images. Numerous comparative studies on four well-known datasets show that the developed algorithm yields better performance and robustness than existing forensics algorithms of the same kind.

Color Filter Array Demosaicking Using Densely Connected Residual Network

Deep convolutional neural networks have been used extensively in recent image processing research, exhibiting drastically improved performance. In this study, we apply convolutional neural networks to color filter array demosaicking, which plays an essential role in single-sensor digital cameras. Contrary to conventional convolutional neural network-based demosaicking models, the proposed model does not require any initial interpolation step for mosaicked input images, which increases the computational complexity. Using a mosaicked image as input, the proposed model is trained in an end-to-end manner to generate demosaicked images outputs. Many deep neural networks experience vanishing-gradient problem, which makes models hard to be trained. To solve this problem, we apply residual learning and densely connected convolutional neural network. Moreover, we apply block-wise convolutional neural networks to consider local features. Finally, we apply a sub-pixel interpolation layer to generate demosaicked output images more efficiently and accurately. Experimental results show that our proposed model outperforms conventional solutions and state-of-the-art models.

Sensor Interpixel Correlation Analysis and Reduction: A Review

Sensor interpixel correlation (IC), as caused by crosstalk, interpixel capacitance, the brighter‐fatter effect, blooming or smear, lowers the sensor MTF and increases color distortion in color filter array sensors. This is generally overlooked, and challenging to mitigate. Recent findings regarding scientific grade (CCD or hybrid CMOS) sensors, describe non-linearities causing pixel charge accumulation dependent IC. This is relevant even for lower grade sensors. High dynamic range imaging (HDRI) is especially susceptible to IC, because the sensor is deliberately operated partly in saturation. Existing HDRI algorithms exclude saturated pixels but generally overlook IC. This review article motivates the need for a generalization of the point spread function (PSF) into an irradiance-exposure dependent (IED) PSF, also considering supersaturation. The IED PSF is challenging to characterize and apply, partly due to its non-linearity. However, doing so can improve the MTF irrespective of image sensor technology, and for conventional as well as HDRI imaging.

Maximum accurate medical image demosaicing using WRGB based Newton Gregory interpolation method

Color image demosaicing is an important image processing technique for obtaining high quality image. This operation is complicated for digital camera, since it converts a Bayer mosaic image to a full color image, which estimates the output image quality of the camera. Nowadays, many techniques are developed to reduce the size and weight of electronic gadgets. In this way, to reduce the size of camera Color Filter Array is used which has only one sensor for three primary colors (Red, Green and Blue). So many demosaicing techniques are developed which reconstruct the full color image from the incomplete color samples in CFA. But till today there is no demosaicing algorithm for medical image re-construction is available in existing literature. In this paper a hybrid medical image demosaicing technique is developed for obtaining better image quality, higher sensitivity as well as higher resolution for high quality medical diagnosis. To improve the medical image quality, Newton Gregory Interpolation is used. This technique mainly focused on Green channel and produce interpolated Red, Green and Blue channels and for enhancing the sensitivity and resolution, updating White channel texture component with RGB channels. In post processing Cross Multilateral Filter is used which decomposes the smoothness component from the texture component. The proposed newton Gregory interpolation based medical image Demosaicing achieves 42.31 db for brain database of NIH and 39.52 db for Iris database of UPOL.

Demosaicking with two-dimensional continuous 3\u2009\xd7\u20093 order hidden Markov model

Since most digital cameras use color filter arrays to sample red, green, and blue colors by a specific pattern, only one color sample would be taken at every pixel location. The process named demosaicking is exploited to recover the full-color image from the incomplete color samples. The paper presents a novel demosaicking method based on two-dimensional continuous 3 × 3 order HMM (2D 3 × 3 CHMM), which incorporates the statistics of high resolution images into the CFA interpolation process. The proposed new method adopts an approach of MAP sequence estimation and exploits high-order statistical dependency between missing pixels. Experiment results demonstrate that the proposed method outperforms several existing or state-of-the-art demosaicking techniques in terms of both objective and subjective evaluations.

Demosaicing using Dual Layer Feedforward Neural Network

Color demosaicing is the problem of recovering full color/spectral channel from a subsampled image captured by single-chip digital cameras covered by Color Filter Array (CFA). Several algorithms have been proposed in the literature, however most of them are tuned to a particular arrangement of color filters. In this paper, we propose a generic algorithm based on a simple Neural Network (NN) architecture, which is trained on a small image database and which gives competitive results. To prove our statement, we test our network on several state of art CFAs and a 5 channel Spectral Filter Array (SFA). We demonstrate our result both on simulated images coming from standard image databases and also RAW images for a 5 channel SFA camera.

Demosaicing of Periodic and Random Color Filter Arrays by Linear Anisotropic Diffusion

The authors develop several versions of the diffusion equation to demosaic color filter arrays of any kind. In particular, they compare isotropic versus anisotropic and linear versus non-linear formulations. Using these algorithms, they investigate the effect of mosaics on the resulting demosaiced images. They perform cross analysis on images, mosaics, and algorithms. They find that random mosaics do not perform the best with their algorithms, but rather pseudo-random mosaics give the best results. The Bayer mosaic also shows equivalent results to good pseudo-random mosaics in terms of peak signal-to-noise ratio but causes visual aliasing artifacts. The linear anisotropic diffusion method performs the best of the diffusion versions, at the level of state-of-the-art algorithms. © 2018 Society for Imaging Science and Technology.

Single-Sensor Video Cameras and the TLCI-2012

Reliable measurements of multiple single-sensor color filter array video cameras have recently been obtained. These can be used to augment the Television Lighting Consistency Index Q a [European Broadcasting Union Recommendation R-137 Television Lighting Consistency Index (TLCI)-2012] and the Standard Camera Model, which were mainly based on modern broadcast HDTV cameras with three-sensor technology. During the development of TLCI-2012, an interest in single-sensor cameras was noted, but reliable measurements were difficult and only one example was available under NonDisclosure Agreement from the manufacturer.

Survey of Demosaicking Methods for Polarization Filter Array Images.

Snapshot polarization imaging has gained interest in the last few decades. Recent research and technology achievements defined the polarization Filter Array (PFA). It is dedicated to division-of-focal plane polarimeters, which permits to analyze the direction of light electric field oscillation. Its filters form a mosaicked pattern, in which each pixel only senses a fraction of the total polarization states, so the other missing polarization states have to be interpolated. As for Color or Spectral Filter Arrays (CFA or SFA), several dedicated demosaicking methods exist in the PFA literature. Such methods are mainly based on spatial correlation disregarding inter-channel correlation. We show that polarization channels are strongly correlated in images. We therefore propose to extend some demosaicking methods from CFA/SFA to PFA, and compare them with those that are PFA-oriented. Objective and subjective analysis show that the pseudo panchromatic image difference method provides the best results and can be used as benchmark for PFA demosaicking.

Lensless broadband diffractive imaging with improved depth of focus: wavefront modulation by multilevel phase masks

This paper introduces a novel lensless full colour diffractive computational imaging system with a planar Multilevel Phase Mask (MPM) as a diffractive optical element (DOE). The novelty concerns: a methodology of MPM design for improved depth of focus (DoF); design of PSFs for RGB imaging and an inverse imaging algorithm with sparse colour image modelling simultaneous for all RGB channels. MPMs are step-wise invariant. The cubic wavefront coding (WFC) is incorporated in MPMs with optimization of number of levels and width of invariant steps. This design of MPM makes the system robust with respect to defocus (improves DoF) and diminish chromatic aberrations typical for DOEs. Broadband multichannel test-images are exploited for design and testing of the lensless system. We consider two alternative optical setups: Wavelength Multiplexing (WM) and Wavelength Division (WD). In WM, the light beam is broadband multichannel with light sources radiating all wavelengths simultaneously and a CMOS sensor is equipped with a Bayer colour filter array (CFA) for registration of spectral measurements. In this setup, a single MPM is designed for the broadband multichannel light beams. In WD, separate exposures of RGB channels are registered by a broadband grey-scale CCD sensor. Different MPMs are designed for each of the RGB channels. Simulation experiments demonstrate the essentially extended DoF of the designed lensless systems and the advanced accuracy and quality of imaging with respect to the corresponding WM and WD systems with refractive lenses. Due to robustness of the designed lensless system to chromatic aberrations, this advantage has a place even with respect to the lens-system.

Copy-Move Forgery Localization Using Convolutional Neural Networks and CFA Features

This article describes how images could be forged using different techniques, and the most common forgery is copy-move forgery, in which a part of an image is duplicated and placed elsewhere in the same image. This article describes a convolutional neural network (CNN)-based method to accurately localize the tampered regions, which combines color filter array (CFA) features. The CFA interpolation algorithm introduces the correlation and consistency among the pixels, which can be easily destroyed by most image processing operations. The proposed CNN method can effectively distinguish the traces caused by copy-move forgeries and some post-processing operations. Additionally, it can utilize the classification result to guide the feature extraction, which can enhance the robustness of the learned features. This article, per the authors, tests the proposed method in several experiments. The results demonstrate the efficiency of the method on different forgeries and quantifies its robustness and sensitivity.

Demosaicking based on channel-correlation adaptive dictionary learning

Image demosaicking is the problem of reconstructing color images from raw images captured using a digital camera with a color filter array. Sparse representation-based demosaicking method achieves superior performance on the commonly used Kodak dataset; however, it performs inferior on the IMAX dataset. We analyze that the factor of the sparse representation-based demosaicking methods that perform inconsistently is channel-correlation, which we define as the mean value of correlation coefficients between the RGB channels. Accordingly, we propose a channel-correlation adaptive dictionary learning-based demosaicking method. Different from the sparse representation-based demosaicking methods that use a fixed dictionary, our method trains a general dictionary on training image patches with various channel-correlations. Then, we learn a function matrix between the general dictionary and channel-correlations. For a raw image patch with an estimated channel-correlation, we compute a dictionary corresponding to its channel-correlation through the function matrix. Finally, we demosaick it with the corresponding dictionary using the sparse representation model. Experiments confirm that the proposed method performs adaptively well on raw images with various channel-correlations.