Demosaicking Algorithm Research Articles

Joint Defogging and Demosaicking.

Image defogging is a technique used extensively for enhancing visual quality of images in bad weather conditions. Even though defogging algorithms have been well studied, defogging performance is degraded by demosaicking artifacts and sensor noise amplification in distant scenes. In order to improve the visual quality of restored images, we propose a novel approach to perform defogging and demosaicking simultaneously. We conclude that better defogging performance with fewer artifacts can be achieved when a defogging algorithm is combined with a demosaicking algorithm simultaneously. We also demonstrate that the proposed joint algorithm has the benefit of suppressing noise amplification in distant scenes. In addition, we validate our theoretical analysis and observations for both synthesized data sets with ground truth fog-free images and natural scene data sets captured in a raw format.

Bayer Demosaicking With Polynomial Interpolation.

Demosaicking is a digital image process to reconstruct full color digital images from incomplete color samples from an image sensor. It is an unavoidable process for many devices incorporating camera sensor (e.g., mobile phones, tablet, and so on). In this paper, we introduce a new demosaicking algorithm based on polynomial interpolation-based demosaicking. Our method makes three contributions: calculation of error predictors, edge classification based on color differences, and a refinement stage using a weighted sum strategy. Our new predictors are generated on the basis of on the polynomial interpolation, and can be used as a sound alternative to other predictors obtained by bilinear or Laplacian interpolation. In this paper, we show how our predictors can be combined according to the proposed edge classifier. After populating three color channels, a refinement stage is applied to enhance the image quality and reduce demosaicking artifacts. Our experimental results show that the proposed method substantially improves over the existing demosaicking methods in terms of objective performance (CPSNR, S-CIELAB ΔE*, and FSIM), and visual performance.

Context-aware joint dictionary learning for color image demosaicking

A self-learning and context-aware demosaicking algorithm is presented.We advance dictionary learning and preserve data locality for modeling image data.We perform favorably against existing filteri...

Multispectral demosaicking considering out-of-focus problem for red-green-blue-near-infrared image sensors

A near-infrared (NIR) band provides information invisible to human eyes for discriminating and recognizing objects more clearly under low lighting conditions. To capture color and NIR images simultaneously, a multispectral filter array (MSFA) sensor is used. However, because lenses have different refractive indices for different wavelengths, lenses may fail to focus all rays to the same convergence. This is the reason an out-of-focus problem occurs and images are blurred. In this paper, a demosaicking algorithm that considers the out-of-focus problem is proposed. This algorithm is used by the MSFA of a red-green-blue-NIR image sensor to obtain color and NIR images. After the energies of the multispectral (MS) channels in the MSFA image are balanced to minimize aliasing, that image is filtered by the estimated low-pass kernel to generate a panchromatic (PAN) image. When an image is acquired, the out-of-focus problem and the formation process of the PAN image are modeled. The desired MS image is estimated by solving the least squares approach of the difference between the PAN and MS images based on the models. The experimental results show that the proposed algorithm performs well in estimating high-quality MS images and reduces the out-of-focus problem.

Context-aware joint dictionary learning for color image demosaicking

Most digital cameras are overlaid with color filter arrays (CFA) on their electronic sensors, and thus only one particular color value would be captured at every pixel location. When producing the output image, one needs to recover the full color image from such incomplete color samples, and this process is known as demosaicking. In this paper, we propose a novel context-constrained demosaicking algorithm via sparse-representation based joint dictionary learning. Given a single mosaicked image with incomplete color samples, we perform color and texture constrained image segmentation and learn a dictionary with different context categories. A joint sparse representation is employed on different image components for predicting the missing color information in the resulting high-resolution image. During the dictionary learning and sparse coding processes, we advocate a locality constraint in our algorithm, which allows us to locate most relevant image data and thus achieve improved demosaicking performance. Experimental results show that the proposed method outperforms several existing or state-of-the-art techniques in terms of both subjective and objective evaluations.

Beyond Color Difference: Residual Interpolation for Color Image Demosaicking.

In this paper, we propose residual interpolation (RI) as an alternative to color difference interpolation, which is a widely accepted technique for color image demosaicking. Our proposed RI performs the interpolation in a residual domain, where the residuals are differences between observed and tentatively estimated pixel values. Our hypothesis for the RI is that if image interpolation is performed in a domain with a smaller Laplacian energy, its accuracy is improved. Based on the hypothesis, we estimate the tentative pixel values to minimize the Laplacian energy of the residuals. We incorporate the RI into the gradient-based threshold free algorithm, which is one of the state-of-the-art Bayer demosaicking algorithms. Experimental results demonstrate that our proposed demosaicking algorithm using the RI surpasses the state-of-the-art algorithms for the Kodak, the IMAX, and the beyond Kodak data sets.

Automatic Design of Color Filter Arrays in the Frequency Domain.

In digital color imaging, the raw image is typically obtained through a single sensor covered by a color filter array (CFA), which allows only one color component to be measured at each pixel. The procedure to reconstruct a full color image from the raw image is known as demosaicking. Since the CFA may cause irreversible visual artifacts, the CFA and the demosaicking algorithm are crucial to the quality of demosaicked images. Fortunately, the design of CFAs in the frequency domain provides a theoretical approach to handling this issue. However, almost all the existing design methods in the frequency domain involve considerable human effort. In this paper, we present a new method to automatically design CFAs in the frequency domain. Our method is based on the frequency structure representation of mosaicked images. We utilize a multi-objective optimization approach to propose frequency structure candidates, in which the overlap among the frequency components of images mosaicked with the CFA is minimized. Then, we optimize parameters for each candidate, which is formulated as a constrained optimization problem. We use the alternating direction method to solve it. Our parameter optimization method is applicable to arbitrary frequency structures, including those with conjugate replicas of chrominance components. Experiments on benchmark images confirm the advantage of the proposed method.

A Demosaicking Algorithm with Adaptive Inter-Channel Correlation

Most common cameras use a CCD sensor device measuring a single color per pixel. Demosaicking is the interpolation process by which one can infer a full color image from such a matrix of values, thus interpolating the two missing components per pixel. Most demosaicking methods take advantage of inter-channel correlation locally selecting the best interpolation direction. The obtained results look convincing except when local geometry cannot be inferred from neighboring pixels or channel correlation is low. In these cases, these algorithms create interpolation artifacts such as zipper effect or color aliasing. This paper discusses the implementation details of the algorithm proposed in [J. Duran, A. Buades, “Self-Similarity and Spectral Correlation Adaptive Algorithm for Color Demosaicking”, IEEE Transactions on Image Processing, 23(9), pp. 4031–4040, 2014]. The proposed method involves nonlocal image self-similarity in order to reduce interpolation artifacts when local geometry is ambiguous. It further introduces a clear and intuitive manner of balancing how much channel-correlation must be taken advantage of. Source Code An ANSI C source code implementation of the described algorithms is accessible at the IPOL web page of this article 1 , together with an on-line demo.

Multidirectional Weighted Interpolation and Refinement Method for Bayer Pattern CFA Demosaicking

This paper presents a novel multidirectional weighted interpolation algorithm for color filter array interpolation. Our proposed method has two contributions to demosaicking. First, different from conventional interpolation methods based on two directions or four directions, the proposed method exploits to greater degree correlations among neighboring pixels along eight directions to improve the interpolation performance. Second, we propose an efficient postprocessing method to reduce interpolation artifacts based on the color difference planes. Compared with conventional state-of-the-art demosaicking algorithms, our experimental results show the proposed algorithm provides superior performance in both objective and subjective image quality. Furthermore, this implementation has moderate computational complexity.

A practical one-shot multispectral imaging system using a single image sensor.

Single-sensor imaging using the Bayer color filter array (CFA) and demosaicking is well established for current compact and low-cost color digital cameras. An extension from the CFA to a multispectral filter array (MSFA) enables us to acquire a multispectral image in one shot without increased size or cost. However, multispectral demosaicking for the MSFA has been a challenging problem because of very sparse sampling of each spectral band in the MSFA. In this paper, we propose a high-performance multispectral demosaicking algorithm, and at the same time, a novel MSFA pattern that is suitable for our proposed algorithm. Our key idea is the use of the guided filter to interpolate each spectral band. To generate an effective guide image, in our proposed MSFA pattern, we maintain the sampling density of the G -band as high as the Bayer CFA, and we array each spectral band so that an adaptive kernel can be estimated directly from raw MSFA data. Given these two advantages, we effectively generate the guide image from the most densely sampled G -band using the adaptive kernel. In the experiments, we demonstrate that our proposed algorithm with our proposed MSFA pattern outperforms existing algorithms and provides better color fidelity compared with a conventional color imaging system with the Bayer CFA. We also show some real applications using a multispectral camera prototype we built.

An edge-adaptive demosaicking method based on image correlation

To reduce the cost, size and complexity, a consumer digital camera usually uses a single sensor overlaid with a color filter array (CFA) to sample one of the red-green-blue primary color values, and uses demosaicking algorithm to estimate the missing color values at each pixel. A novel image correlation and support vector machine (SVM) based edge-adaptive algorithm was proposed, which can reduce edge artifacts and false color artifacts, effectively. Firstly, image pixels were separated into edge region and smooth region with an edge detection algorithm. Then, a hybrid approach switching between a simple demosaicking algorithm on the smooth region and SVM based demosaicking algorithm on the edge region was performed. Image spatial and spectral correlations were employed to create middle planes for the interpolation. Experimental result shows that the proposed approach produced visually pleasing full-color result images and obtained higher CPSNR and smaller S-CIELAB ΔE*ab than other conventional demosaicking algorithms.

Improved directional weighted interpolation method for single-sensor camera imaging

We present an improved directional weighted interpolation method for single-sensor camera imaging. By observing the fact that the conventional directional weighted interpolation methods are based on unreliable assumptions using spectral correlation, a contribution of this work is made using an antialiasing finite impulse response filter to improve the interpolation accuracy by exploiting robust spectral correlation. We also make a contribution toward refining the interpolation result by using the gradient inverse weighted filtering method. An experimental analysis of images revealed that our proposed algorithm provided superior performance in terms of both objective and subjective image quality compared to conventional directional weighted demosaicking algorithms. Our implementation has very low complexity and is, therefore, well suited for real-time applications.

Taylor series and adaptive fusion strategy for Bayer demosaicking

Demosaicking is an important step in generating a full-color image from a color filter array image captured by a single-sensor digital camera. In this paper, we propose a new demosaicking algorithm which uses Taylor series and an adaptive fusion strategy. For generating directional predictors, we present a new interpolator based on Taylor series. The directional predictors are used together with spatial and spectral correlations. Using these predictors and the adaptive fusion strategy improves the green channel interpolation, in turn helping to reconstruct the red and blue channel interpolations. Experiments comparing the proposed algorithm to some other demosaicking algorithms show that the proposed algorithm provides superior performance in terms of both objective and subjective image quality.

Camera processing with chromatic aberration.

Since the refractive index of materials commonly used for lens depends on the wavelengths of light, practical camera optics fail to converge light to a single point on an image plane. Known as chromatic aberration, this phenomenon distorts image details by introducing magnification error, defocus blur, and color fringes. Though achromatic and apochromatic lens designs reduce chromatic aberration to a degree, they are complex and expensive and they do not offer a perfect correction. In this paper, we propose a new postcapture processing scheme designed to overcome these problems computationally. Specifically, the proposed solution is comprised of chromatic aberration-tolerant demosaicking algorithm and post-demosaicking chromatic aberration correction. Experiments with simulated and real sensor data verify that the chromatic aberration is effectively corrected.

Self-similarity and Spectral Correlation Adaptive Algorithm for Color Demosaicking.

Most common cameras use a CCD sensor device measuring a single color per pixel. The other two color values of each pixel must be interpolated from the neighboring pixels in the so-called demosaicking process. State-of-the-art demosaicking algorithms take advantage of inter-channel correlation locally selecting the best interpolation direction. These methods give impressive results except when local geometry cannot be inferred from neighboring pixels or channel correlation is low. In these cases, they create interpolation artifacts. We introduce a new algorithm involving non-local image self-similarity in order to reduce interpolation artifacts when local geometry is ambiguous. The proposed algorithm introduces a clear and intuitive manner of balancing how much channel-correlation must be taken advantage of. Comparison shows that the proposed algorithm gives state-of-the-art methods in several image bases.

Local adaptive directional color filter array interpolation based on inter-channel correlation

In this paper, we present a novel color image demosaicking algorithm. The algorithm consists of two steps: an interpolation step and a refinement step. The missing green color information is first interpolated by using the color channel difference. In the refinement step, a local weighted directional interpolation method guided by the pre-interpolated green channel is applied to refine the interpolation results along the determined interpolation direction. Lastly, post-processing is implemented to output the final demosaicked full color image. Due to well estimating the interpolation directions based on reliable inter-channel correlation, the interpolation accuracy is guaranteed, besides, an effective post-processing was implemented to reduce the interpolation artifacts. As a result, compared with the latest demosaicking algorithms, experiments showed that the proposed method provides superior performance in terms of both objective and subjective image qualities.

Demosaicking using geometric duality and dilated directional differentiation

This paper presents a new demosaicking algorithm which uses two cost terms: the interpolation error of a low resolution image based on geometric duality and the dilated directional differentiation of color differences. Since a given high resolution image and its low resolution image obtained by sampling have similar edge properties, the proposed algorithm computes the interpolation errors for the candidate directions in the low resolution image, and exploits them as a cost term for the direction. In addition, the interpolation direction can be determined accurately even in the vicinity of object boundaries by dilating the directional differentiation of the color difference values. Through dilation, some pixels, which are in the neighborhood of an edge but classified into a flat region by simple edge detection like the Sobel filter, are reclassified. By combining this edge classifier and the weighted sum of the estimates obtained by Taylor approximation, missing pixels are interpolated. Simulation results show that the proposed demosaicking algorithm is superior to other state-of-the-art algorithms in terms of visual and objective qualities. Furthermore, the computational complexity is comparable with the existing algorithms. Therefore, the proposed algorithm is suitable for real-time implementation.

Voting-Based Directional Interpolation Method and Its Application to Still Color Image Demosaicking

In this paper, we present a novel color image demosaicking algorithm using a voting-based edge direction detection method and a directional weighted interpolation method. By introducing the voting strategy, the interpolation direction of the center missing color component can be determined accurately. Along the determined interpolation direction, the center missing color component is interpolated using the gradient weighted interpolation method by exploring the intra-channel gradient correlation of the neighboring pixels. As compared with the latest demosaicking algorithms, experiments show that the proposed algorithm provides superior performance in terms of both objective and subjective image qualities.

Edge Strength based Fuzzification of Colour Demosaicking Algorithms

In this paper we fuzzify the bilinear colour interpolation and the non-linear colour differential correlative interpolation techniques of colour demosaicking, using the edge strength map of the mosaic image. In doing so we ensure that the edges have less participation in the interpolation thereby minimizing the interpolation error. The result is faster and more accurate colour demosaicking algorithms with application to high resolution images. The overall improvement is quite significant when it comes to colour interpolation of high definition/high resolution images containing a lot of detail or edges as proved by the extensive experimental results on state-of-the-art databases and state-of-the-art comparison. Defence Science Journal, Vol. 64, No. 1, January 2014, DOI:10.14429/dsj.64.4732

Color filter array interpolation by successive refinement over color channels using gradient inverse-weighted filtering

In this paper, we present a novel color image demosaicking algorithm based on a directional weighted interpolation method and gradient inverse-weighted filter-based refinement method. By applying a directional weighted interpolation method, the missing center pixel is interpolated, and then using the nearest neighboring pixels of the pre-interpolated pixel within the same color channel, the accuracy of interpolation is refined using a five-point gradient inverse weighted filtering method we proposed. The refined interpolated pixel values can be used to estimate the other missing pixel values successively according to the correlation inter-channels. Experimental analysis of images revealed that our proposed algorithm provided superior performance in terms of both objective and subjective image quality compared to conventional state-of-the-art demosaicking algorithms. Our implementation has very low complexity and is therefore well suited for real-time applications.