Demosaicing Algorithms Research Articles

An Edge-Preserving Regularization Model for the Demosaicing of Noisy Color Images

This paper proposes an edge-preserving regularization technique to solve the color image demosaicing problem in the realistic case of noisy data. We enforce intra-channel local smoothness of the intensity (low-frequency components) and inter-channel local similarity of the depth of object borders and textures (high-frequency components). Discontinuities of both the low-frequency and high-frequency components are accounted for implicitly, i.e., through suitable functions of the proper derivatives. For the treatment of even the finest image details, derivatives of first, second, and third orders are considered. The solution to the demosaicing problem is defined as the minimizer of an energy function, accounting for all these constraints plus a data fidelity term. This non-convex energy is minimized via an iterative deterministic algorithm, applied to a family of approximating functions, each implicitly referring to geometrically consistent image edges. Our method is general because it does not refer to any specific color filter array. However, to allow quantitative comparisons with other published results, we tested it in the case of the Bayer CFA and on the Kodak 24-image dataset, the McMaster (IMAX) 18-image dataset, the Microsoft Demosaicing Canon 57-image dataset, and the Microsoft Demosaicing Panasonic 500-image dataset. The comparisons with some of the most recent demosaicing algorithms show the good performance of our method in both the noiseless and noisy cases.

ФИЛЬТРАЦИЯ RAW-ИЗОБРАЖЕНИЙ НА ОСНОВЕ ОБОБЩЕННОГО МЕТОДА НАИМЕНЬШИХ МОДУЛЕЙ

This article studies the filtering of RAW images obtained from a photo matrix. The article adapts a filter based on the generalized method of least modules (GMLM) for filtering RAW images from digital cameras. Digital images obtained by digitizing an analog signal with a digital photo matrix have a noise component in addition to the useful signal. The image structure obtained directly from the digital matrix is very different from the usual RGB image. It is necessary to apply a demosaicing algorithm to obtain an RGB image, which introduces additional distortions. It is proposed to filter the resulting image before applying the demosaicing algorithm. RAW images use more bits to describe the brightness of each pixel than standard RGB images and have a different structure. The adaptation of existing digital filters is required to work with such images. For GMLM filtering, coefficients for the parameters of the smoothing transformation function are selected. The advantages of filtering RAW images in comparison with standard RGB images and a comparison of GMLM with averaging and median filtering are shown. Based on the obtained dependencies, the practical problem of increasing the contrast in images of glass insulators based on their RAW images for the subsequent detection of defects in insulators is solved.

Experimental and Simulated Study of the Relationship Between Color Camera Imaging and Color-Modeled Equivalence Ratio Measurement

A simulation methodology was proposed to reconstruct color images under different camera spectral sensitivity functions ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CSSFs</i> ) by integrating the hyperspectral images. Based on this proposed simulation method, the effects of different <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CSSFs</i> , Bayer patterns, and demosaicing algorithms on color-modeled <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CH</i> * and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C₂</i> * measurement have been investigated. Firstly, initial full-color images were reconstructed under the CIE 1931 XYZ matching function ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CMF</i> ). Then, the Bayer pattern encoded images were obtained from the initial reconstructed images by removing the pixel values according to different Bayer patterns. Next, three different conventional demosaicing algorithms were employed to calculate the full interpolated color images, and last, the effects of demosaicing algorithms on color-modeled <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CH</i> * and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C₂</i> * measurement were analyzed. It was found that the effects of Bayer patterns and demosaicing algorithms on color-modeled global equivalence ratio (Φ) measurement were limited, whereas the most significant impact is <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CSSF</i> . Therefore, an improved color-modeled Φ measurement model was also proposed using a color correction algorithm under <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CMF</i> , which may provide a quick, efficient, and generalized method for color-modeled Φ measurement.

Polarization Image Laser Line Extraction Methods for Reflective Metal Surfaces

In this work, we propose a novel pipeline method for laser line extraction from images with a polarization image sensor. The proposed method is specifically developed for strong laser beam reflections from metal surfaces. For the preprocessing stage, we propose a demosaicing algorithm for color polarizer filter array (CPFA) sensors. This can be implemented by using either one-quarter or full resolution of the sensor. In addition, we propose two methods for optimizing the information available in a 12-channel color polarization image. The first method is based on the minimum linearly polarized irradiance, and the second method is based on the linear polarization intensity. These preprocessing and optimization methods are combined with laser line extraction methods. The laser line extraction is done with either the polarized finite impulse response (FIR) center of gravity (COG), where the laser line coordinates are computed from the filtered laser intensity distribution, or with the polarized FIR-Peak, where the laser line coordinates are calculated from the first derivative of the filtered laser signal. The performance of the proposed algorithms is studied experimentally using a laser line scanner assembly, made of a polarization camera, and a laser line projector operating in the blue wavelength range.

Content based Adaptive Image Demosaicing using Random Forest Algorithm.

Health care, forgery and engineering are just a few of the many industries that rely on the content of images. Mobile phone cameras use image sensors with Bayer patterning. It is necessary to use a demosaicing algorithm to extract the fullcolour image with requisite quality. Content-based adaptive demosaicing utilising random forest algorithm is proposed in this article, as it has the advantage of being easy to train and evaluate. Interaction curvature was used as the predictor. Interpolation techniques: linear, closest, cubic, rational v4 precede this section. For each pixel, 50 learning cycles are utilised, and all of this work is done using MATLAB software. Using random forest algorithms, ten pictures are used to calculate PSNR, SNR, SSIM, and MSSIM. All of the test photos were more efficient when using Random forest as a filter.

A demosaicing method for compressive color single-pixel imaging based on a generative adversarial network

In conventional full color single-pixel imaging (SPI), multiple times measurement or multiple detectors are required to capture all channels’ information of a color scene. The introduction of color filter array (CFA) provides a new idea for full color SPI and the sampling ratio is reduced to one third. However, the imaging quality and color fidelity largely depend on the demosaicing algorithm and when the sampling ratio decreases, the demosaicing result gets worse sharply or even totally fails. In this paper, we utilize a generative adversarial network (GAN) to implement color recovery specific for compressively sampled mosaic images reconstructed in SPI. Deep-learning based demosaicing method helps restore color SPI with higher imaging quality and color fidelity at lower sampling ratios compared with some state-of-the-art demosaicing algorithms. Both computer simulation and optical experiment have demonstrated the effectiveness and feasibility of the proposed method.

Performance analysis of reconstructed image using colour demosaicing algorithm for digital images

Demosaicing algorithm is emerging as one of the most promising techniques in digital image processing because of providing high-quality image enhancement with low power and high speed. Edge detector, weighting block of anisotropic type, and a compensator with filters are the components of the proposed algorithm. Spatial filters and Laplacian methods are used in the filter-based compensation methodology. These improve edge detection, and the blurring effect is reduced. Hardware sharing reduces the cost of the system. Portable and fast systems are required to produce the test results in a short span. For the above-mentioned purpose, the system is designed in VLSI technology. MATLAB and Xilinx tools are used for the system design. A novel strategy to address shading contrasts in multiview video successions is proposed, and it utilises a thick coordinating-based worldwide enhancement structure. The proposed vitality work guarantees to safeguard nearby structures by controlling deviations from the first picture angles. When compared with the existing works 8% and 90.6% of power reduction is observed in the proposed work. It also improves the average colour signal-to-noise ratio quality by more than 1.6 dB.

Robust camera model identification using demosaicing residual features

In this paper, we propose a new framework for performing accurate and robust camera model identification by fully exploiting demosaicing information in a camera’s output images. Instead of fitting a camera’s demosaicing process into parametric models, our framework works by exposing and extracting a diverse set of intra-channel and inter-channel color value correlations originated from the demosaicing process. To expose these correlations, we first apply a number of diversified baseline demosaicing algorithms to re-demosaic the image under investigation, and gather a set of both linear and nonlinear demosaicing residuals. To further extract demosaicing correlations with respect to the color filter array (CFA) structure, co-occurrence matrices are calculated using a new set of geometric patterns. These patterns are specifically designed to extract different types of color value dependencies within the repeated lattice of the CFA pattern. We design a multi-class ensemble classifier to utilize all extracted color value correlations to perform camera model identification. A series of experiments show that our proposed framework can achieve an accuracy of 98.14% on a database with 68 camera models, and is highly robust to post-JPEG compression and contrast enhancement.

MCFD: A Hardware-Efficient Noniterative Multicue Fusion Demosaicing Algorithm

Color demosaicing is an essential step in the camera image processing pipeline, especially on hardware platforms for real-time video applications. Though many demosaicing algorithms have been proposed in the past two decades, there remains a substantial gap between industrial needs and academic research. On one hand, industry requires a high perceptual quality, artifact-free, and low-cost demosaicing algorithm that is noniterative and small window based ready to be implemented on a hardware platform with a limited line buffer. On the other hand, academia is targeting high PSNR/SSIM, with the computation cost and line buffer receiving second priority, and often a frame buffer and iterative operation are used. The cost, large line buffer and iteration requirement make most existing demosaicing algorithms inapplicable on hardware platforms. In this paper, we introduce a novel low-cost demosaicing algorithm to narrow the gap. We keep the operation window size and computation cost as the first priority and achieve both high PSNR/SSIM and visual perceptual quality compared to previous state-of-the-art methods on standard test datasets. We fully investigate the a priori knowledge of natural scene raw images and find several key cues that are beneficial to demosaicing. Our demosaicing algorithm is a smart fusion of these useful cues. Furthermore, we solve typical demosaicing issues that occur in many traditional methods, including false color artifacts, T-section closing, and zippering artifacts. The proposed method has no learning stage, no iteration operation, a small line buffer and a limited number of parameters, so it can easily be applied to hardware platforms.

A practical approach on non-regular sampling and universal demosaicing of raw image sensor data

Non-regular sampling is a well-known method to avoid aliasing in digital images. However, the vast majority of single sensor cameras use regular organized color filter arrays (CFAs), that require an optical-lowpass filter (OLPF) and sophisticated demosaicing algorithms to suppress sampling errors. In this paper a variety of non-regular sampling patterns are evaluated, and a new universal demosaicing algorithm based on the frequency selective reconstruction is presented. By simulating such sensors it is shown that images acquired with non-regular CFAs and no OLPF can lead to a similar image quality compared to their filtered and regular sampled counterparts. The MATLAB source code and results are available at: http://github. com/PhilippBackes/dFSR

Measuring camera Shannon Information Capacity with a Siemens Star Image

Shannon information capacity, which can be expressed as bits per pixel or megabits per image, is an excellent figure of merit for predicting camera performance for a variety of machine vision applications, including medical and automotive imaging systems. Its strength is that is combines the effects of sharpness (MTF) and noise, but it has not been widely adopted because it has been difficult to measure and has never been standardized. We have developed a method for conveniently measuring information capacity from images of the familiar sinusoidal Siemens Star chart. The key is that noise is measured in the presence of the image signal, rather than in a separate location where image processing may be different—a commonplace occurrence with bilateral filters. The method also enables measurement of SNRI, which is a key performance metric for object detection. Information capacity is strongly affected by sensor noise, lens quality, ISO speed (Exposure Index), and the demosaicing algorithm, which affects aliasing. Information capacity of in-camera JPEG images differs from corresponding TIFF images from raw files because of different demosaicing algorithms and nonuniform sharpening and noise reduction.

Passive Image Forgery Detection Based on the Demosaicing Algorithm and JPEG Compression

Multimedia files play an important role in everyday life. Today, the majority of the population owns state-of-the-art cameras integrated into their mobile devices. Technological development not only facilitates the generation of multimedia content, but also the intentional manipulation of it, and this is where forensic techniques of detecting manipulation on images and videos take on great importance. Although historically there has been confidence in the integrity of images, the advance of technology has begun to erode this confidence. This work proposes a digital image authentication method based on the quadratic mean error of the Color Filter Array interpolation pattern estimated from the analysed image. For the evaluation of the proposed method, experiments were carried out with public databases of forged images that are widely developed for research purposes. The results of the experiments demonstrate the efficiency of the proposed method.

A light field demosaicing method with double guided filtering

Aiming at the problem that the light field multi-view image quality is poor which is resulting from the specific lenslet structure of the light field camera and pixel aliasing at the lenslet edge, a light field demosaicing algorithm based on double-guided filtering is proposed. First, the G image is reconstructed by reweighting the gradient based threshold free (GBTF) algorithm with the white image and lenslet mask information. Then, the reconstructed G image is used to double-guide the R/B image for reconstruction. Finally, the reconstructed R, G, and B images are combined into a full color image. The demosaicing result demonstrates that compared with other advanced demosaicing algorithms, the index CPSNR is increased by 1.68%, the index SSIM is increased by 2%, and the light field multi-view image obtained by our method has clear edges and less color artifacts.

Optimization of Demosaicing Algorithm for Autofluorescence Imaging System

Autofluorescence imaging (AFI) systems are widely used in the detection of precancerous lesions. Fluorescence images of precancerous tissue are usually red (R) or blue (B), so this kind of system has high requirement for colour recovery, especially in R and B channels. Besides, AFI system requires bulk data transmission with no time delay. Existing colour recovery algorithms focus more on green (G) channel, overlooking R and B channels. Although the state-of-art demosaicing algorithms can perform well in colour recovery, they often have high computational cost and high hardware requirements. We propose an efficient interpolation algorithm with low complexity to solve the problem. When calculating R and B channel values, we innovatively propose the diagonal direction to select the interpolation direction, and apply colour difference law to make full use of the correlation between colour channels. The experimental results show that the peak signal-to-noise ratios (PSNRs) of G, R and B channels reach 37.54, 37.40 and 38.22 dB, respectively, which shows good performance in recovery of R and B channels. In conclusion, the algorithm proposed in this paper can be used as an alternative to the existing demosaicing algorithms for AFI system.

Comparison of Deep Learning and Conventional Demosaicing Algorithms for Mastcam Images

Bayer pattern filters have been used in many commercial digital cameras. In National Aeronautics and Space Administration’s (NASA) mast camera (Mastcam) imaging system, onboard the Mars Science Laboratory (MSL) rover Curiosity, a Bayer pattern filter is being used to capture the RGB (red, green, and blue) color of scenes on Mars. The Mastcam has two cameras: left and right. The right camera has three times better resolution than that of the left. It is well known that demosaicing introduces color and zipper artifacts. Here, we present a comparative study of demosaicing results using conventional and deep learning algorithms. Sixteen left and 15 right Mastcam images were used in our experiments. Due to a lack of ground truth images for Mastcam data from Mars, we compared the various algorithms using a blind image quality assessment model. It was observed that no one algorithm can work the best for all images. In particular, a deep learning-based algorithm worked the best for the right Mastcam images and a conventional algorithm achieved the best results for the left Mastcam images. Moreover, subjective evaluation of five demosaiced Mastcam images was also used to compare the various algorithms.

Hardware-efficient color correlation–adaptive demosaicing with multifiltering

Color demosaicing is a key image processing step aiming to reconstruct the missing pixels from a recorded raw image that has a color filter array (CFA) pattern. The color correlation–based guided filters, such as minimized-Laplacian residual interpolation (MLRI), are known as the state-of-the-art demosaicing techniques. However, in the conventional guided filter-based techniques, the artifacts are generated in areas with low color correlation. Furthermore, a large number of line memories are required in the hardware implementation of a conventional guided filter-based technique because of the large effective field size. To overcome these two problems, we propose a color correlation–adaptive demosaicing algorithm that selectively applies a specific intracolor demosaicing to regions with low color correlation. We also propose an algorithm structure that reduces the effective field size in the vertical direction to reduce the number of line memories, while maintaining the image quality performance during the hardware implementation. The experimental results show that the proposed scheme can reduce the line memory to one-third while showing marginal performance degradation compared to the state-of-the-art MLRI weighted framework in terms of the color peak signal-to-noise ratio for the IMAX datasets.

Comparative Study Of Biorthogonal Wavelets Accuracy In Demosaicing Algorithm Based On Wavelet Analysis Of Luminance Component

Critical issues in demosaicing are visual artifacts appearing in reconstructed images. Many algorithms have been proposed to overcome this problem. In this work, we have performed a comparative study of some conventional and recent color image demosaicing algorithms and identified experimentally the one that gives good results. From our investigation, algorithm based on wavelets analysis of the luminance component, is the best. Biorthogonal wavelets are recommended to be used in this algorithm. We then experiment famous bi-orthogonal wavelets to identify the best one. We conclude that Mallat's bi-orthogonal wavelets give the best reconstruction results according to CPSNR and FSIMc measures.

An Essential Image Augmentation Processes for Pattern Based Image Retrieval System

An image retrieval system is an image search engine is most useful for human day to day life. But still, the image retrieval systems are working in the traditional manner. Nowadays increasing the image database in a huge amount of size with the heterogeneous category. At the same time increasing the demands of users in a various manner. The most demanding fields in society are healthcare, agriculture, trademark, and crime. In healthcare field diagnose the disease, in crime field investigate the criminals, in trademark field to make an analysis to recognize the right product and in agriculture field to find the disease affected fruit images. The previous image retrieval systems are working with limitations like choose any one of the fields, any one of image file format and any one of the image feature and also any one of the medical image modality. To overcome the above limitations, the Pattern Based Image Retrieval (PBIR) system has been proposed. The PBIR system works by pattern recognition techniques. A pattern is a visible entity of image. Recognition is a label learning process. It considers mainly the patterns of the image for finding a similar group of images from the heterogeneous image database. And also include all directions of image features for image retrieval processing. The PBIR system is used to find uncertain parts of the image during augmentation steps. The augmentation work is an essential to increase the quality of images and make more reliable for feature extraction. Because currently we are using streaming image data. The images are having many unknown disturbances during capture the image. In this paper, we are mainly focusing image augmentation processes are a demosaicing algorithm, gray slice, gradient magnitude and pattern detection, Viola-Jones face detection algorithm for improving the PBIR system performance. And also PBIR system working with four image file formats are.JPG, BMP, GIF, PNG. The three main medical image modalities are CT scan, MRI scan and PET scan images. The image database of PBIR system is 3D DICOM MRI image, Brand logo image, Mango fruit image and ATM crime image. Finally, Image Quality Assessment (IQA) has been carried out using various evaluation measures are found better in the performance accuracy. And also include the results of this processes for further research directions.

Demosaicing Based on Directional Difference Regression and Efficient Regression Priors.

Color demosaicing is a key image processing step aiming to reconstruct the missing pixels from a recorded raw image. On the one hand, numerous interpolation methods focusing on spatial-spectral correlations have been proved very efficient, whereas they yield a poor image quality and strong visible artifacts. On the other hand, optimization strategies, such as learned simultaneous sparse coding and sparsity and adaptive principal component analysis-based algorithms, were shown to greatly improve image quality compared with that delivered by interpolation methods, but unfortunately are computationally heavy. In this paper, we propose efficient regression priors as a novel, fast post-processing algorithm that learns the regression priors offline from training data. We also propose an independent efficient demosaicing algorithm based on directional difference regression, and introduce its enhanced version based on fused regression. We achieve an image quality comparable to that of the state-of-the-art methods for three benchmarks, while being order(s) of magnitude faster.

A Study on Various Color Filter Array based Techniques

A demosaicing algorithm is just a digital image process used to reconstruct full color image from the incomplete color samples output from a picture sensor overlaid with a color filter array (CFA). It is also known as CFA interpolation or color reconstruction. Most contemporary digital camera models acquire images using a single image sensor overlaid with a CFA, so demosaicing is the main processing pipeline required to render these images into a viewable format. Many modern digital camera models can save images in a natural format allowing the consumer to demosaicit using software, as opposed to utilizing the camera's built-in firmware. Thus demosaicing becomes and major area of research in vision processing applications. The key objective of the paper is to examine and analyze various image demosaicing techniques. The entire aim would be to explore various limitations of the earlier techniques. This paper ends up with the suitable gaps in earlier techniques.