Bayer Pattern Image Research Articles

Direct feature extraction and image co-registration of morphological structure from Bayer pattern raw planetary images

Feature extraction from planetary remote sensing images is a primary image processing task for object recognition, crater counter, morphological structure dimension and unpaired image co-registration. The paper presents a novel methodology to extract features directly from Bayer Pattern raw planetary images. Gradient information is extracted from Bayer Pattern image using standard edge operator that follows Color Difference Constancy (CDC) assumption. The proposed method's advantage is that it can skip the computationally intensive image processing pipeline and can have the Bayer Pattern raw planetary image flow directly for useful information extraction. Sobel Edge Detector is applied on Indian Mars Color Camera (MCC) Bayer Pattern raw images, and Gradient Magnitude Map (GMM) is generated at different Martian terrains. In addition, we have developed a direct image co-registration approach for MCC Bayer intensity raw image with respect to Mars Digital Image Model (MDIM) 2.1 reference using Mode-Mean Combo Patch Filler and Gradient Intensity induced Scale Invariant Feature Transform (GI-SIFT) based feature matching. The outlier matched points are removed by Feature Similarity Score guided Random Sample Consensus (FSS-RANSAC) estimation technique. The visual evaluation and quantitative metrics indicate that GMM from MCC Bayer Pattern raw image has negligible degradation with respect to GMM extracted from MCC demosaic image. The Root Mean Square Error (RMSE) is computed at different Mars regions, and it is found that the average image co-registration accuracy is less than 0.5 pixel.

Gradient-Based Feature Extraction From Raw Bayer Pattern Images.

In this paper, the impact of demosaicing on gradient extraction is studied and a gradient-based feature extraction pipeline based on raw Bayer pattern images is proposed. It is shown both theoretically and experimentally that the Bayer pattern images are applicable to the central difference gradient-based feature extraction algorithms with negligible performance degradation, as long as the arrangement of color filter array (CFA) patterns matches the gradient operators. The color difference constancy assumption, which is widely used in various demosaicing algorithms, is applied in the proposed Bayer pattern image-based gradient extraction pipeline. Experimental results show that the gradients extracted from Bayer pattern images are robust enough to be used in histogram of oriented gradients (HOG)-based pedestrian detection algorithms and shift-invariant feature transform (SIFT)-based matching algorithms. By skipping most of the steps in the image signal processing (ISP) pipeline, the computational complexity and power consumption of a computer vision system can be reduced significantly.

A Low-Complexity End-to-End Stereo Matching Pipeline From Raw Bayer Pattern Images to Disparity Maps

Conventional computer vision algorithms, including stereo matching algorithms, take finely rendered color images as input. However, existing image signal processing (ISP) pipelines for color image generation are designed for photography with a goal of generating pleasing images for human eyes. This paper describes a new end-to-end pipeline for stereo matching from raw Bayer pattern images to disparity maps with customized ISP. Unlike conventional stereo matching systems which need a complete ISP module to render full-size standard RGB (sRGB) images, a subsampling-based demosaicing-downsampling (SDD) operation is introduced in the proposed pipeline to demosaic and downsample the Bayer pattern images. The resultant half-size color image pairs are processed with simple denoising and tone mapping algorithms to generate the final input images of stereo matching algorithms. It is found that the simple nearest neighbor upsampling method is good enough to generate the final full-size disparity maps. Experimental results show that the proposed pipeline is capable of generating comparable or even better stereo matching results than the conventional pipeline. By skipping most of the unnecessary ISP steps and reducing the size of input images, the computational complexity of the end-to-end stereo matching pipeline is significantly reduced.

Low-Light Demosaicking and Denoising for Small Pixels Using Learned Frequency Selection

Low-light imaging is a challenging task because of the excessive photon shot noise. Color imaging in low-light is even more difficult because one needs to demosaick and denoise simultaneously. Existing demosaicking algorithms are mostly designed for well-illuminated scenarios, which fail to work with low-light. Recognizing the recent development of small pixels and low read noise image sensors, we propose a learning-based joint demosaicking and denoising algorithm for low-light color imaging. Our method combines the classical theory of color filter arrays and modern deep learning. We use an explicit carrier to demodulate the color from the input Bayer pattern image. We integrate trainable filters into the demodulation scheme to improve flexibility. We introduce a guided filtering module to transfer knowledge from the luma channel to the chroma channels, thus offering substantially more reliable denoising. Extensive experiments are performed to evaluate the performance of the proposed method, using both synthetic datasets and real data. Results indicate that the proposed method offers consistently better performance over the current state-of-the-art, across several standard evaluation metrics.

Histogram of Oriented Gradients Feature Extraction From Raw Bayer Pattern Images

This brief studies the redundancy in the image processing pipeline for histogram of oriented gradients (HOG) feature extraction. The impact of demosaicing on the extracted HOG features is analyzed and experimented. It is shown that by taking advantage of the inter-channel correlation of natural images, the HOG features can be directly extracted from the Bayer pattern images with proper gamma compression. Due to the elimination of the image processing pipeline, the power consumption and computational complexity of the detection system can be significantly reduced. Experimental results show that the Bayer pattern image-based HOG features can be used in pedestrian detection systems with little performance degradation.

FPGA Acceleration of Color Interpolation Algorithm Based on Gradient and Color Difference

To quickly and efficiently achieve a high-quality color image on the embedded platform, designed and used FPGA to complete Bayer pattern image Color interpolation. In this paper, we proposed a color interpolation algorithm based on image gradient direction and color difference. The algorithm calculates the G component according to the gradient value and color difference at the pixel to be interpolated, and then calculates the missing R component and B component based on the G component interpolation. Finally, the pixels of the image boundary area are interpolated. Tested on the Kodak dataset, the experimental results show that the subjective visual quality and objective image quality evaluation of the algorithm is superior to the conventional algorithm. On the other hand, and reduce problems such as image aliasing and edge blurring. The proposed algorithm has been applied to the actual Xilinx’s FPGA platform, a real-time speed of 120 MHz, enabling real-time image color Interpolation.

Optimized sensing of sparse and small targets using lens-free holographic microscopy.

Lens-free holographic microscopy offers sub-micron resolution over an ultra-large field-of-view >20 mm2, making it suitable for bio-sensing applications that require the detection of small targets at low concentrations. Various pixel super-resolution techniques have been shown to enhance resolution and boost signal-to-noise ratio (SNR) by combining multiple partially-redundant low-resolution frames. However, it has been unclear which technique performs best for small-target sensing. Here, we quantitatively compare SNR and resolution in experiments using no regularization, cardinal-neighbor regularization, and a novel implementation of sparsity-promoting regularization that uses analytically-calculated gradients from Bayer-pattern image sensors. We find that sparsity-promoting regularization enhances the SNR by ~8 dB compared to the other methods when imaging micron-scale beads with surface coverages up to ~4%.

Using online dictionary learning to improve Bayer pattern image coding

Image quality is a fundamental concern in image compression. There is a lot of noise in image compression process, which may impact on users not getting precise identification. It has, thus, always been neglected in image compression in the past researches. In fact, noise takes a beneficial role in image reconstruction. In this paper, we chose noise as considered and recommended as a coding method for Bayer pattern image based on online dictionary learning. Investigations have depicted that the proposed method in Bayer pattern image coding might develop the rate of distortion performance of Bayer pattern image coding at any rate.

Colorization-Based RGB-White Color Interpolation using Color Filter Array with Randomly Sampled Pattern

Recently, several RGB-White (RGBW) color filter arrays (CFAs) have been proposed, which have extra white (W) pixels in the filter array that are highly sensitive. Due to the high sensitivity, the W pixels have better SNR (Signal to Noise Ratio) characteristics than other color pixels in the filter array, especially, in low light conditions. However, most of the RGBW CFAs are designed so that the acquired RGBW pattern image can be converted into the conventional Bayer pattern image, which is then again converted into the final color image by using conventional demosaicing methods, i.e., color interpolation techniques. In this paper, we propose a new RGBW color filter array based on a totally different color interpolation technique, the colorization algorithm. The colorization algorithm was initially proposed for colorizing a gray image into a color image using a small number of color seeds. Here, we adopt this algorithm as a color interpolation technique, so that the RGBW color filter array can be designed with a very large number of W pixels to make the most of the highly sensitive characteristics of the W channel. The resulting RGBW color filter array has a pattern with a large proportion of W pixels, while the small-numbered RGB pixels are randomly distributed over the array. The colorization algorithm makes it possible to reconstruct the colors from such a small number of RGB values. Due to the large proportion of W pixels, the reconstructed color image has a high SNR value, especially higher than those of conventional CFAs in low light condition. Experimental results show that many important information which are not perceived in color images reconstructed with conventional CFAs are perceived in the images reconstructed with the proposed method.

Superpixel based fusion and demosaicing for multi-focus Bayer images

In this paper, a novel superpixel based multi-focus image fusion algorithm for raw images of single-sensor color imaging devices which incorporates the Bayer pattern is proposed. The proposed algorithm is more efficient than traditional fusion schemes since the raw Bayer pattern images are fused before color demosaicing. With the proposed fusion algorithm, the interpolation errors introduced by the repeated demosaicing operation on multi-source images can also be greatly reduced. In detail, a clarity measurement of Bayer pattern image is defined to judge the focus-level of raw Bayer pattern images, and the fusion operator is performed on superpixels which provide powerful grouping cues of local image feature. By comparing the clarities of superpixels, a weight map is constructed and the guided filter is utilized to refine the weight map. The raw images are merged with refined weight map to get the fused Bayer pattern image, which is restored by the demosaicing algorithm to get the full resolution color image. Experimental results demonstrate that the proposed algorithm can obtain better fused results with more natural appearance and fewer artifacts than the traditional algorithms.

Multi-Focus Raw Bayer Pattern Image Fusion for Single-Chip Camera

In this paper, an efficient patch-based image fusion approach for raw images of single-chip imaging devices incorporated with the Bayer CFA pattern is presented. The multi-source raw Bayer pattern images are firstly parted into half overlapped patches. Then, the patches with maximum clarity measurement defined for raw Bayer pattern image are selected as the fused patches. Next, all the fused local patches are merged with weighted average method in order to reduce the blockness effect of fused raw Bayer pattern image. Finally, the real color fused image is obtained by gradient based demosaicing technology. The multi-source raw Bayer pattern data is fused before demosaicing, so that the multi-sensor system will be more efficient and the artifacts introduced in demosaicing processing do not accumulate in image fusion processing. For comparison, the raw images are also interpolated firstly, and then various image fusion methods are used to get the fused color images. Experimental results show that the proposed algorithm is valid and very effective.

A Coding Method of Bayer Pattern Image Based on Wyner-Ziv Structure

该文提出一种基于Wyner-Ziv(WZ)结构的四通道贝尔模板图像分布式编解码方法。在编码端对贝尔模板图像进行结构分离转换，形成4个分量图像并分别执行离散余弦变换，依据拉格朗日代价函数的收敛性，利用Lloyd迭代算法设计了全局优化的量化器，采用Slepian-Wolf(SW)信道编码方法对各分量变换系数的量化输出进行独立编码，在解码端利用亮度分量作为边信息，联合解码重构贝尔模板图像。实验表明，在高速率情况下，其率失真性能得到较好的改善。

컬러 보간 에러 감소를 위한 에지 방향성 컬러 보간 방법과 결합된 디블러링 알고리즘

디지털 이미징 장치는 일반적으로 베이어 패턴(Bayer pattern)을 사용하며, 영상 획득 과정에서 광학적 블러(blur)에 의해 영상의 품질이 손상된다. 블러된 베이어 영상에서 고해상도 컬러 영상을 얻기 위하여, 일반적으로 컬러 보간 방법과 디블러링 방법을 독립적으로 수행한다. 하지만, 베이어 샘플링에 의한 에지 정보가 불충분하여 에지를 가로지르는 방향으로 보간 하게 되고, 이에 따라 컬러 보간 과정에서 에러가 발생한다. 이러한 에러는 디블러링 과정에서 강조되어 영상의 품질을 하락시킨다. 따라서 본 논문은 컬러 보간 방법과 결합된 디블러링 알고리즘을 제안한다. 제안하는 방법은 크게 보간 단계와 영역 결정 단계로 나눌 수 있다. 보간 단계에서는 가정된 에지 방향에 따라 보간 및 디블러링 과정을 수행하고, 영역 결정 단계에서는 각 화소 위치에서 국부 영역의 특성을 추정하고, 보간 단계에서 구한 값을 영역 적응적으로 융합한다. 또한 본 논문에서는 디블러링 성능을 향상시키기 위하여 광학적 블러와 유사한 파동 광학에 근거한 블러 모델을 기반으로 하고, 추정한 국부 영역 특성을 반영하여 디블러링 필터를 추정한다. 실험 결과를 통해 제안하는 방법이 컬러 보간 에러가 확대되는 것을 방지함을 확인할 수 있으며, 기존 방법에 비해 수치적인 면과 시각적인 면에서 뛰어난 결과를 보임을 확인 할 수 있다.

Research and Design of Color Restoration Algorithm for Demosaicing Bayer Pattern Images

This paper analyses the process of obtaining real color image based on a single CCD/CMOS sensor. Bayer CFA Interpolation algorithm and White Balance algorithm are presented. The proposed interpolation algorithm which can obtain better color image and is easier to implement by hardware is provided. In order to cancel chromatic aberration, one White Balance algorithm which aims at sequential image is presented and process of FPGA design is given. Experimental results show that the design works normally. The color of corrected picture is vivid compared to the real world.

Mixture of Gaussians-Based Background Subtraction for Bayer-Pattern Image Sequences

This letter proposes a background subtraction method for Bayer-pattern image sequences. The proposed method models the background in a Bayer-pattern domain using a mixture of Gaussians (MoG) and classifies the foreground in an interpolated red, green, and blue (RGB) domain. This method can achieve almost the same accuracy as MoG using RGB color images while maintaining computational resources (time and memory) similar to MoG using grayscale images. Experimental results show that the proposed method is a good solution to obtain high accuracy and low resource requirements simultaneously. This improvement is important for a low-level task like background subtraction since its accuracy affects the performance of high-level tasks, and is preferable for implementation in real-time embedded systems such as smart cameras.

An adaptive color plane interpolation method based on edge detection

In this letter, an adaptive interpolation algorithm based on edge detection is proposed. With this algorithm, all the missing green values can be reconstructed in Bayer pattern image by using edge detection interpolation method. Reconstructed images composed of green pixels are classified according to the high frequency components in image, and the threshold T needed for all kinds of green images in the edge detection is determined through experiments. The edge detection is carried out based on the one Dimensional (1D) gradient operator. If the gradient value is greater than T, this pixel is located on the edge; otherwise the pixel is in the smooth area of the image. Finally, the simple bilinear interpolation is used for the smooth area while the Laplacian interpolation with the second-order correction term is adopted to reconstruct the other red/blue values on the edge. This algorithm resolves effectively the conflicts between reconstructing high quality color image and reducing computational complexity, and thus largely enhances the processing speed for the reconstructed color image.

Saturation-based adaptive inverse gradient interpolation for Bayer pattern images

A saturation-based adaptive gradient interpolation algorithm is developed for Bayer pattern images obtained in single-sensor digital cameras. The proposed algorithm uses the concept of pseudo-saturation for 2/spl times/2 blocks of the image. The algorithm adapts the scheme of interpolation based on the classification of the pseudo-saturation of Bayer pattern images. Compared with other algorithms, the proposed method preserves fine details, reduces false colours, and distortions around edges and sharp features of the colour image reconstructed from different kinds of Bayer pattern images.