Joint Bilateral Filter Research Articles

The Design of Image Depth Information Extraction Algorithm Based on Joint Bilateral Filtering

With the vigorous development of science and technology, specialists and scholars pay their attention to the 2D-to-3D media conversion more and more. As the next generational display, 3D-TV has lots of problems to solve, which must depend on depth estimation. Here it proposes a novel line tracing method and depth refinement filter as core of depth estimation framework. First, edge detection is performed with the downscaled input image. The line tracing algorithm traces strong edge positions to generate an initial staircase depth map. The initial depth map is further improved by a recursive depth refinement filter. It finally presents visual results from depth estimation and stereo image generation. The experimental results show that the algorithm is effective and feasible which leads a good result.

Switching filtering method for interactive systems

Aiming at the problem that the mixed noise interference caused by the mixed projection noise system is not accurate and the real-time performance is poor, this article proposes an adaptive system switching filtering method based on Bayesian estimation switching rules. The method chooses joint bilateral filtering and improved adaptive median filtering as the filtering subsystems and selects the sub-filtering system suitable for the noise by switching rules to achieve the purpose of effectively removing noise. The simulation experiment was carried out by the self-developed human–computer interactive projection image system platform. Through the subjective evaluation, objective evaluation, and running time comparison analysis, a better filtering effect was achieved, and the balance between the filtering precision and the real-time performance of the interactive system was well obtained. Therefore, the proposed method can be widely applied to various human–computer interactive image filtering systems.

Multi-Scale Frequency Reconstruction for Guided Depth Map Super-Resolution via Deep Residual Network

The depth maps obtained by the consumer-level sensors are always noisy in the low-resolution (LR) domain. Existing methods for the guided depth super-resolution, which are based on the pre-defined local and global models, perform well in general cases (e.g., joint bilateral filter and Markov random field). However, such model-based methods may fail to describe the potential relationship between RGB-D image pairs. To solve this problem, this paper proposes a data-driven approach based on the deep convolutional neural network with global and local residual learning. It progressively upsamples the LR depth map guided by the high-resolution intensity image in multiple scales. A global residual learning is adopted to learn the difference between the ground truth and the coarsely upsampled depth map, and the local residual learning is introduced in each scale-dependent reconstruction sub-network. This scheme can restore the depth structure from coarse to fine via multi-scale frequency synthesis. In addition, batch normalization layers are used to improve the performance of depth map denoising. Our method is evaluated in noise-free and noisy cases. A comprehensive comparison against 17 state-of-the-art methods is carried out. The experimental results show that the proposed method has faster convergence speed as well as improved performances based on the qualitative and quantitative evaluations.

Fast Scale-Adaptive Bilateral Texture Smoothing

In the classical bilateral filter, a range kernel is used together with a spatial kernel for smoothing out fine details while simultaneously preserving edges. More recently, it has been demonstrated that even coarse textures can be smoothed using joint bilateral filtering. In this paper, we demonstrate that the superior texture filtering results can be obtained by adapting the spatial kernel at each pixel. To the best of our knowledge, spatial adaptation (of the bilateral filter) has not been explored for texture smoothing. The rationale behind adapting the spatial kernel is that one cannot smooth beyond a certain level using a fixed spatial kernel, no matter how we manipulate the range kernel. In fact, we should simply aggregate more pixels using a sufficiently wide spatial kernel to locally enhance the smoothing. Based on this reasoning, we propose to use the classical bilateral filter for texture smoothing, where we adapt the width of the spatial kernel at each pixel. We describe a simple and efficient gradient-based rule for the latter task. The attractive aspect is that we are able to develop a fast algorithm that can accelerate the computations by an order without visibly compromising the filtering quality. We demonstrate that our method outperforms classical bilateral filtering, joint bilateral filtering, and other filtering methods, and is competitive with the optimization methods. We also present some applications of texture smoothing using the proposed method.

Consistent pan-sharpening based on multistage joint and dual bilateral filters

Consistent pan-sharpening based on multistage joint and dual bilateral filters

Research on image denoising algorithm of joint bilateral filter and wavelet threshold shrinkage

Research on image denoising algorithm of joint bilateral filter and wavelet threshold shrinkage

Static/Dynamic Filtering for Mesh Geometry.

The joint bilateral filter, which enables feature-preserving signal smoothing according to the structural information from a guidance, has been applied for various tasks in geometry processing. Existing methods either rely on a static guidance that may be inconsistent with the input and lead to unsatisfactory results, or a dynamic guidance that is automatically updated but sensitive to noises and outliers. Inspired by recent advances in image filtering, we propose a new geometry filtering technique called static/dynamic filter, which utilizes both static and dynamic guidances to achieve state-of-the-art results. The proposed filter is based on a nonlinear optimization that enforces smoothness of the signal while preserving variations that correspond to features of certain scales. We develop an efficient iterative solver for the problem, which unifies existing filters that are based on static or dynamic guidances. The filter can be applied to mesh face normals followed by vertex position update, to achieve scale-aware and feature-preserving filtering of mesh geometry. It also works well for other types of signals defined on mesh surfaces, such as texture colors. Extensive experimental results demonstrate the effectiveness of the proposed filter for various geometry processing applications such as mesh denoising, geometry feature enhancement, and texture color filtering.

Depth Image Super Resolution Based on Edge-Guided Method

Depth image super-resolution (SR) is a technique which can reconstruct a high-resolution (HR) depth image from a low-resolution (LR) depth image. Its purpose is to obtain HR details to meet the needs of various applications in computer vision. In general, conventional depth image SR methods often cause edges in the final HR image to be blurred or ragged. To solve this problem, an edge-guided method for depth image SR is presented in this paper. To get high-quality edge information, a pair of sparse dictionaries was applied to reconstruct edges of depth image. Then, with the guidance of these high-quality edges, a depth image was interpolated by using a modified joint bilateral filter. Edge-guided method can preserve the sharpness of edges and effectively avoid generating blurry and ragged edges when SR is performed. Experiments showed that the proposed method can get better results on both subjective and objective evaluation, and the reconstructed performance was superior to conventional depth image SR methods.

Feature-preserving mesh denoising based on guided normal filtering

Mesh denoising is important to improve the quality of the geometry surface acquired by 3D scanning devices. This paper proposes a feature-preserving denoising framework. By classifying the faces into feature and non-feature faces, we use joint bilateral filtering and partial neighborhood filtering to deal with the face normals these two kinds of faces. Experimental results show that our method outperforms the existing methods and achieves higher quality results on the geometry feature.

Multi-scale image fusion through rolling guidance filter

Image fusion is essential in enhancing visual quality by blending complementary images, which are derived from different captured conditions or different sensors in the same scene. The role of image fusion in the Internet of Things has become considerably important in the future. For instance, data captured by multiple visual sensors need further computation or fusion, which is based on a network of making a decision or an analysis. A new image fusion method is proposed by using rolling guidance filter and joint bilateral filter in this paper. First, the saliency maps of two source images are extracted by the Kirsch operator. Subsequently, the two source images are decomposed by rolling guidance filter to obtain multi-scale images. Second, joint bilateral filter and optimal correction are utilized to optimize the saliency maps and obtain the final weight maps. Finally, two fusion rules are used to restore the final fused image. The proposed method not only preserves the details of source images, but also suppresses the artifacts effectively. Experimental results prove that our method generates better effects on both visual perception and objective quantization than traditional methods.

Depth image enhancement algorithm for preserving boundary

The drawback of current depth image enhancement algorithms is poor performance of edge preserving. To solve this drawback, the gradient mask guided joint filtering(GMGJF)algorithm is proposed. The Sobel gradient transform is used to obtain the boundary direction information, and the hole region of the depth images was utilized to generate the hole mask.Furthermore, taking the boundary direction and the cavity mask as the guidance, the color image was jointed to perform iterative Gaussian filtering and hole filling on the depth image. Experimental results show that the peak signal to noise ratio(PSNR)and the structural similarity index measure(SSIM) of GMGJF algorithm are improved by at least 3.50% and 1.07% respectively, compared with the iterative median filter(IMF), guided filter (GF) and joint bilateral filter(JBF) algorithms, it has both the strongest ability of denoising and hole filling, and can remain the boundary features best, which is good for feature extraction and target recognition of depth image.

Rolling Guidance Based Scale-Aware Spatial Sparse Unmixing for Hyperspectral Remote Sensing Imagery

Spatial regularization based sparse unmixing has attracted much attention in the hyperspectral remote sensing image processing field, which combines spatial information consideration with a sparse unmixing model, and has achieved improved fractional abundance results. However, the traditional spatial sparse unmixing approaches can suppress discrete wrong unmixing points and smooth an abundance map with low-contrast changes, and it has no concept of scale difference. In this paper, to better extract the different levels of spatial details, rolling guidance based scale-aware spatial sparse unmixing (namely, Rolling Guidance Sparse Unmixing (RGSU)) is proposed to extract and recover the different levels of important structures and details in the hyperspectral remote sensing image unmixing procedure, as the different levels of structures and edges in remote sensing imagery have different meanings and importance. Differing from the existing spatial regularization based sparse unmixing approaches, the proposed method considers the different levels of edges by combining a Gaussian filter-like method to realize small-scale structure removal with a joint bilateral filtering process to account for the spatial domain and range domain correlations. The proposed method is based on rolling guidance spatial regularization in a traditional spatial regularization sparse unmixing framework, and it accomplishes scale-aware sparse unmixing. The experimental results obtained with both simulated and real hyperspectral images show that the proposed method achieves visual effects better and produces higher quantitative results (i.e., higher SRE values) when compared to the current state-of-the-art sparse unmixing algorithms, which illustrates the effectiveness of the rolling guidance based scale aware method. In the future work, adaptive scale-aware spatial sparse unmixing framework will be studied and developed to enhance the current idea.

A new computational approach for edge-preserving image decomposition

Decomposition of an image into its cartoon part and texture part has been an interesting area of research. It is an important pre-processing step in many computer vision and image processing techniques such as image segmentation, pattern matching, object recognition, tone mapping as both cartoon and texture parts contain two different kinds of information. Significant work is already available in the literature. At the time of decomposition of an input image into its cartoon and texture part, we required texture-smoothing along with edge-preservation. Most of the approaches available in the literature establish a trade-off between the edge preservation and the texture-smoothing. This common drawback of the existing approaches motivates us to design a new image decomposition method by which we can achieve texture-smoothing without having any loss of edge information in cartoon part. To achieve this aim we introduce a new approach based on Joint bilateral filter and DCT (Discrete cosine transform). We show experimental results on several images to show the effectiveness of our method and comparison with some state-of-the-art methods.

Joint bilateral filtering and spectral similarity-based sparse representation: A generic framework for effective feature extraction and data classification in hyperspectral imaging

Classification of hyperspectral images (HSI) has been a challenging problem under active investigation for years especially due to the extremely high data dimensionality and limited number of samples available for training. It is found that hyperspectral image classification can be generally improved only if the feature extraction technique and the classifier are both addressed. In this paper, a novel classification framework for hyperspectral images based on the joint bilateral filter and sparse representation classification (SRC) is proposed. By employing the first principal component as the guidance image for the joint bilateral filter, spatial features can be extracted with minimum edge blurring thus improving the quality of the band-to-band images. For this reason, the performance of the joint bilateral filter has shown better than that of the conventional bilateral filter in this work. In addition, the spectral similarity-based joint SRC (SS-JSRC) is proposed to overcome the weakness of the traditional JSRC method. By combining the joint bilateral filtering and SS-JSRC together, the superiority of the proposed classification framework is demonstrated with respect to several state-of-the-art spectral-spatial classification approaches commonly employed in the HSI community, with better classification accuracy and Kappa coefficient achieved.

Depth maps and high-dynamic range image generation from alternating exposure multiview images

For stereo matching, it is hard to find accurate correspondence for saturated regions, such as too dark or too bright regions, because there is rarely reliable information to match. In this situation, conventional high-dynamic range (HDR) imaging techniques combining multiple exposures for each viewpoint can be adopted to generate well-exposed stereo images. This approach is, however, time-consuming and needs much memory to store multiple exposures for each viewpoint. We propose an efficient method to generate HDR multiview images as well as corresponding accurate depth maps. First, we take a single exposure for each viewpoint with alternating exposure setting, such as short and long exposure, as functions of viewpoint changes. Then, we compute an initial depth map for each view only using neighboring images that have the same exposure. To reduce the error of the initial depth maps for the saturated regions, we adopt the fusion move algorithm fusing neighboring depth maps that have different error regions. Finally, using the enhanced depth maps, we generate artifact-free and sharp HDR images using the joint bilateral filtering and a detail-transfer technique. Experimental results show that our method produces both consistent HDR images and accurate depth maps for various indoor and outdoor multiview images.

A region-based video de-noising algorithm based on temporal and spatial correlations

Video de-noising is important for display and subsequent analysis. However, it remains challenging with regard to structured noise and temporal smoothness. This paper proposes a region-based video de-noising algorithm. First, the luma component of each frame is explicitly divided into a moving region and a non-moving region. Then, 3D structures are formed for different regions according to their different temporal characteristics. A bilateral filter is extended to 3D and subsequently employed for luma component de-noising. Second, de-noising for chroma is proposed instead of a simple transplantation of the de-noising for luma. The edge information within the de-noised luma is referred to during chroma de-noising based on the joint bilateral filter. The experimental results demonstrate that the proposed de-noising algorithm, which outperforms state-of-the-art algorithms, can not only efficiently remove the noise but also maintain temporal smoothness among neighboring frames.

Sky detection‐ and texture smoothing‐based high‐visibility haze removal from images and videos

AbstractTo address the gloomy sky and the low contrast caused by the left fog in the existing image dehazing methods, we propose a robust haze removal algorithm for images and videos. First, a sky detection‐based adaptive atmospheric light estimation method is designed for brighter and cleaner restoration results for the sky regions. Second, in order to reconstruct a transmission map in line with the depth variation, we preprocess the input image with texture smoothing to keep the color consistency inside the same planar object and devise a texture smoothing‐based robust transmission estimation method, with which the contrast and color saturation of fog‐free image are greatly promoted. Finally, the restored results are post‐processed with the joint bilateral filter for the purpose of noise removal. What's more, a guided filter‐based temporally coherent atmospheric light smoothing strategy and a Gaussian filter‐based spatial‐temporally coherent transmission smoothing strategy are put forward for video dehazing, which can ensure the spatial as well as temporal continuity of the haze‐free videos. Experimental results show that the recovered haze‐free images and videos have high contrast and color saturation with cleaner sky regions, and the haze‐free videos are free of jittering and flickering phenomena.

Edge preserving super-resolution algorithm using multi-stage cascaded joint bilateral filter

Super-resolution (SR) algorithms address the inabilities of poor imaging devices, there by producing high quality images with enhanced resolution. We propose a new SR approach which produces sharp high resolution (HR) image using its low resolution (LR) counterparts. The proposed method uses geometric duality for spatially adapting covariance-based interpolation (CBI). To preserve edge information, a multi-stage cascaded joint bilateral filter (MSCJBF) is proposed as an intermediary stage. These edges are incorporated in the high frequency subbands obtained by the stationary wavelet transform (SWT), through nearest neighbor interpolation (NNI) method. Prior to the NNI process, the high frequency subbands undergo two-lobed lanczos interpolation to achieve the desired resolution enhancement. The quantitative and qualitative analysis for various test images prove the superiority of our method.

Simultaneous enhancement and noise reduction of a single low‐light image

Images obtained under low-light conditions tend to have the characteristics of low-grey levels, high-noise levels, and indistinguishable details. Image degradation not only affects the recognition of images, but also influences the performance of the computer vision system. The low-light image enhancement algorithm based on the dark channel prior de-hazing technique can enhance the contrast of images effectively and can highlight the details of images. However, the dark channel prior de-hazing technique ignores the effects of noise, which leads to significant noise amplification after the enhancement process. In this study, a de-hazing-based simultaneous enhancement and noise reduction algorithm of are proposed by analysing the essence of the dark channel prior de-hazing technique and bilateral filter. First, the authors estimate the values of the initial parameters of the hazy image model by de-hazing technique. Then, they correct the parameters of the hazy image model alternately with the iterative joint bilateral filter. Experimental results indicate that the proposed algorithm can simultaneously enhance the low-light images and reduce noise effectively. The proposed algorithm could also perform quite well compared with the current common image enhancement and noise reduction algorithms in terms of the subjective visual effects and objective quality assessments.

An Efficient Structure‐Aware Bilateral Texture Filtering for Image Smoothing

AbstractPhotos contain well‐structured and plentiful visual information. Edges are active and expressive stimuli for human visual perception. However, it is hard to separate structure from details because edge strength and object scale are entirely different concepts. This paper proposes a structure‐aware bilateral texture algorithm to remove texture patterns and preserve structures. Our proposed method is simple and fast, as well as effective in removing textures. Instead of patch shift, smaller patches represent pixels located at structure edges, and original patches represent the texture regions. Specifically, this paper also improves joint bilateral filter to preserve small structures. Moreover, a windowed inherent variation is adapted to distinguish textures and structures for detecting structure edges. Finally, the proposed method produces excellent experimental results. These results are compared to some results of previous studies. Besides, structure‐preserving filtering is a critical operation in many image processing applications. Our proposed filter is also demonstrated in many attractive applications, such as seam carving, detail enhancement, artistic rendering, etc.