Joint Bilateral Filter Research Articles

Shape-aware Mesh Normal Filtering

Mesh denoising is a fundamental yet open problem in geometry processing. The main challenge is to remove noise while recovering the shape of the underlying surface as accurately as possible. In this paper, we propose a novel joint bilateral filter on the face normal field. The key is to estimate a reliable guidance normal field by constructing a shape-aware consistent patch for accurately describing the local shape of each face. To this end, we first select a candidate patch for each face by using a newly defined consistent metric considering both patch flatness and face-to-patch orientation similarity. Then, spectral analysis is used in combination with ℓ0 minimization to refine the candidate patches in a shape-aware manner. The refined patches do not contain any features, and therefore they can accurately describe the local shape of the underlying surface. After smoothing the face normal field, vertex positions are reconstructed to match the filtered face normals. Our mesh denoising method is theoretically rooted and practical for dealing with the meshes containing corners with low sampling rates, multi-scale features, or narrow structure regions. Extensive experimental results demonstrate that our method can significantly improve the feature preserving capability of joint normal filter and outperforms state-of-the-art methods visually and quantitatively.

Fusion of Infrared and Visible Images for Surveillance Applications using Ann

Edge preserving filters were used to Multi-Scale Decomposition (MSD) for fusion of visible and infrared images. Traditional edge preserving MSDs may be unable to achieve satisfactory structural separation from details, resulting in fusion performance degradation. The objective of this work is to propose a MSD Iteration with ANN fusion technique for infrared and visual image which improves the fusion execution. Initially the original image is decomposed by “Gaussian smoothness and joint bilateral filter”. Edge retention and scaling perception attributes are introduced to satisfactorily separate image details from source images. Decomposition includes preserving the attributes of edge and zoom perception, so that the detail information is completely separated from the image details and the improvement of fusion performance is maintained. The rule aims to merge these decomposed layers. A saliency map is constructed by Laplacian and Gaussian low-pass filters to find the initial weight map and further a guided filter is used to determine the final weight map. The enhanced fused image later obtained by using ANN, which eventually increases the act of fusion execution. This work proposes ANN based fusion algorithm for fusing visible and infrared images and obtains better performance by reducing the complexity.

Multimodal medical image fusion based on joint bilateral filter and local gradient energy

• An effective, fast and noise robust multimodal medical image fusion method is proposed. An image is modeled as a superposition of the structure layer and the energy layer decomposed by joint filter, then a novel local gradient energy operator is proposed to fuse the structure layer and the abs-max rule to fuse the energy layer. • 118 co-registered pairs of medical images cover five different categories of medical image fusion problems (MR-T1/MR-T2, CT/MR, MR-Gad/MR-T2, MR/PET, and MR/SPECT) are tested. To the best of our knowledge, this is the most comprehensive experiment both in quantity and classes for medical image fusion. • The merits of noniterative, local, and without multiscale decomposition or reconstruction make the proposed method easy to implementation and achieve high computational efficiency. • Our method can be effectively extended to other image fusion problems, such as multifocus image fusion, infrared and visible image fusion, millimeter-wave and visible image fusion, panchromatic and multispectral image fusion. As a powerful assistance technique for biomedical diagnosis, multimodal medical image fusion has emerged as a hot topic in recent years. Unfortunately, the trade-off among fusion performance, time consumption and noise robustness for many medical image fusion algorithms remains an enormous challenge. In this paper, an effective, fast and robust medical image fusion method is proposed. A two-layer decomposition scheme is introduced by the joint bilateral filter, the energy layer containing rich intensity information, and the structure layer capturing ample details. Then a novel local gradient energy operator based on the structure tensor and neighbor energy is proposed to fuse the structure layer and the l 1 -max rule is introduced to fuse the energy layer. A total of 118 co-registered pairs of medical images covering five different categories of medical image fusion problems are tested in experiments. Seven latest representative medical image fusion methods are compared, and six representative quality evaluation metrics with complementary characteristics are fully employed to objectively evaluate the fused results. Extensive experimental results demonstrate that the proposed method yields better performance than some state-of-the-art methods in both visual quality and quantitative evaluation, and achieves nearly real-time computational efficiency and robustness to noise.

CDDA: color-dominant deep autoencoder for faster and efficient bilateral image filtering

Nonlinear processing of high-dimensional data is quite common in image filtering algorithms. Bilateral, joint bilateral, and non-local means filters are the examples of the same. Real-time implementation of high-dimensional filters has always been a research challenge due to its computational complexity. In this paper, we have proposed a solution utilizing both color sparseness and color dominance in an image which ensures a faster algorithm for generic high-dimensional filtering. The solution speeds up the filtering algorithm further by psycho-visual saliency-based deep encoded dominant color gamut, learned for different subject classes of images. The proposed bilateral filter has been proved to be efficient both in terms of psycho-visual quality and performance for edge-preserving smoothing and denoising of color images. The results demonstrate competitiveness of our proposed solution with the existing fast bilateral algorithms in terms of the CTQ (critical to quality) parameters.

Joint Bilateral Filter for Signal Recovery from Phase Preserved Curvelet Coefficients for Image Denoising

Thresholding of Curvelet Coefficients, for image denoising, drains out subtle signal component in noise subspace. In effect, it also produces ringing artifacts near edges. We found that the noise sensitivity of Curvelet phases — in contrast to their magnitude — reduces with higher noise level. Thus, we preserved the phase of the coefficients below threshold at coarser scale and estimated the corresponding magnitude by Joint Bilateral Filtering (JBF) technique. In contrast to the traditional hard thresholding, the coefficients in the finest scale is estimated using Bilateral Filtering (BF). The proposed filtering approach in the finest scale exhibits better connectedness among the edges, while removing the granular artifacts in the denoised image due to hard thresholding. Finally, the use of Guided Image Filter (GIF) on the Curvelet-based reconstructed image (initial denoised image in spatial domain) ensures the preservation of small image information with sharper edges and textures detail in the final denoised image. The lower noise sensitivity of Curvelet phase at higher noise strength accelerates the performance of proposed method over several state-of-the-art techniques and provides comparable outcome at lower noise levels.

Dense multiview stereo based on image texture enhancement

AbstractIn this paper, we propose a novel Multiview Stereo (MVS) method which can effectively estimate geometry in low‐textured regions. Conventional MVS algorithms predict geometry by performing dense correspondence estimation across multiple views under the constraint of epipolar geometry. As low‐textured regions contain less feature information for reliable matching, estimating geometry for low‐textured regions remains hard work for previous MVS methods. To address this issue, we propose an MVS method based on texture enhancement. By enhancing texture information for each input image via our multiscale bilateral decomposition and reconstruction algorithm, our method can estimate reliable geometry for low‐textured regions that are intractable for previous MVS methods. To densify the final output point cloud, we further propose a novel selective joint bilateral propagation filter, which can effectively propagate reliable geometry estimation to neighboring unpredicted regions. We validate the effectiveness of our method on the ETH3D benchmark. Quantitative and qualitative comparisons demonstrate that our method can significantly improve the quality of reconstruction in low‐textured regions.

Real-time video dehazing via incremental transmission learning and spatial-temporally coherent regularization

Video dehazing is of crucial importance to the outdoor surveillance, and its main challenges mainly come from the spatial-temporal coherence and computational efficiency. This paper presents an efficient real-time video dehazing approach via incremental transmission learning and spatial-temporally coherent regularization, while explicitly suppressing the possible visual artifacts. First, we propose to initialize the transmission map frame-by-frame by a boundary constrained open dark channel model. Then, supposing a scene point yields highly correlated transmission values between adjacent frames, we impose a temporally coherent term to maintain the temporal consistency of consecutive transmission values, and simultaneously derive an incremental transmission adjusting term to adapt the abrupt scene depth changes between the adjacent frames. Accordingly, the highly correlated and spatial-temporally coherent transmission maps can be optimized for each video frame dehazing, whereby the flickering artifacts can be well reduced. Finally, we group each haze/haze-free images in pair and further utilize the guided joint bilateral filter to suppress the visual artifacts that possibly amplified during the dehazing process. The experimental results show that the proposed video dehazing approach is able to well preserve the spatial-temporal coherence, runs sufficiently fast, and also performs favorably compared to the state-of-the-art methods.

CMWMF: Constant Memory Architecture of Weighted Mode/Median Filter for Extremely Large Label Depth Refinement

In this study, we propose a constant memory hardware architecture that can support weighted mode, median, and joint bilateral filters, which is referred to as CMWMF. This work aims to meet the high memory and computation requirements of processing depth maps with a large number of depth candidates. In the proposed architecture, we leverage the geometry smoothing characteristic of natural images to reduce the static random access memory (SRAM) size for hardware implementation. The architecture preserves a constant number of disparity values instead of depending on the label count and size of the local supporting window. A novel weighted median search procedure is proposed, which assigns a computation to each input cycle, thereby rendering the process hardware friendly. An index-checking technique is proposed to process out-of-order joint histograms. We adopted the above-mentioned techniques in our architecture as they consume a constant SRAM size and supports multiple types of filters. As a result, this architecture reduces the SRAM size by 92.4% with a negligible decrease in performance. According to our analysis on the KITTI, and Middlebury datasets, and with actual depth cameras, the preserved information is sufficient. The proposed architecture is one of the most suitable depth refinement architectures for scenarios having a large number of depth candidates.

Guided filter based image enhancement for focal error compensation in low cost automated histopathology microscopic system.

Low-cost automated histopathology microscopy systems usually suffer from optical imperfections, producing images that are slightly Out of Focus (OoF). In this work, a guided filter (GF) based image preprocessing is proposed for compensating focal errors and its efficacy is demonstrated on images of healthy and malaria infected red blood cells (h-RBCs and i-RBCs), and PAP smears. Since contrast enhancement has been widely used as an image preprocessing step for the analysis of histopathology images, a systematic comparison is made with six such prominently used methods, namely Contrast Limited Adaptive Histogram Equalization (CLAHE), RIQMC-based optimal histogram matching (ROHIM), modified L0, Morphological Varying(MV)-Bitonic filter, unsharp mask filter and joint bilateral filter. The images enhanced using GF approach lead to better segmentation accuracy (upto 50% improvement over native images) and visual quality compared to other approaches, without any change in the color tones. Thus, the proposed GF approach is a viable solution for rectifying the OoF microscopy images without the loss of the valuable diagnostic information presented by the color tone.

Entropy-Based Image Fusion with Joint Sparse Representation and Rolling Guidance Filter.

Image fusion is a very practical technology that can be applied in many fields, such as medicine, remote sensing and surveillance. An image fusion method using multi-scale decomposition and joint sparse representation is introduced in this paper. First, joint sparse representation is applied to decompose two source images into a common image and two innovation images. Second, two initial weight maps are generated by filtering the two source images separately. Final weight maps are obtained by joint bilateral filtering according to the initial weight maps. Then, the multi-scale decomposition of the innovation images is performed through the rolling guide filter. Finally, the final weight maps are used to generate the fused innovation image. The fused innovation image and the common image are combined to generate the ultimate fused image. The experimental results show that our method’s average metrics are: mutual information ()—5.3377, feature mutual information ()—0.5600, normalized weighted edge preservation value ()—0.6978 and nonlinear correlation information entropy ()—0.8226. Our method can achieve better performance compared to the state-of-the-art methods in visual perception and objective quantification.

Spatiotemporal Tree Filtering for Enhancing Image Change Detection.

Change detection has received extensive attention because of its realistic significance and broad application fields. However, none of the existing change detection algorithms can handle all scenarios and tasks so far. Different from the most of contributions from the research community in recent years, this paper does not work on designing new change detection algorithms. We, instead, solve the problem from another perspective by enhancing the raw detection results after change detection. As a result, the proposed method is applicable to various kinds of change detection methods, and regardless of how the results are detected. In this paper, we propose Fast Spatiotemporal Tree Filter (FSTF), a purely unsupervised detection method, to enhance coarse binary detection masks obtained by different kinds of change detection methods. In detail, the proposed FSTF has adopted a volumetric structure to effectively synthesize spatiotemporal information of the same target from the current time and history frames to enhance detection. The computational complexity analyzed in the view of graph theory also show that the fast realization of FSTF is a linear time algorithm, which is capable of handling efficient on-line detection tasks. Finally, comprehensive experiments based on qualitative and quantitative analysis verify that FSTF-based change detection enhancement is superior to several other state-of-the-art methods including fully connected Conditional Random Field (CRF), joint bilateral filter, and guided filter. It is illustrated that FSTF is versatile enough to also improve saliency detection as well as semantic image segmentation.

Edge-Guided Depth Image Super-Resolution Based on KSVD

This paper proposes an edge-guided super-resolution algorithm for single frame depth images based on K singular value decomposition (KSVD). Compared with conventional algorithms, the proposed algorithm has two key contributions. Firstly it suppresses the jagged edge effect of up-sampled depth images by KSVD, which learns a complete dictionary to describe the mapping between the jagged edges and corresponding smooth ones. Secondly it improves the joint bilateral filter based on connectivity. The improved filter can not only preserve the sharpness of the edges during the interpolation, but also suppress noise. The proposed algorithm has been extensively tested on the Middlebury dataset and compared with some existing state-of-the-art methods. Both quantitative and qualitative experimental results show its performance superiority.

水下二维及三维距离选通成像去噪技术研究

Compared with traditional underwater cameras, the detection distance of underwater range-gated imaging can be increased by 2-3 times. Furthermore, based on this technology, fast high-resolution 3D imaging can be achieved. It has great potentials in underwater target detection and recognition, automatic navigation, marine scientific research and natural resources exploration. Although range-gated imaging can suppress the backscattering noise by space slicing, and achieve higher quality images, the backscattering noise of the sampling water volume still exists in gated images, resulting in low image signal-to-noise ratio and contrast, especially for distant targets or low reflectance targets. This paper systematically introduced author's research of deblurring methods for 2D and 3D range-gated imaging. In 2D imaging, two methods were proposed:the first was an algorithm of self-adaptive double-plateau histogram equalization to improve image contrast and better meet human vision; the second was to use a water-noise-reference denoising algorithm to improve the signal-to-noise ratio of target gated images. In 3D imaging, there were also two methods proposed:one was to use the water-noise-reference denoising algorithm for 3D reconstruction, and the other was to use the threshold-dependent joint bilateral filter algorithm for enhancing 3D images. The four proposed methods can be used independently or in combination for denoising enhancement of underwater range gated imaging.

Video super-resolution via pre-frame constrained and deep-feature enhanced sparse reconstruction

This paper presents a new video super-resolution (SR) method that can generate high-quality and temporally coherent high-resolution (HR) videos. Starting from the traditional sparse reconstruction framework that works well for image SR, we improve it significantly from the following aspects to obtain an effect video SR method. Firstly, to enhance the temporal coherence between adjacent HR frames, once a HR frame is estimated, we use it to guide the sparse reconstruction of the next low-resolution frame. Secondly, instead of using just hand-craft features, we further incorporate deep features generated by VGG16 into our sparse reconstruction based video SR method. Thirdly, we constantly update the dictionary, which is the core of the sparse reconstruction, by making use of the previously estimated HR frame. Finally, after the HR video is reconstructed, we use a joint bilateral filter to post-process it to remove artifacts and transfer image details. Experiments demonstrate that the proposed four strategies effectively improve our final results. In most of the experiments of this paper, our results are better than those produced by the latest deep learning based approaches.

Single Image Super-Resolution Algorithm Possessing Edge and Contrast Preservation

In real time surveillance video applications, it is often required to identify a region of interest in a degraded low resolution (LR) image. State-of-the-art super-resolution (SR) techniques produce images with poor illumination and degraded high frequency details. In this paper, we present a different approach for SISR by correcting the dual-tree complex wavelet transform (DT-CWT) subbands using the multi-stage cascaded joint bilateral filter (MSCJBF) and singular value decomposition (SVD). The proposed method exploits geometric regularity for implementing the covariance-based interpolation in the spatial domain. We decompose the interpolated LR image into different image and wavelet coefficients by employing DT-CWT. To preserve edges, we alter the wavelet sub-bands with the high frequency details obtained from the MSCJBF. Simultaneously, we retain uniform illumination by improving the image coefficients using SVD. In addition, the wavelet sub-bands undergo lanczos interpolation prior to the subband refinement. Experimental results demonstrate the effectiveness of our method.

Progressive complex illumination image appearance transfer based on CNN

Image appearance transfer is the process of transferring the appearance features from the user-supplied reference image to the target image. Since traditional appearance transfer methods are based on low-level feature, it is difficult to obtain natural effect in the case of complex illumination between the target image and the reference image. This is mainly because that the appearance mapping relation with complex illumination is a highly nonlinear problem. Although traditional methods are good at dealing with linear transformations, they are less suitable for solving the highly nonlinear problems caused by such complex illumination. Moreover, convolutional neural network (CNN) can extract the hierarchical abstraction features at different levels of the network layer, so it can be conveniently used to realize progressive transfer. In this paper, we propose a novel method for progressive appearance transfer for images with complex illumination. Firstly, we convert the input images from the RGB color space to the HSV color space. The illumination transfer is carried out only in the illumination channel of the image, and for the other two channels, the color distribution of the reference image is transferred to the target image. Secondly, CNN is specially designed to extract the hierarchical feature maps. To achieve progressive transfer, the histogram reshaping method is carried out by using the hierarchical feature maps extracted from the CNN. The appearance transfer results are obtained after the illumination transfer and color transfer. To optimize the transfer results, we adopt the joint bilateral filter to smooth the noises. The experimental results show that our method can effectively solve the problem of progressive appearance transfer for images with complex illumination.

Anisotropic Joint Trilateral Rolling Filter for Image Smoothing

Filtering an image by eliminating noise and irrelevant details while preserving prominent structure edges is an important pre-processing task in many fields, such as image processing and computer vision. In this study, a novel approach called anisotropic joint trilateral rolling filter (AJTRF) is proposed for the smoothing of an image while preserving important structure edges. AJTRF works in an iterative manner and extends the joint bilateral filter to a joint trilateral filter by employing additional weights in eigen space. During filtering, the range weights are recalculated in each iteration, while the spatial Gaussian weights are computed in anisotropic orientations instead of isotropic orientations. Furthermore, the inner products between anisotropic orientations (expressed as eigenvectors) are used as weights to measure orientation similarities of pixels. Structure tensors at each pixel are calculated to determine the structure orientations, which are used as the anisotropic orientations. A range of experiments are conducted. The results demonstrate that the proposed AJTRF has stronger smoothing and structure preserving ability than established methods.

Image colourisation using deep feature‐guided image retrieval

In this study, the authors aim to colourise a greyscale image using a fully automated framework which retrieves similar images from a reference database and then transfers the colour from the most similar retrieved images to perform colourisation. Inspired by the recent success of deep learning techniques in extracting semantic information from images, they first use fc7 features from AlexNet to retrieve similar images from the reference database. Top- k retrieved images are considered for colour transfer to the target greyscale image, using various pixel level features. The images which result from the previous step are given a colour enhancement with Reinhard stain normalisation. They follow a pixel-wise colour saturation based averaging technique to impart colour at pixel level. The final image is rectified using joint bilateral filtering. The resulting coloured images have a realistic appearance, similar in quality to the original coloured images. The proposed method outperforms several previous colourisation techniques, yielding superior performance both quantitatively and qualitatively. The method also enhances low-contrast images.

Optimization of the proposed hybrid denoising technique to overcome over-filtering issue.

Image denoising has become a crucial task in medical ultrasound (US) imaging due to the presence of speckle or multiplicative noise and additive Gaussian noise. Recently, several denoising techniques such as adaptive wavelet thresholding & joint bilateral (AWT + JB) filter, adaptive fuzzy switching weighted mean (AFSWM) filter and median patch-based locally optimal Wiener (MPBLOW) filter have been proposed to remove the speckle noise. However, these denoising techniques were found to remove noise along with the essential parts of the actual image data which is known as over-filtering. Thereby, it reduces the accuracy of the recognition process. In this paper, a new hybrid filter technique is proposed by combining anisotropic diffusion (AD) with Butterworth band pass filter to overcome over-filtering of the image. In addition, the performance of the proposed hybrid filter and its design parameters are enhanced using the particle swarm optimization (PSO) algorithm. The simulation results show that the proposed filtering technique achieves a better denoising performance when compared with other filtering techniques in terms of peak signal-to-noise ratio (PSNR), signal-to-noise ratio (SNR), structural similarity index (SSIM) and edge preservation index (EPI). Moreover, the results validated that the proposed filtering technique using PSO achieves effective performance than using the harmony search algorithm (HSA) and other filtering techniques.

Multimodal image seamless fusion

An iterative joint bilateral filter is used to obtain a natural weight map. Images from different modalities are merged by a weighted-sum rule in the spatial domain. Saliency maps are determined by the gradient of the pairwise raw images. Comparing the pairwise values of saliency maps, a coarse weight map is attained to determine which pixel is preferred. Since such a coarse weight map obtained by pairwise comparison is not a natural weight map subjectively, i.e., it is inconsistent with human visual system, the weight map is modified by using an iterative joint bilateral filter. With the iterative joint bilateral filter, the weight map becomes natural. We use the refined weight map to obtain the fused image and we seamlessly merge images from different modalities effectively. Experiments were conducted on several pairs of multimodal images to verify the effectiveness and superiority of the proposed image fusion algorithm compared to the state-of-the-art methods. (C) 2019 SPIE and IS&T