Joint Bilateral Filter Research Articles

A two-stage defect detection method for unevenly illuminated self-adhesive printed materials

The process of printing defect detection usually suffers from challenges such as inaccurate defect extraction and localization, caused by uneven illumination and complex textures. Moreover, image difference-based defect detection methods often result in numerous small-scale pseudo defects. To address these challenges, this paper proposes a comprehensive defect detection approach that integrates brightness correction and a two-stage defect detection strategy for self-adhesive printed materials. Concretely, a joint bilateral filter coupled with brightness correction corrects uneven brightness properly, meanwhile smoothing the grid-like texture in complex printed material images. Then, in the first detection stage, an image difference method based on a bright–dark difference template group is designed to effectively locate printing defects despite slight brightness fluctuations. Afterward, a discriminative method based on feature similarity is employed to filter out small-scale pseudo-defects in the second detection stage. The experimental results show that the improved difference method achieves an average precision of 99.1% in defect localization on five different printing pattern samples. Furthermore, the second stage reduces the false detection rate to under 0.5% while maintaining the low missed rate.

Expanding Sparse Radar Depth Based on Joint Bilateral Filter for Radar-Guided Monocular Depth Estimation.

Radar data can provide additional depth information for monocular depth estimation. It provides a cost-effective solution and is robust in various weather conditions, particularly when compared with lidar. Given the sparse and limited vertical field of view of radar signals, existing methods employ either a vertical extension of radar points or the training of a preprocessing neural network to extend sparse radar points under lidar supervision. In this work, we present a novel radar expansion technique inspired by the joint bilateral filter, tailored for radar-guided monocular depth estimation. Our approach is motivated by the synergy of spatial and range kernels within the joint bilateral filter. Unlike traditional methods that assign a weighted average of nearby pixels to the current pixel, we expand sparse radar points by calculating a confidence score based on the values of spatial and range kernels. Additionally, we propose the use of a range-aware window size for radar expansion instead of a fixed window size in the image plane. Our proposed method effectively increases the number of radar points from an average of 39 points in a raw radar frame to an average of 100 K points. Notably, the expanded radar exhibits fewer intrinsic errors when compared with raw radar and previous methodologies. To validate our approach, we assess our proposed depth estimation model on the nuScenes dataset. Comparative evaluations with existing radar-guided depth estimation models demonstrate its state-of-the-art performance.

Robust Remote Sensing Image Classification utilizing Multiverse Optimization Model with Deep Transfer Learning Approach

The classifying process of Remote Sensing Image (RSI) has been crucial in diverse applications, involving land cover mapping, environmental monitoring, disaster assessment, and urban planning. Conventionally, image classification in Remote Sensing (RS) relies on Machine Learning (ML) and handcrafted feature extraction approaches. But, lately, Deep Learning (DL) has become an effective and powerful method for RSI classification. DL algorithms, especially Convolutional Neural Network (CNN), have shown great performance in different Computer Vision (CV) tasks, involving image classification. This study proposes a Robust RSI Classification using Multiverse Optimization Algorithm with Deep Transfer Learning (RRSIC-MVO) model. The RRSIC-MVO technique exploits the DL with hyper-parameter selection mechanism to classify the RSI. In the presented RRSIC-MVO technique, the noise occurrence can be handled by joint bilateral filter (JBF). Following, MobileNetv3 model is applied for feature extractor and its hyper-parameters can be elected by the use of FA. Finally, soft-margin support vector machine (SM-SVM) model is applied for classification purposes. A series of investigations are accomplished on RS dataset for assessing the RRSIC-MVO model. The outputs demonstrate the effectiveness of RRSIC-MVO technique, outperforming traditional machine learning methods.

Enhancing Anomaly Detection in Pedestrian Walkways using Improved Sparrow Search Algorithm with Parallel Features Fusion Model

Anomaly detection in pedestrian walkways is a vital research area, widely employed to enhance the safety of the pedestrians. Because of the widespread usage of the video surveillance systems and the increasing number of captured videos, the conventional manual examination of labeling abnormal events is a laborious process. Therefore, an automatic surveillance system to accurately detect anomalies becomes essential among computer vision researchers. Presently, the development of deep learning (DL) models has gained significant interest in different computer vision processes namely object classification and object detection, and these applications were depending on supervised learning that required labels. This article develops an Improved Meta-heuristic with Parallel Features Fusion Model for Anomaly Detection in Pedestrian Walkways (IMPFF-ADPW) method. The main aim of the IMPFF-ADPW approach is to recognize the existence of anomalies in pedestrian walkways. To obtain this, the IMPFF-ADPW method applies a joint bilateral filter (JBF) for the process of noise removal. Besides, a parallel fusion process comprising NasNet Mobile and Darknet-53 models can be utilized for feature extraction. For the anomaly detection method, the deep autoencoder (DAE) model is applied and its hyperparameters are finetuned by using an improved sparrow search algorithm (ISSA). A wide of experimental outcomes can be applied to the UCSD database to illustrate the betterment of the IMPFF-ADPW methodology. The simulation values indicated the enhanced performance of the IMPFF-ADPW method over other existing techniques.

SIERRA: A robust bilateral feature upsampler for dense prediction

Feature upsampling is a fundamental operation in modern deep network architectures. Existing upsamplers, however, are prone to cause negative upsampling—performance of an upsampler falls behind naive interpolation. For instance, the recent dynamic upsampler CARAFE is the best performing operator in semantic segmentation, but it turns out to be the worst one in image matting. In this work, we present robuSt bIlatERal featuRe upsAmpler (SIERRA), a simple, task-robust, plug-and-play, and ultra lightweight upsampler. Its key idea is to use an efficient gradient-prior kernel to modulate a (shifted) distance-prior kernel to control which feature points participate in interpolation, which shares a similar spirit to joint bilateral filtering (JBF). Yet, in contrast to JBF that requires high-res guidance, SIERRA generates kernels from the low-res decoder feature alone. Extensive experiments demonstrate the superiority and robustness of SIERRA on five dense prediction tasks. Code will be available online.

Blind quality assessment of multi-exposure fused images considering the detail, structure and color characteristics.

In the process of multi-exposure image fusion (MEF), the appearance of various distortions will inevitably cause the deterioration of visual quality. It is essential to predict the visual quality of MEF images. In this work, a novel blind image quality assessment (IQA) method is proposed for MEF images considering the detail, structure, and color characteristics. Specifically, to better perceive the detail and structure distortion, based on the joint bilateral filtering, the MEF image is decomposed into two layers (i.e., the energy layer and the structure layer). Obviously, this is a symmetric process that the two decomposition results can independently and almost completely describe the information of MEF images. As the former layer contains rich intensity information and the latter captures some image structures, some energy-related and structure-related features are extracted from these two layers to perceive the detail and structure distortion phenomena. Besides, some color-related features are also obtained to present the color degradation which are combined with the above energy-related and structure-related features for quality regression. Experimental results on the public MEF image database demonstrate that the proposed method achieves higher performance than the state-of-the-art quality assessment ones.

Structure-Preserving Texture Smoothing via Scale-Aware Bilateral Total Variation

The purpose of texture smoothing is to preserve the prominent structure in the image while smoothing the salient texture. However, the existing methods are difficult to achieve a satisfactory balance between filtering out salient textures and preserving weak edge structures and small structures. To this end, we propose a structure-preserving texture smoothing method via scale-aware bilateral total variation. First, the joint bilateral filter is introduced to construct the window bilateral variation, and combined with the window total variation, a regularizer called the bilateral total variation is formed, which accurately quantifies the characteristics of texture and structure, to finely smooth salient textures while preserving weak edge structures and small structures. Subsequently, we proposed a scale-aware scheme to make the proposed regularizer more powerful in preserving small structures and adopted an optimization scheme to convert the original non-convex optimization problem into a least squares regression problem. The effectiveness of the proposed regularizer is verified in the dataset. The experimental results demonstrate the superiority of the proposed method in texture smoothing and other applications compared to other state-of-the-art approaches.

Real-Time Visualization of Large-Scale Geological Models With Nonlinear Feature-Preserving Levels of Detail.

The rapidly growing size and complexity of 3D geological models has increased the need for level-of-detail techniques and compact encodings to facilitate interactive visualization. For large-scale hexahedral meshes, state-of-the-art approaches often employ wavelet schemes for level of detail as well as for data compression. Here, wavelet transforms serve two purposes: (1) they achieve substantial compression for data reduction; and (2) the multiresolution encoding provides levels of detail for visualization. However, in coarser detail levels, important geometric features, such as geological faults, often get too smoothed out or lost, due to linear translation-invariant filtering. The same is true for attribute features, such as discontinuities in porosity or permeability. We present a novel, integrated approach addressing both purposes above, while preserving critical data features of both model geometry and its attributes. Our first major contribution is that we completely decouple the computation of levels of detail from data compression, and perform nonlinear filtering in a high-dimensional data space jointly representing the geological model geometry with its attributes. Computing detail levels in this space enables us to jointly preserve features in both geometry and attributes. While designed in a general way, our framework specifically employs joint bilateral filters, computed efficiently on a high-dimensional permutohedral grid. For data compression, after the computation of all detail levels, each level is separately encoded with a standard wavelet transform. Our second major contribution is a compact GPU data structure for the encoded mesh and attributes that enables direct real-time GPU visualization without prior decoding.

Multi-modal MRI image fusion of the brain based on joint bilateral filter and non-subsampled shearlet transform

Multi-modal MRI image fusion of the brain based on joint bilateral filter and non-subsampled shearlet transform

Push-broom compressive hyperspectral imaging method based on self-fusion refinement

Compressive hyperspectral imaging (CHI) with random encoding mask usually suffers from various noises and artifacts. Inspired by the dual-camera CHI techniques based on hyperspectral (HS) and multispectral (MS) image fusion, herein, we present a single-camera push-broom CHI method based on self-fusion refinement (SFR). In this work, the MS guidance image for data fusion is derived directly from the raw solved HS data cube itself rather than any additional data source, which turns cross-fusion into self-fusion; furthermore, a modified joint bilateral filtering (JBF) fusion algorithm is developed to adapt this self-fusion problem, and an adaptive range Gaussian radius is adopted to avoid the invalidation or over-smoothing effects so as to ensure spatial and spectral improvement. The visualized and quantitative assessment results both demonstrate that the proposed method achieves high-quality HS imaging in terms of noise and artifact removal and spatial–spectral fidelity. Furthermore, the proposed method has a great flexibility and extensibility, whose performances highly depend on the exact fusion algorithm adopted, and a more suitable fusion algorithm will lead to better reconstruction quality; herein, the SFR process by the modified JBF achieves better performances than SFR by guided filtering (GF) or Markov random field (MRF).

Refinement method for compressive hyperspectral data cubes based on self-fusion.

Compressive hyperspectral images often suffer from various noises and artifacts, which severely degrade the imaging quality and limit subsequent applications. In this paper, we present a refinement method for compressive hyperspectral data cubes based on self-fusion of the raw data cubes, which can effectively reduce various noises and improve the spatial and spectral details of the data cubes. To verify the universality, flexibility, and extensibility of the self-fusion refinement (SFR) method, a series of specific simulations and practical experiments were conducted, and SFR processing was performed through different fusion algorithms. The visual and quantitative assessments of the results demonstrate that, in terms of noise reduction and spatial-spectral detail restoration, the SFR method generally is much better than other typical denoising methods for hyperspectral data cubes. The results also indicate that the denoising effects of SFR greatly depend on the fusion algorithm used, and SFR implemented by joint bilateral filtering (JBF) performs better than SRF by guided filtering (GF) or a Markov random field (MRF). The proposed SFR method can significantly improve the quality of a compressive hyperspectral data cube in terms of noise reduction, artifact removal, and spatial and spectral detail improvement, which will further benefit subsequent hyperspectral applications.

Trainable joint bilateral filters for enhanced prediction stability in low-dose CT

Low-dose computed tomography (CT) denoising algorithms aim to enable reduced patient dose in routine CT acquisitions while maintaining high image quality. Recently, deep learning (DL)-based methods were introduced, outperforming conventional denoising algorithms on this task due to their high model capacity. However, for the transition of DL-based denoising to clinical practice, these data-driven approaches must generalize robustly beyond the seen training data. We, therefore, propose a hybrid denoising approach consisting of a set of trainable joint bilateral filters (JBFs) combined with a convolutional DL-based denoising network to predict the guidance image. Our proposed denoising pipeline combines the high model capacity enabled by DL-based feature extraction with the reliability of the conventional JBF. The pipeline’s ability to generalize is demonstrated by training on abdomen CT scans without metal implants and testing on abdomen scans with metal implants as well as on head CT data. When embedding RED-CNN/QAE, two well-established DL-based denoisers in our pipeline, the denoising performance is improved by 10%/82% (RMSE) and 3%/81% (PSNR) in regions containing metal and by 6%/78% (RMSE) and 2%/4% (PSNR) on head CT data, compared to the respective vanilla model. Concluding, the proposed trainable JBFs limit the error bound of deep neural networks to facilitate the applicability of DL-based denoisers in low-dose CT pipelines.

Illuminant Adaptive Wideband Image Synthesis Using Separated Base-Detail Layer Fusion Maps

In this study, we present a wideband image synthesis technique for day and night object identification. To synthesize the visible and near-infrared images, a base component and a detailed component are first decomposed using a bilateral filter, and the detailed component is synthesized using a local variance map. In addition, considering the difference in the near-infrared image characteristics between daytime and nighttime, the base components are synthesized using a luminance saturation region map and depth and penetration map using a joint bilateral filter. The proposed method overcomes the partial over- or under-exposure caused by sunlight and infrared auxiliary light, which is experienced variously in wideband imaging, and improves the identification of objects in various indoor and outdoor images compared with that achieved by existing methods by emphasizing detailed components.

Masked Joint Bilateral Filtering via Deep Image Prior for Digital X-Ray Image Denoising.

Medical image denoising faces great challenges. Although deep learning methods have shown great potential, their efficiency is severely affected by millions of trainable parameters. The non-linearity of neural networks also makes them difficult to be understood. Therefore, existing deep learning methods have been sparingly applied to clinical tasks. To this end, we integrate known filtering operators into deep learning and propose a novel Masked Joint Bilateral Filtering (MJBF) via deep image prior for digital X-ray image denoising. Specifically, MJBF consists of a deep image prior generator and an iterative filtering block. The deep image prior generator produces plentiful image priors by a multi-scale fusion network. The generated image priors serve as the guidance for the iterative filtering block, which is utilized for the actual edge-preserving denoising. The iterative filtering block contains three trainable Joint Bilateral Filters (JBFs), each with only 18 trainable parameters. Moreover, a masking strategy is introduced to reduce redundancy and improve the understanding of the proposed network. Experimental results on the ChestX-ray14 dataset and real data show that the proposed MJBF has achieved superior performance in terms of noise suppression and edge preservation. Tests on the portability of the proposed method demonstrate that this denoising modality is simple yet effective, and could have a clinical impact on medical imaging in the future.

Joint self-supervised and reference-guided learning for depth inpainting

Depth information can benefit various computer vision tasks on both images and videos. However, depth maps may suffer from invalid values in many pixels, and also large holes. To improve such data, we propose a joint self-supervised and reference-guided learning approach for depth inpainting. For the self-supervised learning strategy, we introduce an improved spatial convolutional sparse coding module in which total variation regularization is employed to enhance the structural information while preserving edge information. This module alternately learns a convolutional dictionary and sparse coding from a corrupted depth map. Then, both the learned convolutional dictionary and sparse coding are convolved to yield an initial depth map, which is effectively smoothed using local contextual information. The reference-guided learning part is inspired by the fact that adjacent pixels with close colors in the RGB image tend to have similar depth values. We thus construct a hierarchical joint bilateral filter module using the corresponding color image to fill in large holes. In summary, our approach integrates a convolutional sparse coding module to preserve local contextual information and a hierarchical joint bilateral filter module for filling using specific adjacent information. Experimental results show that the proposed approach works well for both invalid value restoration and large hole inpainting.

Adaptive side window joint bilateral filter

Adaptive side window joint bilateral filter

Two-Scale Multimodal Medical Image Fusion Based on Structure Preservation

Medical image fusion has an indispensable value in the medical field. Taking advantage of structure-preserving filter and deep learning, a structure preservation-based two-scale multimodal medical image fusion algorithm is proposed. First, we used a two-scale decomposition method to decompose source images into base layer components and detail layer components. Second, we adopted a fusion method based on the iterative joint bilateral filter to fuse the base layer components. Third, a convolutional neural network and local similarity of images are used to fuse the components of the detail layer. At the last, the final fused result is got by using two-scale image reconstruction. The contrast experiments display that our algorithm has better fusion results than the state-of-the-art medical image fusion algorithms.

Superpixel Guided Deformable Convolution Network for Hyperspectral Image Classification.

Convolutional neural networks are widely used in the field of hyperspectral image classification because of their excellent nonlinear feature extraction ability. However, as the sampling position of the regular convolution kernel is unchangeable, the regular convolution cannot distinctively extract the spatial and spectral information around the central pixel, which makes the classification results at the boundaries of ground objects over-smoothed and the classification performance degraded. Thus, we propose a novel superpixel guided deformable convolution network (SGDCN) for hyperspectral image classification. Firstly, the superpixel region fusion filter (SRF-Filter) is designed to fuse the initial superpixel region segmented by the simple linear iterative clustering (SLIC), making the fused superpixel region have a high homogeneity and also contain spatial features of diverse scales. Then, the superpixel guided deformable convolution (SGD-Conv) is proposed to make the shape of deformable convolution consistent with the real shape of land covers, and the SGD-Conv can extract pure neighborhood spatial-spectral features. Finally, a superpixel joint bilateral filter (SPJBF) is designed to solve the pixel-level and region-level misclassification problem, which can effectively utilize the superpixel region's homogeneity and improve the classification accuracy. Experiments on three HSI datasets indicate that the SGDCN can obtain better classification performance when compared with other twelve state-of-the-art methods.

Hyperspectral Image Classification Based on Kernel-Guided Deformable Convolution and Double-Window Joint Bilateral Filter

Convolutional neural networks (CNNs) have been widely used in hyperspectral image (HSI) classification. However, a shape-fixed convolution kernel cannot extract appropriate spatial-spectral features. Thus, we propose a novel two-stage classification method based on kernel-guided deformable convolution networks and double-window joint bilateral filter (KDCDWBF) for HSIs. First, according to the calculated similarity map, the shape of the kernel-guided deformable convolution (KDC) is more consistent with the real shape of land covers, so the KDC can extract more pure neighborhood spatial-spectral information. Then, using the piecewise smoothness property of the HSI, a double-window joint bilateral filter (DWJBF) is designed to complete the coarse-to-fine classification stage, which can solve the misclassification problem of single pixels and small regions. Experiments on two HSI datasets demonstrate that the proposed network can achieve better classification performance when compared with other state-of-the-art methods.

Fast and effective single-scan dual-energy cone-beam CT reconstruction and decomposition denoising based on dual-energy vectorization.

Flat-panel detector (FPD) based dual-energy cone-beam computed tomography (DE-CBCT) is a promising imaging technique for dedicated clinical applications. In this paper, we proposed a fully analytical method for fast and effective single-scan DE-CBCT image reconstruction and decomposition. A rotatable Mo filter was inserted between an x-ray source and imaged object to alternately produce low and high-energy x-ray spectra. First, filtered-backprojection (FBP) method was applied on down-sampled projections to reconstruct low and high-energy images. Then, the two images were converted into a vectorized form represented with an amplitude and an argument image. Using amplitude image as a guide, a joint bilateral filter was applied to denoise the argument image. Then, high-quality dual-energy images were recovered from the amplitude image and the denoised argument image. Finally, the recovered dual-energy images were further used for low-noise material decomposition and electron density synthesis. Imaging was conducted on a Catphan® 600 phantom and an anthropomorphic head phantom. The proposed method was evaluated via comparison with the traditional two-scan method and a commonly used filtering method (HYPR-LR). On the Catphan® 600 phantom, the proposed method successfully reduced streaking artifacts and preserved spatial resolution and noise-power-spectrum (NPS) pattern. In the electron density image, the proposed method increased contrast-to-noise ratio (CNR) by more than 2.5 times and achieved <1.2% error for electron density values. On the anthropomorphic head phantom, the proposed method greatly improved the soft-tissue contrast and the fine detail differentiation ability. In the selected ROIs on different human tissues, the differences between the CT number obtained by the proposed method and that by the two-scan method were less than 4HU. In the material images, the proposed method suppressed noise by over 75.5% compared with two-scan results, and by over 40.4% compared with HYPR-LR results. Implementation of the whole algorithm took 44.5s for volumetric imaging, including projection preprocessing, FBP reconstruction, joint bilateral filtering, and material decomposition. Using down-sampled projections in single-scan DE-CBCT, the proposed method could effectively and efficiently produce high-quality DE-CBCT images and low-noise material decomposition images. This method demonstrated superior performance on spatial resolution enhancement, NPS preservation, noise reduction, and electron density accuracy, indicating better prospect in material differentiation and dose calculation.