Edge-preserving Filter Research Articles

Multi exposure fusion for high dynamic range imaging via multi-channel gradient tensor

Multi-exposure fusion (MEF) is an effective technique for directly fusing a sequence of low dynamic range (LDR) images from a high dynamic range (HDR) natural scene. The goal is to generate an information enriched LDR image. Despite its effectiveness, current MEF methods often encounter issues such as detail loss and color degradation. Additionally, existing algorithms often struggle to balance image quality and computation time, particularly for large-sized images. This paper introduces an innovative MEF algorithm that address these challenges, offering improved performance and computational time across all image sizes. The algorithm employs a multi-channel gradient tensor on RGB images to effectively capture the contrast information among the three channels. This mechanism allows an edge-preserving image filter to maintain edges while smoothing weight maps. To enhance computational efficiency, the algorithm uses a fast approximation method suitable for large sized images. Our comprehensive experimental results demonstrate that the proposed method outperforms existing MEF techniques both quantitatively and qualitatively. Furthermore, our method reduces computational time by approximately 30% compared to the most recent state-of-the-art techniques.

Gaussian error loss function for image smoothing

Edge-preserving image smoothing plays an important role in the fields of image processing and computational photography, and is widely used for a variety of applications. The edge-preserving filters based on global optimization models have attracted widespread attention due to their nice smoothing quality. According to existing research, the edge-preserving capability is strongly correlated to the penalty function used for gradient regularization. By analyzing the edge-stopping function of existing penalties, we demonstrate that existing image smoothing models are not adequately edge-preserving. In this paper, based on a Gaussian error function (ERF), we propose a Gaussian error loss function (ERLF), which shows stronger edge-preserving capability. We embed the proposed loss function into a global optimization model for edge-preserving image smoothing. In addition, we propose an efficient solution based on additive half-quadratic minimization and Fourier-domain optimization that is capable of processing 720P color images (over 20 fps) in real-time on an NVIDIA RTX 3070 GPU. We have experimented with the proposed filter on a number of low-level vision tasks. Both quantitative and qualitative experimental results show that the proposed filter outperforms existing filters. Therefore, it can be practical for real applications.

Detail-preserving Joint Image Upsampling

Image operators can be instrumental to computational imaging and photography. However, many of them are computationally intensive. In this article, we propose an effective yet efficient joint upsampling method to accelerate various image operators. We show that edge-preserving filtering can be facilitated with a downsampling-and-upsampling process. Moreover, when the extent of smoothing is mild, the process is detail preserving, i.e., the fine details lost in the low-resolution (LR) images can be accurately restored in the high-resolution (HR) images. Given an HR input and an LR output of an operator, we downsample the HR input and calculate its affinities to the HR input. By applying the affinities to the LR output, we promote its resolution. Due to the strong detail-preserving property, the HR output derived in the previous step may exhibit aliasing artifacts around the salient edges. We further refine it based on the linear relations in a small neighborhood to rid the artifacts. Experiments on various image operators show that our method achieves superior quality over the state-of-the-art joint upsampling methods. Furthermore, the running time of our method is linear to the number of pixels. Our naive implementation derives 1080P images in real time (24 fps) on an NVIDIA GTX 3070 GPU.

Stereo matching algorithm using deep learning and edge-preserving filter for machine vision

Machine vision research began with a single-camera system, but these systems had various limitations from having just one point-of-view of the environment and no depth information, therefore stereo cameras were invented. This paper proposes a hybrid method of a stereo matching algorithm with the goal of generating an accurate disparity map critical for applications such as 3D surface reconstruction and robot navigation to name a few. Convolutional neural network (CNN) is utilised to generate the matching cost, which is then input into cost aggregation to increase accuracy with the help of a bilateral filter (BF). Winner-take-all (WTA) is used to generate the preliminary disparity map. An edge-preserving filter (EPF) is applied to that output based on a transform that defines an isometry between curves on the 2D image manifold in 5D and the real line to eliminate these artefacts. The transform warps the input signal adaptively to allow linear 1D filtering. Due to the filter's resistance to high contrast and brightness, it is effective in refining and removing noise from the output image. Based on experimental research employing a Middlebury standard validation benchmark, this approach gives high accuracy with an average non-occluded error of 6.71% comparable to other published methods.

Single-shot enhanced imaging through multimode fiber based on self-learning networks

High-fidelity imaging through a multimode fiber (MMF) plays a crucial role in various fields such as medicine, communication, and detection. However, the optical transmission matrix of the MMF suffers from dimensionality reduction. This case results in poor reconstruction quality such as low resolution and noise. In this paper, an unsupervised self-learning circulate learning network is employed to enhance a single degraded image without ground truth images. Moreover, an edge-preserving smoothing filter is applied to address the heavy noise problem of the reconstructed images. Experimental results demonstrate that the proposed method can improve the dimensionality and fidelity of the reconstructed target. Compared to traditional transmission matrix-based reconstruction methods, we have a competitive advantage in terms of evaluation metrics. The proposed method further advances the development of imaging through a multimode fiber.

Image Segmentation Technology Based on Ant Colony Algorithm

Image segmentation is a key task in computer vision, with applications ranging from medical diagnosis to autonomous driving. The Ant Colony Algorithm (ACO), modeled after ant foraging behavior, has emerged as a viable segmentation methodology. However, ACO-based segmentation algorithms frequently generate segmented outputs with jagged or uneven boundaries, which reduces their interpretability and usability. To alleviate this problem, they study the use of boundary-smoothing approaches in ACO-based segmentation. In this paper, they investigate image segmentation technology based on the Ant Colony Algorithm, with a focus on border smoothing. They examine the fundamentals of ACO and its application to image segmentation, emphasizing its strengths and limits. They also look at several boundary smoothing strategies, such as morphological operations, edge-preserving filters, and active contours (snakes), and how they affect segmentation performance. Through experimental validation and comparative analysis, they show that boundary smoothing improves the accuracy and visual quality of segmented images produced by ACO-based segmentation algorithms. These results help to design more robust and visually appealing segmentation algorithms, which have potential applications in medical imaging, remote sensing, and industrial automation.

Edge preserving infrared and visible image fusion with three layer decomposition based on multi-level co-occurrence filtering

By merging different portraits of a particular scene, image fusion attempts to create a blended image that combines details from all the images. Infrared (IR) and visible image fusion can be accomplished in a variety of ways, including recent deep-learning-based techniques. However, edge-preserving filter (EPF) based fusion works well since it retains all the information from both images. Local filtering-based techniques, on the other hand, limit the fusion performance by introducing multiple gradient reversal artifacts and halos. This work presents an advanced IR and visible image fusion approach depending on three-level decomposition using multi-level co-occurrence filtering, which aims to overcome the common shortfalls such as halo effects seen in existing EPF based fusion. The reference images are decomposed in to base layer, small-scale layers and large-scale layers using multi-level co-occurrence filtering (MLCoF). Since most of the low frequency details are contained in the base layer, the conventional merging strategy by averaging is replaced with novel foreground information map (FIM) based fusion strategy. Small-scale layers are combined by applying max-absolute fusion strategy. A novel weight-map guided edge preserving fusion strategy is put forward for the integration of large-scale layers. Later, fused image is generated by the superposition of these different layers. Subjective visual and objective quantitative analysis shows that the suggested technique attains more notable performance in contrast with other modern fusion methods including many deep-learning techniques. In terms of visual perspective view, the results produced by the proposed approach are superior and include all details from both images. Additionally, it produces outcomes free of gradient reversal and halo artifacts.

X-ray image enhancement with multi-scale local edge preserving filter based on fuzzy entropy.

Recently, X-rays have been widely used to detect complex structural workpieces. Due to the uneven thickness of the workpiece and the high dynamic range of the X-ray image itself, the detailed internal structure of the workpiece cannot be clearly displayed. To solve this problem, we propose an image enhancement algorithm based on a multi-scale local edge-preserving filter. Firstly, the global brightness of the image is enhanced through logarithmic transformation. Then, to enhance the local contrast, we propose utilizing the gradient decay function based on fuzzy entropy to process the gradient and then incorporate the gradient into the energy function of the local edge-preserving filter (LEP) as a constraint term. Finally, multiple base layers and detail layers are obtained through filtering multi-scale decomposition. All detail layers are enhanced and fused using S-curve mapping to improve contrast further. This method is competitive in both quantitative indices and visual perception quality. The experimental results demonstrate that the proposed method significantly enhances various complex workpieces and is highly efficient.

Hyperspectral Image Classification Using Improved Active Deep Learning Framework Including Edge Preserving Filters

To extract valuable information from satellite data for applications such as agriculture, geological research, and environmental monitoring, the classification of hyperspectral images is an essential task. Labeling each pixel in this process is time-consuming and requires financial resources. To this end, working with a small number of samples is very important. In order to provide high classification performances with a limited number of samples, this paper aims to enhance the performance with an active learning framework. The framework incorporates dimensionality reduction, an edge-preserving filter, and active learning steps. From this perspective, we investigated different edge-preserving filter methods to analyze the effects on performance. By combining edge-preserving filters with dimensionality reduction, the study presents a unique method that improves classification performance while maintaining image quality and reducing noise. The following five edge-preserving smoothing filters are evaluated: weighted least squares (WLS), Joint-Histogram weighted median filter (Joint WMF), fast global image smoother (FGS), bilateral filter (BF), and static/dynamic (SD). Our experiments demonstrate that compared to the reference research (CNN+AL+MRF), the proposed framework increased overall and average accuracies about 2-5% for Indian Pines, Pavia University, and Salinas datasets.

Decomposed Multilateral Filtering for Accelerating Filtering with Multiple Guidance Images.

This paper proposes an efficient algorithm for edge-preserving filtering with multiple guidance images, so-called multilateral filtering. Multimodal signal processing for sensor fusion is increasingly important in image sensing. Edge-preserving filtering is available for various sensor fusion applications, such as estimating scene properties and refining inverse-rendered images. The main application is joint edge-preserving filtering, which can preferably reflect the edge information of a guidance image from an additional sensor. The drawback of edge-preserving filtering lies in its long computational time; thus, many acceleration methods have been proposed. However, most accelerated filtering cannot handle multiple guidance information well, although the multiple guidance information provides us with various benefits. Therefore, we extend the efficient edge-preserving filters so that they can use additional multiple guidance images. Our algorithm, named decomposes multilateral filtering (DMF), can extend the efficient filtering methods to the multilateral filtering method, which decomposes the filter into a set of constant-time filtering. Experimental results show that our algorithm performs efficiently and is sufficient for various applications.

Multimodal Medical Image Fusion Utilizing Two-scale Image Decomposition via Saliency Detection.

Modern medical imaging modalities used by clinicians have many applications in the diagnosis of complicated diseases. These imaging technologies reveal the internal anatomy and physiology of the body. The fundamental idea behind medical image fusion is to increase the image's global and local contrast, enhance the visual impact, and change its format so that it is better suited for computer processing or human viewing while preventing noise magnification and accomplishing excellent real-time performance. The top goal is to combine data from various modal images (CT/MRI and MR-T1/MR-T2) into a solitary image that, to the greatest degree possible, retains the key characteristics (prominent features) of the source images. The clinical accuracy of medical issues is compromised because innumerable classical fusion methods struggle to conserve all the prominent features of the original images. Furthermore, complex implementation, high computation time, and more memory requirements are key problems of transform domain methods. With the purpose of solving these problems, this research suggests a fusion framework for multimodal medical images that makes use of a multi-scale edge-preserving filter and visual saliency detection. The source images are decomposed using a two-scale edge-preserving filter into base and detail layers. Base layers are combined using the addition fusion rule, while detail layers are fused using weight maps constructed using the maximum symmetric surround saliency detection algorithm. The resultant image constructed by the presumed method has improved objective evaluation metrics than other classical methods, as well as unhindered edge contour, more global contrast, and no ringing effect or artifacts. The methodology offers a dominant and symbiotic arsenal of clinical symptomatic, therapeutic, and biomedical research competencies that have the prospective to considerably strengthen medical practice and biological understanding.

Robust multi-scale weighting-based edge-smoothing filter for single image dehazing

The guided image filter (GIF) and weighted guided image filter (WGIF) are local linear model-based good edge-preserving filters. However, due to fixed regularization parameter, they suffer from halo artifacts (morphological artifacts) in the sharp regions. To overcome this issue, a robust multi-scale weighting-based edge-smoothing filter (RMWEF) for single image dehazing is proposed in this paper. It removes morphological artifacts and over-smoothness strongly and preserves edge information precisely in both flat and sharp regions. The proposed dehaze method has four-steps. First, initial transmission map and atmospheric map are estimated by using a novel dark channel prior (DCP) method. Then, the morphological artifacts of initial transmission map are reduced by using non-local haze line averaging (NL-HLA) method. In the third step, transmission map is refined by using the proposed RMWEF. Finally, the haze free image is restored. Theoretical and experimental analysis proves that the proposed algorithm produce effective dehaze results quicker than the existing methods.

Linear Time Non-Local Cost Aggregation on Complementary Spatial Tree Structures

Studies on many dense correspondence tasks in the field of computer vision attempt to find spatially smooth results. A typical way to solve these problems is by smoothing the matching costs using edge-preserving filters. However, local filters generate locally optimal results, in that they only take the costs over a small support window into account, and non-local filters based on a minimum spanning tree (MST) tend to overuse the piece-wise constant assumption. In this paper, we propose a linear time non-local cost aggregation method based on two complementary spatial tree structures. The geodesic distances in both the spatial and intensity spaces along the tree structures are used to evaluate the similarity of pixels, and the final aggregated cost is the sum of the outputs from these two trees. The filtering output of a pixel on each tree can be obtained by recursively aggregating the costs along eight sub-trees with linear time complexity. The only difference between the filtering procedures on these two spatial tree structures is the order of the filtering. Experimental results in optical flow estimation and stereo matching on the Middlebury and KITTI datasets demonstrate the effectiveness and efficiency of our method. It turns out that our method outperforms typical non-local filters based on the MST in cost aggregation. Moreover, a comparison of handcrafted features and deep features learned by convolutional neural networks (CNNs) in calculating the matching cost is also provided. The code will be available soon.

Weighted guided image filtering with entropy evaluation weighting

Although the guided image filter (GIF) is an excellent edge-preserving filter, it generally suffers from halo artifacts due to the local property and the fixed regularization parameter. To address the problem, a weighted guided image filter (WGIF) was proposed by incorporating an edge-aware weighting into the GIF. In the filtering process, WGIF employs an averaging strategy for edge-aware weighting. Although the averaging strategy is a highly efficient method, it is susceptible to extreme values and tends to obscure critical factors, so it often leads to inaccurate results. Consequently, the output results quality of the WGIF is often degraded. To remedy the deficiency, a weighted guided image filter with entropy evaluation weighting (EEW-WGIF) is proposed in this paper. EEW-WGIF employs an edge-aware weighting strategy based on entropy evaluation method to detect edges more accurately, and incorporates an explicit constraint based on the gradient variation to better preserve edges. To verify the filtering effectiveness of the EEW-WGIF, it was applied to edge-preserving smoothing filtering, exposure images fusion, single image detail enhancement, structure-transferring filtering and image denoising. Experimental results show that the proposed filter can achieve excellent performance in both visual quality and objective evaluation.

Efficient private key generation from iris data for privacy and security applications

Private key generation based on biometric data is gaining popularity in several cryptographic applications. Some key generation approaches using fingerprint and face data are known for this purpose. The existing approaches, in general, follow direct features from raw data, which may have an issue of revealing users’ biometric data. Further, iris biometrics being a better source for stable and secure key generation, is yet to be explored. In this work, an iris biometric-based key generation approach is proposed. In the proposed method, first, an ensemble L0 Gradient Minimization and an edge-preserving filter are used to extract iris structure from the iris images. Secondly, the iris texture data is normalized, and the consistent region is selected based on a novel statistical method. Thirdly, feature vectors are generated using the ensemble local space-filling curve descriptors and their variants. Fourth, a discriminant feature vector is generated using hybrid feature selection and neighbourhood component analysis. Finally, an interval-based encoding scheme is used to generate a key from the discriminant feature vector. To substantiate the efficacy of the proposed approach, extensive experiments have been carried out with near-infrared images, visible wavelength images, at-a-distance, and on-move images. Further, distinctiveness, dissimilarity, randomness, and security analyses have been carried out to validate the resilience of keys against different attacks. More significantly, a unique key can be generated dynamically, that is, from an online image. The proposed approach is applicable to different cryptographic applications, such as data storage security, remote authentication protocol, blockchain system security, etc.

Multisensor Infrared and Visible Image Fusion via Double Joint Edge Preservation Filter and Nonglobally Saliency Gradient Operator

Multi-sensor infrared and visible image fusion is to extract image features captured by multi-sensor devices to generate a fusion image that contains both infrared thermal radiation and visible texture details. Based on this purpose, this paper proposes a novel infrared and visible image fusion method via double joint edge preservation filter and non-globally saliency gradient operator. First, the source images are finely decomposed by the proposed filter, which has the merits of multi-level scale awareness and double joint edge preservation, into the base, infrared detail, and visible detail layers. Furthermore, to ensure that the overall appearance of the fused image is maximally similar to the source images, a maximum-absolute rule is introduced to process the base layers. Then, a non-globally saliency gradient operator constructed by three different kinds of basic operators is presented to dominate the fusion of the detail layers, aiming to filter out the redundant information and recover the saliency features. Finally, the merged image is obtained by summating the fused base and detail layers. The experimental results demonstrate that the proposed method not only fully extracts the visible details and infrared thermal target of the source images but also generates a fused image with excellent subjective performance and objective indicators compared to other state-of-the-art fusion methods.

Edge-preserving smoothing filter using fast M-estimation method with an automatic determination algorithm for basic width

Digital filtering is essential for digital imaging, image recognition, and super-resolution technology. For example, the presence of noise in images captured by digital cameras causes deterioration of the image quality and image recognition rate. In order to improve the image recognition rate, noise reduction and edge preservation must be performed during preprocessing. Noise is generally reduced using low-pass filters, such as the Gaussian filter. Although they reduce noise, such filters also have the properties of blurring edge. A strong edge blur reduces the accuracy of the feature detection in image recognition. Therefore, in our previous study, a fast M-estimation Gaussian filter for images (FMGFI) was proposed as an image filter that simultaneously achieves denoising and edge preservation. In the FMGFI, the setting of the optimal basic width of the 2nd order B-spline basis functions is important for achieving simultaneous denoising and edge preservation. In this method, the optimal basic width of the FMGFI was determined not only by manually setting the basic width but also by human judgment of the filtered images. Consequently, the inability to automatically determine the optimal basic width hindered efficient denoising during image processing Therefore, in this research, we develop and propose a method that can automatically determine the optimal basic width of the FMGFI. The previously proposed method calculates using the same basic width for all the pixels over the entire image; in contrast, the proposed method calculates using the basic width automatically determined for each pixel. The experiments confirmed that the method proposed in this study achieves higher denoising and edge preservation performance than the ones used in previous research. The results also showed that it has the highest denoising performance against salt-and-pepper noise as compared to other filters: non-local mean filter, Gaussian filter, median filter, bilateral filter, adaptive bilateral filter, and FMGFI. The experimental results for the Gaussian noise sowed that the proposed method has the same denoising and edge preservation performance as the other filters in visual evaluation. From the above, the proposed method is expected to contribute to efficient denoising and improvement of image quality by using it as a preprocessing.

An Image Denoising Method for Arc-Scanning SAR for Airport Runway Foreign Object Debris Detection

Arc-scanning synthetic aperture radar (AS-SAR) is an emerging technical means for detecting foreign object debris (FOD). Most FOD are small and appear as weak targets with a low signal-to-noise ratio (SNR) in AS-SAR images. Therefore, image noise is a fundamental challenge in detecting FOD on airport runways that leads to many false alarms. A weak scattering denoising method is proposed to aim at the noise caused by speckle and rough surface scattering. To enhance FOD detection, a transformation parameter concept is offered and adopted, which has different characteristics for the target and background. This paper estimates the transformation parameter through logarithms, normalization, and morphological erosion and optimizes them with edge-preserving filtering. The results show that despeckling and runway scattering suppression can be simultaneously implemented, and that field experiments validate the performance of this method.

Underwater image enhancement by modified color correction and adaptive Look-Up-Table with edge-preserving filter

Particles suspended in water cause poor visibility and color distortions in the images that are captured in underwater. These suspended particles cause light to be scattered that results in non-uniform illumination due to dispersion and refraction of light. The illumination variation cause color casts in the underwater images. This paper proposes an underwater image enhancement method that improves the visibility, contrast and color cast in the underwater images. The proposed method includes a modified color correction, an adaptive Look-Up-Table (LUT) and an edge-preserving filter. First, a color correction is performed on individual LAB color space components. The improvement of the luminance (L) component is processed with a sequence of steps such as CLAHE, sharpening, gamma and color correction. In addition, the chrominance (A and B) components are modified by further color correction. Second, the contrast of the color-corrected image is enhanced using the adaptive LUT based on probability threshold. This probability threshold increases the image contrast by limiting the pixel intensity values in each RGB channel. In addition, the probability threshold controls the halo effects and artifacts. Finally, the underwater image is enhanced with a fast local Laplacian filter (FLLF) to preserve the edge details with texture smoothing. The denoised image is adjusted with histogram normalization for a further color balancing. The experimental results obtained from the proposed method on different underwater image datasets shows a supremacy over color casts, visibility, halo effects, free of artifacts and edge-preserving in the underwater images compared to the existing methods.

Retinex-Based Relighting for Night Photography

The lighting up of buildings is one form of entertainment that makes a city more colorful, and photographers sometimes change this lighting using photo-editing applications. This paper proposes a method for automatically performing such changes that follows the Retinex theory. Retinex theory indicates that the complex scenes caught by the human visual system are affected by surrounding colors, and Retinex-based image processing uses these characteristics to generate images. Our proposed method follows this approach. First, we propose a method for extracting a relighting saliency map using Retinex with edge-preserving filtering. Second, we propose a sampling method to specify the lighting area. Finally, we composite the additional light to match the human visual perception. Experimental results show that the proposed sampling method is successful in keeping the illuminated points in bright locations and equally spaced apart. In addition, the proposed various diffusion methods can enhance nighttime skyline photographs with various expressions. Finally, we can add in a new light by considering Retinex theory to represent the perceptual color.