Inverse Non-subsampled Shearlet Transform Research Articles

Multi-modal medical image fusion using improved dual-channel PCNN.

This paper proposes a medical image fusion method in the non-subsampled shearlet transform (NSST) domain to combine a gray-scale image with the respective pseudo-color image obtained through different imaging modalities. The proposed method applies a novel improved dual-channel pulse-coupled neural network (IDPCNN) model to fuse the high-pass sub-images, whereas the Prewitt operator is combined with maximum regional energy (MRE) to construct the fused low-pass sub-image. First, the gray-scale image and luminance of the pseudo-color image are decomposed using NSST to find the respective sub-images. Second, the low-pass sub-images are fused by the Prewitt operator and MRE-based rule. Third, the proposed IDPCNN is utilized to get the fused high-pass sub-images from the respective high-pass sub-images. Fourth, the luminance of the fused image is obtained by applying inverse NSST on the fused sub-images, which is combined with the chrominance components of the pseudo-color image to construct the fused image. A total of 28 diverse medical image pairs, 11 existing methods, and nine objective metrics are used in the experiment. Qualitative and quantitative fusion results show that the proposed method is competitive with and even outpaces some of the existing medical fusion approaches. It is also shown that the proposed method efficiently combines two gray-scale images.

TDFusion: When Tensor Decomposition Meets Medical Image Fusion in the Nonsubsampled Shearlet Transform Domain.

In this paper, a unified optimization model for medical image fusion based on tensor decomposition and the non-subsampled shearlet transform (NSST) is proposed. The model is based on the NSST method and the tensor decomposition method to fuse the high-frequency (HF) and low-frequency (LF) parts of two source images to obtain a mixed-frequency fused image. In general, we integrate low-frequency and high-frequency information from the perspective of tensor decomposition (TD) fusion. Due to the structural differences between the high-frequency and low-frequency representations, potential information loss may occur in the fused images. To address this issue, we introduce a joint static and dynamic guidance (JSDG) technique to complement the HF/LF information. To improve the result of the fused images, we combine the alternating direction method of multipliers (ADMM) algorithm with the gradient descent method for parameter optimization. Finally, the fused images are reconstructed by applying the inverse NSST to the fused high-frequency and low-frequency bands. Extensive experiments confirm the superiority of our proposed TDFusion over other comparison methods.

A pansharpening method combining iterative filtering and NSST-NSML-PAPCNN to optimize spatial detail extraction and injection

ABSTRACT Spatial injection-based pansharpening methods are prone to spatial or spectral distortions in pansharpening images due to insufficient extraction of spatial details and a mismatch between the amount of spatial detail information injected and the required amount. To this end, this paper proposes a pansharpening method that optimizes spatial detail extraction and injection. Firstly, a method to optimize the amount of spatial detail injection is proposed, that is, to extract the high-frequency information of the image through iterative filtering and determine the optimal number of iterations based on the global analysis of the method. Then, to fully extract and combine the spatial detail information of the source image, the detailed high-frequency image extracted corresponding to the optimal iterative filtering times is decomposed by non-subsampled shearlet transform (NSST), and a new multi-scale sum-modified-Laplacian (NSML) as an external stimulus to a parameter-adaptive pulse-coupled neural network model (PAPCNN). A fusion rule based on multi-scale morphological gradients is designed to extract a small amount of detailed information for the low-frequency subband. The fused spatial detail image can be obtained by combining the fused low-frequency and high-frequency subbands and inverse NSST transformation. Finally, pansharpening can be realized by combining spatial detail image, injection coefficient, and MS image. In this paper, many experiments are carried out on the QuickBird, GeoEye-1, and WorldView-4 datasets, and quantitative and qualitative comparisons are made with eight advanced methods. Experimental results show that the method proposed in this paper can achieve better fusion results.

Multimodal medical image fusion in NSST domain with structural and spectral features enhancement

For the past 25 years, medical imaging has been extensively used for clinical diagnosis. The main difficulties in medicine are accurate disease recognition and improved therapy. Using a single imaging modality to diagnose disease is challenging for clinical personnel. In this paper, a novel structural and spectral feature enhancement method in NSST Domain for multimodal medical image fusion (MMIF) is proposed. Initially, the proposed method uses the Intensity, Hue, Saturation (IHS) method to generate two pairs of images. The input images are then decomposed using the Non-Subsampled Shearlet Transform (NSST) method to obtain low frequency and high frequency sub-bands. Next, a proposed Structural Information (SI) fusion strategy is employed to Low Frequency Sub-bands (LFS's). It will enhance the structural (texture, background) information. Then, Principal Component Analysis (PCA) is employed as a fusion rule to High Frequency Sub-bands (HFS's) to obtain the pixel level information. Finally, the fused final image is obtained by employing inverse NSST and IHS. The proposed algorithm was validated using different modalities containing 120 image pairs. The qualitative and quantitative results demonstrated that the algorithm proposed in this research work outperformed numerous state-of-the-art MMIF approaches.

Multimodality Medical Image Fusion Using Clustered Dictionary Learning in Non-Subsampled Shearlet Transform.

Imaging data fusion is becoming a bottleneck in clinical applications and translational research in medical imaging. This study aims to incorporate a novel multimodality medical image fusion technique into the shearlet domain. The proposed method uses the non-subsampled shearlet transform (NSST) to extract both low- and high-frequency image components. A novel approach is proposed for fusing low-frequency components using a modified sum-modified Laplacian (MSML)-based clustered dictionary learning technique. In the NSST domain, directed contrast can be used to fuse high-frequency coefficients. Using the inverse NSST method, a multimodal medical image is obtained. Compared to state-of-the-art fusion techniques, the proposed method provides superior edge preservation. According to performance metrics, the proposed method is shown to be approximately 10% better than existing methods in terms of standard deviation, mutual information, etc. Additionally, the proposed method produces excellent visual results regarding edge preservation, texture preservation, and more information.

Parameter adaptive unit-linking pulse coupled neural network based MRI–PET/SPECT image fusion

Medical image fusion has many applications to healthcare that is accomplished by extracting and then combining the complementary information from multiple medical images into a single image. The pulse coupled neural network (PCNN) is greatly applied to image fusion due to its efficient coupling of surrounding neurons, but suffers from network complexity and manual parameter settings. This paper proposes a novel parameter adaptive unit-linking PCNN (PAULPCNN) model, whose parameters are automatically obtained from the external stimulus and exhibits a simpler structure than the original PCNN. The multi-scale decomposition-based methods, particularly based on the non-subsampled shearlet transform (NSST), are widely employed by researchers to perform medical image fusion due to their efficient separation of spatial details at different scales. Motivated by the advantages of the PCNN and multi-scale decomposition-based methods, a hybrid medical image fusion method based on the PAULPCNN model is introduced in the NSST domain that merges the salient complementary details from a gray-scale and the corresponding pseudo-color medical image captured at different modalities to produce a more informative image that is more useful to medical experts in computer-aided diagnosis of diseases. The proposed method first employs NSST to transform the source images into a low-pass and several high-pass sub-bands, respectively. The high-pass sub-bands are combined using the firing times of the proposed PAULPCNN model, whereas a new distance-weighted regional energy-based rule is applied to construct the fused low-pass sub-band, which while estimating regional energy, assigns weight to different neighboring pixels depending on their distance from the central pixel. Finally, inverse NSST is applied on the fused sub-bands to construct the fused image. The effectiveness of the proposed technique is shown using the fusion results of eleven state-of-the-art methods on thirty gray-scale and pseudo-color brain medical image pairs comprising mild Alzheimer’s disease, Huntington’s disease, motor neuron disease, sagittal plane, coronal plane, transaxial plane, and glioma image pairs, where eight objective metrics are considered for the quantitative assessment. Experimental results demonstrate that the proposed method is competitive with the state-of-the-art methods, which even outperforms some of these methods by providing fused images with better visual quality and greater objective performance, which are more suitable for specialists to diagnose brain diseases.

MGFuse: An Infrared and Visible Image Fusion Algorithm Based on Multiscale Decomposition Optimization and Gradient-Weighted Local Energy

Existing image fusion algorithms have difficulty in effectively preserving valuable target features in infrared and visible images, which easily introduces blurry edges and unremarkable notable targets during their fusion process. We propose the MGFuse algorithm as a solution to this problem, which is a novel fusion algorithm that utilizes multiscale decomposition optimization and gradient-weighted local energy. Initially, non-subsampled shearlet transform (NSST) is applied to partition both the infrared and visible images into several high-frequencies and low-frequencies components. Subsequently, the acquired low frequencies continue to be decomposed via proposed optimization function to get base layers and texture layers, which can optimize the quality of image edges and preserve fine-grained details, respectively. In addition, we have formulated an intrinsic attribute-based energy (IAE) fusion scheme to merge the two base layers. The texture layers and high-frequencies are extracted by gradient-weighted local energy (GE) operator based on structure tensor, which are employed to construct the fusion strategy for these parts. At last, the acquired texture and base parts are linearly combined to get the integrated low-frequency layer on which the final image is acquired using inverse NSST. Numerous experimental observations demonstrate that our MGFuse algorithm achieve superior fusion capability than the reference nine advanced algorithms in both qualitative and quantitative assessment, and robustness to noisy images with different noise levels.

An effective nonlocal means image denoising framework based on non-subsampled shearlet transform

Image denoising is a fundamental task in computer vision and image processing system with an aim of estimating the original image by eliminating the noise and artefact from the noise-corrupted version of the image. In this study, a nonlocal means (NLM) algorithm with NSST (non-subsampled shearlet transform) has been designed to surface a computationally simple image denoising algorithm. Initially, NSST is employed to decompose source image into coarser and finer layers. The number of decomposition levels of NSST is set to two, resulting in set of low-frequency coefficients (coarser layer) and four sets high-frequency coefficients (finer layers). The two number of levels of decomposition are used in order to preserve memory, reduce processing time, and mitigate the influence of noise and misregistration errors. The finer layers are then processed using NLM algorithm, while the coarser layer is left as it is. The NL-Means algorithm reduces noise in finer layers while maintaining the sharpness of strong edges, such as the image silhouette. When compared to noisy images, this filter preserves textured regions, resulting in retaining more information. To obtain a final denoised image, inverse NSST is performed to the coarser layer and the NL-means filtered finer layers. The robustness of our method has been tested on the different multisensor and medical image dataset with diverse levels of noise. In the context of both subjective assessment and objective measurement, our method outperforms numerous other existing denoising algorithms notably in terms of retaining fine image structures. It is also clearly exhibited that the proposed method is computationally more effective as compared to other prevailing algorithms.

HID: The Hybrid Image Decomposition Model for MRI and CT Fusion.

Multimodal medical image fusion can combine salient information from different source images of the same part and reduce the redundancy of information. In this paper, an efficient hybrid image decomposition (HID) method is proposed. It combines the advantages of spatial domain and transform domain methods and breaks through the limitations of the algorithms based on single category features. The accurate separation of base layer and texture details is conducive to the better effect of the fusion rules. First, the source anatomical images are decomposed into a series of high frequencies and a low frequency via nonsubsampled shearlet transform (NSST). Second, the low frequency is further decomposed using the designed optimization model based on structural similarity and structure tensor to get an energy texture layer and a base layer. Then, the modified choosing maximum (MCM) is designed to fuse base layers. The sum of modified Laplacian (SML) is used to fuse high frequencies and energy texture layers. Finally, the fused low frequency can be obtained by adding fused energy texture layer and base layer. And the fused image is reconstructed by the inverse NSST. The superiority of the proposed method is verified by amounts of experiments on 50 pairs of magnetic resonance imaging (MRI) images and computed tomography (CT) images and others, and compared with 12 state-of-the-art medical image fusion methods. It is demonstrated that the proposed hybrid decomposition model has a better ability to extract texture information than conventional ones.

NSTBNet: Toward a nonsubsampled shearlet transform for broad convolutional neural network image denoising

Deep convolutional neural networks (CNNs) have achieved huge success in the image denoising fields. However, there still have three drawbacks to be overcome: firstly, it is a big cost to train a deeper CNN model for better denoising performance, secondly, most models often lack the flexibility to remove spatially variant additive Gaussian noise (AWGN), finally, it is difficult to achieve a good trade-off between denoising performance and details preservation. In this paper, we propose a novel network integrated with nonsubsampled shearlet transform (NSST) and a broad convolutional neural network, namely NSTBNet. The proposed NSTBNet has the following desirable properties: (1) a single model has the ability to deal with different noise levels and spatially variant AWGN; (2) the model combines the two networks make it wider to improve denoising performance without increasing too much computational cost; (3) the NSST and inverse NSST are employed to acquire more texture and to avoid gridding effect. Extensive simulation experiments on AWGN and realistic noise show that NSTBNet has state-of-the-art denoising performance, making it a high application value.

Guided filter random walk and improved spiking cortical model based image fusion method in NSST domain

Image fusion has become a hot issue in the field of information processing. In this paper, a novel image fusion method based on guided filter random walk and improved spiking cortical model (ISCM) in non-subsampled shearlet transform (NSST) is presented. The core process is composed of three steps. Firstly, the source images to be fused are decomposed into the low-frequency parts and high-frequency parts via NSST. Then, two models including guided filter and random walk are combined to complete the fusion of low-frequency sub-images. As for the fusion of the high-frequency parts, the traditional spiking cortical model is improved to be ISCM to capture and fuse the details information of the source images. Finally, the fused image can be obtained by inverse NSST. In order to verify the effectiveness of the proposed method, lots of datasets with different categories are selected as the source images to conduct the simulation experiments. Experimental results demonstrate that the proposed method own obvious superiorities over the representative ones in terms of both subjective visual performance and objective evaluation data.

Combining Regional Energy and Intuitionistic Fuzzy Sets for Infrared and Visible Image Fusion.

To get more obvious target information and more texture features, a new fusion method for the infrared (IR) and visible (VIS) images combining regional energy (RE) and intuitionistic fuzzy sets (IFS) is proposed, and this method can be described by several steps as follows. Firstly, the IR and VIS images are decomposed into low- and high-frequency sub-bands by non-subsampled shearlet transform (NSST). Secondly, RE-based fusion rule is used to obtain the low-frequency pre-fusion image, which allows the important target information preserved in the resulting image. Based on the pre-fusion image, the IFS-based fusion rule is introduced to achieve the final low-frequency image, which enables more important texture information transferred to the resulting image. Thirdly, the ‘max-absolute’ fusion rule is adopted to fuse high-frequency sub-bands. Finally, the fused image is reconstructed by inverse NSST. The TNO and RoadScene datasets are used to evaluate the proposed method. The simulation results demonstrate that the fused images of the proposed method have more obvious targets, higher contrast, more plentiful detailed information, and local features. Qualitative and quantitative analysis results show that the presented method is superior to the other nine advanced fusion methods.

Saliency-Guided Nonsubsampled Shearlet Transform for Multisource Remote Sensing Image Fusion.

The rapid development of remote sensing and space technology provides multisource remote sensing image data for earth observation in the same area. Information provided by these images, however, is often complementary and cooperative, and multisource image fusion is still challenging. This paper proposes a novel multisource remote sensing image fusion algorithm. It integrates the contrast saliency map (CSM) and the sum-modified-Laplacian (SML) in the nonsubsampled shearlet transform (NSST) domain. The NSST is utilized to decompose the source images into low-frequency sub-bands and high-frequency sub-bands. Low-frequency sub-bands reflect the contrast and brightness of the source images, while high-frequency sub-bands reflect the texture and details of the source images. Using this information, the contrast saliency map and SML fusion rules are introduced into the corresponding sub-bands. Finally, the inverse NSST reconstructs the fusion image. Experimental results demonstrate that the proposed multisource remote image fusion technique performs well in terms of contrast enhancement and detail preservation.

A multisource image fusion method for multimodal pig-body feature detection

The multisource image fusion has become an active topic in the last few years owing to its higher segmentation rate. To enhance the accuracy of multimodal pig-body feature segmentation, a multisource image fusion method was employed. Nevertheless, the conventional multisource image fusion methods can not extract superior contrast and abundant details of fused image. To superior segment shape feature and detect temperature feature, a new multisource image fusion method was presented and entitled as NSST-GF-IPCNN. Firstly, the multisource images were resolved into a range of multiscale and multidirectional subbands by Nonsubsampled Shearlet Transform (NSST). Then, to superior describe fine-scale texture and edge information, even-symmetrical Gabor filter and Improved Pulse Coupled Neural Network (IPCNN) were used to fuse low and high-frequency subbands, respectively. Next, the fused coefficients were reconstructed into a fusion image using inverse NSST. Finally, the shape feature was extracted using automatic threshold algorithm and optimized using morphological operation. Nevertheless, the highest temperature of pig-body was gained in view of segmentation results. Experiments revealed that the presented fusion algorithm was able to realize 2.102-4.066% higher average accuracy rate than the traditional algorithms and also enhanced efficiency.

Multimodal image fusion based on global-regional-local rule in NSST domain

An accurate representation of the source image is vital in image fusion processing. However, among most of the traditional multi-scale decomposition based image fusion methods, they fail to accurately interpret the source images, only capturing local information without considering the regional and global information. To overcome this drawback, a novel image fusion method based on the global-regional-local rule is proposed in the nonsubsampled shearlet transform (NSST) domain. First, the source paired images are decomposed into the low-pass subbands and high-pass subbands with the NSST transformation. Second, for comprehensively representing the statistical correlation of source image, three-level statistical models, that is, global contextual hidden Markov model (G-CHMM), regional contextual hidden Markov model (R-CHMM), and local contextual hidden Markov model (L-CHMM) are established for the high-pass subbands. In the R-CHMM, a novel feature vector is extracted for getting the region map of source image by the fuzzy cluster method (FCM). Third, to get accurate active measures of the source image, a global-regional-local fusion rule based on the statistical characteristics extracted from global-regional-local CHMM is designed, which is used to fuse the high-pass subbands. The low-pass subbands are fused with the choose-max rule based on the local gradient measure. Finally, the fused image is obtained by the inverse NSST. Experimental results on serials of infrared and visible images, medical images and multi-focus images demonstrate the advantage of the proposed method in terms of detail preserving.

Infrared and visible image fusion based on nonlinear enhancement and NSST decomposition

In multi-scale geometric analysis (MGA)-based fusion methods for infrared and visible images, adopting the same representation for the two types of images will result in the non-obvious thermal radiation target in the fused image, which can hardly be distinguished from the background. To solve the problem, a novel fusion algorithm based on nonlinear enhancement and non-subsampled shearlet transform (NSST) decomposition is proposed. Firstly, NSST is used to decompose the two source images into low- and high-frequency sub-bands. Then, the wavelet transform (WT) is used to decompose high-frequency sub-bands to obtain approximate sub-bands and directional detail sub-bands. The “average” fusion rule is performed for fusion for approximate sub-bands. And the “max-absolute” fusion rule is performed for fusion for directional detail sub-bands. The inverse WT is used to reconstruct the high-frequency sub-bands. To highlight the thermal radiation target, we construct a non-linear transform function to determine the fusion weight of low-frequency sub-bands, and whose parameters can be further adjusted to meet different fusion requirements. Finally, the inverse NSST is used to reconstruct the fused image. The experimental results show that the proposed method can simultaneously enhance the thermal target in infrared images and preserve the texture details in visible images, and which is competitive with or even superior to the state-of-the-art fusion methods in terms of both visual and quantitative evaluations.

A novel approach for multi-focus image fusion based on SF-PAPCNN and ISML in NSST domain

In order to further improve the contrast and sharpness of fused image, a novel multi-focus image fusion algorithm based on spatial frequency-motivated parameter-adaptive pulse coupled neural network (SF-PAPCNN) and improved sum-modified-laplacian (ISML) in nonsubsampled shearlet transform (NSST) domain is proposed in this paper. In its procedural steps, at first, the source images are decomposed into low-frequency and high-frequency components by NSST. The low-frequency components are fused by SF-PAPCNN model, the PAPCNN is designed to estimate the PCNN parameters adaptively according to the input information, and the high-frequency components are fused by ISML model. Finally, the inverse NSST is employed to the fused coefficients to reconstruct the fused image. The superiority of the proposed fusion technique is confirmed by many analytical experimentations on the gray and color multi-focus image data sets. Compared with the state-of-the-art image fusion methods, the proposed fusion algorithm has superior performance in terms of visual inspection and objective evaluation.

A phase congruency-based green fluorescent protein and phase contrast image fusion method in nonsubsampled shearlet transform domain.

Image fusion technique is an effective way to merge the information contained in different imaging modalities by generating a more informative composite image. Fusion of green fluorescent protein (GFP) and phase contrast images is of great significance to the subcellular localization, the functional analysis of protein, and the expression of gene. In this article, a phase congruency (PC)-based GFP and phase contrast image fusion method in nonsubsampled shearlet transform (NSST) domain is presented. The input images are decomposed by the NSST to acquire the multiscale and multidirection representations. The high-frequency coefficients are fused with a strategy based on PC and parameter-adaptive pulse coupled neural network (PA-PCNN), while the low-frequency coefficients are integrated through a local energy (LE)-based rule. Finally, the fused image is generated by conducting the inverse NSST on the merged high- and low-frequency coefficients. Experimental results illustrate that the presented method outperforms several state-of-the-art GFP and phase contrast image fusion algorithms on both qualitative and quantitative assessments.

A new image denoising framework using bilateral filtering based non-subsampled shearlet transform

In this paper, we propose an advanced framework for image denoising using bilateral filtering based non-subsampled shearlet transform (NSST). Initially, we apply the NSST to decompose noisy input image into high and low frequency coefficients. The weighted bilateral filter (WBF) is then used to remove the noise from the low frequency coefficients while; thresholding is used to remove noise from the high frequency coefficients. The outputs of both the process are combined to form the resultant image. Finally, the inverse NSST is applied on the resultant output to estimate the final denoised image. To ensure validity of the proposed model, we conduct several experiments by considering different grayscale images with various noise variances. The qualitative and quantitative comparison is illustrated and it shows the improved performance of the proposed method as compared to the other conventional image denoising methods. Mathematical and simulation results are presented to show the validity of our work.

Brain Medical Image Fusion Based on Dual-Branch CNNs in NSST Domain.

Computed tomography (CT) images show structural features, while magnetic resonance imaging (MRI) images represent brain tissue anatomy but do not contain any functional information. How to effectively combine the images of the two modes has become a research challenge. In this paper, a new framework for medical image fusion is proposed which combines convolutional neural networks (CNNs) and non-subsampled shearlet transform (NSST) to simultaneously cover the advantages of them both. This method effectively retains the functional information of the CT image and reduces the loss of brain structure information and spatial distortion of the MRI image. In our fusion framework, the initial weights integrate the pixel activity information from two source images that is generated by a dual-branch convolutional network and is decomposed by NSST. Firstly, the NSST is performed on the source images and the initial weights to obtain their low-frequency and high-frequency coefficients. Then, the first component of the low-frequency coefficients is fused by a novel fusion strategy, which simultaneously copes with two key issues in the fusion processing which are named energy conservation and detail extraction. The second component of the low-frequency coefficients is fused by the strategy that is designed according to the spatial frequency of the weight map. Moreover, the high-frequency coefficients are fused by the high-frequency components of the initial weight. Finally, the final image is reconstructed by the inverse NSST. The effectiveness of the proposed method is verified using pairs of multimodality images, and the sufficient experiments indicate that our method performs well especially for medical image fusion.