Nonsubsampled Contourlet Transform Domain Research Articles

Multimodal Medical Image Fusion based on the VGG19 Model in the NSCT Domain

Aim: To deal with the drawbacks of the traditional medical image fusion methods, such as the low preservation ability of the details, the loss of edge information, and the image distortion, as well as the huge need for the training data for deep learning, a new multi-modal medical image fusion method based on the VGG19 model and the non-subsampled contourlet transform (NSCT) is proposed, whose overall objective is to simultaneously make the full use of the advantages of the NSCT and the VGG19 model. Methodology: Firstly, the source images are decomposed into the high-pass and low-pass subbands by NSCT, respectively. Then, the weighted average fusion rule is implemented to produce the fused low-pass sub-band coefficients, while an extractor based on the pre-trained VGG19 model is constructed to obtain the fused high-pass subband coefficients. Result and Discussion: Finally, the fusion results are reconstructed by the inversion transform of the NSCT on the fused coefficients. To prove the effectiveness and the accuracy, experiments on three types of medical datasets are implemented. Conclusion: By comparing seven famous fusion methods, both of the subjective and objective evaluations demonstrate that the proposed method can effectively avoid the loss of detailed feature information, capture more medical information from the source images, and integrate them into the fused images.

Comparative evaluation of volumetry estimation from plain and contrast enhanced computed tomography liver images.

Surgery planning for liver tumour is carried out using contrast enhanced computed tomography (CECT) images to determine the optimal resection strategy and to assess the volume of liver and tumour. Current surgery planning tools interpret even the functioning liver cells present within the tumour boundary as tumour. Plain CT images provide inadequate information for treatment planning. This work attempts to address two shortcomings of existing surgery planning tools: (i) to delineate functioning liver cells from the non-functioning tumourous tissues within the tumour boundary and (ii) to provide 3D visualization and actual tumour volume from the plain CT images. All slices of plain CT images containing liver are enhanced by means of fuzzy histogram equalization in Non-Subsampled Contourlet Transform (NSCT) domain prior to 3D reconstruction to clearly delineate liver, non-functioning tumourous tissues and functioning liver cells within the tumour boundary. The 3D analysis from plain and CECT images was carried out on five types of liver lesions viz. HCC, metastasis, hemangioma, cyst, and abscess along with normal liver. The study resulted in clear delineation of functional liver tissues from non-functioning tumourous tissues within the tumour boundary from CECT as well as plain CT images. The volume of liver calculated using the proposed approach is found comparable with that obtained using Myrian-XP, a currently followed surgery planning tool in clinical practice. The obtained results from plain CT images will undoubtedly provide valuable diagnostic assistance and surgery planning even for the subset of patients for whom CECT acquisition is not advisable.

Adaptive convolutional sparsity with sub-band correlation in the NSCT domain for MRI image fusion

Objective. Multimodal medical image fusion (MMIF) technologies merges diverse medical images with rich information, boosting diagnostic efficiency and accuracy. Due to global optimization and single-valued nature, convolutional sparse representation (CSR) outshines the standard sparse representation (SR) in significance. By addressing the challenges of sensitivity to highly redundant dictionaries and robustness to misregistration, an adaptive convolutional sparsity scheme with measurement of the sub-band correlation in the non-subsampled contourlet transform (NSCT) domain is proposed for MMIF. Approach. The fusion scheme incorporates four main components: image decomposition into two scales, fusion of detail layers, fusion of base layers, and reconstruction of the two scales. We solved a Tikhonov regularization optimization problem with source images to obtain the base and detail layers. Then, after CSR processing, detail layers were sparsely decomposed using pre-trained dictionary filters for initial coefficient maps. NSCT domain’s sub-band correlation was used to refine fusion coefficient maps, and sparse reconstruction produced the fused detail layer. Meanwhile, base layers were fused using averaging. The final fused image was obtained via two-scale reconstruction. Main results. Experimental validation of clinical image sets revealed that the proposed fusion scheme can not only effectively eliminate the interference of partial misregistration, but also outperform the representative state-of-the-art fusion schemes in the preservation of structural and textural details according to subjective visual evaluations and objective quality evaluations. Significance. The proposed fusion scheme is competitive due to its low-redundancy dictionary, robustness to misregistration, and better fusion performance. This is achieved by training the dictionary with minimal samples through CSR to adaptively preserve overcompleteness for detail layers, and constructing fusion activity level with sub-band correlation in the NSCT domain to maintain CSR attributes. Additionally, ordering the NSCT for reverse sparse representation further enhances sub-band correlation to promote the preservation of structural and textural details.

Statistical image watermark decoder using NSM-HMT in NSCT-FGPCET magnitude domain

The image watermarking technology is an effective technology for protecting image copyright at present. Image watermarking techniques have a constraint relationship between imperceptibility, robustness, and watermark capacity. Balancing imperceptibility, robustness, and watermark capacity becomes a tricky problem. This paper proposes a statistical image watermarking scheme using nonsubsampled Contourlet transform (NSCT) domain fast generic polar complex exponential transform (FGPCET) magnitudes and nonsymmetric mixtures (NSM) based hidden Markov tree (HMT). In the embedding part, to enhance robustness and imperceptibility, we insert the watermark signal into the robust local NSCT domain FGPCET magnitudes through the multiplicative method. In the decoding part, a statistical image watermarking decoder is designed using the NSM-HMT model and maximum likelihood criterion. Here, marginal characteristics and strong correlations of local NSCT domain FGPCET magnitudes are described by NSM-HMT. In addition, the generalized expectation-maximization (GEM)-clusterized method of moments (CMM) approach is used for accurate parameters. Extensive Monte Carlo experiments validate the better performance of the proposed image watermarking method compared to other well-known methods.

Multimodal Medical Image Fusion Using Stacked Auto-encoder in NSCT Domain.

Medical image fusion is a process that aims to merge the important information from images with different modalities of the same organ of the human body to create a more informative fused image. In recent years, deep learning (DL) methods have achieved significant breakthroughs in the field of image fusionbecause of their great efficiency. The DL methods in image fusion have become an active topic due to their high feature extraction and data representation ability. In this work, stacked sparse auto-encoder (SSAE), a general category of deep neural networks, is exploited inmedical image fusion. The SSAE is an efficient technique for unsupervised feature extraction. It has high capability of complex data representation. The proposed fusion method is carried as follows. Firstly, the source images are decomposed into low- and high-frequency coefficient sub-bands with the non-subsampled contourlet transform (NSCT). The NSCT is a flexible multi-scale decomposition technique, and it issuperior to traditional decomposition techniques in several aspects. After that, the SSAE is implemented for feature extraction to obtain a sparse and deep representation from high-frequency coefficients. Then, thespatial frequencies are computed for the obtained features to be used for high-frequency coefficient fusion. After that, a maximum-based fusion rule is applied to fuse the low-frequency sub-band coefficients. The final integrated image is acquired by applying theinverse NSCT. The proposed method has been applied and assessed on various groups of medical image modalities. Experimental results prove that the proposed method could effectively merge the multimodal medical images, while preserving the detail information, perfectly.

MRI image enhancement based on feature clustering in the NSCT domain

<abstract> <p>The noise and low clarity in magnetic resonance imaging (MRI) images impede the doctor's diagnosis. An MRI image enhancement method is proposed in the non-subsampled contourlet transform (NSCT) domain. The coefficients of the NSCT are classified as noise component, weak edges component and strong edges component by feature clustering. We modified the transform coefficients to enhance the MRI images. The coefficients corresponding to noise are set to zero, the coefficients corresponding to strong edges are essentially unchanged, and the coefficients corresponding to weak edges are enhanced by a simplified nonlinear gain function. It is shown that the proposed MRI image enhancement method has advantages in visual quality and objective evaluation indexes compared to the state-of-the-art methods.</p> </abstract>

Multimodal image fusion and denoising in NSCT domain using CNN and FOTGV

Multimodal medical image fusion is the most popular tool to integrate important information of multimodal medical images into a single complete informative image. Fusion provides an effective way for medical image diagnosis and treatment. However the acquired medical images may be corrupted with noise due to patient movement or faulty transmission, which misguides the image analysis and requires denoising. Therefore, in the suggested scheme non-subsampled contourlet transform (NSCT) is first used to extract features from the noisy source images. Then, a Siamese convolutional neural network (sCNN) is utilized for weighted fusion of important features from the two multimodal images. Simultaneously, a fractional order total generalized variation (FOTGV) is implemented for noise removal with improved degree of freedom. The image processing results demonstrate that the NSCT + sCNN + FOTGV scheme performs effectively for clean as well as noisy images in comparison to the other state-of-the-art conventional techniques on the basis of visual and quantitative analysis.

Multi-focus image fusion approach based on CNP systems in NSCT domain

Coupled neural P (CNP) systems are recently developed distributed and parallel computing models that are abstracted by the mechanisms of coupled and spiking neurons. CNP systems differ from spiking neural P (SNP) systems in two main ways, namely the utilization of three data units, and a coupled firing and dynamic threshold mechanism for neurons. This paper focuses on the application of CNP systems to solve multi-focus image fusion problems, and proposes a novel image fusion approach based on CNP systems. Based on two CNP systems with local topology, a multi-focus image fusion framework in the non-subsampled contourlet transform (NSCT) domain is developed, where the two CNP systems are utilized to control the fusion of low-frequency coefficients in the NSCT domain. The proposed fusion approach is evaluated on an open data set of 19 multi-focus images based on five fusion quality indices, and compared to 11 state-of-the-art fusion approaches. Quantitative and qualitative experimental results demonstrate the advantages of the proposed fusion approach in terms of visual quality and fusion performance.

Pulse Coupled Neural Network-Based Multimodal Medical Image Fusion via Guided Filtering and WSEML in NSCT Domain.

Multimodal medical image fusion aims to fuse images with complementary multisource information. In this paper, we propose a novel multimodal medical image fusion method using pulse coupled neural network (PCNN) and a weighted sum of eight-neighborhood-based modified Laplacian (WSEML) integrating guided image filtering (GIF) in non-subsampled contourlet transform (NSCT) domain. Firstly, the source images are decomposed by NSCT, several low- and high-frequency sub-bands are generated. Secondly, the PCNN-based fusion rule is used to process the low-frequency components, and the GIF-WSEML fusion model is used to process the high-frequency components. Finally, the fused image is obtained by integrating the fused low- and high-frequency sub-bands. The experimental results demonstrate that the proposed method can achieve better performance in terms of multimodal medical image fusion. The proposed algorithm also has obvious advantages in objective evaluation indexes VIFF, QW, API, SD, EN and time consumption.

Poisson-Gaussian Noise Reduction for X-Ray Images Based on Local Linear Minimum Mean Square Error Shrinkage in Nonsubsampled Contourlet Transform Domain

Noise reduction is important for X-ray images because it can reduce radiation exposure to patients. X-ray image noise has a Poisson-Gaussian distribution, and recently, noise analysis and removal in multiscale transformations have been widely implemented. The nonsubsampled contourlet transform (NSCT) is a multiscale transformation suitable for medical images that separates the scale and direction. This study proposes a Poisson-Gaussian noise-removal method using NSCT shrinkage that is based on the characteristics of Poisson-Gaussian noise in NSCT domain. It has the structure of a block-matching 3D filtering algorithm in the form of basic estimation and noise removal process; however, the main processes are modified to consider Poisson-Gaussian noise characteristics. In the basic estimation process, an NSCT shrinkage method that is suitable for Poisson-Gaussian noise characteristics is developed by optimizing the local linear minimum mean square error estimator in the NSCT domain. In the denoising step, the noise term of the Wiener filter is determined using the result of the NSCT shrinkage, and finally, the denoised image is obtained. The proposed method is applied to simulated and real X-ray images and is compared with other state-of-the-art Poisson-Gaussian noise removal methods; it exhibits excellent results in both quantitative and qualitative aspects.

Infrared and Visible Image Fusion Method Based on ResNet in a Nonsubsampled Contourlet Transform Domain

Although the traditional image fusion method can obtain rich image results, obvious artificial noise and artifacts are often present in the resulting image. Fusion algorithms based on neural networks can avoid the shortcomings of traditional methods, but they are more complex and less flexible. In this study, we proposed a fusion method using the deep residual neural network ResNet152, which can not only effectively suppress artificial noise but also preserve the edge details of the image and improve the efficiency of the neural network. The proposed method is characterized by a multiscale transformation of an infrared image and visible light image in the optimized nonsubsampled contourlet transformation domain, and the deep residual neural network ResNet152 is used to extract the deep features of the low-pass component to guide the fusion of the low-pass component. The bandpass component is fused by taking the modulus maximum. This method can fully retain the global features and structural information of the source image in the result image. Compared to existing fusion methods using public test image sets, the experimental results show that on a subjective level, the fusion method creates sharper depth edges and fewer noise artifacts than traditional fusion methods. From an objective perspective, the average value for the results of the evaluation function is greater than that of other fusion methods.

IHS-based pan-sharpening technique for visual quality improvement using KPCA and enhanced SML in the NSCT domain

ABSTRACT Due to the different characteristics of satellite imagery, pan-sharpening has become an important field of remote sensing science. It is a reliable method to fuse the high-resolution panchromatic (Pan) image with the low-resolution multispectral (MS) images to generate a composite high-resolution MS (pan-sharpened) image. In this paper, we propose a robust pan-sharpening technique which combines the nonsubsampled contourlet transform (NSCT) with kernel principal component analysis (KPCA). An enhancement method is executed on the source MS image to retain the maximum spatial information present in the MS bands based on the intensity hue saturation (IHS) colour space transform. The proposed pan-sharpening technique is performed as follows: first, the source Pan and MS images are divided into high and low-pass coefficients by NSCT. Second, KPCA is used to propose an effective fusion rule for choosing appropriate low-pass NSCT coefficients for fusion. The high-pass coefficients are fused by proposing an enhanced sum modified Laplacian method (SML). Finally, the final pan-sharpened image is obtained by performing an inverse NSCT and inverse IHS transform on the fused low and high-pass coefficients. Four different groups of satellite image datasets are utilized in the experiments, which show that the proposed technique can both preserve the spatial details of the source images well and avoid spectral distortion. In addition, different image fusion quality metrics are adopted to evaluate the spectral and spatial qualities of the pan-sharpened image. Compared to the state-of-the-art pan-sharpening methods, the proposed technique achieved better performance in balancing spectral and spatial information and improved pan-sharpening results in terms of both visual quality and objective measurements.

Intensity image quality assessment based on multiscale gradient magnitude similarity deviation

Laser active imaging is widely used in many fields. The intensity image quality of laser active imaging is affected by various degradations, such as speckle effect and noise. Most existing objective image quality assessment (IQA) methods that consider only a single distorted image are not suitable for an intensity image. A multiscale full-reference intensity IQA method is presented. The proposed method is based on an improved gradient magnitude similarity deviation (GMSD) in the nonsubsampled contourlet transform (NSCT) domain. The reference and distorted images are decomposed by NSCT to emulate the multichannel structure of the human visual system. Then, the GMSD of each sub-band is computed to capture the intensity image quality. At last, the contrast sensitivity function implementation is employed in the sub-bands of the NSCT domain. All sub-bands’ GMSD is evaluated and pooled together to yield the objective quality index of a distorted intensity image. Experimental results show that the proposed method can effectively and accurately evaluate the quality of intensity images, and it is highly consistent with subjective perception.

Multi‐modal image fusion based on saliency guided in NSCT domain

Image fusion aims at aggregating the redundant and complementary information in multiple original images, the most challenging aspect is to design robust features and discriminant model, which enhances saliency information in the fused image. To address this issue, the authors develop a novel image fusion algorithm for preserving the invariant knowledge of the multimodal image. Specifically, they formulate a novel unified architecture based on non-subsampled contourlet transform (NSCT). Their method introduces Quadtree decomposition and Bezier interpolation to extract crucial infrared features. Furthermore, they propose a saliency advertising phase congruency-based rule and local Laplacian energy-based rule for low- and high-pass sub-bands fusion, respectively. In this approach, the fusion image could not only combine the local and global features of the source image to avoid smoothing the edge of the target, but also retain the minor scales details and resists the interference noise of the multi-modal image. Both objective assessments and subjective visions of experimental results indicate that the proposed algorithm performs competitively in both objective evaluation criteria and visual quality.

Change Detection in Multispectral Remote Sensing Images with Leader Intelligence PSO and NSCT Feature Fusion

Change detection (CD) using Remote sensing images have been a challenging problem over the years. Particularly in the unsupervised domain it is even more difficult. A novel automatic change detection technique in the unsupervised framework is proposed to address the real challenges involved in remote sensing change detection. As the accuracy of change map is highly dependent on quality of difference image (DI), a set of Normalized difference images and a complementary set of Normalized Ratio images are fused in the Nonsubsampled Contourlet Transform (NSCT) domain to generate high quality difference images. The NSCT is chosen as it is efficient in suppressing noise by utilizing its unique characteristics such as multidirectionality and shift-invariance that are suitable for change detection. The low frequency sub bands are fused by averaging to combine the complementary information in the two DIs, and, the higher frequency sub bands are merged by minimum energy rule, for preserving the edges and salient features in the image. By employing a novel Particle Swarm Optimization algorithm with Leader Intelligence (LIPSO), change maps are generated from fused sub bands in two different ways: (i) single spectral band, and (ii) combination of spectral bands. In LIPSO, the concept of leader and followers has been modified with intelligent particles performing Lévy flight randomly for better exploration, to achieve global optima. The proposed method achieved an overall accuracy of 99.64%, 98.49% and 97.66% on the three datasets considered, which is very high. The results have been compared with relevant algorithms. The quantitative metrics demonstrate the superiority of the proposed techniques over the other methods and are found to be statistically significant with McNemar’s test. Visual quality of the results also corroborate the superiority of the proposed method.

A novel multi-source image fusion method for pig-body multi-feature detection in NSCT domain

The multi-source image fusion has been a hot topic during the recent years because of its higher detection rate. To improve the accuracy of pig-body multi-feature detection, a multi-source image fusion method was adopted in this field. However, the traditional multi-source image fusion methods could not obtain better contrast and more details of the fused image. To better detect shape and temperature feature of pig-body, a novel infrared and visible image fusion method was proposed in non-subsampled contourlet transform (NSCT) domain and named NSCT-GF-IAG. Through this technique, the visible and infrared images were first decomposed into a series of multi-scale and multi-directional sub-bands using NSCT. Then, to better represent the fine-scale of texture information and coarse-scale detail information, Gabor filter with even-symmetry and improved average gradient (IAG) were employed to fuse low-frequency and high-frequency sub-bands, respectively. Next, the fused coefficients were reconstructed into a final fusion image by inverse NSCT. Finally, the shape feature of pig-body was obtained by automatic threshold segmentation and optimized by morphological processing. Moreover, the highest temperature was extracted based on shape segmentation of pig-body. Experimental results showed that the proposed fusion method for detecting multi-feature was capable of achieving 2.175–5.129% higher average segmentation rate than the prevailing conventional methods. Besides this, the proposed method also improved efficiency in terms of time consumption.

Smart media and application

Smart media and application

An adaptive enhancement method for breast X-ray images based on the nonsubsampled contourlet transform domain and whale optimization algorithm.

We propose a new method for breast X-ray image adaptive enhancement that combines nonsubsampled contourlet transform (NSCT) with the whale optimization algorithm (WOA). First, the mammography X-ray image was processed by histogram equalization to ensure global image contrast. The processed image was then decomposed into three layers in the NSCT domain. Each layer was each decomposed into two, four, and eight directions. A median filter was used to remove noise in the first and second layers. Then, a special edge filter was adopted to enhance each sub-band image, and two parameters are involved. WOA is used to automatically search the optimal two parameters. Blind image quality index (BIQI) adaptive function was used as an objective function of WOA. Then, inverse NSCT was employed to reconstruct the processed image, generating the final adaptive enhancement image. The digital database for screening mammography (DDSM) was used to verify the performance of the proposed method. Five objective evaluation indexes, including information entropy, average gradient, standard deviation, contrast improvement index (CII), and BIQI, are combined together to construct a new comprehensive index to evaluate the visual quality of the enhanced image. The results show that the proposed method has a good enhancement effect for mammography X-ray images. The overall performance of the proposed method is better than some existing similar methods. Graphical abstract .

Retinal Vasculature Extraction using Non-Subsampled Contourlet Transform and Multi-structure Element Morphology by Reconstruction

Retinal vasculature extraction is an area of utmost interest in ophthalmology. It helps to diagnose various diseases and also play a crucial role in treatment planning and accomplishment.In this work, we suggest an algorithm to segment retinal vasculature from retinal Fundus Images(FI) using multi-structure element morphology after enhancing the image using Normal Inverse Gaussian (NIG) model in the fuzzified Non-Subsampled Contourlet Transform (NSCT) domain. Since both noises and weak edges produce low magnitude NSCT coefficients, image enhancement methods amplify weak edges as well as noises. Direct application of image boosting technique in the NSCT domain causes over enhancement. So a novel image enhancement method is employed by interpreting the term “contrast” as a qualitative instead of a quantitative measure of the image. Membership values of NSCT coefficients are modified using NIG model. Mathematical Morphology(MM) by Multi-structure Elements (MEs) is used to extract the edges of image. False vessel ridges are expunged, and the thin vessel edges are preserved using opening by reconstruction. Connected component analysis followed by length filtering is used to filter the still remaining false edges. In most of the available literature, low-resolution fundus image databases are used for evaluating the algorithm. In our work, we evaluate our algorithm not only utilizing the DRIVE database, a low-resolution retinal image (RI) database, but also using an openly available High-Resolution Fundus (HRF) image database. Our result illustrates that the proposed method outperforms the other techniques considered with average accuracy (ACC) of 96.71%. In addition to ACC, we also use F1-Score and Mathews Correlation Coefficient (MCC) to evaluate our method. The average values of the results obtained with the HRF image database for F1-Score and MCC are 0.8172 and 0.8031, respectively, which are very much encouraging.

Coefficient difference based watermark detector in nonsubsampled contourlet transform domain

Imperceptibility and robustness are two main requirements of any image watermarking systems to guarantee desired functionalities, but there is a tradeoff between them from the information–theoretic perspective. It is a challenging work to design a high performance digital watermarking scheme to keep a trade-off between imperceptibility and robustness. By modeling the nonsubsampled Contourlet transform (NSCT) coefficient differences with ranked set sample (RSS) based Cauchy distribution and employing locally most powerful (LMP) test, we propose a locally optimum image watermark detector in NSCT domain. In the proposed scheme, we first compute the robust coefficient differences according to the inter-scale dependency between NSCT coefficients, and then embed the digital watermark into the significant NSCT coefficient difference subband. At the watermark receiver, robust NSCT coefficient differences are firstly modeled by employing the Cauchy distribution, where the statistical properties of NSCT coefficient differences are captured accurately. Then the RSS approach is introduced to estimate the location parameter and shape parameter of Cauchy distribution. And finally an optimal detector for multiplicative watermarking is developed using the LMP decision rule and RSS-based Cauchy distribution. Also, we utilize the statistical model to derive the closed-form expressions for the watermark detector. Experimental results demonstrate the high efficiency of our watermarking scheme, which can provide better imperceptibility and outstanding robustness against various attacks, in comparison with the state-of-the-art approaches recently proposed in the literature.