Convolutional Sparse Representation Research Articles

A new multi-focus image fusion quality assessment method with convolutional sparse representation

A new multi-focus image fusion quality assessment method with convolutional sparse representation

Adaptive convolutional sparsity with sub-band correlation in the NSCT domain for MRI image fusion

Objective. Multimodal medical image fusion (MMIF) technologies merges diverse medical images with rich information, boosting diagnostic efficiency and accuracy. Due to global optimization and single-valued nature, convolutional sparse representation (CSR) outshines the standard sparse representation (SR) in significance. By addressing the challenges of sensitivity to highly redundant dictionaries and robustness to misregistration, an adaptive convolutional sparsity scheme with measurement of the sub-band correlation in the non-subsampled contourlet transform (NSCT) domain is proposed for MMIF. Approach. The fusion scheme incorporates four main components: image decomposition into two scales, fusion of detail layers, fusion of base layers, and reconstruction of the two scales. We solved a Tikhonov regularization optimization problem with source images to obtain the base and detail layers. Then, after CSR processing, detail layers were sparsely decomposed using pre-trained dictionary filters for initial coefficient maps. NSCT domain’s sub-band correlation was used to refine fusion coefficient maps, and sparse reconstruction produced the fused detail layer. Meanwhile, base layers were fused using averaging. The final fused image was obtained via two-scale reconstruction. Main results. Experimental validation of clinical image sets revealed that the proposed fusion scheme can not only effectively eliminate the interference of partial misregistration, but also outperform the representative state-of-the-art fusion schemes in the preservation of structural and textural details according to subjective visual evaluations and objective quality evaluations. Significance. The proposed fusion scheme is competitive due to its low-redundancy dictionary, robustness to misregistration, and better fusion performance. This is achieved by training the dictionary with minimal samples through CSR to adaptively preserve overcompleteness for detail layers, and constructing fusion activity level with sub-band correlation in the NSCT domain to maintain CSR attributes. Additionally, ordering the NSCT for reverse sparse representation further enhances sub-band correlation to promote the preservation of structural and textural details.

Learning the sparse prior: Modern approaches

AbstractThe sparse prior has been widely adopted to establish data models for numerous applications. In this context, most of them are based on one of three foundational paradigms: the conventional sparse representation, the convolutional sparse representation, and the multi‐layer convolutional sparse representation. When the data morphology has been adequately addressed, a sparse representation can be obtained by solving the sparse coding problem specified by the data model. This article presents a comprehensive overview of these three models and their corresponding sparse coding problems and demonstrates that they can be solved using convex and non‐convex optimization approaches. When the data morphology is not known or cannot be analyzed, it must be learned from training data, thereby formulating dictionary learning problems. This article addresses two different dictionary learning paradigms. In an unsupervised scenario, dictionary learning involves the alternating or joint resolution of sparse coding and dictionary updating. Another option is to create a recurrent neural network by unrolling algorithms designed to solve sparse coding problems. These networks can then be used in a supervised learning setting to facilitate the training of dictionaries via forward‐backward optimization. This article lists numerous applications in various domains and outlines several directions for future research related to the sparse prior.This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Modeling Methods Statistical and Graphical Methods of Data Analysis > Modeling Methods and Algorithms Statistical Models > Nonlinear Models

Multi-modal medical image fusion via multi-dictionary and truncated Huber filtering

Multi-modal medical image fusion provides comprehensive and objective descriptions of lesions for clinical medical assistance. However, retaining useful information while achieving noise robustness remains challenging for existing techniques. In this paper, we propose a novel medical image fusion algorithm based on multi-dictionary convolutional sparse representation. Especially, truncated Huber filtering is first introduced to achieve detail-base layer decomposition of source images. Subsequently, multiple-dictionary decisions and nuclear energy-based rules are proposed to fuse the details and base layers, respectively. The fused image is reconstructed by synthesizing the fused detail and base components. The proposed model effectively fuses the source global structure and texture information and exhibits strong robustness against noise. Experiments involving extensive noise-free and noisy anatomical and functional medical image fusion on a public dataset covering five fusion categories demonstrate that the proposed method outperforms other state-of-the-art methods in subjective and objective evaluations. The source code of this study is publicly available at https://github.com/JEI981214/MDHU-fusion.

Recent advances via convolutional sparse representation model for pixel-level image fusion

Recent advances via convolutional sparse representation model for pixel-level image fusion

A novel endoscopy image fusion system: combine white light imaging and compound band imaging.

White light imaging (WLI) is a commonly seen examination mode in endoscopy. The particular light in compound band imaging (CBI) can highlight delicate structures, such as capillaries and tiny structures on the mucosal surface. These two modes complement each other, and doctors switch between them manually to complete the examination. This paper proposes an endoscopy image fusion system to combine WLI and CBI. We add a real-time rotatable color wheel in the light source device of the AQ-200 endoscopy system to achieve rapid imaging of two modes at the same position of living tissue. The two images corresponding to the pixel level can avoid registration and lay the foundation for image fusion. We propose a multi-scale image fusion framework, which involves Laplacian pyramid (LP) and convolutional sparse representation (CSR) and strengthens the details in the fusion rule. Volunteer experiments and ex vivo pig stomach trials are conducted to verify the feasibility of our proposed system. We also conduct comparative experiments with other image fusion methods, evaluate the quality of the fused images, and verify the effectiveness of our fusion framework. The results show that our fused image has rich details, high color contrast, apparent structures, and clear lesion boundaries. An endoscopy image fusion system is proposed, which does not change the doctor's operation and makes the fusion of WLI and CBI optical staining technology a reality. We change the light source device of the endoscope, propose an image fusion framework, and verify the feasibility and effectiveness of our scheme. Our method fully integrates the advantages of WLI and CBI, which can help doctors make more accurate judgments than before. The endoscopy image fusion system is of great significance for improving the detection rate of early lesions and has broad application prospects.

A CSR-based visible and infrared image fusion method in low illumination conditions for sense and avoid

AbstractMachine vision has been extensively researched in the field of unmanned aerial vehicles (UAV) recently. However, the ability of Sense and Avoid (SAA) largely limited by environmental visibility, which brings hazards to flight safety in low illumination or nighttime conditions. In order to solve this critical problem, an approach of image enhancement is proposed in this paper to improve image qualities in low illumination conditions. Considering the complementarity of visible and infrared images, a visible and infrared image fusion method based on convolutional sparse representation (CSR) is a promising solution to improve the SAA ability of UAVs. Firstly, the source image is decomposed into a texture layer and structure layer since infrared images are good at characterising structural information, and visible images have richer texture information. Both the structure and the texture layers are transformed into the sparse convolutional domain through the CSR mechanism, and then CSR coefficient mapping are fused via activity level assessment. Finally, the image is synthesised through the reconstruction results of the fusion texture and structure layers. In the experimental simulation section, a series of visible and infrared registered images including aerial targets are adopted to evaluate the proposed algorithm. Experimental results demonstrates that the proposed method increases image qualities in low illumination conditions effectively and can enhance the object details, which has better performance than traditional methods.

An intervention study of VR sports games on the perceptual and motor abilities of children with incomplete intellectual development

Abstract Childhood is a critical stage for the development of perceptual and motor abilities, and strengthening the training of children with incomplete mental development at this stage will affect the development of motor skills during their growth. In this paper, we take VR technology as a starting point to build a perceptual model and introduce a convolutional sparse representation algorithm. First, a sparse representation with few non-zero elements is found to optimize a function consisting of a data fidelity term and a sparse induced penalty function. Then, the sum of the convolution of the filter and the convolution sparse feature map, i.e., the convolution operation, is computed to generate the translation invariants. Then the convolutional sparse coding method is introduced to the traditional unsupervised problem by calculating the minimization objective function and solving it in an iterative manner alternatively. Finally, the constituents of the signal are analyzed and the discrete equivalence of the convolution is derived based on the Fourier transform to derive the intervening variables. The experimental results showed that the mean value of the post-test of motor ability compared with the pre-test increased by 4.6 through an eight-week VR sports game training intervention study test on different children with incomplete mental development. Therefore, it is of great theoretical and practical significance to understand the characteristics of perceptual and motor abilities of children with incomplete intellectual development and to develop corresponding programs for VR sports game training according to their characteristics.

Multimodal Medical Image Fusion: The Perspective of Deep Learning

Multimodal medical image fusion involves the integration of medical images originating from distinct modalities and captured by various sensors, with the aim to enhance image quality, minimize redundant information, and preserve specific features, ultimately leading to increased efficiency and accuracy in clinical diagnoses. In recent years, the emergence of deep learning techniques has propelled significant advancements in image fusion, addressing the limitations of conventional methods that necessitate manual design of activity level measurement and fusion rules. This paper initially presents a systematic description of the multimodal medical image fusion problem, delineating the interrelationships between different fusion modalities while summarizing their characteristics and functions. Subsequently, it reviews the theories and enhancement approaches associated with deep learning in the medical image fusion domain, striving for a comprehensive overview of the state-of-the-art developments in this field from a deep learning perspective. These developments encompass multimodal feature extraction methods based on convolutional techniques, adversarial learning-based methods, convolutional sparse representation and stacked autoencoder-based signal processing methods, and unified models. Lastly, the paper summarizes the enhancement techniques for multimodal medical image fusion methods, highlighting the pressing issues and challenges encountered by deep learning approaches in this domain.

Bearing Fault Diagnosis Using Convolutional Sparse Representation Combined With Nonlocal Similarity

Conducting fault diagnosis to bearing is of great significance to ensure their stable working and reduce economic losses of equipment in industrial manufacturing. However, the existing methods are not accurate enough to detect fault frequency in real operating conditions affected by noise. To detect the fault frequency more accurately, in this paper, a novel fault diagnosis algorithm is proposed by using convolutional sparse representation under the non-local similarity of vibration signal. Firstly, the denoised similar subblocks is collected in non-local region with correlation coefficient to speed up the training process of adaptive filters and reconstruct the original signal more closely. Secondly, to remove noise contained in each component, a denoising algorithm based on adaptive threshold is employed to shrink the coefficients of each component decomposed by convolutional sparse representation. Then, a measure function indicating the activity of fault frequency is designed to select the optimal signal subband which contains the main fault information. Finally, the envelope spectrum is calculated to detect the fault frequency. In order to verify the performance of the proposed algorithm, we conduct a series of experiments on two public datasets and one real collected data, and compare the results with some state-of-the-art fault diagnosis algorithms. Experimental results show that the proposed algorithm can accurately detect the fault frequency of rolling bearings and its harmonics and exhibits advantages in fault frequency detection compared with other fault diagnosis methods.

Compressive Spectral Video Sensing using the Convolutional Sparse Coding framework CSC4D

Spectral Videos (SV) contain a scene’s spatial–spectral-time information. Just as with Spectral Images (SI), SVs require expensive sensing hardware, storage plus high frame ratios. Although Super Resolution techniques improve the quality of low-resolution SVs, Compressive Spectral Video Sensing (CSVS) senses high-quality SVs by extending the Compressive Sensing Image (CSI) techniques. CSI uses the universal Sparse Signal Representation (SSR) model for SVs and SIs despite the limited quality of the recovered signals. On the other hand, dictionaries synthesis models are used successfully for representing SIs, SVs, and in CSI. This work proposes the 4D convolutional sparse representation (CSC4D) for recovering full-resolution SV from CSVS measurements. It is based on a multidimensional formulation of the CSC model, profiting from its robustness without additional optical flow information. Extensive numerical simulations (two CSI architectures and noise models) show that the proposed CSC4D+CSVS improves the state-of-the-art in both quality and border sharpness by up to 1.5 dB.

Simultaneous denoising and resolution enhancement of seismic data based on elastic convolution dictionary learning

Enhancing seismic resolution is a key component in seismic data processing, which plays a valuable role in raising the prospecting accuracy of oil reservoirs. However, in noisy situations, existing resolution enhancement methods are difficult to yield satisfactory processing outcomes for reservoir characterization. To solve this problem, we develop a new approach for simultaneous denoising and resolution enhancement of seismic data based on convolution dictionary learning. First, an elastic convolution dictionary learning algorithm is presented to efficiently learn a convolution dictionary with stronger representation capability from the noisy data to be processed. Specifically, the algorithm introduces the elastic L1/2 norm as a sparsity constraint and employs a steepest gradient descent strategy to efficiently solve the frequency-domain linear system with substantial computational cost in a half-quadratic splitting framework. Then, based on the learned convolution dictionary, a weighted convolutional sparse representation paradigm is designed to encode the noisy data to acquire an optimal sparse approximation of the effective signal. Subsequently, a high-resolution dictionary with a broadband spectrum is constructed by the proposed parameter scaling strategy and matched filtering technique on the basis of atomic spectrum modeling. Finally, the optimal sparse approximation of the effective signal and the constructed high-resolution dictionary are used for data reconstruction to obtain the seismic signal with high resolution and high signal-to-noise ratio. Synthetic and field dataset examples are executed to check the effectiveness and reliability of the developed method. The results indicate that this method has a more competitive performance in seismic applications compared with the conventional deconvolution and spectral whitening methods.

A multimodal fusion method for Alzheimer's disease based on DCT convolutional sparse representation.

The medical information contained in magnetic resonance imaging (MRI) and positron emission tomography (PET) has driven the development of intelligent diagnosis of Alzheimer's disease (AD) and multimodal medical imaging. To solve the problems of severe energy loss, low contrast of fused images and spatial inconsistency in the traditional multimodal medical image fusion methods based on sparse representation. A multimodal fusion algorithm for Alzheimer' s disease based on the discrete cosine transform (DCT) convolutional sparse representation is proposed. The algorithm first performs a multi-scale DCT decomposition of the source medical images and uses the sub-images of different scales as training images, respectively. Different sparse coefficients are obtained by optimally solving the sub-dictionaries at different scales using alternating directional multiplication method (ADMM). Secondly, the coefficients of high-frequency and low-frequency subimages are inverse DCTed using an improved L1 parametric rule combined with improved spatial frequency novel sum-modified SF (NMSF) to obtain the final fused images. Through extensive experimental results, we show that our proposed method has good performance in contrast enhancement, texture and contour information retention.

Denoising and Inpainting of Sonar Images Using Convolutional Sparse Representation

Recently sparse representation (SR) techniques have been extensively applied for sonar images restoration tasks such as denoising and inpainting. However, the fine details such as edges and texture information present in sonar images are not fully restored using these techniques. This is due to ineffective dictionary formation and learning of sparse components and dictionaries. In this paper, the author proposes the convolutional sparse representation (CSR)-based methods for sonar image denoising and inpainting tasks. To minimize the Gaussian noise from sonar images, an alternative direction method of the multiplier (ADMM) optimization algorithm is used for solving the convolutional basis pursuit denoising (CBPDN) problem with joint sparsity through an L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> norm term. For the removal of impulse noise from sonar images, an ADMM algorithm is used to solve a CSR problem with an L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> data fidelity term and both L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> and L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> gradient regularization terms. For the inpainting of sonar images, the additive mask simulation technique is used for solving CSR problems related to boundary masking. From the experimental results, it is found that the proposed sonar image denoising and inpainting methods have created better visual quality sonar images, higher peak signal-to-noise ratio (PSNR), and structure similarity (SSIM) values, as compared to state-of-the-art sonar image denoising and inpainting methods.

An Improved Sparse Representation Based on Local Orthogonal Matching Pursuit for Bearing Compound Fault Diagnosis

Convolutional sparse representation based on local orthogonal matching pursuit (SR-LocOMP) can recover wheelset-bearing fault impulses without being affected by random slippage and plays an important role in wheelset-bearing fault diagnosis. However, the performance of SR-LocOMP at a low signal-to-noise ratio (SNR) is not satisfied. In addition, when the analyzed signals contain compound faults, the fault feature extraction ability of SR-LocOMP will be greatly affected. To further improve the performance of SR-LocOMP, this article proposes an improved SR-LocOMP algorithm, named improved SR-LocOMP. The proposed method introduces scale-space representation (SSR) for frequency band segmentation and then performs SR-LocOMP for signals in each segmented frequency band. Reconstructed signals corresponding to each segmented frequency band are obtained separately. Envelope spectral <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${L}2/{L}1$ </tex-math></inline-formula> norm is used to measure the fault information content of each reconstructed signal to extract more accurate and richer impulse response features caused by bearing faults. The proposed method not only has the ability to detect compound faults but also can effectively reduce noise interference. Through simulation and experimental analysis, the superiority of the proposed method is verified.

AN INFRARED AND VISIBLE IMAGE FUSION METHOD BASED ON FUZZY PULSE COUPLED NEURAL NETWORK AND SALIENCE DETECTION

Existing image fusion methods based on pulse coupled neural network (PCNN) ignore the fact that the human brain is essential to the organic combination of neural network and fuzzy system. As a result, some pixels that carry detailed information cannot activate neurons, thereby resulting in loss of details of source images and lowering the quality of fused images. To solve this problem, the present study proposed an infrared and visible image fusion method based on fuzzy pulse coupled neural network (FPCNN) and salience detection. This proposed method constructed FPCNN by combining the fuzzy theory and pulse coupled neural network. The cartoon parts obtained from the decomposition of source images by the total variation model were fused based on FPCNN. Then, salience detection of coefficients of the convolution sparse representation of the texture parts was conducted. The texture parts were fused on the basis of salience detection results. Finally, the fused cartoon and texture parts were added to obtain the fused image. The effectiveness and superiority of the proposed method were verified by experiments. Results demonstrate that compared with the most advanced seven algorithms, the proposed algorithm is superior in terms of objective evaluation indexes such as information entropy, average gradient, structural similarity, and fusion performance based on gradient. Moreover, the proposed method has good visual effect. This study provides a reference for producing high-quality all-weather images in whole-scene description. Keywords: Image fusion, Pulse coupled neural network, Salience detection, Convolution sparse representation

Multimodal medical image fusion with convolution sparse representation and mutual information correlation in NSST domain

Multimodal medical image is an effective method to solve a series of clinical problems, such as clinical diagnosis and postoperative treatment. In this study, a medical image fusion method based on convolutional sparse representation (CSR) and mutual information correlation is proposed. In this method, the source image is decomposed into one high-frequency and one low-frequency sub-band by non-subsampled shearlet transform. For the high-frequency sub-band, CSR is used for high-frequency coefficient fusion. For the low-frequency sub-band, different fusion strategies are used for different regions by mutual information correlation analysis. Analysis of two kinds of medical image fusion problems, namely, CT–MRI and MRI–SPECT, reveals that the performance of this method is robust in terms of five common objective metrics. Compared with the other six advanced medical image fusion methods, the experimental results show that the proposed method achieves better results in subjective vision and objective evaluation metrics.

Nursing Analysis Based on Medical Imaging Technology before and after Coronary Angiography in Cardiovascular Medicine.

With the advancement of technology, medical imaging technology has been greatly improved. This article mainly studies the nursing before and after coronary angiography in cardiovascular medicine based on medical imaging technology. This paper proposes a multimodal medical image fusion algorithm based on multiscale decomposition and convolution sparse representation. The algorithm first decomposes the preregistered source medical image by NSST, takes the subimages of different scales as training images, and optimizes the subdictionaries of different scales; then convolution and sparse the subimages on each scale encoding to obtain the sparse coefficients of different subimages; secondly, the combination of improved L1 norm and improved spatial frequency (novel sum-modified SF (NMSF)) is used for high-frequency subimage coefficients, and the fusion of low-frequency subimages improved the rule of combining the L1 norm and the regional energy; finally, the final fused image is obtained by inverse NSST of the fused low-frequency subband and high-frequency subband. Experimental analysis found that the bifurcation angle has nothing to do with the damage of the branch vessels after the main branch stent is placed. The bifurcation angle greater than 50° is an independent predictor of MACE after stent extrusion for bifurcation lesions. Experimental results show that the proposed method has good performance in contrast enhancement, detail extraction, and information retention, and it improves the quality of the fusion image.

Supervised anomaly detection by convolutional sparse representation

Supervised anomaly detection by convolutional sparse representation

A Competent Convolutional Sparse Representation Model for Pan-Sharpening of Multi-Spectral Images

Two types of images are produced by Earth observation satellites, each having complementing spatial andspectral characteristics. Pan-sharpening (PS) is based on remote sensing and image fusion approach thatproduces a high spatial resolution multi-spectral image by merging spectral information from a low spatialresolution multispectral (MS) image with intrinsic spatial details from a high spatial resolution panchromatic(PAN) image. Traditional pan-sharpening methods continue to seek for a fused image that contains thenecessary spatial and spectral information. This work proposes a pan-sharpening method based on a recentinvention, convolutional sparse representation (CSR). Geometric structural characteristics are extracted fromthe PAN image using a CSR-based filtering procedure. The challenge of learning filters, convolutional basispursuit denoising (CBPDN), is handled using a modified dictionary learning method based on the concept ofAlternating Direction Method of Multipliers (ADMM). The retrieved details are put into MS bands usingapplicable weighting coefficients. Because the proposed fusion model avoids the standard patch-basedmethod, spatial and structural features are preserved while spectral quality is maintained. The spectraldistortion index SAM and the spatial measure ERGAS improve by 4.4 and 6.2 percent, respectively, whencompared to SR-based techniques. The computational complexity is reduced by 200 seconds when compared&#x0D; to the most recent SR-based fusion technique. The proposed method's efficacy is demonstrated by reduced-scale and full-scale experimental findings utilising the QuickBird and GeoEye-1 datasets.