Dictionary Learning Research Articles

CROSS STYLE IMAGE SYNTHESIS THROUGH SEMI COUPLED DICTIONARY LEARNING AND GENERALIZED CROSS VALIDATION ALGORITHM

In various Computer vision applications for better visualization, interpretation and better recognition, we frequently want to change an image in one style into another one which is a large research area of compressive sensing. Algorithms on compressed sensing are in demand to create better reconstruction of the images while taking less computational time and requiring less storage capacity. In the present work attempts are put in the same direction and brought an algorithm which includes semi coupled dictionary learning (SCDL) model of compressed sensing with Singular Value Decomposition (SVD) based Generalized Cross Validation (GCV) algorithm. Using this algorithm one can find the different value of regularization parameters for different sizes of images and that helps to get better reconstruction of the images with ease while doing experiments on cross style image synthesis problems.

Quantitative MR Image Reconstruction Using Parameter-Specific Dictionary Learning With Adaptive Dictionary-Size and Sparsity-Level Choice.

We propose a method for the reconstruction of parameter-maps in Quantitative Magnetic Resonance Imaging (QMRI). Because different quantitative parameter-maps differ from each other in terms of local features, we propose a method where the employed dictionary learning (DL) and sparse coding (SC) algorithms automatically estimate the optimal dictionary-size and sparsity level separately for each parameter-map. We evaluated the method on a T1-mapping QMRI problem in the brain using the BrainWeb data as well as in-vivo brain images acquired on an ultra-high field 7 T scanner. We compared it to a model-based acceleration for parameter mapping (MAP) approach, other sparsity-based methods using total variation (TV), Wavelets (Wl), and Shearlets (Sh) to a method which uses DL and SC to reconstruct qualitative images, followed by a non-linear (DL+Fit). Our algorithm surpasses MAP, TV, Wl, and Sh in terms of RMSE and PSNR. It yields better or comparable results to DL+Fit by additionally significantly accelerating the reconstruction by a factor of approximately seven. The proposed method outperforms the reported methods of comparison and yields accurate T1-maps. Although presented for T1-mapping in the brain, our method's structure is general and thus most probably also applicable for the the reconstruction of other quantitative parameters in other organs. From a clinical perspective, the obtained T1-maps could be utilized to differentiate between healthy subjects and patients with Alzheimer's disease. From a technical perspective, the proposed unsupervised method could be employed to obtain ground-truth data for the development of data-driven methods based on supervised learning.

Transformer‐based reranking for improving Korean morphological analysis systems

AbstractThis study introduces a new approach in Korean morphological analysis combining dictionary‐based techniques with Transformer‐based deep learning models. The key innovation is the use of a BERT‐based reranking system, significantly enhancing the accuracy of traditional morphological analysis. The method generates multiple suboptimal paths, then employs BERT models for reranking, leveraging their advanced language comprehension. Results show remarkable performance improvements, with the first‐stage reranking achieving over 20% improvement in error reduction rate compared with existing models. The second stage, using another BERT variant, further increases this improvement to over 30%. This indicates a significant leap in accuracy, validating the effectiveness of merging dictionary‐based analysis with contemporary deep learning. The study suggests future exploration in refined integrations of dictionary and deep learning methods as well as using probabilistic models for enhanced morphological analysis. This hybrid approach sets a new benchmark in the field and offers insights for similar challenges in language processing applications.

3-D Point Cloud Attribute Compression With -Laplacian Embedding Graph Dictionary Learning.

3-D point clouds facilitate 3-D visual applications with detailed information of objects and scenes but bring about enormous challenges to design efficient compression technologies. The irregular signal statistics and high-order geometric structures of 3-D point clouds cannot be fully exploited by existing sparse representation and deep learning based point cloud attribute compression schemes and graph dictionary learning paradigms. In this paper, we propose a novel p-Laplacian embedding graph dictionary learning framework that jointly exploits the varying signal statistics and high-order geometric structures for 3-D point cloud attribute compression. The proposed framework formulates a nonconvex minimization constrained by p-Laplacian embedding regularization to learn a graph dictionary varying smoothly along the high-order geometric structures. An efficient alternating optimization paradigm is developed by harnessing ADMM to solve the nonconvex minimization. To our best knowledge, this paper proposes the first graph dictionary learning framework for point cloud compression. Furthermore, we devise an efficient layered compression scheme that integrates the proposed framework to exploit the correlations of 3-D point clouds in a structured fashion. Experimental results demonstrate that the proposed framework is superior to state-of-the-art transform-based methods in M-term approximation and point cloud attribute compression and outperforms recent MPEG G-PCC reference software.

A new model for bearing fault diagnosis based on mutual mapping of signals and images and sparse representation

For the issue of significant noise in the collected bearing fault signals, a new bearing fault diagnosis model based on mutual mapping of signals and images (MMSI) and sparse representation (SR) denoising is proposed. Firstly, the fault signal is divided into several segments with the same number of sampling points, and then arrange these segments in ascending order of rows. Secondly, convert the arranged signals into grayscale image and use dictionary learning for block denoising. Then, the de-noised grayscale image is restored to a signal in line order. Finally, k-nearest neighbor (KNN) is used for fault classification. To verify the performance of the proposed model, experiments are tested on 12 single working conditions and 30 multi working conditions on the Case Western Reserve University dataset and the Paderborn dataset. The experimental results indicate that compared with some existing models, the MMSI–SR–KNN model can not only accurately diagnose bearing faults in artificial damage experiments, but also performs better in real damage faults. This indicates that the model has good generalization ability between different datasets and working conditions.

Multi-layer convolutional dictionary learning network for signal denoising and its application to explainable rolling bearing fault diagnosis

As a vital mechanical sub-component, the health monitoring of rolling bearings is important. Vibration signal analysis is a commonly used approach for fault diagnosis of bearings. Nevertheless, the collected vibration signals cannot avoid interference from noises which has a negative influence on fault diagnosis. Thus, denoising needs to be utilized as an essential step of vibration signal processing. Traditional denoising methods need expert knowledge to select hyperparameters. And data-driven methods based on deep learning lack interpretability and a clear justification for the design of architecture in a “black-box” deep neural network. An approach to systematically design neural networks is by unrolling algorithms, such as learned iterative soft-thresholding (LISTA). In this paper, the multi-layer convolutional LISTA (ML-CLISTA) algorithm is derived by embedding a designed multi-layer sparse coder to the convolutional extension of LISTA. Then the multi-layer convolutional dictionary learning (ML-CDL) network for mechanical vibration signal denoising is proposed by unrolling ML-CLISTA. By combining ML-CDL network with a classifier, the proposed denoising method is applied to the explainable rolling bearing fault diagnosis. The experiments on two bearing datasets show the superiority of the ML-CDL network over other typical denoising methods.

Low‐frequency seismic deghosting in a compressed domain using parabolic dictionary learning

AbstractDeghosting is an important technique in the marine seismic industry, as it plays a crucial role in mitigating the effects of ghost reflections from the sea surface, which can significantly impact the accuracy and resolution of subsurface imaging. In recent years, various acquisition‐based techniques have been developed to tackle the challenge of removing receiver–ghost reflections, which is the focus of our paper. These state‐of‐the‐art approaches, such as dual‐sensor or multicomponent towed streamer acquisitions, have demonstrated exceptional accuracy by combining pressure and particle motion data. However, such methods face limitations when dealing with low frequencies due to heavy noise contamination in the particle motion data. Consequently, ghost‐free data reconstruction at low frequencies typically relies on processing‐based approaches, which exclusively utilize recorded pressure data. This study presents a novel deghosting method for low‐frequency applications based on parabolic dictionary learning, which relies solely on recorded pressure data. The proposed method has the advantage of being applicable directly in a compressed domain, eliminating the need for data decompression prior to the deghosting process when compression is applied before the processing steps. This not only reduces costs related to data storage and transfer but also provides a cost‐effective alternative to conventional deghosting by operating directly on the compressed data format, which is smaller in size. The effectiveness of the proposed method was evaluated using both synthetic and field datasets. The results obtained from a synthetic data example indicate that the proposed method achieves similar results to an industry‐standard frequency–wavenumber method, while achieving a compression rate of over 7. Furthermore, the method was tested using a field dataset consisting of a full sail‐line of marine seismic acquisition. The comparison of 2D pre‐stack migrated images between the proposed method and the industry‐standard frequency–wavenumber method revealed insignificant differences, while achieving a compression ratio higher than 5 when our method was used.

Single image super-resolution based on Bendlets analysis and structural dictionary learning

High-resolution biological tissue slice images provide opportunities for more precise observation and analysis of histopathological features. These images typically consist of enclosed contours, and Bendlets transform is an effective means to approximate such structures. In this study, we leverage Bendlet transform to incorporate multi-level image data, thereby establishing a multi-scale pyramid feature set. This approach aids in achieving a sparser representation of image textures and structures. We apply Bendlets transform to extract multi-frequency wavelet subband features and formulate structured dictionaries for both high and low resolutions based on these features. After creating these dictionaries, we compute sparse representation coefficients using the low-resolution dictionary and subsequently integrate them with the high-resolution dictionary to generate high-resolution subbands. Ultimately, through the process of inverse wavelet transformation, we completed the reconstruction of high-resolution images. This method not only significantly enhances the restoration of image details and clarity but also effectively preserves the overall structural characteristics of the original images.

Exploiting sparsity of hyperspectral image: A novel approach for compressive hyperspectral image reconstruction using deep learning

Compressive hyperspectral imaging is an emerging technology that captures compressed two-dimensional (2D) measurements and subsequently reconstructs three-dimensional (3D) hyperspectral images, capitalizing on the sparsity prior. Numerous end-to-end networks have been proposed, yielding notable improvements in reconstruction speed and quality compared to traditional algorithms. However, these networks fall short in exploiting the sparsity of hyperspectral images in their direct mapping from 2D measurements to 3D hyperspectral images. To address this limitation, this paper introduces a novel approach for compressive hyperspectral image reconstruction using deep learning. The deep learning network comprises two components: the sparse coefficient recovery net and the dictionary training net. The dictionary training net incorporates a pre-trained dictionary, whose parameters can also be updated during the backpropagation process. Concurrently, the sparse coefficient recovery net generates sparse coefficients under this dictionary. Our approach draws inspiration from conventional dictionary learning algorithms, enabling effective utilization of signal sparsity. In comparison to several state-of-the-art methods, our method achieves superior reconstruction quality and demonstrates enhanced robustness against noise. The proposed approach is envisioned to offer fresh insights into hyperspectral images reconstruction using deep learning models.

Dictionary learning with cone atoms and application to anomaly detection

We propose a variant of dictionary learning (DL) for sparse representations where the atoms are cones instead of simple vectors. The most convenient vector from a cone, called actual atom, is used to build the linear sparse representation of a given signal. We present a DL algorithm suited for cone atoms, which can update the dictionary without storing all the actual atoms that are used in the representations of the training signals. Also, the algorithm ensures that the cone atoms are disjoint and thus the representation problem is well posed. We use the proposed cone DL for anomaly detection. On a specific type of anomaly, called “dependency”, the DL methods involving cone atoms are better than those from a reputed benchmark. They are also better than standard DL.

Research on Mushroom Image Classification Algorithm Based on Deep Sparse Dictionary Learning

The traditional mushroom feature extraction method has low classification efficiency and unsatisfactory effect. Dictionary learning is widely used in image classification. However, the previous work is to learn dictionaries in the original space, which limits the performance of sparse representation classification. In order to solve the problem of spatial redundancy in traditional convolutional neural networks and the weak performance of deep learning in small samples, an improved dictionary learning algorithm, Deep Sparse Dictionary learning (DSDL), is proposed. The input to DSDL is not a matrix gathered from the original grayscale image or a hand-created feature, but rather a relatively deeper feature extraction via a stack autoencoder. Then, a structured dictionary is designed to reconstruct the deep features according to different categories of distinguishing features. In addition, it is necessary to learn the associated structured projection sparse dictionary to ensure that the decoder updates in the direction of the deconvolution operator error is minimal. By utilizing sparse dictionary learning loss functions and autoencoder loss functions, DSDL can simultaneously learn deep latent features and corresponding dictionary pairs. In the testing phase of DSDL, the minimum errors of deep feature and structured projection components for different classes can be directly represented by basic matrix multiplication operations. Experimental results show that the proposed method achieves a good classification effect on mushroom images, which shows the effectiveness of the method.

Multimodal classification of Alzheimer's disease and mild cognitive impairment using custom MKSCDDL kernel over CNN with transparent decision-making for explainable diagnosis

The study presents an innovative diagnostic framework that synergises Convolutional Neural Networks (CNNs) with a Multi-feature Kernel Supervised within-class-similar Discriminative Dictionary Learning (MKSCDDL). This integrative methodology is designed to facilitate the precise classification of individuals into categories of Alzheimer's Disease, Mild Cognitive Impairment (MCI), and Cognitively Normal (CN) statuses while also discerning the nuanced phases within the MCI spectrum. Our approach is distinguished by its robustness and interpretability, offering clinicians an exceptionally transparent tool for diagnosis and therapeutic strategy formulation. We use scandent decision trees to deal with the unpredictability and complexity of neuroimaging data. Considering that different people's brain scans are different, this enables the model to make more detailed individualised assessments and explains how the algorithm illuminates the specific neuroanatomical regions that are indicative of cognitive impairment. This explanation is beneficial for clinicians because it gives them concrete ideas for early intervention and targeted care. The empirical review of our model shows that it makes diagnoses with a level of accuracy that is unmatched, with a classification efficacy of 98.27%. This shows that the model is good at finding important parts of the brain that may be damaged by cognitive diseases.

Additional Value From Free-Text Diagnoses in Electronic Health Records: Hybrid Dictionary and Machine Learning Classification Study.

Physicians are hesitant to forgo the opportunity of entering unstructured clinical notes for structured data entry in electronic health records. Does free text increase informational value in comparison with structured data? This study aims to compare information from unstructured text-based chief complaints harvested and processed by a natural language processing (NLP) algorithm with clinician-entered structured diagnoses in terms of their potential utility for automated improvement of patient workflows. Electronic health records of 293,298 patient visits at the emergency department of a Swiss university hospital from January 2014 to October 2021 were analyzed. Using emergency department overcrowding as a case in point, we compared supervised NLP-based keyword dictionaries of symptom clusters from unstructured clinical notes and clinician-entered chief complaints from a structured drop-down menu with the following 2 outcomes: hospitalization and high Emergency Severity Index (ESI) score. Of 12 symptom clusters, the NLP cluster was substantial in predicting hospitalization in 11 (92%) clusters; 8 (67%) clusters remained significant even after controlling for the cluster of clinician-determined chief complaints in the model. All 12 NLP symptom clusters were significant in predicting a low ESI score, of which 9 (75%) remained significant when controlling for clinician-determined chief complaints. The correlation between NLP clusters and chief complaints was low (r=-0.04 to 0.6), indicating complementarity of information. The NLP-derived features and clinicians' knowledge were complementary in explaining patient outcome heterogeneity. They can provide an efficient approach to patient flow management, for example, in an emergency medicine setting. We further demonstrated the feasibility of creating extensive and precise keyword dictionaries with NLP by medical experts without requiring programming knowledge. Using the dictionary, we could classify short and unstructured clinical texts into diagnostic categories defined by the clinician.

Structured dictionary learning of rating migration matrices for credit risk modeling

Structured dictionary learning of rating migration matrices for credit risk modeling

Deep convolutional dictionary learning network for sparse view CT reconstruction with a group sparse prior

Purpose Numerous techniques based on deep learning have been utilized in sparse view computed tomography (CT) imaging. Nevertheless, the majority of techniques are instinctively constructed utilizing state-of-the-art opaque convolutional neural networks (CNNs) and lack interpretability. Moreover, CNNs tend to focus on local receptive fields and neglect nonlocal self-similarity prior information. Obtaining diagnostically valuable images from sparsely sampled projections is a challenging and ill-posed task.Method To address this issue, we propose a unique and understandable model named DCDL-GS for sparse view CT imaging. This model relies on a network comprised of convolutional dictionary learning and a nonlocal group sparse prior. To enhance the quality of image reconstruction, we utilize a neural network in conjunction with a statistical iterative reconstruction framework and perform a set number of iterations. Inspired by group sparsity priors, we adopt a novel group thresholding operation to improve the feature representation and constraint ability and obtain a theoretical interpretation. Furthermore, our DCDL-GS model incorporates filtered backprojection (FBP) reconstruction, fast sliding window nonlocal self-similarity operations, and a lightweight and interpretable convolutional dictionary learning network to enhance the applicability of the model.Results The efficiency of our proposed DCDL-GS model in preserving edges and recovering features is demonstrated by the visual results obtained on the LDCT-P and UIH datasets. Compared to the results of the most advanced techniques, the quantitative results are enhanced, with increases of 0.6-0.8 dB for the peak signal-to-noise ratio (PSNR), 0.005-0.01 for the structural similarity index measure (SSIM), and 1-1.3 for the regulated Fréchet inception distance (rFID) on the test dataset. The quantitative results also show the effectiveness of our proposed deep convolution iterative reconstruction module and nonlocal group sparse prior.Conclusion In this paper, we create a consolidated and enhanced mathematical model by integrating projection data and prior knowledge of images into a deep iterative model. The model is more practical and interpretable than existing approaches. The results from the experiment show that the proposed model performs well in comparison to the others.

Multi-layer representation for cross-view action recognition

Daily action recognition is an important application domain in computer vision. Considering the complex temporal structure and view changes of daily actions, we propose a multi-layer representation for cross-view action recognition (MRCAR) based on two-stage optimization using a training set and a validation set. In the first stage, a cross-view atomic dictionary learning model is constructed based on jointly sparse constraints, utilizing the original features of the training set. Subsequently, a shared atomic dictionary, containing representative view-independent information, is obtained through alternating optimization. In the second stage, a reconstruction function based on the obtained shared atomic dictionary is developed. This function is employed to acquire sparse codes for the training and validation sets, enhancing the representativeness of motion atoms and improving overall recognition performance. Finally, experimental results from the WVU (West Virginia University) dataset and the NTU (Nanyang Technological University) RGB-D 120 dataset demonstrate that the proposed MRCAR method achieves an accuracy improvement of 3%-20% over recent state-of-the-art methods.

Image super-resolution reconstruction based on deep dictionary learning and A+

Image super-resolution reconstruction based on deep dictionary learning and A+

Convolutional sparse coding for compressed sensing photoacoustic CT reconstruction with partially known support.

In photoacoustic tomography (PAT), imaging speed is an essential metric that is restricted by the pulse laser repetition rate and the number of channels on the data acquisition card (DAQ). Reconstructing the initial sound pressure distribution with fewer elements can significantly reduce hardware costs and back-end acquisition pressure. However, undersampling will result in artefacts in the photoacoustic image, degrading its quality. Dictionary learning (DL) has been utilised for various image reconstruction techniques, but they disregard the uniformity of pixels in overlapping blocks. Therefore, we propose a compressive sensing (CS) reconstruction algorithm for circular array PAT based on gradient domain convolutional sparse coding (CSCGR). A small number of non-zero signal positions in the sparsely encoded feature map are used as partially known support (PKS) in the reconstruction procedure. The CS-CSCGR-PKS-based reconstruction algorithm can use fewer ultrasound transducers for signal acquisition while maintaining image fidelity. We demonstrated the effectiveness of this algorithm in sparse imaging through imaging experiments on the mouse torso, brain, and human fingers. Reducing the number of array elements while ensuring imaging quality effectively reduces equipment hardware costs and improves imaging speed.

Interactive and discriminative analysis dictionary learning for image classification

Interactive and discriminative analysis dictionary learning for image classification

Learning the sparse prior: Modern approaches

AbstractThe sparse prior has been widely adopted to establish data models for numerous applications. In this context, most of them are based on one of three foundational paradigms: the conventional sparse representation, the convolutional sparse representation, and the multi‐layer convolutional sparse representation. When the data morphology has been adequately addressed, a sparse representation can be obtained by solving the sparse coding problem specified by the data model. This article presents a comprehensive overview of these three models and their corresponding sparse coding problems and demonstrates that they can be solved using convex and non‐convex optimization approaches. When the data morphology is not known or cannot be analyzed, it must be learned from training data, thereby formulating dictionary learning problems. This article addresses two different dictionary learning paradigms. In an unsupervised scenario, dictionary learning involves the alternating or joint resolution of sparse coding and dictionary updating. Another option is to create a recurrent neural network by unrolling algorithms designed to solve sparse coding problems. These networks can then be used in a supervised learning setting to facilitate the training of dictionaries via forward‐backward optimization. This article lists numerous applications in various domains and outlines several directions for future research related to the sparse prior.This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Modeling Methods Statistical and Graphical Methods of Data Analysis > Modeling Methods and Algorithms Statistical Models > Nonlinear Models