Synthesis Dictionary Research Articles

Discriminative Dictionary Pair Learning With Scale-Constrained Structured Representation for Image Classification.

The dictionary pair learning (DPL) model aims to design a synthesis dictionary and an analysis dictionary to accomplish the goal of rapid sample encoding. In this article, we propose a novel structured representation learning algorithm based on the DPL for image classification. It is referred to as discriminative DPL with scale-constrained structured representation (DPL-SCSR). The proposed DPL-SCSR utilizes the binary label matrix of dictionary atoms to project the representation into the corresponding label space of the training samples. By imposing a non-negative constraint, the learned representation adaptively approximates a block-diagonal structure. This innovative transformation is also capable of controlling the scale of the block-diagonal representation by enforcing the sum of within-class coefficients of each sample to 1, which means that the dictionary atoms of each class compete to represent the samples from the same class. This implies that the requirement of similarity preservation is considered from the perspective of the constraint on the sum of coefficients. More importantly, the DPL-SCSR does not need to design a classifier in the representation space as the label matrix of the dictionary can also be used as an efficient linear classifier. Finally, the DPL-SCSR imposes the l2,p -norm on the analysis dictionary to make the process of feature extraction more interpretable. The DPL-SCSR seamlessly incorporates the scale-constrained structured representation learning, within-class similarity preservation of representation, and the linear classifier into one regularization term, which dramatically reduces the complexity of training and parameter tuning. The experimental results on several popular image classification datasets show that our DPL-SCSR can deliver superior performance compared with the state-of-the-art (SOTA) dictionary learning methods. The MATLAB code of this article is available at https://github.com/chenzhe207/DPL-SCSR.

Subject-Independent EEG Emotion Recognition Based on Genetically Optimized Projection Dictionary Pair Learning.

One of the primary challenges in Electroencephalogram (EEG) emotion recognition lies in developing models that can effectively generalize to new unseen subjects, considering the significant variability in EEG signals across individuals. To address the issue of subject-specific features, a suitable approach is to employ projection dictionary learning, which enables the identification of emotion-relevant features across different subjects. To accomplish the objective of pattern representation and discrimination for subject-independent EEG emotion recognition, we utilized the fast and efficient projection dictionary pair learning (PDPL) technique. PDPL involves the joint use of a synthesis dictionary and an analysis dictionary to enhance the representation of features. Additionally, to optimize the parameters of PDPL, which depend on experience, we applied the genetic algorithm (GA) to obtain the optimal solution for the model. We validated the effectiveness of our algorithm using leave-one-subject-out cross validation on three EEG emotion databases: SEED, MPED, and GAMEEMO. Our approach outperformed traditional machine learning methods, achieving an average accuracy of 69.89% on the SEED database, 24.11% on the MPED database, 64.34% for the two-class GAMEEMO, and 49.01% for the four-class GAMEEMO. These results highlight the potential of subject-independent EEG emotion recognition algorithms in the development of intelligent systems capable of recognizing and responding to human emotions in real-world scenarios.

Manifolds-Based Low-Rank Dictionary Pair Learning for Efficient Set-Based Video Recognition

As an important research direction in image and video processing, set-based video recognition requires speed and accuracy. However, the existing static modeling methods focus on computational speed but ignore accuracy, whereas the dynamic modeling methods are higher-accuracy but ignore the computational speed. Combining these two types of methods to obtain fast and accurate recognition results remains a challenging problem. Motivated by this, in this study, a novel Manifolds-based Low-Rank Dictionary Pair Learning (MbLRDPL) method was developed for a set-based video recognition/image set classification task. Specifically, each video or image set was first modeled as a covariance matrix or linear subspace, which can be seen as a point on a Riemannian manifold. Second, the proposed MbLRDPL learned discriminative class-specific synthesis and analysis dictionaries by clearly imposing the nuclear norm on the synthesis dictionaries. The experimental results show that our method achieved the best classification accuracy (100%, 72.16%, 95%) on three datasets with the fastest computing time, reducing the errors of state-of-the-art methods (JMLC, DML, CEBSR) by 0.96–75.69%.

Periodicity-Aware Signal Denoising Using Capon-Optimized Ramanujan Filter Banks and Pruned Ramanujan Dictionaries

We propose a ‘periodicity-aware’ hybrid analysis-synthesis framework for denoising discrete-time periodic signals. Our method uses Ramanujan filter banks (RFB) for analysis and dictionaries for synthesis. The synthesis dictionary retains appropriate subspaces for signal reconstruction, by pruning the Ramanujan dictionary based on the outputs of the RFB. Unlike the other existing denoising methods, a unique advantage of the proposed method is that the denoised output is guaranteed to be composed of integer-periodic components with periods smaller than a pre-selected value. Our method works well even when the signal length is small, and has a high SNR gain over a wide range of input signal SNRs. Furthermore, we propose to adapt each filter in the analysis bank to the incoming data, by optimizing the filter coefficients through a multi-band Capon formulation. This helps in suppressing the spurious energy peaks generated from higher period filters in the analysis bank, further improving the denoising performance. Implementing multiband Capon filters requires inverses of several autocorrelation matrices. To reduce computations, a way to recursively compute these inverses based on Levinson's recursion is discussed. Next, we prove several multirate properties of Ramanujan subspace signals. An important property among these is that after decimation, a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$q$</tex-math></inline-formula> -th Ramanujan subspace signal still remains in the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$q$</tex-math></inline-formula> -th Ramanujan space, if and only if the decimation rate <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$M$</tex-math></inline-formula> is coprime to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$q$</tex-math></inline-formula> . This is helpful to further reduce computations required in the analysis part of the denoising framework by downsampling the filter outputs. Extensive Monte-Carlo simulations comparing different variants of the proposed method and several existing denoising methods are also provided.

Support vector machine embedding discriminative dictionary pair learning for pattern classification

Discriminative dictionary learning (DDL) aims to address pattern classification problems via learning dictionaries from training samples. Dictionary pair learning (DPL) based DDL has shown superiority as compared with most existing algorithms which only learn synthesis dictionaries or analysis dictionaries. However, in the original DPL algorithm, the discrimination capability is only promoted via the reconstruction error and the structures of the learned dictionaries, while the discrimination of coding coefficients is not considered in the process of dictionary learning. To address this issue, we propose a new DDL algorithm by introducing an additional discriminative term associated with coding coefficients. Specifically, a support vector machine (SVM) based term is employed to enhance the discrimination of coding coefficients. In this model, a structured dictionary pair and SVM classifiers are jointly learned, and an optimization method is developed to address the formulated optimization problem. A classification scheme based on both the reconstruction error and SVMs is also proposed. Simulation results on several widely used databases demonstrate that the proposed method can achieve competitive performance as compared with some state-of-the-art DDL algorithms.

Transfer Discriminative Dictionary Pair Learning Approach for Across-Subject EEG Emotion Classification.

Electroencephalogram (EEG) signals are not easily camouflaged, portable, and noninvasive. It is widely used in emotion recognition. However, due to the existence of individual differences, there will be certain differences in the data distribution of EEG signals in the same emotional state of different subjects. To obtain a model that performs well in classifying new subjects, traditional emotion recognition approaches need to collect a large number of labeled data of new subjects, which is often unrealistic. In this study, a transfer discriminative dictionary pair learning (TDDPL) approach is proposed for across-subject EEG emotion classification. The TDDPL approach projects data from different subjects into the domain-invariant subspace, and builds a transfer dictionary pair learning based on the maximum mean discrepancy (MMD) strategy. In the subspace, TDDPL learns shared synthesis and analysis dictionaries to build a bridge of discriminative knowledge from source domain (SD) to target domain (TD). By minimizing the reconstruction error and the inter-class separation term for each sub-dictionary, the learned synthesis dictionary is discriminative and the learned low-rank coding is sparse. Finally, a discriminative classifier in the TD is constructed on the classifier parameter, analysis dictionary and projection matrix, without the calculation of coding coefficients. The effectiveness of the TDDPL approach is verified on SEED and SEED IV datasets.

Self-paced non-convex regularized analysis–synthesis dictionary learning for unsupervised feature selection

Due to the ability to prevent over-fitting, reduce computational complexity and storage cost, and enhance interpretability, unsupervised feature selection (UFS) has received widespread attention in a large number of application scenarios. However, a majority of the existing UFS methods cannot make full use of the diversity between data points, which usually leads to a suboptimal solution. In this paper, a novel UFS approach, referred to as self-paced analysis–synthesis dictionary learning (SPASDL) is proposed by integrating dictionary learning and self-paced learning into a joint framework. Specifically, we reconstruct and code data with the synthesis dictionary and analysis dictionary, respectively. This strategy can avoid using pseudo labels of data, which may mislead the feature selection process. Further, we introduce self-paced learning into dictionary learning to realize the reconstruction of data points from easy to hard in the training process. To identify more discriminative features, we impose the non-convex sparse constraint on the synthesis dictionary. For the purpose of optimizing the resulting model, we exploit an efficient iterative optimization algorithm based on alternative search strategy (ASS). Besides, the theoretical results on the optimization algorithm, including the convergence analysis and computational complexity, are investigated. Finally, experimental results on public available real-world datasets demonstrate the effectiveness and the superiority of the proposed method.

Power Quality Disturbance Signal Denoising Based on Overcomplete Representation

A denoising method for power quality (PQ) disturbance signal based on optimized decomposition of overcomplete dictionary is proposed. First, based on the idea of sparse decomposition, an overcomplete synthesis dictionary matching the time‐frequency characteristics of PQ disturbance signal is constructed. Then, the matching pursuit (MP) algorithm is used to search the disturbance parameters in Gabor dictionary, and the improved particle swarm optimization (PSO) algorithm is combined with the improved Nelder–Mead simplex extremum algorithm to improve the search speed and accuracy. Finally, the obtained disturbance parameters and the overcomplete synthesis dictionary are used to extract the disturbance characteristic waveform, and then the noise is separated. This method is used to reduce the noisy PQ signals and compared with the results of wavelet and other noise reduction methods, the results show that the method can effectively denoise PQ signal, and better retain the original characteristics of PQ disturbances. © 2021 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.

Efficient and robust discriminant dictionary pair learning for pattern classification

Dictionary learning has been widely applied in the field of pattern classification. Although many analysis-synthesis dictionary pair learning methods have been proposed in recent years, most of them ignore the influence of noises and training their model is usually time-consuming. To overcome these issues, we propose an efficient and robust discriminant analysis-synthesis dictionary pair learning (ERDDPL) method for pattern classification, which can efficiently learn a structured analysis-synthesis dictionary pair with powerful discrimination and representation capabilities. Specifically, we design a coding coefficient discriminant term to ensure that the coding coefficient matrix has an approximate block diagonal structure, which can enhance the discrimination capability of the structured analysis dictionary. To weaken the influence of noises on the structured synthesis dictionary, we impose a low-rank constraint on each synthesis sub-dictionary. Besides, we develop a fast and efficient iterative algorithm to solve the optimization problem of ERDDPL. Extensive experiments on five image datasets sufficiently verify that our method has higher classification accuracy and efficiency than the state-of-the-art dictionary learning methods.

Relaxed Block-Diagonal Dictionary Pair Learning With Locality Constraint for Image Recognition.

We propose a novel structured analysis-synthesis dictionary pair learning method for efficient representation and image classification, referred to as relaxed block-diagonal dictionary pair learning with a locality constraint (RBD-DPL). RBD-DPL aims to learn relaxed block-diagonal representations of the input data to enhance the discriminability of both analysis and synthesis dictionaries by dynamically optimizing the block-diagonal components of representation, while the off-block-diagonal counterparts are set to zero. In this way, the learned synthesis subdictionary is allowed to be more flexible in reconstructing the samples from the same class, and the analysis dictionary effectively transforms the original samples into a relaxed coefficient subspace, which is closely associated with the label information. Besides, we incorporate a locality-constraint term as a complement of the relaxation learning to enhance the locality of the analytical encoding so that the learned representation exhibits high intraclass similarity. A linear classifier is trained in the learned relaxed representation space for consistent classification. RBD-DPL is computationally efficient because it avoids both the use of class-specific complementary data matrices to learn discriminative analysis dictionary, as well as the time-consuming l1/l0 -norm sparse reconstruction process. The experimental results demonstrate that our RBD-DPL achieves at least comparable or better recognition performance than the state-of-the-art algorithms. Moreover, both the training and testing time are significantly reduced, which verifies the efficiency of our method. The MATLAB code of the proposed RBD-DPL is available at https://github.com/chenzhe207/RBD-DPL.

Twin-Incoherent Self-Expressive Locality-Adaptive Latent Dictionary Pair Learning for Classification.

The projective dictionary pair learning (DPL) model jointly seeks a synthesis dictionary and an analysis dictionary by extracting the block-diagonal coefficients with an incoherence-constrained analysis dictionary. However, DPL fails to discover the underlying subspaces and salient features at the same time, and it cannot encode the neighborhood information of the embedded coding coefficients, especially adaptively. In addition, although the data can be well reconstructed via the minimization of the reconstruction error, useful distinguishing salient feature information may be lost and incorporated into the noise term. In this article, we propose a novel self-expressive adaptive locality-preserving framework: twin-incoherent self-expressive latent DPL (SLatDPL). To capture the salient features from the samples, SLatDPL minimizes a latent reconstruction error by integrating the coefficient learning and salient feature extraction into a unified model, which can also be used to simultaneously discover the underlying subspaces and salient features. To make the coefficients block diagonal and ensure that the salient features are discriminative, our SLatDPL regularizes them by imposing a twin-incoherence constraint. Moreover, SLatDPL utilizes a self-expressive adaptive weighting strategy that uses normalized block-diagonal coefficients to preserve the locality of the codes and salient features. SLatDPL can use the class-specific reconstruction residual to handle new data directly. Extensive simulations on several public databases demonstrate the satisfactory performance of our SLatDPL compared with related methods.

Structured discriminant analysis dictionary learning for pattern classification

Dictionary learning has been widely used in the field of pattern recognition. Although the existing synthesis dictionary learning methods have achieved impressive results, they need compute sparse codes using a time-consuming sparse coding procedure. As a dual viewpoint of synthesis dictionary learning, analysis dictionary learning (ADL) has attracted much attention due to its high efficiency and intuitive meaning in recent years. However, how to associate analysis dictionary atoms with class labels and learn a structured discriminant analysis dictionary, is still a challenging problem. In this paper, we propose a structured discriminant analysis dictionary learning (SDADL) method to learn a structured discriminant analysis dictionary that consists of the class-specific analysis sub-dictionaries associated with the corresponding classes. Specifically, we introduce a classification error term into SDADL model to learn an optimal linear classifier for classification. To obtain the discriminant analysis sparse codes, we introduce a discriminant analysis sparse code error term into SDADL model, which forces the samples from the same class to have similar analysis sparse codes. Moreover, we also introduce a structured discriminant term into SDADL model to improve the discrimination capability of both each class-specific analysis sub-dictionary and analysis sparse codes. An efficient iterative algorithm is also developed to solve the optimization problem of SDADL. In addition, we design a novel scheme for classification. Extensive experiments on five image datasets verify the effectiveness of SDADL for pattern classification.

A Novel Discriminative Dictionary Pair Learning Constrained by Ordinal Locality for Mixed Frequency Data Classification

A dilemma faced by classification is that the data is not collected at the same frequency in some applications. We investigate the mixed frequency data in a new way and recognize them as a special style of multi-view data, in which each view data is collected at a different sampling frequency. This article proposes a discriminative dictionary pair learning method constrained by ordinal locality for mixed frequency data classification (shorted by DPLOL-MF). This method integrates synthesis dictionary and analysis dictionary into a dictionary pair, which not only improves computational cost caused by the <inline-formula><tex-math notation="LaTeX">${\ell _0}$</tex-math></inline-formula> or <inline-formula><tex-math notation="LaTeX">${\ell _1}$</tex-math></inline-formula> -norm constraint, but also can deal with the sampling frequency inconsistency. The DPLOL-MF utilizes a synthesis dictionary to learn class-specified reconstruction information and employs an analysis dictionary to generate coding coefficients by analyzing samples. Particularly, the ordinal locality preserving term is leveraged to constrain the atoms of dictionaries pair to further facilitate the learned dictionary pair to be more discriminative. Besides, we design a specific classification scheme for the inconsistent sample size of mixed frequency data. This paper illustrates a novel idea to solve the classification task of mixed frequency data and the experimental results demonstrate the effectiveness of the proposed method.

DISCRIMINATIVE DICTIONARY PAIR LEARNING FOR IMAGE CLASSIFICATION

Dictionary learning (DL) for sparse coding has been widely applied in the field of computer vision. Many DL approaches have been developed recently to solve pattern classification problems and have achieved promising performance. In this paper, to improve the discriminability of the popular dictionary pair learning (DPL) algorithm, we propose a new method called discriminative dictionary pair learning (DDPL) for image classification. To achieve the goal of signal representation and discrimination, we impose the incoherence constraints on the synthesis dictionary and the low-rank regularization on the analysis dictionary. The DDPL method ensures that the learned dictionary has the powerful discriminative ability and the signals are more separable after coding. We evaluate the proposed method on benchmark image databases in comparison with existing DL methods. The experimental results demonstrate that our method outperforms many recently proposed dictionary learning approaches.

Low‐Rank Combined Adaptive Sparsifying Transform for Blind Compressed Sensing Image Recovery

Compressed sensing (CS) exploits the sparsity of images or image patches in a transform domain or synthesis dictionary to reconstruct images from undegraded images. Because the synthesis dictionary learning methods involves NP-hard sparse coding and expensive learning steps, sparsifying transform based blind compressed sending (BCS) has been shown to be effective and efficient in applications, while also enjoying good convergence guarantees. By minimizing the rank of an overlapped patch group matrix to efficiently exploit the nonlocal self-similarity features of the image, while the sparsifying transform model imposes the local features of the image. We propose a combined low-rank and adaptive sparsifying transform (LRAST) BCS method to better represent natural images. We utilized the patch coordinate (PCD) descent algorithm to optimize the method, and this enforced the intrinsic local sparsity and nonlocal self-similarity of the images simultaneously in a unified framework. The experimental results indicated a promising performance, even in comparison to state-of-theart methods.

Combining synthesis sparse with analysis sparse for single image super-resolution

Sparse coding based-methods show great effectiveness in single image super-resolution (SR). Existing methods generally use only synthesis sparse coding. However, the analysis sparse coding model, which is an alternative to the synthesis sparse coding model, is typically neglected In this paper, we propose a novel single image SR method by combining synthesis sparse coding with analysis sparse coding. In contrast to the existing sparse coding-based SR methods, we replace synthesis sparse coding with analysis sparse coding in the low-resolution (LR) feature representation phase. Thus, we use an analysis dictionary to obtain the LR coefficients and use a synthesis dictionary to reconstruct the high-resolution (HR) patches. Due to the introducing of analysis sparse representation, the l0 or l1 optimization is replaced by soft threshold shrinkage, which is a more time-saving method. To improve the convergence of the training model, we introduce a linear mapping function that reveals the relationship between the HR coefficients and LR coefficients. An alternating optimization strategy is adopted to solve the improved training model. Furthermore, global and nonlocal constraints are taken into account to improve the quality of the reconstructed images. Compared with some existing SR methods, the experimental results demonstrate that our proposed method can obtain better performance.

Learning Hybrid Representation by Robust Dictionary Learning in Factorized Compressed Space.

In this paper, we investigate the robust dictionary learning (DL) to discover the hybrid salient low-rank and sparse representation in a factorized compressed space. A Joint Robust Factorization and Projective Dictionary Learning (J-RFDL) model is presented. The setting of J-RFDL aims at improving the data representations by enhancing the robustness to outliers and noise in data, encoding the reconstruction error more accurately and obtaining hybrid salient coefficients with accurate reconstruction ability. Specifically, J-RFDL performs the robust representation by DL in a factorized compressed space to eliminate the negative effects of noise and outliers on the results, which can also make the DL process efficient. To make the encoding process robust to noise in data, J-RFDL clearly uses sparse L2, 1-norm that can potentially minimize the factorization and reconstruction errors jointly by forcing rows of the reconstruction errors to be zeros. To deliver salient coefficients with good structures to reconstruct given data well, J-RFDL imposes the joint low-rank and sparse constraints on the embedded coefficients with a synthesis dictionary. Based on the hybrid salient coefficients, we also extend J-RFDL for the joint classification and propose a discriminative J-RFDL model, which can improve the discriminating abilities of learnt coefficients by minimizing the classification error jointly. Extensive experiments on public datasets demonstrate that our formulations can deliver superior performance over other state-of-the-art methods.

Classification-Friendly Sparse Encoder and Classifier Learning

Sparse representation (SR) and dictionary learning (DL) have been extensively used for feature encoding, aiming to extract the latent classification-friendly feature of observed data. Existing methods use sparsity penalty and learned dictionary to enhance discriminative capability of sparse codes. However, training dictionary for SR is time consuming and the resulted discriminative capability is limited. Rather than learning dictionary, we propose to employ the dictionary at hand, e.g., the training set as the class-specific synthesis dictionary to pursue an ideal discriminative property of SR of the training samples: each data can be represented only by data-in-class. In addition to the discriminative property, we also introduce a smoothing term to enforce the representation vectors to be uniform within class. The discriminative property helps to separate the data from different classes while the smoothing term tends to group the data from the same class and further strengthen the separation. The SRs are used as new features to train a sparse encoder and a classifier. Once the sparse encoder and the classifier are learnt, the test stage is very simple and highly efficient. Specifically, the label of a test sample can be easily computed by multiplying the test sample with the sparse encoder and the classifier. We call our method Classification-Friendly Sparse Encoder and Classifier Learning (CF-SECL). Extensive experiments show that our method outperforms some state-of the-art model-based methods.

Class-Specific Sparse Principal Component Analysis for Visual Classification

Extensive research has demonstrated that dictionary learning is active in improving the performance of the representation based classification. However, dictionary learning suffers from lacking an effective dictionary structure that can well tradeoff the reducing reconstruction error and enhancing the representative information. In this paper, we focus on designing capable dictionary learning architecture for the visual classification task with few-shot training samples. First, we propose a class-specific sparse PCA approach to extend the conventional dictionary learning to dictionary pair learning for visual classification. The orthogonal synthesis dictionary ensures fewer reconstruction errors with fixed dictionary size while the sparse analysis dictionary guarantees the representative features. Second, we develop the Alternating Direction Method of Multipliers algorithm to optimize the analysis dictionary to improve computational efficiency. Extensive experiments conducted on multiple image datasets demonstrate that the proposed class-specific PCA method performs favorably against the state-of-art methods on eight widely used image datasets.

Learning Deep Analysis Dictionaries for Image Super-Resolution

Inspired by the recent success of deep neural networks and the recent efforts to develop multi-layer dictionary models, we propose a Deep Analysis dictionary Model (DeepAM) which is optimized to address a specific regression task known as single image super-resolution. Contrary to other multi-layer dictionary models, our architecture contains L layers of analysis dictionary and soft-thresholding operators to gradually extract high-level features and a layer of synthesis dictionary which is designed to optimize the regression task at hand. In our approach, each analysis dictionary is partitioned into two sub-dictionaries: an Information Preserving Analysis Dictionary (IPAD) and a Clustering Analysis Dictionary (CAD). The IPAD together with the corresponding soft-thresholds is designed to pass the key information from the previous layer to the next layer, while the CAD together with the corresponding soft-thresholding operator is designed to produce a sparse feature representation of its input data that facilitates discrimination of key features. DeepAM uses both supervised and unsupervised setup. Simulation results show that the proposed deep analysis dictionary model achieves better performance compared to a deep neural network that has the same structure and is optimized using back-propagation when training datasets are small.