Synthesis Dictionary Learning Research Articles

Self-paced non-convex regularized analysis–synthesis dictionary learning for unsupervised feature selection

Due to the ability to prevent over-fitting, reduce computational complexity and storage cost, and enhance interpretability, unsupervised feature selection (UFS) has received widespread attention in a large number of application scenarios. However, a majority of the existing UFS methods cannot make full use of the diversity between data points, which usually leads to a suboptimal solution. In this paper, a novel UFS approach, referred to as self-paced analysis–synthesis dictionary learning (SPASDL) is proposed by integrating dictionary learning and self-paced learning into a joint framework. Specifically, we reconstruct and code data with the synthesis dictionary and analysis dictionary, respectively. This strategy can avoid using pseudo labels of data, which may mislead the feature selection process. Further, we introduce self-paced learning into dictionary learning to realize the reconstruction of data points from easy to hard in the training process. To identify more discriminative features, we impose the non-convex sparse constraint on the synthesis dictionary. For the purpose of optimizing the resulting model, we exploit an efficient iterative optimization algorithm based on alternative search strategy (ASS). Besides, the theoretical results on the optimization algorithm, including the convergence analysis and computational complexity, are investigated. Finally, experimental results on public available real-world datasets demonstrate the effectiveness and the superiority of the proposed method.

Structured discriminant analysis dictionary learning for pattern classification

Dictionary learning has been widely used in the field of pattern recognition. Although the existing synthesis dictionary learning methods have achieved impressive results, they need compute sparse codes using a time-consuming sparse coding procedure. As a dual viewpoint of synthesis dictionary learning, analysis dictionary learning (ADL) has attracted much attention due to its high efficiency and intuitive meaning in recent years. However, how to associate analysis dictionary atoms with class labels and learn a structured discriminant analysis dictionary, is still a challenging problem. In this paper, we propose a structured discriminant analysis dictionary learning (SDADL) method to learn a structured discriminant analysis dictionary that consists of the class-specific analysis sub-dictionaries associated with the corresponding classes. Specifically, we introduce a classification error term into SDADL model to learn an optimal linear classifier for classification. To obtain the discriminant analysis sparse codes, we introduce a discriminant analysis sparse code error term into SDADL model, which forces the samples from the same class to have similar analysis sparse codes. Moreover, we also introduce a structured discriminant term into SDADL model to improve the discrimination capability of both each class-specific analysis sub-dictionary and analysis sparse codes. An efficient iterative algorithm is also developed to solve the optimization problem of SDADL. In addition, we design a novel scheme for classification. Extensive experiments on five image datasets verify the effectiveness of SDADL for pattern classification.

Low‐Rank Combined Adaptive Sparsifying Transform for Blind Compressed Sensing Image Recovery

Compressed sensing (CS) exploits the sparsity of images or image patches in a transform domain or synthesis dictionary to reconstruct images from undegraded images. Because the synthesis dictionary learning methods involves NP-hard sparse coding and expensive learning steps, sparsifying transform based blind compressed sending (BCS) has been shown to be effective and efficient in applications, while also enjoying good convergence guarantees. By minimizing the rank of an overlapped patch group matrix to efficiently exploit the nonlocal self-similarity features of the image, while the sparsifying transform model imposes the local features of the image. We propose a combined low-rank and adaptive sparsifying transform (LRAST) BCS method to better represent natural images. We utilized the patch coordinate (PCD) descent algorithm to optimize the method, and this enforced the intrinsic local sparsity and nonlocal self-similarity of the images simultaneously in a unified framework. The experimental results indicated a promising performance, even in comparison to state-of-theart methods.

Image Recovery via Transform Learning and Low-Rank Modeling: The Power of Complementary Regularizers.

Recent works on adaptive sparse and on low-rank signal modeling have demonstrated their usefulness in various image/video processing applications. Patch-based methods exploit local patch sparsity, whereas other works apply low-rankness of grouped patches to exploit image non-local structures. However, using either approach alone usually limits performance in image reconstruction or recovery applications. In this work, we propose a simultaneous sparsity and low-rank model, dubbed STROLLR, to better represent natural images. In order to fully utilize both the local and non-local image properties, we develop an image restoration framework using a transform learning scheme with joint low-rank regularization. The approach owes some of its computational efficiency and good performance to the use of transform learning for adaptive sparse representation rather than the popular synthesis dictionary learning algorithms, which involve approximation of NP-hard sparse coding and expensive learning steps. We demonstrate the proposed framework in various applications to image denoising, inpainting, and compressed sensing based magnetic resonance imaging. Results show promising performance compared to state-of-the-art competing methods.

Low-rank analysis–synthesis dictionary learning with adaptively ordinal locality

Analysis dictionary learning (ADL) has been successfully applied to a variety of learning systems. However, the ordinal locality of analysis dictionary has rarely been explored in constructing discriminative terms. In this paper, a discriminative low-rank analysis–synthesis dictionary learning (LR-ASDL) algorithm with the adaptively ordinal locality is proposed for object classification. Specifically, we first explicitly introduce the relations between the analysis atoms and profiles (i.e., row vectors of the coefficients matrix). That is, the similarity between two profiles depends on that between the corresponding analysis atoms. Moreover, an adaptively ordinal locality preserving(AOLP) term is constructed by simultaneously exploiting the profiles and analysis atoms, which can be learned in a supervised way. In this way, the neighborhood correlations between analysis atoms and the high-order ranking information of each analysis atom’s neighbors can be simultaneously preserved in the learning process. Particularly, this helps to uncover the intrinsic underlying data factors and inherit the geometry structure information of training samples. Furthermore, the low-rank model is imposed on the synthesis atoms to further facilitate the learned dictionaries to be more discriminative. Extensive experimental results on eight databases demonstrate that the LR-ASDL algorithm clearly outperforms some analysis and synthesis dictionary learning algorithms using deep and hand-crafted features.

Online single and multiple analysis dictionary learning-based approach for visual object tracking

In recent years, various synthesis dictionary learning-based techniques have been applied for the task of visual object tracking. However, there are challenges in the representation ability of the target and the background using a synthesis dictionary. Owing to such challenges, the target in a given video sequence is not tracked very efficiently. We propose an online analysis dictionary learning framework for visual tracking. Our main motivation stems from the fact that analysis dictionary is able to capture significantly more variability in the data as compared to a synthesis dictionary of the same dimensions. We make use of single and multiple analysis dictionaries in our work. In order to leverage the advantage of multiple dictionaries, we adaptively fuse them. To the best of our knowledge, this is the first work that uses analysis dictionary for visual tracking. We use the entire OTB-50 dataset to evaluate the performance of proposed trackers. Our experiments show a significant improvement over existing synthesis dictionary learning approaches. This makes our approach much more preferable over the conventional approaches that use synthesis dictionary learning model for visual tracking.

VIDOSAT: High-Dimensional Sparsifying Transform Learning for Online Video Denoising.

Techniques exploiting the sparsity of images in a transform domain are effective for various applications in image and video processing. In particular, transform learning methods involve cheap computations and have been demonstrated to perform well in applications, such as image denoising and medical image reconstruction. Recently, we proposed methods for online learning of sparsifying transforms from streaming signals, which enjoy good convergence guarantees and involve lower computational costs than online synthesis dictionary learning. In this paper, we apply online transform learning to video denoising. We present a novel framework for online video denoising based on high-dimensional sparsifying transform learning for spatio-temporal patches. The patches are constructed either from corresponding 2D patches in successive frames or using an online block matching technique. The proposed online video denoising requires little memory and offers efficient processing. Numerical experiments evaluate the performance of the proposed video denoising algorithms on multiple video data sets. The proposed methods outperform several related and recent techniques, including denoising with 3D DCT, prior schemes based on dictionary learning, non-local means, background separation, and deep learning, as well as the popular VBM3D and VBM4D.

Weakly Supervised Dictionary Learning

We present a probabilistic modeling and inference framework for discriminative analysis dictionary learning under a weak supervision setting. Dictionary learning approaches have been widely used for tasks such as low-level signal denoising and restoration as well as high-level classification tasks, which can be applied to audio and image analysis. Synthesis dictionary learning aims at jointly learning a dictionary and corresponding sparse coefficients to provide accurate data representation. This approach is useful for denoising and signal restoration, but may lead to sub-optimal classification performance. By contrast, analysis dictionary learning provides a transform that maps data to a sparse discriminative representation suitable for classification. We consider the problem of analysis dictionary learning for time-series data under a weak supervision setting in which signals are assigned with a global label instead of an instantaneous label signal. We propose a discriminative probabilistic model that incorporates both label information and sparsity constraints on the underlying latent instantaneous label signal using cardinality control. We present the expectation maximization (EM) procedure for maximum likelihood estimation (MLE) of the proposed model. To facilitate a computationally efficient E-step, we propose both a chain and a novel tree graph reformulation of the graphical model. The performance of the proposed model is demonstrated on both synthetic and real-world data.

Synthesis K-SVD based analysis dictionary learning for pattern classification

In the fields of computer vision and pattern recognition, dictionary learning techniques have been widely applied. In classification tasks, synthesis dictionary learning is usually time-consuming during the classification stage because of the sparse reconstruction procedure. Analysis dictionary learning, which is another research line, is more favorable due to its flexible representative ability and low classification complexity. In this paper, we propose a novel discriminative analysis dictionary learning method to enhance classification performance. Particularly, we incorporate a linear classifier and the supervised information into the traditional analysis dictionary learning framework by adding a discrimination error term. A synthesis K-SVD based algorithm which can effectively constrain the sparsity is presented to solve the proposed model. Extensive comparison experiments on benchmark databases validate the satisfactory performance of our method.

FRIST—flipping and rotation invariant sparsifying transform learning and applications

Features based on sparse representation, especially using the synthesis dictionary model, have been heavily exploited in signal processing and computer vision. However, synthesis dictionary learning typically involves NP-hard sparse coding and expensive learning steps. Recently, sparsifying transform learning received interest for its cheap computation and its optimal updates in the alternating algorithms. In this work, we develop a methodology for learning flipping and rotation invariant sparsifying transforms, dubbed FRIST, to better represent natural images that contain textures with various geometrical directions. The proposed alternating FRIST learning algorithm involves efficient optimal updates. We provide a convergence guarantee, and demonstrate the empirical convergence behavior of the proposed FRIST learning approach. Preliminary experiments show the promising performance of FRIST learning for sparse image representation, segmentation, denoising, robust inpainting, and compressed sensing-based magnetic resonance image reconstruction.

Discriminative analysis-synthesis dictionary learning for image classification

Dictionary learning has played an important role in the success of sparse representation. Although discriminative synthesis dictionary learning for sparse representation with a high-computational-complexity l0 or l1 norm constraint has been well studied for image classification, jointly and discriminatively learning an analysis dictionary and a synthesis dictionary is still in its infant stage. As a dual of synthesis dictionary, the recently developed analysis dictionary can provide a complementary view of data representation, which can have a much lower time complexity than sparse synthesis representation. Although several class-specific analysis-synthesis dictionary, which may have a big correlation between different classes' dictionaries, have been developed, how to learn a more compact and discriminative universal analysis-synthesis dictionary is still open. In this paper, to provide a more complete view of discriminative data representation, we propose a novel model of discriminative analysis-synthesis dictionary learning (DASDL), in which a linear classifier based on the coding coefficient is jointly learned with the dictionary pair, thus the performance of the classifier and the representational power of the dictionary pair being considered at the same time by the same optimization procedure. The size of the learned dictionaries can be very small since the analysis-synthesis dictionary is shared by all class data. An iterative algorithm to efficiently solve the proposed DASDL is presented in this paper. The experiments on face recognition, gender classification, action recognition and image classification clearly show the superiority of the proposed DASDL.

Analysis-Synthesis Dictionary Learning for Universality-Particularity Representation Based Classification

Dictionary learning has played an important role in the success of sparse representation. Although synthesis dictionary learning for sparse representation has been well studied for universality representation (i.e., the dictionary is universal to all classes) and particularity representation (i.e., the dictionary is class-particular), jointly learning an analysis dictionary and a synthesis dictionary is still in its infant stage. Universality-particularity representation can well match the intrinsic characteristics of data (i.e., different classes share commonality and distinctness), while analysis-synthesis dictionary can give a more complete view of data representation (i.e., analysis dictionary is a dual-viewpoint of synthesis dictionary). In this paper, we proposed a novel model of analysis-synthesis dictionary learning for universality-particularity (ASDL-UP) representation based classification. The discrimination of universality and particularity representation is jointly exploited by simultaneously learning a pair of analysis dictionary and synthesis dictionary. More specifically, we impose a label preserving term to analysis coding coefficients for universality representation. Fisher-like regularizations for analysis coding coefficients and the subsequent synthesis representation are introduced to particularity representation. Compared with other state-of-the-art dictionary learning methods, ASDL-UP has shown better or competitive performance in various classification tasks.

Online Sparsifying Transform Learning—Part I: Algorithms

Techniques exploiting the sparsity of signals in a transform domain or dictionary have been popular in signal processing. Adaptive synthesis dictionaries have been shown to be useful in applications such as signal denoising, and medical image reconstruction. More recently, the learning of sparsifying transforms for data has received interest. The sparsifying transform model allows for cheap and exact computations. In this paper, we develop a methodology for online learning of square sparsifying transforms. Such online learning can be particularly useful when dealing with big data, and for signal processing applications such as real-time sparse representation and denoising. The proposed transform learning algorithms are shown to have a much lower computational cost than online synthesis dictionary learning. In practice, the sequential learning of a sparsifying transform typically converges faster than batch mode transform learning. Preliminary experiments show the usefulness of the proposed schemes for sparse representation, and denoising.