Supervised Dictionary Learning Method Research Articles

Structured dictionary learning using mixed-norms and group-sparsity constraint

Recently, sparse representation and dictionary learning have shown significant performance in machine vision. In particular, several supervised dictionary learning methods have been proposed for classification aim and increasing its accuracy. Among them, structured dictionary learning is an interesting approach which captures the discriminative properties of each class and common features among all classes in class-specific sub-dictionaries and a distinct shared sub-dictionary, respectively. It extracts the structural information that exists in samples of each class to increase the classification accuracy. Therefore, in this paper, a group-based structured dictionary learning method is proposed that captures structural information in each class and learns class-specific and shared sub-dictionaries based on mixed $$l_{2,1}$$ norm. Also, mixed $$l_{2,1}$$ norm is used for acquiring the sparse coefficients of data samples based on the learned sub-dictionaries. Then, classification is done by finding the class with (1) minimum reconstruction error or (2) maximum number of nonzero groups based on $$l_{1,0}$$ norm. The proposed method is evaluated by conducting experiments on Extended YaleB, AR and CMU-PIE face databases and the USPS handwritten digits database. The experimental results demonstrate the effectiveness of the proposed method in data representation and classification.

Supervised Dictionary Learning With Regularization for Near-Infrared Spectroscopy Classification

Near-infrared spectroscopy (NIRS) has been widely used in many fields due to its advantages with fast analysis speed, non-destructive testing, and on-site detection. However, NIRS has some shortcomings, such as low signal-to-noise ratio, weak absorption intensity, and overlapping peaks. The research of near-infrared spectral modeling method becomes the core of analyzing NIRS. In order to improve the accuracy of prediction model for NIRS, this paper proposes a novel sparse classification mechanism by designing appropriate regularization factors. The existing supervised dictionary learning methods have been proposed for classification aim and increasing its accuracy, the proposed method addresses some defects existing in this area through designing the representation-constrained term and the coefficients incoherence term, and the added two terms can get the reconstruction error of coding coefficients and correlations between similar samples by sharing dictionary under more stable control. Then, based on the proposed model, a supervised class-specific dictionary learning algorithm is developed by choosing appropriate samples with class labels. Finally, a classification scheme integrating the novel sparse model is designed to exploit such discriminative information. The proposed method is evaluated by conducting experiments on drug and tobacco leaves NIRS datasets. The experimental results show that the proposed sparse classification mechanism is promising for classifying NIRS and may be an alternative method to the traditional ones.

Discriminative Semi-Supervised Dictionary Learning with Entropy Regularization for Pattern Classification

Dictionary learning has played an important role in the success of sparse representation, which triggers the rapid developments of unsupervised and supervised dictionary learning methods. However, in most practical applications, there are usually quite limited labeled training samples while it is relatively easy to acquire abundant unlabeled training samples. Thus semi-supervised dictionary learning that aims to effectively explore the discrimination of unlabeled training data has attracted much attention of researchers. Although various regularizations have been introduced in the prevailing semi-supervised dictionary learning, how to design an effective unified model of dictionary learning and unlabeled-data class estimating and how to well explore the discrimination in the labeled and unlabeled data are still open. In this paper, we propose a novel discriminative semi-supervised dictionary learning model (DSSDL) by introducing discriminative representation, an identical coding of unlabeled data to the coding of testing data final classification, and an entropy regularization term. The coding strategy of unlabeled data can not only avoid the affect of its incorrect class estimation, but also make the learned discrimination be well exploited in the final classification. The introduced regularization of entropy can avoid overemphasizing on some uncertain estimated classes for unlabeled samples. Apart from the enhanced discrimination in the learned dictionary by the discriminative representation, an extended dictionary is used to mainly explore the discrimination embedded in the unlabeled data. Extensive experiments on face recognition, digit recognition and texture classification show the effectiveness of the proposed method.

Bilevel Model-Based Discriminative Dictionary Learning for Recognition.

Most supervised dictionary learning methods optimize the combinations of reconstruction error, sparsity prior, and discriminative terms. Thus, the learnt dictionaries may not be optimal for recognition tasks. Also, the sparse codes learning models in the training and the testing phases are inconsistent. Besides, without utilizing the intrinsic data structure, many dictionary learning methods only employ the l0 or l1 norm to encode each datum independently, limiting the performance of the learnt dictionaries. We present a novel bilevel model-based discriminative dictionary learning method for recognition tasks. The upper level directly minimizes the classification error, while the lower level uses the sparsity term and the Laplacian term to characterize the intrinsic data structure. The lower level is subordinate to the upper level. Therefore, our model achieves an overall optimality for recognition in that the learnt dictionary is directly tailored for recognition. Moreover, the sparse codes learning models in the training and the testing phases can be the same. We further propose a novel method to solve our bilevel optimization problem. It first replaces the lower level with its Karush-Kuhn-Tucker conditions and then applies the alternating direction method of multipliers to solve the equivalent problem. Extensive experiments demonstrate the effectiveness and robustness of our method.

Entropy Optimized Feature-Based Bag-of-Words Representation for Information Retrieval

In this paper, we present a supervised dictionary learning method for optimizing the feature-based Bag-of-Words (BoW) representation towards Information Retrieval. Following the cluster hypothesis, which states that points in the same cluster are likely to fulfill the same information need, we propose the use of an entropy-based optimization criterion that is better suited for retrieval instead of classification. We demonstrate the ability of the proposed method, abbreviated as EO-BoW, to improve the retrieval performance by providing extensive experiments on two multi-class image datasets. The BoW model can be applied to other domains as well, so we also evaluate our approach using a collection of 45 time-series datasets, a text dataset, and a video dataset. The gains are three-fold since the EO-BoW can improve the mean Average Precision, while reducing the encoding time and the database storage requirements. Finally, we provide evidence that the EO-BoW maintains its representation ability even when used to retrieve objects from classes that were not seen during the training.

Semi-Supervised Dictionary Learning via Structural Sparse Preserving

While recent techniques for discriminative dictionary learning have attained promising results on the classification tasks, their performance is highly dependent on the number of labeled samples available for training. However, labeling samples is expensive and time consuming due to the significant human effort involved. In this paper, we present a novel semi- supervised dictionary learning method which utilizes the structural sparse relationships between the labeled and unlabeled samples. Specifically, by connecting the sparse reconstruction coefficients on both the original samples and dictionary, the unlabeled samples can be automatically grouped to the different labeled samples, and the grouped samples share a small number of atoms in the dictionary via mixed l2p- norm regularization. This makes the learned dictionary more representative and discriminative since the shared atoms are learned by using the labeled and unlabeled samples potentially from the same class. Minimizing the derived objective function is a challenging task because it is non-convex and highly non-smooth. We propose an efficient optimization algorithm to solve the problem based on the block coordinate descent method. Moreover, we have a rigorous proof of the convergence of the algorithm. Extensive experiments are presented to show the superior performance of our method in classification applications.

Learning a structure adaptive dictionary for sparse representation based classification

Dictionary learning (DL), playing a key role in the success of sparse representation, has led to state-of-the-art results in image classification tasks. Among the existing supervised dictionary learning methods, the label of each dictionary atom is predefined and fixed, i.e., each dictionary atom is either associated to all classes or assigned to a single class. In this paper, we propose a structure adaptive dictionary learning (SADL) method to learn the relationship between dictionary atoms and classes, which is indicated by a binary association matrix and jointly optimized with the dictionary. The binary association matrix can not only represent class-specific dictionary atoms, but also hyper-class dictionary atoms shared by multiple classes. Furthermore, discrimination is explored by introducing Fisher criterion on coding coefficient and reducing between-class dictionary coherence. The extensive experimental results have shown that the proposed SADL can achieve better performance than previous supervised dictionary learning methods on various classification databases.

Multi-level discriminative dictionary learning with application to large scale image classification.

The sparse coding technique has shown flexibility and capability in image representation and analysis. It is a powerful tool in many visual applications. Some recent work has shown that incorporating the properties of task (such as discrimination for classification task) into dictionary learning is effective for improving the accuracy. However, the traditional supervised dictionary learning methods suffer from high computation complexity when dealing with large number of categories, making them less satisfactory in large scale applications. In this paper, we propose a novel multi-level discriminative dictionary learning method and apply it to large scale image classification. Our method takes advantage of hierarchical category correlation to encode multi-level discriminative information. Each internal node of the category hierarchy is associated with a discriminative dictionary and a classification model. The dictionaries at different layers are learnt to capture the information of different scales. Moreover, each node at lower layers also inherits the dictionary of its parent, so that the categories at lower layers can be described with multi-scale information. The learning of dictionaries and associated classification models is jointly conducted by minimizing an overall tree loss. The experimental results on challenging data sets demonstrate that our approach achieves excellent accuracy and competitive computation cost compared with other sparse coding methods for large scale image classification.

Task-Driven Dictionary Learning for Hyperspectral Image Classification With Structured Sparsity Constraints

Sparse representation models a signal as a linear combination of a small number of dictionary atoms. As a generative model, it requires the dictionary to be highly redundant in order to ensure both a stable high sparsity level and a low reconstruction error for the signal. However, in practice, this requirement is usually impaired by the lack of labelled training samples. Fortunately, previous research has shown that the requirement for a redundant dictionary can be less rigorous if simultaneous sparse approximation is employed, which can be carried out by enforcing various structured sparsity constraints on the sparse codes of the neighboring pixels. In addition, numerous works have shown that applying a variety of dictionary learning methods for the sparse representation model can also improve the classification performance. In this paper, we highlight the task-driven dictionary learning algorithm, which is a general framework for the supervised dictionary learning method. We propose to enforce structured sparsity priors on the task-driven dictionary learning method in order to improve the performance of the hyperspectral classification. Our approach is able to benefit from both the advantages of the simultaneous sparse representation and those of the supervised dictionary learning. We enforce two different structured sparsity priors, the joint and Laplacian sparsity, on the task-driven dictionary learning method and provide the details of the corresponding optimization algorithms. Experiments on numerous popular hyperspectral images demonstrate that the classification performance of our approach is superior to sparse representation classifier with structured priors or the task-driven dictionary learning method.

Combining affinity propagation with supervised dictionary learning for image classification

Recently support vector machines (SVM) using spatial pyramid matching (SPM) kernel have been highly successful in image classification applications. And linear spatial pyramid matching using sparse coding (ScSPM) scheme has been proposed to enhance the performance of SPM both in time and classification accuracy. In order to reduce the time complexity of dictionary construction process, sparse coding with affinity propagation method has been proposed in this paper. Because the dictionary used for sparse coding plays a key role in these methods, we also adopt supervised dictionary learning method to construct dictionary. The coding coefficients of each class have greater separability for SVM classification. Substantial experiments on Scene15 and CalTech101 image datasets have been conducted to investigate the performance of proposed approach in multi-class image classification; the results show that the approach can reach higher accuracy compared with ScSPM.