Dictionary Learning Framework Research Articles

Data-driven two-layer visual dictionary structure learning

An important issue in statistical modeling is to determine the complexity of the model based on the scale of data so as to effectively mitigate the model’s overfitting problems without big data. We adopt a data-driven approach to automatically determine the number of components of the model. In order to better extract robust features, we propose a framework of data-driven two-layer structure visual dictionary learning (DTSVDL). It works by dividing the visual dictionary structure learning into two levels: the attribute layer and the detail layer. In the attribute layer, the attributes of the image dataset are learned, and these attributes are obtained by a data-driven Bayesian nonparametric model. Then, in the detail layer, the detailed information over attributes is further explored and refined, and the attributes are weighted by the number of effective observations associated with each attribute. Our proposed approach has three main advantages: (1) the two-layer structure makes our building visual dictionary be more expressive; (2) the number of components in the attribute layer can be determined automatically from the data; (3) the components are automatically determined based on the scale of visual words; therefore, our model can well mitigate the overfitting problem. In addition, by comparing with stacked autoencoders, stacked denoising autoencoders, LeNet-5, speeded-up robust features, and pretrained deep learning model ImageNet-VGG-F algorithms, we find that our approach achieves satisfactory image categorization results on two benchmark datasets. Specifically, higher categorization performance is achieved than by the classical approaches on 15 scene categories and action datasets. We conclude that the resulting DTSVDL possesses a good generality derived from attribute information as well as an excellent distinction derived from detailed information. In other words, the visual dictionary learned by our algorithm is more expressive and discriminatory.

Online single and multiple analysis dictionary learning-based approach for visual object tracking

In recent years, various synthesis dictionary learning-based techniques have been applied for the task of visual object tracking. However, there are challenges in the representation ability of the target and the background using a synthesis dictionary. Owing to such challenges, the target in a given video sequence is not tracked very efficiently. We propose an online analysis dictionary learning framework for visual tracking. Our main motivation stems from the fact that analysis dictionary is able to capture significantly more variability in the data as compared to a synthesis dictionary of the same dimensions. We make use of single and multiple analysis dictionaries in our work. In order to leverage the advantage of multiple dictionaries, we adaptively fuse them. To the best of our knowledge, this is the first work that uses analysis dictionary for visual tracking. We use the entire OTB-50 dataset to evaluate the performance of proposed trackers. Our experiments show a significant improvement over existing synthesis dictionary learning approaches. This makes our approach much more preferable over the conventional approaches that use synthesis dictionary learning model for visual tracking.

Row-Sparse Discriminative Deep Dictionary Learning for Hyperspectral Image Classification

In recent studies in hyperspectral imaging, biometrics, and energy analytics, the framework of deep dictionary learning has shown promise. Deep dictionary learning outperforms other traditional deep learning tools when training data are limited; therefore, hyperspectral imaging is one such example that benefits from this framework. Most of the prior studies were based on the unsupervised formulation; and in all cases, the training algorithm was greedy and hence suboptimal. This is the first work that shows how to learn the deep dictionary learning problem in a joint fashion. Moreover, we propose a new discriminative penalty to the said framework. The third contribution of this work is showing how to incorporate stochastic regularization techniques into the deep dictionary learning framework. Experimental results on hyperspectral image classification shows that the proposed technique excels over all state-of-the-art deep and shallow (traditional) learning based methods published in recent times.

Unsupervised Joint Subspace and Dictionary Learning for Enhanced Cross-Domain Person Re-Identification

Person re-identification (Re-ID) has drawn increasing attention from both academia and industry due to its great potentials in surveillance applications. Most existing research efforts have attempted to tackle cross-view variation in single-domain person Re-ID. However, there is still a lack of effective approaches to cross-domain person Re-ID problem. In this paper, an unsupervised joint subspace and dictionary learning (UJSDL) framework is proposed to address the cross-domain person Re-ID problem, where both cross-view (i.e., across different cameras in the same network of cameras) and cross-domain (i.e., across different network of cameras) variation are jointly addressed. In particular, to reduce the impact of cross-view distribution variation, the graph Laplacian approach is used to project the images from different camera views in each domain into a shared subspace. To alleviate the impact of cross-domain distribution variation, a shared dictionary is learned from all the projection subspaces such that the discriminative information from both the labeled source datasets and the unlabeled target dataset are well encoded. To efficiently solve the joint subspace and dictionary learning task, an alternating optimization algorithm is presented. We used multiple different feature sets and conducted experiments on multiple benchmark datasets as the target domain. The results demonstrate that UJSDL outperforms the state-of-the-art approaches.

Unified No-Reference Quality Assessment of Singly and Multiply Distorted Stereoscopic Images.

A challenging problem in the no-reference quality assessment of multiply distorted stereoscopic images (MDSIs) is to simulate the monocular and binocular visual properties under a mixed type of distortions. Due to the joint effects of multiple distortions in MDSIs, the underlying monocular and binocular visual mechanisms have different manifestations with those of singly distorted stereoscopic images (SDSIs). This paper presents a unified no-reference quality evaluator for SDSIs and MDSIs by learning monocular and binocular local visual primitives (MB-LVPs). The main idea is to learn MB-LVPs to characterize the local receptive field properties of the visual cortex in response to SDSIs and MDSIs. Furthermore, we also consider that the learning of primitives should be performed in a task-driven manner. For this, two penalty terms including reconstruction error and quality inconsistency are jointly minimized within a supervised dictionary learning framework, generating a set of quality-oriented MB-LVPs for each single and multiple distortion modality. Given an input stereoscopic image, feature encoding is performed using the learned MB-LVPs as codebooks, resulting in the corresponding monocular and binocular responses. Finally, responses across all the modalities are fused with probabilistic weights which are determined by the modality-specific sparse reconstruction errors, yielding the final monocular and binocular features for quality regression. The superiority of our method has been verified on several SDSI and MDSI databases.

Multi-view coupled dictionary learning for person re-identification

In recent years, person re-identification is becoming an important technique, which can be applied in computer vision, pedestrian tracking and intelligent monitoring. Due to the large variations of visual appearance caused by view angle, pose changing, light changing, background clutter and occlusion, person re-identification is very challenging. In practice, there exist large differences among different types of features and among different cameras. To improve the favorable representation of different features, we propose a multi-view based coupled dictionary pair learning framework, which can learn dictionary pairs for multiple categories of features, e.g., the color features, texture features and hybrid features etc. Specifically, with the learned color feature dictionary pair, we can obtain the color feature representation coefficients of each person from different cameras. The texture feature dictionary pair seeks to learn the texture feature representation coefficients of each person from both cameras. The hybrid feature dictionary pair aims to learn the hybrid feature coefficients for each person. The learned coupled dictionary pairs can demonstrate the intrinsic relationship of different cameras and different types of features. When the resolution of image is too low, the texture information will be lost to some extent. There is few high-resolution person dataset so far, we contribute a newly collected dataset, named High-Resolution Pedestrian re-Identification Dataset (HRPID) on the campus of Wuhan University. The size of person images is normalized to 230 × 560 pixels, which is bigger than existing person re-identification datasets. Experimental results on a new dataset and two public pedestrian datasets demonstrate that our proposed approach can perform better than the other competing methods.

Class-specific synthesized dictionary model for Zero-Shot Learning

Zero-Shot Learning (ZSL) aims at recognizing unseen classes that are absent during the training stage. Unlike the existing approaches that learn a visual-semantic embedding model to bridge the low-level visual space and the high-level class prototype space, we propose a novel synthesized approach for addressing ZSL within a dictionary learning framework. Specifically, it learns both a dictionary matrix and a class-specific encoding matrix for each seen class to synthesize pseudo instances for unseen classes with auxiliary of seen class prototypes. This allows us to train the classifiers for the unseen classes with these pseudo instances. In this way, ZSL can be treated as a traditional classification task, which makes it applicable for traditional and generalized ZSL settings simultaneously. Extensive experimental results on four benchmark datasets (AwA, CUB, aPY, and SUN) demonstrate that our method yields competitive performances compared to state-of-the-art methods on both settings.

Linearized kernel dictionary learning with group sparse priors for action recognition

Classification-driven dictionary has been successfully used in pattern recognition and computer vision in recent years. In this paper, a discriminative dictionary is constructed by concatenating all class specific sub-dictionaries and one sub-dictionary containing the common patterns. To further enhance the discriminative power, we also propose to use group sparse priors in the coding stage of the dictionary learning process. A kernel dictionary is learned to solve the same direction distribution problem existing in the traditional dictionary learning framework. Actually, the kernel dictionary is learned in a linearized manner by using virtual features. We evaluate our method on three public action datasets including facial expression, Hand Gesture and UCF Sports. Experimental results demonstrate that our method can achieve the better or at least competitive performance when compared with other action recognition methods.

A Cortical Folding Pattern-Guided Model of Intrinsic Functional Brain Networks in Emotion Processing.

There have been increasing studies demonstrating that emotion processing in humans is realized by the interaction within or among the large-scale intrinsic functional brain networks. Identifying those meaningful intrinsic functional networks based on task-based functional magnetic resonance imaging (task fMRI) with specific emotional stimuli and responses, and exploring the underlying functional working mechanisms of interregional neural communication within the intrinsic functional networks are thus of great importance to understand the neural basis of emotion processing. In this paper, we propose a novel cortical folding pattern-guided model of intrinsic networks in emotion processing: gyri serve as global functional connection centers that perform interregional neural communication among distinct regions via long distance dense axonal fibers, and sulci serve as local functional units that directly communicate with neighboring gyri via short distance fibers and indirectly communicate with other distinct regions via the neighboring gyri. We test the proposed model by adopting a computational framework of dictionary learning and sparse representation of emotion task fMRI data of 68 subjects in the publicly released Human Connectome Project. The proposed model provides novel insights of functional mechanisms in emotion processing.

Patient-specific seizure detection in long-term EEG using signal-derived empirical mode decomposition (EMD)-based dictionary approach

Objective. The objective of the work described in this paper is the development of a computationally efficient methodology for patient-specific automatic seizure detection in long-term multi-channel EEG recordings. Approach. A novel patient-specific seizure detection approach based on a signal-derived empirical mode decomposition (EMD)-based dictionary approach is proposed. For this purpose, we use an empirical framework for EMD-based dictionary creation and learning, inspired by traditional dictionary learning methods, in which the EMD-based dictionary is learned from the multi-channel EEG data being analyzed for automatic seizure detection. We present the algorithm for dictionary creation and learning, whose purpose is to learn dictionaries with a small number of atoms. Using training signals belonging to seizure and non-seizure classes, an initial dictionary, termed as the raw dictionary, is formed. The atoms of the raw dictionary are composed of intrinsic mode functions obtained after decomposition of the training signals using the empirical mode decomposition algorithm. The raw dictionary is then trained using a learning algorithm, resulting in a substantial decrease in the number of atoms in the trained dictionary. The trained dictionary is then used for automatic seizure detection, such that coefficients of orthogonal projections of test signals against the trained dictionary form the features used for classification of test signals into seizure and non-seizure classes. Thus no hand-engineered features have to be extracted from the data as in traditional seizure detection approaches. Main results. The performance of the proposed approach is validated using the CHB–MIT benchmark database, and averaged accuracy, sensitivity and specificity values of 92.9%, 94.3% and 91.5%, respectively, are obtained using support vector machine classifier and five-fold cross-validation method. These results are compared with other approaches using the same database, and the suitability of the approach for seizure detection in long-term multi-channel EEG recordings is discussed. Significance. The proposed approach describes a computationally efficient method for automatic seizure detection in long-term multi-channel EEG recordings. The method does not rely on hand-engineered features, as are required in traditional approaches. Furthermore, the approach is suitable for scenarios where the dictionary once formed and trained can be used for automatic seizure detection of newly recorded data, making the approach suitable for long-term multi-channel EEG recordings.

Learning a discriminative dictionary for classification with outliers

Abstract Dictionary learning aims to find a dictionary where signals in some ensemble have sparse representations, and has been successfully applied for classification. However, traditional dictionary learning methods for classification assume there is no outlier in the training data, which may not be the case in practical applications. In this paper, we propose a new discriminative dictionary learning framework for classification, which simultaneously learns a discriminative dictionary and detects outliers in the data. The dictionary learning framework is formulated into an optimization problem with designed regularizers to promote both the discrimination and outlier-detection capability. We demonstrate the superior performance of the proposed approach in comparison with state-of-the-art alternatives by conducting extensive experiments on various image classification tasks.

Baseline-free guided wave damage detection with surrogate data and dictionary learning.

In guided wave structural health monitoring, damage detection is often accomplished by comparing measurements before damage (i.e., baseline data) and after damage (i.e., test data). Yet, in practical scenarios, baseline data is often unavailable. Data from surrogate structures (structures similar to the test structure) could replace baseline data, but due to small differences in material properties, such as thickness, temperature, and other effects, this data is often unreliable. In this paper, a dictionary learning framework overcomes this challenge and detects damage with surrogate information. The framework combines wave propagation characteristics of a test structure with geometric information from surrogate structures to create a synthetic damage-free baseline. The test data is compared with the synthetic baseline to detect damage. The framework is evaluated with four 108 mm ×108 mm plates: two 1.6-mm thick aluminum plates, one 1.6-mm thick steel plate, and one 6.25 mm thick aluminum plate. The framework is applied to each test structure after learning from plates with different material properties and thicknesses. With full wavefield data, this paper successfully isolates reflections from a mass without using explicit baseline data. Furthermore, with sparse guided wave data, this paper shows that a drop in a correlation coefficient can detect a mass without using explicit baseline data.

Face recognition based on manifold constrained joint sparse sensing with K-SVD

Face recognition based on Sparse representation idea has recently become an important research topic in computer vision community. However, the dictionary learning process in most of the existing approaches suffers from the perturbations brought by the variations of the input samples, since the consistence of the learned dictionaries from similar input samples based on K-SVD are not well addressed in the existing literature. In this paper, we will propose a novel technique for dictionary learning based on K-SVD to address the consistence issue. In particular, the proposed method embeds the manifold constraints into a standard dictionary learning framework based on k-SVD and force the optimization process to satisfy the structure preservation requirement. Therefore, this new approach can consistently integrate the manifold constraints during the optimization process, and it can contribute a better solution which is robust to the variance of the input samples. Extensive experiments on several popular face databases show a consistent performance improvement in comparison to some related state-of-the-art algorithms.

Cascade residuals guided nonlinear dictionary learning

In this paper, we aim to extend dictionary learning onto hierarchical image representations in a principled way. To achieve dictionary atoms capture additional information from extended receptive fields and attain improved descriptive capacity, we present a two-pass multi-resolution cascade framework for dictionary learning and sparse coding. This cascade method allows collaborative reconstructions at different resolutions using only the same dimensional dictionary atoms. The jointly learned dictionary comprises atoms that adapt to the information available at the coarsest layer, where the support of atoms reaches a maximum range, and the residual images, where the supplementary details refine progressively a reconstruction objective. The residual at a layer is computed by the difference between the aggregated reconstructions of the previous layers and the downsampled original image at that layer. Our method generates flexible and accurate representations using only a small number of coefficients. It is computationally efficient since it encodes the image at the coarsest resolution while yielding very sparse residuals. Our extensive experiments on multiple image coding, denoising, inpainting and artifact removal tasks demonstrate that our method provides superior results.

Weakly Paired Multimodal Fusion for Object Recognition

The ever-growing development of sensor technology has led to the use of multimodal sensors to develop robotics and automation systems. It is therefore highly expected to develop methodologies capable of integrating information from multimodal sensors with the goal of improving the performance of surveillance, diagnosis, prediction, and so on. However, real multimodal data often suffer from significant weak-pairing characteristics, i.e., the full pairing between data samples may not be known, while pairing of a group of samples from one modality to a group of samples in another modality is known. In this paper, we establish a novel projective dictionary learning framework for weakly paired multimodal data fusion. By introducing a latent pairing matrix, we realize the simultaneous dictionary learning and the pairing matrix estimation, and therefore improve the fusion effect. In addition, the kernelized version and the optimization algorithms are also addressed. Extensive experimental validations on some existing data sets are performed to show the advantages of the proposed method. Note to Practitioners —In many industrial environments, we usually use multiple heterogeneous sensors, which provide multimodal information. Such multimodal data usually lead to two technical challenges. First, different sensors may provide different patterns of data. Second, the full-pairing information between modalities may not be known. In this paper, we develop a unified model to tackle such problems. This model is based on a projective dictionary learning method, which efficiently produces the representation vector for the original data by an explicit form. In addition, the latent pairing relation between samples can be learned automatically and be used to improve the classification performance. Such a method can be flexibly used for multimodal fusion with full-pairing, partial-pairing and weak-pairing cases.

Image restoration via Bayesian dictionary learning with nonlocal structured beta process

Nonparametric Bayesian dictionary learning has shown a powerful potential in image restoration. However, it still lacks exploiting image structure to improve the performance. In this work, we propose a sparse Bayesian dictionary learning framework with structure prior called nonlocal structured beta process factor analysis (NLS-BPFA) which connects nonlocal self-similarity and sparse Bayesian dictionary learning. A nonlocal structured beta process is proposed to introduce the nonlocal self-similarity as a structure prior for image denoising and inpainting. Unlike most of the existing image denoising methods, our proposed method does not need to know noise variance in advance like an unsupervised learning. The experimental results demonstrate the effectiveness of our proposed model.

Weakly Supervised Dictionary Learning

We present a probabilistic modeling and inference framework for discriminative analysis dictionary learning under a weak supervision setting. Dictionary learning approaches have been widely used for tasks such as low-level signal denoising and restoration as well as high-level classification tasks, which can be applied to audio and image analysis. Synthesis dictionary learning aims at jointly learning a dictionary and corresponding sparse coefficients to provide accurate data representation. This approach is useful for denoising and signal restoration, but may lead to sub-optimal classification performance. By contrast, analysis dictionary learning provides a transform that maps data to a sparse discriminative representation suitable for classification. We consider the problem of analysis dictionary learning for time-series data under a weak supervision setting in which signals are assigned with a global label instead of an instantaneous label signal. We propose a discriminative probabilistic model that incorporates both label information and sparsity constraints on the underlying latent instantaneous label signal using cardinality control. We present the expectation maximization (EM) procedure for maximum likelihood estimation (MLE) of the proposed model. To facilitate a computationally efficient E-step, we propose both a chain and a novel tree graph reformulation of the graphical model. The performance of the proposed model is demonstrated on both synthetic and real-world data.

Ship Classification Based on MSHOG Feature and Task-Driven Dictionary Learning with Structured Incoherent Constraints in SAR Images

In this paper, we present a novel method for ship classification in synthetic aperture radar (SAR) images. The proposed method consists of feature extraction and classifier training. Inspired by SAR-HOG feature in automatic target recognition, we first design a novel feature named MSHOG by improving SAR-HOG, adapting it to ship classification, and employing manifold learning to achieve dimensionality reduction. Then, we train the classifier and dictionary jointly in task-driven dictionary learning (TDDL) framework. To further improve the performance of TDDL, we enforce structured incoherent constraints on it and develop an efficient algorithm for solving corresponding optimization problem. Extensive experiments performed on two datasets with TerraSAR-X images demonstrate that the proposed method, MSHOG feature and TDDL with structured incoherent constraints, outperforms other existing methods and achieves state-of-art performance.

Illumination Normalization for Face Recognition via Jointly Optimized Dictionary-Learning and Sparse Representation

In this paper, we present a novel patch-based dictionary learning (DL) framework for face illumination normalization (IN). The proposed method is based on sparse representation that is from the coupled dictionaries (DL-IN). The used dictionary pairs in this paper are jointly optimized from paired face images with normal and irregular illuminations. A Gaussian mixture model (GMM) clustering was further used to improve the capability of modeling data with more complex distributions. The proposed GMM adapts individual model based on a portion of the samples and lastly fuses all the images together. Experimental results show that the proposed framework by jointly optimizing DL and incorporating structural sparsity as a prior can improve the accuracy for face recognition.

Sparsity and Low-Rank Dictionary Learning for Sparse Representation of Monogenic Signal

This paper proposes a new framework of dictionary learning for a recently developed study, sparse representation of monogenic signal. The proposed framework is applied to target recognition in SAR image. Unlike the preceding works, where the sparse model is formed via an overcomplete dictionary whose atoms are the training samples themselves, a new approach to learn a more discriminative dictionary is proposed. To achieve target classification, two specific implementation schemes, global learning, and local learning, are developed. The global learning generates a single, universal dictionary for all target class. Since the class membership committed to each atom (element) is lost, the reconstruction error committed to each class is incapable to be computed. The conventional decision rule by the minimal residual could not be applied any more. Hence, this paper develops two decision rules for global learning. The first resorts to a third-party classifier, while the other recommends a nonparametric criterion. Different from the global learning, the local learning learns a subdictionary for each target class. These subdictionaries are concatenated to form a global one. Each subdictionary has been learned within certain target class, the generated atoms can be therefore labeled. The identification is predicted by evaluating which class of atom could produce the minimal reconstruction error. The effectiveness of proposed scheme is verified with multiple comparative studies.