Nonparametric Bayesian Dictionary Learning Research Articles

Non-parametric Bayesian dictionary learning based on Laplace noise

Sparse representation based on over-complete dictionaries is a hot issue in the field of computer vision and machine learning. In probability theory, over-complete dictionary can be learned by non-parametric Bayesian techniques with Beta Process. However, traditional probabilistic dictionary learning method assumes noise follows Gaussian distribution, which can only remove Gaussain noise. In order to remove outlier or complex noise, we propose a dictionary learning method based on non-parametric Bayesian technology by assuming the noise follows Laplacian distribution. Because the non-conjugacy of Laplacian distribution makes the calculation of posteriors of latent variables more complicate, thus we utilize a superposition of an infinite number of Gaussian distributions to substitute for L1 density function. The weights of mixture Gaussian distribution are controlled by an extra hidden variable. Then the Bayesian inference is applied to learn all the key parameters in the proposed probabilistic model, which avoids the processing of parameter setting and fine tuning. In the experiments, we mainly test the performance of different algorithms in removing salt-and-pepper noise and mixture noises. The experimental results show that the PSNRs of our algorithm are higher 2-4 dB at least than other classic algorithms.

Orthogonality-Promoting Dictionary Learning via Bayesian Inference

Dictionary Learning (DL) plays a crucial role in numerous machine learning tasks. It targets at finding the dictionary over which the training set admits a maximally sparse representation. Most existing DL algorithms are based on solving an optimization problem, where the noise variance and sparsity level should be known as the prior knowledge. However, in practice applications, it is difficult to obtain these knowledge. Thus, non-parametric Bayesian DL has recently received much attention of researchers due to its adaptability and effectiveness. Although many hierarchical priors have been used to promote the sparsity of the representation in non-parametric Bayesian DL, the problem of redundancy for the dictionary is still overlooked, which greatly decreases the performance of sparse coding. To address this problem, this paper presents a novel robust dictionary learning framework via Bayesian inference. In particular, we employ the orthogonality-promoting regularization to mitigate correlations among dictionary atoms. Such a regularization, encouraging the dictionary atoms to be close to being orthogonal, can alleviate overfitting to training data and improve the discrimination of the model. Moreover, we impose Scale mixture of the Vector variate Gaussian (SMVG) distribution on the noise to capture its structure. A Regularized Expectation Maximization Algorithm is developed to estimate the posterior distribution of the representation and dictionary with orthogonality-promoting regularization. Numerical results show that our method can learn the dictionary with an accuracy better than existing methods, especially when the number of training signals is limited.

Cross-Domain Collaborative Learning via Discriminative Nonparametric Bayesian Model

Cross-domain data analysis has been becoming more and more important, and can be effectively adopted for many applications. However, it is difficult to propose a unified cross-domain collaborative learning framework for cross-domain analysis in social multimedia, because cross-domain data have multidomain, multimodal, sparse, and supervised properties. In this paper, we propose a generic cross-domain collaborative learning (CDCL) framework via a discriminative nonparametric Bayesian dictionary learning model for cross-domain data analysis. Compared with existing cross-domain learning methods, our proposed model mainly has four advantages: First, to address the domain discrepancy, we utilize the shared domain priors among multiple domains to make them share a common feature space. Second, to exploit the multimodal property, we use the shared modality priors to model the relationship between different modalities. Third, to deal with the sparse property of media data in one domain, our goal is to learn a shared dictionary to bridge different domains and complement each other. Finally, to make use of the supervised property, we exploit class label information to learn the shared discriminative dictionary, and utilize a latent probability vector to select different dictionary elements for representation of each class. Therefore, the proposed model can investigate the superiorities of different sources to supplement and improve each other effectively. In experiments, we have evaluated our model for two important applications including cross-platform event recognition and cross-network video recommendation. The experimental results have showed the effectiveness of our CDCL model for cross-domain analysis.

Image restoration via Bayesian dictionary learning with nonlocal structured beta process

Nonparametric Bayesian dictionary learning has shown a powerful potential in image restoration. However, it still lacks exploiting image structure to improve the performance. In this work, we propose a sparse Bayesian dictionary learning framework with structure prior called nonlocal structured beta process factor analysis (NLS-BPFA) which connects nonlocal self-similarity and sparse Bayesian dictionary learning. A nonlocal structured beta process is proposed to introduce the nonlocal self-similarity as a structure prior for image denoising and inpainting. Unlike most of the existing image denoising methods, our proposed method does not need to know noise variance in advance like an unsupervised learning. The experimental results demonstrate the effectiveness of our proposed model.

The nonparametric Bayesian dictionary learning based interpolation method for WSNs missing data

The conventional data interpolation methods based on sparse representation usually assume that the signal is sparse under the overcomplete dictionary. Specially, they must confirm the dimensions of dictionary and the signal sparse level in advance. However, it is hard to know them if the signal is complicated or dynamically changing. In this paper, we proposed a nonparametric Bayesian dictionary learning based interpolation method for WSNs missing data, which is the combination of sparse representation and data interpolation. This method need not preset sparse degrees and dictionary dimensions, and our dictionary atoms are drawn from a multivariate normal distribution. In this case, the dictionary size will be learned adaptively by the nonparametric Bayesian method. In addition, we implement the Dirichlet process to exploit the spatial similarity of the sensing data in WSNs, thus to improve the interpolation accuracy. The interpolation model parameters, the optimal dictionary and sparse coefficients, can be inferred by the means of Gibbs sampling. The missing data will be estimated commendably through the derived parameters. The experimental results show that the data interpolation method we proposed outperforms the conventional methods in terms of interpolation accuracy and robustness.

Seismic Signal Compression Using Nonparametric Bayesian Dictionary Learning via Clustering

We introduce a seismic signal compression method based on nonparametric Bayesian dictionary learning method via clustering. The seismic data is compressed patch by patch, and the dictionary is learned online. Clustering is introduced for dictionary learning. A set of dictionaries could be generated, and each dictionary is used for one cluster’s sparse coding. In this way, the signals in one cluster could be well represented by their corresponding dictionaries. A nonparametric Bayesian dictionary learning method is used to learn the dictionaries, which naturally infers an appropriate dictionary size for each cluster. A uniform quantizer and an adaptive arithmetic coding algorithm are adopted to code the sparse coefficients. With comparisons to other state-of-the art approaches, the effectiveness of the proposed method could be validated in the experiments.