beta-Bernoulli Process Research Articles

Deep Latent Factor Model for Spatio-Temporal Forecasting

Latent factor models can perform spatio-temporal forecasting (i.e., predicting future responses at unmeasured as well as measured locations) by modeling temporal dependence using latent factors and considering spatial dependence using a spatial prior on factor loadings. However, they may fail to capture complex spatio-temporal dependence because the latent factors are typically assumed to follow a classical linear time series model, such as a vector autoregressive model. In this article, we propose a deep latent factor model for spatio-temporal forecasting that can model complex spatio-temporal dependence more flexibly by leveraging the high expressive power of a deep neural network. Specifically, the latent factors are modeled using a recurrent neural network and the factor loadings are modeled using a distance-based Gaussian process. The proposed model allows the number of latent factors to be inferred from the data using a beta-Bernoulli process, which enables computationally more efficient implementation compared to previous methods. We derive a stochastic variational inference algorithm for scalable inference of the proposed model and validate the model using simulated and real data examples.

Complex Disease Individual Molecular Characterization Using Infinite Sparse Graphical Independent Component Analysis.

Identifying individual mechanisms involved in complex diseases, such as cancer, is essential for precision medicine. Their characterization is particularly challenging due to the unknown relationships of high-dimensional omics data and their inter-patient heterogeneity. We propose to model individual gene expression as a combination of unobserved molecular mechanisms (molecular components) that may differ between the individuals. Considering a baseline molecular profile common to all individuals, these molecular components may represent molecular pathways differing from the population background. We defined an infinite sparse graphical independent component analysis (isgICA) to identify these molecular components. This model relies on double sparseness: the source matrix sparseness defines the subset of genes involved in each molecular component, whereas the weight matrix sparseness identifies the subset of molecular components associated with each patient. As the number of molecular components is unknown but likely high, we simultaneously inferred it and the weight matrix sparseness using the beta-Bernoulli process (BBP). We simulated data from a double sparse ICA with 10/30 components with specific sparseness structures for 100/500 individuals and 500/1000/5000 genes with different noise variance levels to evaluate the reconstruction of the latent structures by our model. For all simulations, the isgICA was able to reconstruct with higher accuracy than 2 state-of-the-art methods (ica and fastICA) the number of components, the weight and source matrix sparsenesses (correlation simulated/estimated >.8). Applying our model to the expression of 1063 genes of 614 breast cancer patients, the isgICA identified 22 components. According to the source matrix, 7 of these 22 components seemed to be specifically related to 3 known molecular pathways with a prognostic effect in early breast cancer (immune system, proliferation, and stroma invasion). This proposed algorithm provides an insight into individual molecular heterogeneity to better understand complex disease mechanisms.

The APP for estimating population proportion based on skew normal approximations and the Beta-Bernoulli process

The a priori procedure (APP) is concerned with determining appropriate sample sizes to ensure that sample statistics to be obtained are likely to be good estimates of corresponding population parameters. Previous APP work pertaining to proportions has used the normal approximation to the binomial distribution, but this is problematic when the population proportion is near zero or one. The present contribution addresses the issue in four ways. First, we add a skew normal approximation that does a better job than the normal approximation. Second, we add a Bayesian component making use of a prior beta distribution that is conjugate to the binomial distribution. Third, we provide simulations and real data examples, one of them is a set of Covid-19 data. Finally, we include free and user-friendly computer programs to aid researchers in making the calculations.

Robust Clustering With Subpopulation-Specific Deviations

The National Birth Defects Prevention Study (NBDPS) is a case-control study of birth defects conducted across 10 U.S. states. Researchers are interested in characterizing the etiologic role of maternal diet, collected using a food frequency questionnaire. Because diet is multidimensional, dimension reduction methods such as cluster analysis are often used to summarize dietary patterns. In a large, heterogeneous population, traditional clustering methods, such as latent class analysis, used to estimate dietary patterns can produce a large number of clusters due to a variety of factors, including study size and regional diversity. These factors result in a loss of interpretability of patterns that may differ due to minor consumption changes. Based on adaptation of the local partition process, we propose a new method, robust profile clustering, to handle these data complexities. Here, participants may be clustered at two levels: (1) globally, where women are assigned to an overall population-level cluster via an overfitted finite mixture model, and (2) locally, where regional variations in diet are accommodated via a beta-Bernoulli process dependent on subpopulation differences. We use our method to analyze the NBDPS data, deriving prepregnancy dietary patterns for women in the NBDPS while accounting for regional variability. Supplementary materials for this article, including a standardized description of the materials available for reproducing the work, are available as an online supplement.

Fusion of Hyperspectral and Multispectral Images Based on a Bayesian Nonparametric Approach

This paper presents a new approach to fusion of hyperspectral and multispectral images based on Bayesian nonparametric sparse representation. The approach formulates the image fusion problem within a constrained optimization framework, while assuming that the target image lives in a lower dimensional subspace. The subspace transform matrix is determined by principal component analysis, and the sparse regularization term is designed depending on a set of dictionaries and sparse coefficients associated with the observed images. Specifically, the dictionary elements and sparse coefficients are learned by the Bayesian nonparametric approach with the beta-Bernoulli process, which establishes the probability distribution models for each latent variable and calculates the posterior distributions by Gibbs sampling. Finally, serving the obtained posterior distributions as a priori, the fusion problem is solved via an alternate optimization process, where the alternate direction method of multipliers is applied to perform the optimization with respect to the target image. The Bayesian nonparametric method is used to optimize the sparse coefficients. Exhaustive experiments using both two public datasets and one real-world dataset of remote sensing images show that the proposed approach outperforms the existing state-of-the-art methods.

The Exact Inference of Beta Process and Beta Bernoulli Process From Finite Observations

Beta Process is a typical nonparametric Bayesian model. and the Beta Bernoulli Process provides a Bayesian nonparametric prior for models involving collections of binary valued features. Some previous studies considered the Beta Process inference problem by giving the Stick-Breaking sampling method. This paper focuses on analyzing the form of precise probability distribution based on a Stick-Breaking approach, that is, the joint probability distribution is derived from any finite number of observable samples: It not only determines the probability distribution function of the Beta Process with finite observation (represented as a group of number between [0,1]), but also gives the distribution function of the Beta Bernoulli Process with the same finite dimension (represented as a matrix with element value of 0 or, 1) by using this distribution as a prior.

Hyperspectral Recovery from RGB Images using Gaussian Processes

We propose to recover spectral details from RGB images of known spectral quantization by modeling natural spectra under Gaussian Processes and combining them with the RGB images. Our technique exploits Process Kernels to model the relative smoothness of reflectance spectra, and encourages non-negativity in the resulting signals for better estimation of the reflectance values. The Gaussian Processes are inferred in sets using clusters of spatio-spectrally correlated hyperspectral training patches. Each set is transformed to match the spectral quantization of the test RGB image. We extract overlapping patches from the RGB image and match them to the hyperspectral training patches by spectrally transforming the latter. The RGB patches are encoded over the transformed Gaussian Processes related to those hyperspectral patches and the resulting image is constructed by combining the codes with the original processes. Our approach infers the desired Gaussian Processes under a fully Bayesian model inspired by Beta-Bernoulli Process, for which we also present the inference procedure. A thorough evaluation using three hyperspectral datasets demonstrates the effective extraction of spectral details from RGB images by the proposed technique.

Joint bayesian convolutional sparse coding for image super-resolution.

We propose a convolutional sparse coding (CSC) for super resolution (CSC-SR) algorithm with a joint Bayesian learning strategy. Due to the unknown parameters in solving CSC-SR, the performance of the algorithm depends on the choice of the parameter. To this end, a coupled Beta-Bernoulli process is employed to infer appropriate filters and sparse coding maps (SCM) for both low resolution (LR) image and high resolution (HR) image. The filters and the SCMs are learned in a joint inference. The experimental results validate the advantages of the proposed approach over the previous CSC-SR and other state-of-the-art SR methods.

Image de-noising with subband replacement and fusion process using bayes estimators,

A hybrid image de-noising framework with an automatic parameter selection scheme is proposed to handle substantially high noise with an unknown variance. The impetus of the framework is to preserve the latent detail information of the noisy image while removing the noise with an appropriate smoothing and feasible sharpening. The proposed method is executed in two steps. First, the sub-band replacement and fusion process based on accelerated version of the Bayesian non local means method are implemented to enhance the weak edges that often result in low gradient magnitude and fade out during the de-noising process. Then, a truncated beta-Bernoulli process is employed to infer an appropriate dictionary of the edge enhanced data to obtain de-noising results precisely. Numerical simulations are performed to substantiate the restoration of the weak edges through sub-band replacement and fusion process. The proposed de-noising scheme is validated through visual and quantitative results using well established metrics.

Hierarchical Sparse Learning with Spectral-Spatial Information for Hyperspectral Imagery Denoising.

During the acquisition process hyperspectral images (HSI) are inevitably corrupted by various noises, which greatly influence their visual impression and subsequent applications. In this paper, a novel Bayesian approach integrating hierarchical sparse learning and spectral-spatial information is proposed for HSI denoising. Based on the structure correlations, spectral bands with similar and continuous features are segmented into the same band-subset. To exploit local similarity, each subset is then divided into overlapping cubic patches. All patches can be regarded as consisting of clean image component, Gaussian noise component and sparse noise component. The first term is depicted by a linear combination of dictionary elements, where Gaussian process with Gamma distribution is applied to impose spatial consistency on dictionary. The last two terms are utilized to fully depict the noise characteristics. Furthermore, the sparseness of the model is adaptively manifested through Beta-Bernoulli process. Calculated by Gibbs sampler, the proposed model can directly predict the noise and dictionary without priori information of the degraded HSI. The experimental results on both synthetic and real HSI demonstrate that the proposed approach can better suppress the existing various noises and preserve the structure/spectral-spatial information than the compared state-of-art approaches.

Aspect Learning for Multimedia Summarization via Nonparametric Bayesian

Summarization is desirable for efficient comprehension of an increasingly vast amount of data. A summary of multiple documents is a concise description of the main topic. Generally speaking, a topic delivers various aspects. For example, the natural disaster topic is likely to imply the aspects of casualties and rescue. Therefore, a good summary is expected to cover all the informative aspects of a topic in order to enhance both diversity and coverage of the topic. However, for the real-world data, the profile of aspects in a given topic (e.g., the number of the aspects as well as their appropriate describing sentences or images) is hardly specified in advance. To address this problem, this paper proposes an approach to learn the hidden aspects in the topics via a nonparametric Bayesian model for multimedia summarization, namely, aspect learning for multimedia summarization via nonparametric Bayesian (ALSNB). More specifically, we introduce the priors of beta-Bernoulli process and Dirichlet process into the traditional dictionary learning. As a result, the proposed approach is able to adaptively identify the particular aspects of an individual topic. The experimental results on several datasets for text summarization and image summarization show the superiority of the proposed ALSNB over other methods.

Class-specific mid-level feature learning with the Discriminative Group-wise Beta-Bernoulli process restricted Boltzmann machines

In this paper, we propose a Discriminative Group-wise Beta-Bernoulli process restricted Boltzmann machine (DG-BBP RBM), an approach to learn class-specific mid-level features based on the Beta-Bernoulli process restricted Boltzmann machine (BBP RBM), which imposes class-specific sparsity that has discriminative characteristics across different classes to eliminate redundancy among extracted features. With this method, we learn mid-level features that are characteristic of each class and that are shared rarely or not at all with other classes (i.e., are discriminative of that class). In experiments on image classification tasks, our DG-BBP RBM showed much better results than did BBP RBM and related methods and could capture semantic attributes that can be used to discriminate between classes.

A Bayesian Nonparametric Approach to Image Super-Resolution.

Super-resolution methods form high-resolution images from low-resolution images. In this paper, we develop a new Bayesian nonparametric model for super-resolution. Our method uses a beta-Bernoulli process to learn a set of recurring visual patterns, called dictionary elements, from the data. Because it is nonparametric, the number of elements found is also determined from the data. We test the results on both benchmark and natural images, comparing with several other models from the research literature. We perform large-scale human evaluation experiments to assess the visual quality of the results. In a first implementation, we use Gibbs sampling to approximate the posterior. However, this algorithm is not feasible for large-scale data. To circumvent this, we then develop an online variational Bayes (VB) algorithm. This algorithm finds high quality dictionaries in a fraction of the time needed by the Gibbs sampler.

Bayesian joint analysis of heterogeneous genomics data

A non-parametric Bayesian factor model is proposed for joint analysis of multi-platform genomics data. The approach is based on factorizing the latent space (feature space) into a shared component and a data-specific component with the dimensionality of these components (spaces) inferred via a beta-Bernoulli process. The proposed approach is demonstrated by jointly analyzing gene expression/copy number variations and gene expression/methylation data for ovarian cancer patients, showing that the proposed model can potentially uncover key drivers related to cancer. The source code for this model is written in MATLAB and has been made publicly available at https://sites.google.com/site/jointgenomics/. catherine.ll.zheng@gmail.com Supplementary data are available at Bioinformatics online.

MR image reconstruction based on comprehensive sparse prior

In this paper, a novel Magnetic Resonance (MR) reconstruction framework which combines image-wise and patch-wise sparse prior is proposed. For addressing, a truncated beta-Bernoulli process is firstly employed to enforce sparsity on overlapping image patches emphasizing local structures. Due to its properties, beta-Bernoulli process can adaptive infer the sparsity (number of nonzero coefficients) of each patch, an appropriate dictionary, and the noise variance simultaneously, which are prerequisite for iterative image reconstruction. Secondly, a General Gaussian Distribution (GGD) prior is introduced to engage image-wise sparsity for wavelet coefficients, which can be then estimated by a threshold denoising algorithm. Finally, MR image is reconstructed by patch-wise estimation, image-wise estimation and under-sampled k-space data with least square data fitting. Experimental results have demonstrated that proposed approach exhibits excellent reconstruction performance. Moreover, if the image is full of similar low-dimensional-structures, proposed algorithm has dramatically improved Peak Signal to Noise Ratio (PSNR) 7∼9 dB, with comparisons to other state-of-art compressive sampling methods.

Beta Processes, Stick-Breaking and Power Laws

The beta-Bernoulli process provides a Bayesian nonparametric prior for models involving collections of binary-valued features. A draw from the beta process yields an infinite collection of probabilities in the unit interval, and a draw from the Bernoulli process turns these into binary-valued features. Recent work has provided stick-breaking representations for the beta process analogous to the well-known stick-breaking representation for the Dirichlet process. We derive one such stick-breaking representation directly from the characterization of the beta process as a completely random measure. This approach motivates a three-parameter generalization of the beta process, and we study the power laws that can be obtained from this generalized beta process. We present a posterior inference algorithm for the beta-Bernoulli process that exploits the stick-breaking representation, and we present experimental results for a discrete factor-analysis model.

Nonparametric Bayesian Dictionary Learning for Analysis of Noisy and Incomplete Images

Nonparametric Bayesian methods are considered for recovery of imagery based upon compressive, incomplete, and/or noisy measurements. A truncated beta-Bernoulli process is employed to infer an appropriate dictionary for the data under test and also for image recovery. In the context of compressive sensing, significant improvements in image recovery are manifested using learned dictionaries, relative to using standard orthonormal image expansions. The compressive-measurement projections are also optimized for the learned dictionary. Additionally, we consider simpler (incomplete) measurements, defined by measuring a subset of image pixels, uniformly selected at random. Spatial interrelationships within imagery are exploited through use of the Dirichlet and probit stick-breaking processes. Several example results are presented, with comparisons to other methods in the literature.