Sparse Bayesian Factor Research Articles

When It Counts—Econometric Identification of the Basic Factor Model Based on GLT Structures

Despite the popularity of factor models with simple loading matrices, little attention has been given to formally address the identifiability of these models beyond standard rotation-based identification such as the positive lower triangular (PLT) constraint. To fill this gap, we review the advantages of variance identification in simple factor analysis and introduce the generalized lower triangular (GLT) structures. We show that the GLT assumption is an improvement over PLT without compromise: GLT is also unique but, unlike PLT, a non-restrictive assumption. Furthermore, we provide a simple counting rule for variance identification under GLT structures, and we demonstrate that within this model class, the unknown number of common factors can be recovered in an exploratory factor analysis. Our methodology is illustrated for simulated data in the context of post-processing posterior draws in sparse Bayesian factor analysis.

Generalized cumulative shrinkage process priors with applications to sparse Bayesian factor analysis.

The paper discusses shrinkage priors which impose increasing shrinkage in a sequence of parameters. We review the cumulative shrinkage process (CUSP) prior of Legramanti et al. (Legramanti et al. 2020 Biometrika 107, 745-752. (doi:10.1093/biomet/asaa008)), which is a spike-and-slab shrinkage prior where the spike probability is stochastically increasing and constructed from the stick-breaking representation of a Dirichlet process prior. As a first contribution, this CUSP prior is extended by involving arbitrary stick-breaking representations arising from beta distributions. As a second contribution, we prove that exchangeable spike-and-slab priors, which are popular and widely used in sparse Bayesian factor analysis, can be represented as a finite generalized CUSP prior, which is easily obtained from the decreasing order statistics of the slab probabilities. Hence, exchangeable spike-and-slab shrinkage priors imply increasing shrinkage as the column index in the loading matrix increases, without imposing explicit order constraints on the slab probabilities. An application to sparse Bayesian factor analysis illustrates the usefulness of the findings of this paper. A new exchangeable spike-and-slab shrinkage prior based on the triple gamma prior of Cadonna et al. (Cadonna et al. 2020 Econometrics 8, 20. (doi:10.3390/econometrics8020020)) is introduced and shown to be helpful for estimating the unknown number of factors in a simulation study. This article is part of the theme issue 'Bayesian inference: challenges, perspectives, and prospects'.

Single-Cell Differential Network Analysis with Sparse Bayesian Factor Models.

Differential network analysis plays an important role in learning how gene interactions change under different biological conditions, and the high resolution of single-cell RNA (scRNA-seq) sequencing provides new opportunities to explore these changing gene-gene interactions. Here, we present a sparse hierarchical Bayesian factor model to identify differences across network structures from different biological conditions in scRNA-seq data. Our methodology utilizes latent factors to impact gene expression values for each cell to help account for zero-inflation, increased cell-to-cell variability, and overdispersion that are unique characteristics of scRNA-seq data. Condition-dependent parameters determine which latent factors are activated in a gene, which allows for not only the calculation of gene-gene co-expression within each group but also the calculation of the co-expression differences between groups. We highlight our methodology’s performance in detecting differential gene-gene associations across groups by analyzing simulated datasets and a SARS-CoV-2 case study dataset.

Sparse Bayesian factor analysis for structural damage detection under unknown environmental conditions

Damage detection of civil engineering structures needs to consider the effect of normal environmental variations on structural dynamic properties. This study develops a novel structural damage detection method using factor analysis in the sparse Bayesian learning framework. The unknown changing environmental factors that affect the structural dynamic properties are treated as latent variables in the model. The automatic relevance determination prior is adopted for the factor loading matrix for model selection. All variables and parameters, including the factor loading matrix, error vector and latent variables, are solved using the iterative expectation-maximization technique. The variables are then used to reconstruct structural responses. The Euclidean norm of the error vector is calculated as the damage indicator to detect possible damage when limited vibration data are available. Two laboratory-tested examples are utilized to verify the effectiveness of the proposed method. Results demonstrate that the number of underlying environmental factors and structural damage can be accurately identified, even though the changing environmental data are unavailable. The proposed method has the advantages of online monitoring and automatic identification of underlying environmental factors.

A sparse Bayesian factor model for the construction of gene co-expression networks from single-cell RNA sequencing count data

BackgroundGene co-expression networks (GCNs) are powerful tools that enable biologists to examine associations between genes during different biological processes. With the advancement of new technologies, such as single-cell RNA sequencing (scRNA-seq), there is a need for developing novel network methods appropriate for new types of data.ResultsWe present a novel sparse Bayesian factor model to explore the network structure associated with genes in scRNA-seq data. Latent factors impact the gene expression values for each cell and provide flexibility to account for common features of scRNA-seq: high proportions of zero values, increased cell-to-cell variability, and overdispersion due to abnormally large expression counts. From our model, we construct a GCN by analyzing the positive and negative associations of the factors that are shared between each pair of genes.ConclusionsSimulation studies demonstrate that our methodology has high power in identifying gene-gene associations while maintaining a nominal false discovery rate. In real data analyses, our model identifies more known and predicted protein-protein interactions than other competing network models.

Generalized Bayesian Factor Analysis for Integrative Clustering with Applications to Multi-Omics Data.

Integrative clustering is a clustering approach for multiple datasets, which provide different views of a common group of subjects. It enables analyzing multi-omics data jointly to, for example, identify the subtypes of diseases, cells, and so on, capturing the complex underlying biological processes more precisely. On the other hand, there has been a great deal of interest in incorporating the prior structural knowledge on the features into statistical analyses over the past decade. The knowledge on the gene regulatory network (pathways) can potentially be incorporated into many genomic studies. In this paper, we propose a novel integrative clustering method which can incorporate the prior graph knowledge. We first develop a generalized Bayesian factor analysis (GBFA) framework, a sparse Bayesian factor analysis which can take into account the graph information. Our GBFA framework employs the spike and slab lasso (SSL) prior to impose sparsity on the factor loadings and the Markov random field (MRF) prior to encourage smoothing over the adjacent factor loadings, which establishes a unified shrinkage adaptive to the loading size and the graph structure. Then, we use the framework to extend iCluster+, a factor analysis based integrative clustering approach. A novel variational EM algorithm is proposed to efficiently estimate the MAP estimator for the factor loadings. Extensive simulation studies and the application to the NCI60 cell line dataset demonstrate that the propose method is superior and delivers more biologically meaningful outcomes.

Posterior contraction in sparse Bayesian factor models for massive covariance matrices

Sparse Bayesian factor models are routinely implemented for parsimonious dependence modeling and dimensionality reduction in high-dimensional applications. We provide theoretical understanding of such Bayesian procedures in terms of posterior convergence rates in inferring high-dimensional covariance matrices where the dimension can be larger than the sample size. Under relevant sparsity assumptions on the true covariance matrix, we show that commonly-used point mass mixture priors on the factor loadings lead to consistent estimation in the operator norm even when $p\gg n$. One of our major contributions is to develop a new class of continuous shrinkage priors and provide insights into their concentration around sparse vectors. Using such priors for the factor loadings, we obtain similar rate of convergence as obtained with point mass mixture priors. To obtain the convergence rates, we construct test functions to separate points in the space of high-dimensional covariance matrices using insights from random matrix theory; the tools developed may be of independent interest. We also derive minimax rates and show that the Bayesian posterior rates of convergence coincide with the minimax rates upto a $\sqrt{\log n}$ term.

Automated annotation of gene expression image sequences via non-parametric factor analysis and conditional random fields

Motivation: Computational approaches for the annotation of phenotypes from image data have shown promising results across many applications, and provide rich and valuable information for studying gene function and interactions. While data are often available both at high spatial resolution and across multiple time points, phenotypes are frequently annotated independently, for individual time points only. In particular, for the analysis of developmental gene expression patterns, it is biologically sensible when images across multiple time points are jointly accounted for, such that spatial and temporal dependencies are captured simultaneously.Methods: We describe a discriminative undirected graphical model to label gene-expression time-series image data, with an efficient training and decoding method based on the junction tree algorithm. The approach is based on an effective feature selection technique, consisting of a non-parametric sparse Bayesian factor analysis model. The result is a flexible framework, which can handle large-scale data with noisy incomplete samples, i.e. it can tolerate data missing from individual time points.Results: Using the annotation of gene expression patterns across stages of Drosophila embryonic development as an example, we demonstrate that our method achieves superior accuracy, gained by jointly annotating phenotype sequences, when compared with previous models that annotate each stage in isolation. The experimental results on missing data indicate that our joint learning method successfully annotates genes for which no expression data are available for one or more stages.Contact: uwe.ohler@duke.edu

Identification of novel melanoma gene expression signatures using a Bayesian approach to filter out gene expression noise from normal tissues.

e19053 Background: Identification of pertinent gene expression signatures may be confounded by failure to account for admixture of tumor specimens with normal tissue. We propose a novel method for filtering out gene expression “noise” from normal tissues in a set of melanoma samples derived from the NCBI Gene Expression Omnibus (GEO) public database. Methods: Using customized software implemented in MATLAB, we performed a preliminary search of GEO using the query term “melanoma”. We manually identified 695 clinical samples within 19 different gene expression studies (median 43 samples per series). Gene expression results were merged across different platforms using rank normalization for the target gene of the respective probes. Standardized gene expression signatures for 36 normal tissues were identified in GEO (GSE2361). We performed a modified sparse Bayesian factor regression analysis that incorporated signatures of normal tissues. Additional regression allowed for discovery of latent factors characteristic of melanoma subtypes. Pathway analysis was performed using Ingenuity Pathway Analysis (IPA) on characteristic genes for inferred melanoma-specific latent factors. Results: Normal tissue contributed significantly to the reported gene expression profiles, explaining a median of 11.5% of specimen variability (range 1.3-55.0%). In a subset analysis of 40 primary or in situ melanoma lesions versus 16 metastatic lesions (GSE7553), primary lesions were more likely contaminated with genes characteristic of normal skin (p<0.001). The top 5 latent factors explained a median of 17.0% of residual sample variance (range 0.6-64%). On IPA analysis, latent factor 1 was strongly associated with dermatologic disease (p = 7.46 e-36), latent factor 2 was associated with inflammatory response (p = 9.71e-25), and latent factor 3 was associated with upregulation of the MITF transcription factor regulation (p = 4.16e-19). Conclusions: Appropriate methods that filter out normal tissue contamination of melanoma specimens can improve our understanding of gene expression patterns in melanoma subtypes, and potentially improve future translational studies.

Automatic Annotation of Spatial Expression Patterns via Sparse Bayesian Factor Models

Advances in reporters for gene expression have made it possible to document and quantify expression patterns in 2D–4D. In contrast to microarrays, which provide data for many genes but averaged and/or at low resolution, images reveal the high spatial dynamics of gene expression. Developing computational methods to compare, annotate, and model gene expression based on images is imperative, considering that available data are rapidly increasing. We have developed a sparse Bayesian factor analysis model in which the observed expression diversity of among a large set of high-dimensional images is modeled by a small number of hidden common factors. We apply this approach on embryonic expression patterns from a Drosophila RNA in situ image database, and show that the automatically inferred factors provide for a meaningful decomposition and represent common co-regulation or biological functions. The low-dimensional set of factor mixing weights is further used as features by a classifier to annotate expression patterns with functional categories. On human-curated annotations, our sparse approach reaches similar or better classification of expression patterns at different developmental stages, when compared to other automatic image annotation methods using thousands of hard-to-interpret features. Our study therefore outlines a general framework for large microscopy data sets, in which both the generative model itself, as well as its application for analysis tasks such as automated annotation, can provide insight into biological questions.

Uncovering Transcriptional Regulatory Networks by Sparse Bayesian Factor Model

The problem of uncovering transcriptional regulation by transcription factors (TFs) based on microarray data is considered. A novel Bayesian sparse correlated rectified factor model (BSCRFM) is proposed that models the unknown TF protein level activity, the correlated regulations between TFs, and the sparse nature of TF-regulated genes. The model admits prior knowledge from existing database regarding TF-regulated target genes based on a sparse prior and through a developed Gibbs sampling algorithm, a context-specific transcriptional regulatory network specific to the experimental condition of the microarray data can be obtained. The proposed model and the Gibbs sampling algorithm were evaluated on the simulated systems, and results demonstrated the validity and effectiveness of the proposed approach. The proposed model was then applied to the breast cancer microarray data of patients with Estrogen Receptor positive (ER+) status and Estrogen Receptor negative (ER-) status, respectively.