Hierarchical Priors Research Articles

Bayesian semiparametric analysis for two-phase studies of gene-environment interaction

The two-phase sampling design is a cost-efficient way of collecting expensive covariate information on a judiciously selected sub-sample. It is natural to apply such a strategy for collecting genetic data in a sub-sample enriched for exposure to environmental factors for gene-environment interaction (G × E) analysis. In this paper, we consider two-phase studies of G × E interaction where phase I data are available on exposure, covariates and disease status. Stratified sampling is done to prioritize individuals for genotyping at phase II conditional on disease and exposure. We consider a Bayesian analysis based on the joint retrospective likelihood of phase I and phase II data. We address several important statistical issues: (i) we consider a model with multiple genes, environmental factors and their pairwise interactions. We employ a Bayesian variable selection algorithm to reduce the dimensionality of this potentially high-dimensional model; (ii) we use the assumption of gene-gene and gene-environment independence to trade-off between bias and efficiency for estimating the interaction parameters through use of hierarchical priors reflecting this assumption; (iii) we posit a flexible model for the joint distribution of the phase I categorical variables using the non-parametric Bayes construction of Dunson and Xing (2009). We carry out a small-scale simulation study to compare the proposed Bayesian method with weighted likelihood and pseudo likelihood methods that are standard choices for analyzing two-phase data. The motivating example originates from an ongoing case-control study of colorectal cancer, where the goal is to explore the interaction between the use of statins (a drug used for lowering lipid levels) and 294 genetic markers in the lipid metabolism/cholesterol synthesis pathway. The sub-sample of cases and controls on which these genetic markers were measured is enriched in terms of statin users. The example and simulation results illustrate that the proposed Bayesian approach has a number of advantages for characterizing joint effects of genotype and exposure over existing alternatives and makes efficient use of all available data in both phases.

An Empirical Investigation of Bayesian Hierarchical Modeling with Unidimensional IRT Models

Assuming specific values for item hyperparameters, Bayesian nonhierarchical modeling for unidimensional IRT models suffers from problems in that it relies on the availability of appropriate prior information for the three-parameter model or for small datasets. These problems can be resolved by specifying priors in a hierarchical fashion so that the item hyperparameters are unknown and have their own prior distributions. This study investigated the performance of such hierarchical modeling by comparing it with the nonhierarchical approach using Monte Carlo simulations. Their results provided empirical evidence for the advantage of using hierarchical priors in modeling unidimensional item response data when appropriate prior information is not readily available and when datasets are not sufficiently large.

HIERARCHICAL BAYES PREDICTION FOR THE 2008 US PRESIDENTIAL ELECTION

In this paper a procedure is developed to derive the predictive density function of a future observation for prediction in a multiple regression model under hierarchical priors for the vector parameter. The derived predictive density function is applied for prediction in a multiple regression model given in Fair (2002) to study the effect of fluctuations in economic variables on voting behavior in U.S. presidential election. Numerical illustrations suggest that the predictive performance of Fair’s model is good under hierarchical Bayes setup, except for the 1992 election. Fair’s model under hierarchical Bayes setup indicates that the forthcoming 2008 US presidential election is likely to be a very close election slightly tilted towards Republicans. It is likely that republicans will get 50.90% vote with probability for win 0.550 in the 2008 US presidential election

Estimation of covariance matrices based on hierarchical inverse-Wishart priors

This paper focuses on Bayesian shrinkage methods for covariance matrix estimation. We examine posterior properties and frequentist risks of Bayesian estimators based on new hierarchical inverse-Wishart priors. More precisely, we give the conditions for the existence of the posterior distributions. Advantages in terms of numerical simulations of posteriors are shown. A simulation study illustrates the performance of the estimation procedures under three loss functions for relevant sample sizes and various covariance structures.

Hierarchical shrinkage priors for dynamic regressions with many predictors

This paper builds on a simple unified representation of shrinkage Bayes estimators based on hierarchical Normal-Gamma priors. Various popular penalized least squares estimators for shrinkage and selection in regression models can be recovered using this single hierarchical Bayes formulation. Using 129 U.S. macroeconomic quarterly variables for the period 1959 ‐ 2010 I exhaustively evaluate the forecasting properties of Bayesian shrinkage in regressions with many predictors. Results show that for particular data series hierarchical shrinkage dominates factor model forecasts, and hence it becomes a valuable addition to existing methods for handling large dimensional data.

A Bayesian Whittaker–Henderson smoother for general‐purpose and sample‐based spectral baseline estimation and peak extraction

Raman spectroscopy is a well‐established technique that allows both chemical and structural analysis of materials. Raman spectra are often complex and extracting meaningful information is easily hindered by spectral interferences; one of the most significant sources being variations in background. Raman spectra have diverse sources of background making it hard to eliminate them or theoretically to predict the form of the baseline, which frequently varies between samples. Although many different methods for baseline removal have been proposed, most require some form of user input. User input is also subjective and consequently less reproducible than automated methods and variations in baseline subtraction can distort peak heights leading to erroneous results. We present a Bayesian Whittaker–Henderson smoother for spectral baseline estimation and peak extraction. It is a generalisation of the Whittaker–Henderson smoother, a regularised regression algorithm. We introduce hierarchical priors for model parameters of the smoother and propose a global aligner for consistent peak extraction across multiple spectra. We show that this novel smoother significantly outperforms several existing smoothers. Copyright © 2012 John Wiley & Sons, Ltd.

Microwave Imaging Within the First-Order Born Approximation by Means of the Contrast-Field Bayesian Compressive Sensing

A new approach based the contrast field (CF) formulation of the microwave imaging problem that exploits the Bayesian compressive sampling (BCS) paradigm is proposed for the reconstruction of sparse distributions of weak scatterers. Towards this end, the original inverse scattering problem is recast to a probabilistic sparseness constrained optimization by introducing suitable hierarchical priors as sparsity constraints. A fast relevance vector machine (RVM) is then employed to reconstruct the scatterers as well as to estimate the “confidence level” of the inversion. Representative numerical results are presented to illustrate the method as well as to assess its potentialities and limitations in terms of inversion accuracy, computational efficiency, and robustness. Comparisons with state-of-the-art deterministic and stochastic reconstruction methodologies still within the Born approximation (BA) are discussed, as well.

Towards hierarchical blackboard mapping on a whiskered robot

The paradigm case for robotic mapping assumes large quantities of sensory information which allow the use of relatively weak priors. In contrast, the present study considers the mapping problem for a mobile robot, CrunchBot, where only sparse, local tactile information from whisker sensors is available. To compensate for such weak likelihood information, we make use of low-level signal processing and strong hierarchical object priors. Hierarchical models were popular in classical blackboard systems but are here applied in a Bayesian setting as a mapping algorithm. The hierarchical models require reports of whisker distance to contact and of surface orientation at contact, and we demonstrate that this information can be retrieved by classifiers from strain data collected by CrunchBot’s physical whiskers. We then provide a demonstration in simulation of how this information can be used to build maps (but not yet full SLAM) in an zero-odometry-noise environment containing walls and table-like hierarchical objects.

Hierarchical shrinkage priors and model fitting for high-dimensional generalized linear models.

Abstract Genetic and other scientific studies routinely generate very many predictor variables, which can be naturally grouped, with predictors in the same groups being highly correlated. It is desirable to incorporate the hierarchical structure of the predictor variables into generalized linear models for simultaneous variable selection and coefficient estimation. We propose two prior distributions: hierarchical Cauchy and double-exponential distributions, on coefficients in generalized linear models. The hierarchical priors include both variable-specific and group-specific tuning parameters, thereby not only adopting different shrinkage for different coefficients and different groups but also providing a way to pool the information within groups. We fit generalized linear models with the proposed hierarchical priors by incorporating flexible expectation-maximization (EM) algorithms into the standard iteratively weighted least squares as implemented in the general statistical package R. The methods are illustrated with data from an experiment to identify genetic polymorphisms for survival of mice following infection with Listeria monocytogenes. The performance of the proposed procedures is further assessed via simulation studies. The methods are implemented in a freely available R package BhGLM (http://www.ssg.uab.edu/bhglm/).

Hierarchical priors for bias parameters in Bayesian sensitivity analysis for unmeasured confounding

Recent years have witnessed new innovation in Bayesian techniques to adjust for unmeasured confounding. A challenge with existing methods is that the user is often required to elicit prior distributions for high-dimensional parameters that model competing bias scenarios. This can render the methods unwieldy. In this paper, we propose a novel methodology to adjust for unmeasured confounding that derives default priors for bias parameters for observational studies with binary covariates. The confounding effects of measured and unmeasured variables are treated as exchangeable within a Bayesian framework. We model the joint distribution of covariates by using a log-linear model with pairwise interaction terms. Hierarchical priors constrain the magnitude and direction of bias parameters. An appealing property of the method is that the conditional distribution of the unmeasured confounder follows a logistic model, giving a simple equivalence with previously proposed methods. We apply the method in a data example from pharmacoepidemiology and explore the impact of different priors for bias parameters on the analysis results.

Bayesian Approaches to Copula Modelling

Copula models have become one of the most widely used tools in the applied modelling of multivariate data. Similarly, Bayesian methods are increasingly used to obtain efficient likelihood-based inference. However, to date, there has been only limited use of Bayesian approaches in the formulation and estimation of copula models. This article aims to address this shortcoming in two ways. First, to introduce copula models and aspects of copula theory that are especially relevant for a Bayesian analysis. Second, to outline Bayesian approaches to formulating and estimating copula models, and their advantages over alternative methods. Copulas covered include Archimedean, copulas constructed by inversion, and vine copulas; along with their interpretation as transformations. A number of parameterisations of a correlation matrix of a Gaussian copula are considered, along with hierarchical priors that allow for Bayesian selection and model averaging for each parameterisation. Markov chain Monte Carlo sampling schemes for fitting Gaussian and D-vine copulas, with and without selection, are given in detail. The relationship between the prior for the parameters of a D-vine, and the prior for a correlation matrix of a Gaussian copula, is discussed. Last, it is shown how to compute Bayesian inference when the data are discrete-valued using data augmentation. This approach generalises popular Bayesian methods for the estimation of models for multivariate binary and other ordinal data to more general copula models. Bayesian data augmentation has substantial advantages over other methods of estimation for this class of models.

Applying Bayesian mixtures-of-experts models to statistical description of smart power semiconductor reliability

Reliability prediction of semiconductor devices gains importance, since demand increases and resources, e.g. time, are restricted. Normally, methods focusing on technology aspects are applied. This work presents a more mathematical approach by using Bayesian statistics. Physical failure inspection and past research indicate that the data follow a bimodal distribution. Therefore, we suggest using a heteroscedastic mixture of two normal distributions to model the given data. To incorporate the dependency on different test settings, linear models are used for the means and the mixing proportion. Gamma distributions are proposed as priors for the model parameters, due to the physical restrictions concerning the sample space. For the variances hierarchical inverse gamma priors are applied. Sampling from the posterior is done by using Monte Carlo Markov Chain methods. The proposed mixtures-of-experts model shows good adaption to the behavior of the measurements as well as good prediction quality.

Hierarchical Commensurate and Power Prior Models for Adaptive Incorporation of Historical Information in Clinical Trials

Bayesian clinical trial designs offer the possibility of a substantially reduced sample size, increased statistical power, and reductions in cost and ethical hazard. However when prior and current information conflict, Bayesian methods can lead to higher than expected type I error, as well as the possibility of a costlier and lengthier trial. This motivates an investigation of the feasibility of hierarchical Bayesian methods for incorporating historical data that are adaptively robust to prior information that reveals itself to be inconsistent with the accumulating experimental data. In this article, we present several models that allow for the commensurability of the information in the historical and current data to determine how much historical information is used. A primary tool is elaborating the traditional power prior approach based upon a measure of commensurability for Gaussian data. We compare the frequentist performance of several methods using simulations, and close with an example of a colon cancer trial that illustrates a linear models extension of our adaptive borrowing approach. Our proposed methods produce more precise estimates of the model parameters, in particular, conferring statistical significance to the observed reduction in tumor size for the experimental regimen as compared to the control regimen.

3D segmentation of rodent brain structures using hierarchical shape priors and deformable models.

In this paper, we propose a method to segment multiple rodent brain structures simultaneously. This method combines deformable models and hierarchical shape priors within one framework. The deformation module employs both gradient and appearance information to generate image forces to deform the shape. The shape prior module uses Principal Component Analysis to hierarchically model the multiple structures at both global and local levels. At the global level, the statistics of relative positions among different structures are modeled. At the local level, the shape statistics within each structure is learned from training samples. Our segmentation method adaptively employs both priors to constrain the intermediate deformation result. This prior constraint improves the robustness of the model and benefits the segmentation accuracy. Another merit of our prior module is that the size of the training data can be small, because the shape prior module models each structure individually and combines them using global statistics. This scheme can preserve shape details better than directly applying PCA on all structures. We use this method to segment rodent brain structures, such as the cerebellum, the left and right striatum, and the left and right hippocampus. The experiments show that our method works effectively and this hierarchical prior improves the segmentation performance.

Hierarchical Shrinkage Priors for Dynamic Regressions with Many Predictors

This paper builds on a simple unified representation of shrinkage Bayes estimators based on hierarchical Normal-Gamma priors. Various popular penalized least squares estimators for shrinkage and selection in regression models can be recovered using this single hierarchical Bayes formulation. Using 129 U.S. macroeconomic quarterly variables for the period 1959-2010 I exhaustively evaluate the forecasting properties of Bayesian shrinkage in regressions with many predictors. Results show that for particular data series hierarchical shrinkage dominates factor model forecasts, and hence it becomes a valuable addition to existing methods for handling large dimensional data.

A Bayesian Model for Biclustering with Applications

Summary The paper proposes a Bayesian method for biclustering with applications to gene microarray studies, where we want to cluster genes and experimental conditions simultaneously. We begin by embedding bicluster analysis into the framework of a plaid model with random effects. The corresponding likelihood is then regularized by the hierarchical priors in each layer. The resulting posterior, which is asymptotically equivalent to a penalized likelihood, can attenuate the effect of high dimensionality on cluster predictions. We provide an empirical Bayes algorithm for sampling posteriors, in which we estimate the cluster memberships of all genes and samples by maximizing an explicit marginal posterior of these memberships. The new algorithm makes the estimation of the Bayesian plaid model computationally feasible and efficient. The performance of our procedure is evaluated on both simulated and real microarray gene expression data sets. The numerical results show that our proposal substantially outperforms the original plaid model in terms of misclassification rates across a range of scenarios. Applying our method to two yeast gene expression data sets, we identify several new biclusters which show the enrichment of known annotations of yeast genes.

Neighbor-Dependent Ramachandran Probability Distributions of Amino Acids Developed from a Hierarchical Dirichlet Process Model

Distributions of the backbone dihedral angles of proteins have been studied for over 40 years. While many statistical analyses have been presented, only a handful of probability densities are publicly available for use in structure validation and structure prediction methods. The available distributions differ in a number of important ways, which determine their usefulness for various purposes. These include: 1) input data size and criteria for structure inclusion (resolution, R-factor, etc.); 2) filtering of suspect conformations and outliers using B-factors or other features; 3) secondary structure of input data (e.g., whether helix and sheet are included; whether beta turns are included); 4) the method used for determining probability densities ranging from simple histograms to modern nonparametric density estimation; and 5) whether they include nearest neighbor effects on the distribution of conformations in different regions of the Ramachandran map. In this work, Ramachandran probability distributions are presented for residues in protein loops from a high-resolution data set with filtering based on calculated electron densities. Distributions for all 20 amino acids (with cis and trans proline treated separately) have been determined, as well as 420 left-neighbor and 420 right-neighbor dependent distributions. The neighbor-independent and neighbor-dependent probability densities have been accurately estimated using Bayesian nonparametric statistical analysis based on the Dirichlet process. In particular, we used hierarchical Dirichlet process priors, which allow sharing of information between densities for a particular residue type and different neighbor residue types. The resulting distributions are tested in a loop modeling benchmark with the program Rosetta, and are shown to improve protein loop conformation prediction significantly. The distributions are available at http://dunbrack.fccc.edu/hdp.

Bayesian forecasting of parts demand

As supply chains for high technology products increase in complexity, and as the performance expectations of these supply chains also increase, forecasts of parts demands have become indispensable to effective operations management in these markets. Unfortunately, rapid technological change and an abundance of product configurations mean that the demand for parts in high-tech products is frequently volatile and hard to forecast. The paper describes a Bayesian statistical model which was developed to forecast the parts demand for Sun Microsystems, Inc., a major vendor of enterprise computer products. The model embodies a parametric description of the part’s life cycle, allowing it to anticipate changes in demand over time. Furthermore, using hierarchical priors, the model is able to pool demand patterns for a collection of parts, producing calibrated forecasts for new parts with little or no demand history. The paper discusses the problem addressed by the model, the model itself, and a procedure for calibrating it, then compares its forecast performance with those of alternatives.

High-Dimensional Sparse Factor Modeling: Applications in Gene Expression Genomics

We describe studies in molecular profiling and biological pathway analysis that use sparse latent factor and regression models for microarray gene expression data. We discuss breast cancer applications and key aspects of the modeling and computational methodology. Our case studies aim to investigate and characterize heterogeneity of structure related to specific oncogenic pathways, as well as links between aggregate patterns in gene expression profiles and clinical biomarkers. Based on the metaphor of statistically derived “factors” as representing biological “subpathway” structure, we explore the decomposition of fitted sparse factor models into pathway subcomponents and investigate how these components overlay multiple aspects of known biological activity. Our methodology is based on sparsity modeling of multivariate regression, ANOVA, and latent factor models, as well as a class of models that combines all components. Hierarchical sparsity priors address questions of dimension reduction and multiple comparisons, as well as scalability of the methodology. The models include practically relevant non-Gaussian/nonparametric components for latent structure, underlying often quite complex non-Gaussianity in multivariate expression patterns. Model search and fitting are addressed through stochastic simulation and evolutionary stochastic search methods that are exemplified in the oncogenic pathway studies. Supplementary supporting material provides more details of the applications, as well as examples of the use of freely available software tools for implementing the methodology.

On adaptive Bayesian inference

We study the rate of Bayesian consistency for hierarchical priors consisting of prior weights on a model index set and a prior on a density model for each choice of model index. Ghosal, Lember and Van der Vaart [2] have obtained general in-probability theorems on the rate of convergence of the resulting posterior distributions. We extend their results to almost sure assertions. As an application we study log spline densities with a finite number of models and obtain that the Bayes procedure achieves the optimal minimax rate n−γ/(2γ+1) of convergence if the true density of the observations belongs to the Hölder space Cγ[0,1]. This strengthens a result in [1; 2]. We also study consistency of posterior distributions of the model index and give conditions ensuring that the posterior distributions concentrate their masses near the index of the best model.