Hierarchical Priors Research Articles

Bayesian Spatiotemporal Modeling Using Hierarchical Spatial Priors, with Applications to Functional Magnetic Resonance Imaging (with Discussion)

We propose a spatiotemporal Bayesian variable selection model for detecting activation in functional magnetic resonance imaging (fMRI) settings. Following recent research in this area, we use binary indicator variables for classifying active voxels. We assume that the spatial dependence in the images can be accommodated by applying an areal model to parcels of voxels. The use of parcellation and a spatial hierarchical prior (instead of the popular Ising prior) results in a posterior distribution amenable to exploration with an efficient Markov chain Monte Carlo (MCMC) algorithm. We study the properties of our approach by applying it to simulated data and an fMRI data set.

Automatic Building Rooftop Extraction From Aerial Images via Hierarchical RGB-D Priors

Accurate building rooftop extraction from high-resolution aerial images is of crucial importance in a wide range of applications. Owing to the varying appearance and large-scale range of scene objects, especially for building rooftops in different scales and heights, single-scale or individual prior-based extraction technique is insufficient in pursuing efficient, generic, and accurate extraction results. The trend toward integrating multiscale or several cue techniques appears to be the best way; thus, such integration is the focus of this paper. We first propose a novel salient rooftop detector integrating four correlative RGB-D priors (depth cue, uniqueness prior, shape prior, and transition surface prior) for improved rooftop extraction to address the preceding complex issues mentioned. Then, these correlative cues are computed from image layers created by our multilevel segmentation and further fused into the state-of-the-art high-order conditional random field (CRF) framework to locate the rooftop. Finally, an iterative optimization strategy is applied for high-quality solving, which can robustly handle varying appearance of building rooftops. Performance evaluations in the SZTAKI-INRIA benchmark data sets show that our method outperforms the traditional color-based algorithm and the original high-order CRF algorithm and its variants. The proposed algorithm is also evaluated and found to produce consistently satisfactory results for various large-scale, real-world data sets.

Bayesian inference and prediction of a multiple-change-point panel model with nonparametric priors

Change point models using hierarchical priors have been very successful estimating the parameter values of short-lived regimes. However, hierarchical priors have been parametric which leads to shrinkage in the estimates of extraordinary regime parameters. We overcome this by modeling the hierarchical priors nonparametrically. We also extend the change point to a panel of change point processes where the prior shares in the probabilities of changing regimes. When applied to the returns from a panel of actively managed, US equity, mutual funds our multiple-change-point panel model finds mutual fund skill is not persistent but changes over time.

High-Resolution Sparse Subband Imaging Based on Bayesian Learning With Hierarchical Priors

To obtain higher range resolution without incurring significant hardware costs, this paper proposes a novel method for high-resolution sparse subband imaging based on Bayesian learning. The signal model is derived and a probabilistic model is constructed. In particular, hierarchical sparse-promoting priors are imposed on the distribution of scattering centers, which is conjugate to the likelihood function. Then, a closed-form solution is derived based on the MAP-expectation–maximization framework. A multilevel dictionary which automatically adjusts the distance between adjacent atoms is adopted to achieve refined estimation with moderate computational burden. Finally, a coherent processing method is addressed. Experimental results have demonstrated the effectiveness of the proposed method in low signal-to-noise ratio and complex target scenarios.

Experiments with learning graphical models on text

A rich variety of models are now in use for unsupervised modelling of text documents, and, in particular, a rich variety of graphical models exist, with and without latent variables. To date, there is inadequate understanding about the comparative performance of these, partly because they are subtly different, and they have been proposed and evaluated in different contexts. This paper reports on our experiments with a representative set of state of the art models: chordal graphs, matrix factorisation, and hierarchical latent tree models. For the chordal graphs, we use different scoring functions. For matrix factorisation models, we use different hierarchical priors, asymmetric priors on components. We use Boolean matrix factorisation rather than topic models, so we can do comparable evaluations. The experiments perform a number of evaluations: probability for each document, omni-directional prediction which predicts different variables, and anomaly detection. We find that matrix factorisation performed well at anomaly detection but poorly on the prediction task. Chordal graph learning performed the best generally, and probably due to its lower bias, often out-performed hierarchical latent trees.

Mountaintop removal mining alters stream salamander population dynamics

AbstractAimPopulation dynamics are often tightly linked to the condition of the landscape. Focusing on a landscape impacted by mountaintop removal coal mining (MTR), we ask the following questions: (1) How does MTR influence vital rates including occupancy, colonization and persistence probabilities, and conditional abundance of stream salamander species and life stages? (2) Do species and life stages respond similar to MTR mining or is there significant variation among species and life stages?LocationFreshwater and terrestrial habitats in Central Appalachia (South‐eastern Kentucky, USA).MethodsWe conducted salamander counts for three consecutive years in 23 headwater stream reaches in forested or previously mined landscapes. We used a hierarchical, N‐mixture model with dynamic occupancy to calculate species‐ and life stage‐specific occupancy, colonization and persistence rates, and abundance given occupancy. We examined the coefficients of the hierarchical priors to determine population variation among species and life stages.ResultsOver 3 years, reference sites had greater salamander abundances and were occupied at a much higher rate than streams impacted by MTR. At sites impacted by MTR mining, most salamander species and life stages exhibited reduced initial occupancy, colonization rates, persistence rates and conditional abundance relative to reference stream reaches. Furthermore, the rates in MTR sites showed low variance, reinforcing that species and life stages were responding similar to MTR.Main conclusionsSalamander populations in landscapes modified by MTR mining exhibited significantly reduced vital rates compared to reference sites. Yet, similarity in responses across species suggests that management or restoration may benefit the entire salamander assemblage. For example, reforestation could reduce landscape resistance, repair altered hydrologic regimes and allow for higher rates of colonization and persistence in streams impacted by MTR.

Debt regimes and the effectiveness of monetary policy

This paper develops a medium-scale non-linear model of the US economy. Our proposed model, a threshold vector autoregression with stochastic volatility, assumes that changes in government debt-to-GDP ratios drive the transition between regimes, capturing low and high debt regimes. The introduction of hierarchical priors enables us to flexibly shrink the empirical model towards the moments implied by a dynamic stochastic general equilibrium model. In the empirical application, we analyze regime-specific monetary policy shocks. Our findings indicate that the effect of monetary policy is less pronounced in ‘high’ debt regimes, pointing towards differences in the underlying monetary policy transmission mechanisms. Forecast error variance decompositions enable us to shed further light on the relative importance of monetary policy shocks within different debt regimes in terms of explaining the variance of forecast errors.

Bayesian Inference and Prediction of a Multiple-Change-Point Panel Model with Nonparametric Priors

Change point models using hierarchical priors share in the information of each regime when estimating the parameter values of a regime. Because of this sharing, hierarchical priors have been very successful when estimating the parameter values of short-lived regimes and predicting the out-of-sample behavior of the regime parameters. However, the hierarchical priors have been parametric. Their parametric nature leads to global shrinkage that biases the estimates of the parameter coefficient of extraordinary regimes toward the value of the average regime. To overcome this shrinkage, we model the hierarchical prior nonparametrically by letting the hyperparameter's prior—in other words, the hyperprior—be unknown and modeling it with a Dirichlet processes prior. To apply a nonparametric hierarchical prior to the probability of a break occurring, we extend the change point model to a multiple-change-point panel model. The hierarchical prior then shares in the cross-sectional information of the break processes to estimate the transition probabilities. We apply our multiple-change-point panel model to a longitudinal data set of actively managed, U.S. equity, mutual fund returns to measure fund performance and investigate the chances of a skilled fund being skilled in the future.

High-Dimensional Macroeconomic Forecasting Using Message Passing Algorithms

This paper proposes two distinct contributions to econometric analysis of large information sets and structural instabilities. First, it treats a regression model with time-varying coefficients, stochastic volatility and exogenous predictors, as an equivalent high-dimensional static regression problem with thousands of covariates. Inference in this specification proceeds using Bayesian hierarchical priors that shrink the high-dimensional vector of coefficients either towards zero or time-invariance. Second, it introduces the frameworks of factor graphs and message passing as a means of designing efficient Bayesian estimation algorithms. In particular, a Generalized Approximate Message Passing (GAMP) algorithm is derived that has low algorithmic complexity and is trivially parallelizable. The result is a comprehensive methodology that can be used to estimate time-varying parameter regressions with arbitrarily large number of exogenous predictors. In a forecasting exercise for U.S. price inflation this methodology is shown to work very well.

Variational Inference based Automatic Relevance Determination Kernel for Embedded Feature Selection of Noisy Industrial Data

In this paper, an embedded feature selection based on variational relevance vector machines is proposed to simultaneously perform feature selection and model construction. With the settings of specific hierarchical priors over the parameters of an automatic relevance determination kernel (ARDK) function, an approximate posterior distribution over these parameters is here derived and expressed as a multivariate Gaussian distribution, in which a first-order Taylor expansion-based Laplace approximation with respect to the parameters is introduced into the variational inference procedure. The posterior distributions, rather than generic pointwise estimates, over the rest of parameters of the model are also derived. The proposed method can simultaneously select relevant features and samples by adjusting the parameters of ARDK and the weighting vector, respectively. To verify the effectiveness of the proposed method, a synthetic dataset and a number of benchmark datasets, as well as a practical industrial dataset, are employed to solve the regression and classification problems. These experimental results indicate that the proposed method supports the mechanisms of feature selection and model construction while maintaining prediction performance, particularly in an industrial environment.

Heterogeneous reciprocal graphical models.

We develop novel hierarchical reciprocal graphical models to infer gene networks from heterogeneous data. In the case of data that can be naturally divided into known groups, we propose to connect graphs by introducing a hierarchical prior across group-specific graphs, including a correlation on edge strengths across graphs. Thresholding priors are applied to induce sparsity of the estimated networks. In the case of unknown groups, we cluster subjects into subpopulations and jointly estimate cluster-specific gene networks, again using similar hierarchical priors across clusters. We illustrate the proposed approach by simulation studies and three applications with multiplatform genomic data for multiple cancers.

AdOn HDP-HMM: An Adaptive Online Model for Segmentation and Classification of Sequential Data.

Recent years have witnessed an increasing need for the automated classification of sequential data, such as activities of daily living, social media interactions, financial series, and others. With the continuous flow of new data, it is critical to classify the observations on-the-fly and without being limited by a predetermined number of classes. In addition, a model should be able to update its parameters in response to a possible evolution in the distributions of the classes. This compelling problem, however, does not seem to have been adequately addressed in the literature, since most studies focus on offline classification over predefined class sets. In this paper, we present a principled solution for this problem based on an adaptive online system leveraging Markov switching models and hierarchical Dirichlet process priors. This adaptive online approach is capable of classifying the sequential data over an unlimited number of classes while meeting the memory and delay constraints typical of streaming contexts. In this paper, we introduce an adaptive "learning rate" that is responsible for balancing the extent to which the model retains its previous parameters or adapts to new observations. Experimental results on stationary and evolving synthetic data and two video data sets, TUM Assistive Kitchen and collated Weizmann, show a remarkable performance in terms of segmentation and classification, particularly for sequences from evolutionary distributions and/or those containing previously unseen classes.

Latent Class Mixture Models of Treatment Effect Heterogeneity

We provide a general Bayesian framework for modeling treatment effect heterogeneity in experiments with non-categorical outcomes. Our modeling approach incorporates latent class mixture components to capture discrete heterogeneity and regression interaction terms to capture continuous heterogeneity. Flexible error distributions allow robust posterior inference on parameters of interest. Hierarchical shrinkage priors on relevant parameters address multiple comparisons concerns. Leave-one-out cross validation estimates of expected posterior predictive density obtained through importance sampling, together with posterior predictive checks, provide a convenient method for model selection and evaluation. We apply our approach to a clinical trial comparing two HIV treatments and to an instrumental variable analysis of a natural experiment on the effect of Medicaid enrollment on emergency department utilization.

Holographic SAR Tomography Image Reconstruction by Combination of Adaptive Imaging and Sparse Bayesian Inference

In this letter, we propose an imaging algorithm for the holographic synthetic aperture radar tomography in the circumstance of sparse and nonuniform elevation circular passes. Considering the anisotropic behavior of scatterers and the off-grid effect of sparse signal recovery, the algorithm combines the 2-D adaptive imaging method for circular SAR and the sparse Bayesian inference-based method for elevation reconstruction. For each circular pass, the azimuth-range 2-D image can be formed by the adaptive imaging method, which depends on the preretrieved maximum azimuth response angle and the azimuth persistence width. To deal with the off-grid effect in elevation reconstruction, which is caused by the deviation between the true scatterers and the discretized imaging grids, the off-grid sparse Bayesian inference method jointly estimates the scatterers and elevation off-grid error by applying their hierarchical priors. Compared with the conventional compressive sensing method that does not concern the off-grid effect, the proposed algorithm can provide more accurate 3-D reconstruction for pointlike targets, which is verified by the real-data experiments.

Forecasting and Estimating Multiple Change-Point Models with an Unknown Number of Change Points

This paper develops a new approach to change-point modeling that allows for an unknown number of change points in the observed sample. Our model assumes that regime durations have a Poisson distribution. The model approximately nests the two most common approaches: the time-varying parameter model with a change point every period and the change-point model with a small number of regimes. We focus on the construction of reasonable hierarchical priors both for regime durations and for the parameters that characterize each regime. A Markov Chain Monte Carlo posterior sampler is constructed to estimate a change-point model for conditional means and variances. We find that our techniques work well in an empirical exercise involving U.S. inflation and GDP growth. Empirical results suggest that the number of change points is larger than previously estimated in these series and the implied model is similar to a time-varying parameter model with stochastic volatility.

Forecasting with Many Predictors Using Message Passing Algorithms

Machine learning methods are becoming increasingly popular in economics, due to the increased availability of large datasets. In this paper I evaluate a recently proposed algorithm called Generalized Approximate Message Passing (GAMP), which has been popular in signal processing and compressive sensing. I show how this algorithm can be combined with Bayesian hierarchical shrinkage priors typically used in economic forecasting, resulting in computationally efficient schemes for estimating high-dimensional regression models. Using Monte Carlo simulations I establish that in certain scenarios GAMP can achieve estimation accuracy comparable to traditional Markov chain Monte Carlo methods, at a tiny fraction of the computing time. In a forecasting exercise involving a large set of orthogonal macroeconomic predictors, I show that Bayesian shrinkage estimators based on GAMP perform very well compared to a large set of alternatives.

Ship Detection in Polarimetric SAR Images via Variational Bayesian Inference

In this paper, we propose a novel ship detection approach in polarimetric synthetic aperture radar (SAR) images via variational Bayesian inference. First, we express the polarimetric SAR image as a tensor, and decompose the SAR image as the sum of a sparse component associated with ships and a sea clutter component. These components are denoted by some latent variables. Then, we introduce hierarchical priors of the latent variables to establish the probabilistic model of ship detection. By using variational Bayesian inference, we estimate the posterior distributions of the latent variables. Finally, the ship detection result is obtained in the iterative Bayesian inference process. By virtue of the tensor representation of polarimetric SAR image, the proposed approach explicitly uses all the polarization channels of the SAR image, and avoids the possible information loss in scalar polarimetric feature representation. Moreover, the proposed approach needs no sliding windows. The variational Bayesian inference process actually uses all the pixels instead of the limited pixels in sliding windows. Thus, the proposed approach has good ship detection performance and shape preserving ability, which is especially suitable for congested sea areas. Experimental results accomplished over C-band RADARSAT-2 polarimetric SAR images demonstrate that the proposed approach can achieve state-of-the-art ship detection performance.

Bayesian Hierarchical Regression Models for QoE Estimation and Prediction in Audiovisual Communications

This paper presents a trial of establishing a framework for Bayesian statistical modeling of quality of experience (QoE) estimation and prediction in multimedia IP communications. As an example, we take a bandwidth guaranteed interactive audiovisual communication system with a QoE enhancement mechanism. By Bayesian statistical analysis, we demonstrate how QoE is affected by factors of the channel bandwidth, contents of tasks, customization of playout buffering control for QoE enhancement, users' individualities, and gender. Adopting a latent variable approach, we build Bayesian regression models with covariates of the factors and random effect terms having hierarchical priors for the users' attributes. We estimate the posterior probabilities of parameters in the models by performing Markov chain Monte Carlo simulations with WinBUGS. From among the models, we select the most plausible one and explore the sensitivity to priors. We then evaluate the QoE measure, which is the posterior mean of overall satisfaction measured as five-point scores, and examine the effects of the factors on the QoE through odds ratios. Also, the Bayesian model makes predictions of score frequencies and compares them with measured ones to reveal high accuracy of the prediction. In addition, a cross-validatory approach to the model checking is taken for investigating the adequacy of the model; the posterior predictive p-values of a discrepancy function of the model indicate that the model is satisfactory.

A Bayesian method for risk window estimation with application to HPV vaccine trial

In biomedical studies, it is often of interest to estimate how the risk profile of an adverse event is related to the timing of an intervention. For example, in randomized controlled clinical trials of bivalent human papillomavirus (HPV) vaccine, investigators are interested to know how miscarriage rate relates to the timing of HPV vaccination. A risk window is defined as an interval for the covariate where the risk of adverse event is elevated. Existing methods cannot make simultaneous inference on both the risk window and the magnitude of the risk. A hierarchical Bayesian logistic regression model is developed to estimate the risk window of miscarriage on the time of conception with respect to vaccination. Hierarchical priors are proposed and used in Markov Chain Monte Carlo for statistical inference. The performance of the Bayesian model and two existing methods is evaluated in simulation settings with varying risk windows and relative risks. The proposed model provides both point and interval estimates for the risk window regarding to vaccination, and captures its effect modification on miscarriage risk in pregnancy. Analysis of the vaccine trial using the proposed model shows no significant evidence of an association between the HPV vaccine and miscarriage risk. The hierarchical Bayesian model is useful in general in analyzing a randomized trial or an epidemiological study in which the effect of an agent is potentially modified by a temporal factor.

Hierarchical Region-Network Sparsity for High-Dimensional Inference in Brain Imaging.

Structured sparsity penalization has recently improved statistical models applied to high-dimensional data in various domains. As an extension to medical imaging, the present work incorporates priors on network hierarchies of brain regions into logistic-regression to distinguish neural activity effects. These priors bridge two separately studied levels of brain architecture: functional segregation into regions and functional integration by networks. Hierarchical region-network priors are shown to better classify and recover 18 psychological tasks than other sparse estimators. Varying the relative importance of region and network structure within the hierarchical tree penalty captured complementary aspects of the neural activity patterns. Local and global priors of neurobiological knowledge are thus demonstrated to offer advantages in generalization performance, sample complexity, and domain interpretability.