Mixtures Of G-priors Research Articles

Model Selection-Based Estimation for Generalized Additive Models Using Mixtures of g-priors: Towards Systematization

Model Selection-Based Estimation for Generalized Additive Models Using Mixtures of g-priors: Towards Systematization

Power-Expected-Posterior Priors as Mixtures of g-Priors in Normal Linear Models

One of the main approaches used to construct prior distributions for objective Bayes methods is the concept of random imaginary observations. Under this setup, the expected-posterior prior (EPP) offers several advantages, among which it has a nice and simple interpretation and provides an effective way to establish compatibility of priors among models. In this paper, we study the power-expected-posterior prior as a generalization to the EPP in objective Bayesian model selection under normal linear models. We prove that it can be represented as a mixture of g-prior, like a wide range of prior distributions under normal linear models, and thus posterior distributions and Bayes factors are derived in closed form, keeping therefore its computational tractability. Following this result, we can naturally prove that desiderata (criteria for objective Bayesian model comparison) hold for the PEP prior. Comparisons with other mixtures of g-prior are made and results are presented in simulated and real-life datasets.

Posterior consistency of [formula omitted]-prior for variable selection with a growing number of parameters

Recently, Bayesian procedures based on mixtures of g-priors have been widely studied for the variable selection problem in linear models. Maruyama and George (2011) proposed an explicit Bayesian approach without integral representation and showed its posterior model selection consistency when the number of parameters, k, is fixed. Given that linear models with a growing number of parameters have also received increasing attention in practice, we further concentrate on its corresponding posterior model selection consistency when k grows with the sample size, n, at the rate of k=O(nb),0≤b≤1. Specifically, we consider the Bayesian approach with two most commonly used types of priors on the class of models and derive conditions under which the resulting Bayesian approaches achieve such consistency. In addition, we study the case for linear models with the non-normal errors. The proposed results are compared with the existing ones in the literature.

Power-Expected-Posterior Priors for Generalized Linear Models

The power-expected-posterior (PEP) prior provides an objective, automatic, consistent and parsimonious model selection procedure. At the same time it resolves the conceptual and computational problems due to the use of imaginary data. Namely, (i) it dispenses with the need to select and average across all possible minimal imaginary samples, and (ii) it diminishes the effect that the imaginary data have upon the posterior distribution. These attributes allow for large sample approximations, when needed, in order to reduce the computational burden under more complex models. In this work we generalize the applicability of the PEP methodology, focusing on the framework of generalized linear models (GLMs), by introducing two new PEP definitions which are in effect applicable to any general model setting. Hyper-prior extensions for the power parameter that regulates the contribution of the imaginary data are introduced. We further study the validity of the predictive matching and of the model selection consistency, providing analytical proofs for the former and empirical evidence supporting the latter. For estimation of posterior model and inclusion probabilities we introduce a tuning-free Gibbs-based variable selection sampler. Several simulation scenarios and one real life example are considered in order to evaluate the performance of the proposed methods compared to other commonly used approaches based on mixtures of g-priors. Results indicate that the GLM-PEP priors are more effective in the identification of sparse and parsimonious model formulations.

On the correspondence from Bayesian log-linear modelling to logistic regression modelling with g-priors

Consider a set of categorical variables where at least one of them is binary. The log-linear model that describes the counts in the resulting contingency table implies a specific logistic regression model, with the binary variable as the outcome. Within the Bayesian framework, the g-prior and mixtures of g-priors are commonly assigned to the parameters of a generalized linear model. We prove that assigning a g-prior (or a mixture of g-priors) to the parameters of a certain log-linear model designates a g-prior (or a mixture of g-priors) on the parameters of the corresponding logistic regression. By deriving an asymptotic result, and with numerical illustrations, we demonstrate that when a g-prior is adopted, this correspondence extends to the posterior distribution of the model parameters. Thus, it is valid to translate inferences from fitting a log-linear model to inferences within the logistic regression framework, with regard to the presence of main effects and interaction terms.

A mixture of g-priors for variable selection when the number of regressors grows with the sample size

We consider the problem of variable selection in linear regression using mixtures of g-priors. A number of mixtures have been proposed in the literature which work well, especially when the number of regressors p is fixed. In this paper, we propose a mixture of g-priors suitable for the case when p grows with the sample size n, more specifically when $$p=O(n^b)$$ , $$0<b<1$$ . The marginal density based on the proposed mixture has a nice approximation with a closed form expression, which makes application of the method as tractable as an information criterion-based method. The proposed method satisfies fundamental properties like model selection consistency when the true model lies in the model space, and also consistency in an appropriate sense, under misspecified models setup. The method is quite robust in the sense that the above properties are not confined to normal linear models; they continue to hold under reasonable conditions for a general class of error distributions. Finally, we compare the performance of the proposed prior theoretically with that of some other mixtures of g-priors. We also compare it with several other Bayesian methods of model selection using simulated data sets. Theoretically, as well as in simulations, it emerges that unlike most of the other methods of model selection, the proposed prior is competent enough while selecting the true model irrespective of its dimension.

Inference From Intrinsic Bayes’ Procedures Under Model Selection and Uncertainty

In this article, we present a fully coherent and consistent objective Bayesian analysis of the linear regression model using intrinsic priors. The intrinsic prior is a scaled mixture of g-priors and promotes shrinkage toward the subspace defined by a base (or null) model. While it has been established that the intrinsic prior provides consistent model selectors across a range of models, the posterior distribution of the model parameters has not previously been investigated. We prove that the posterior distribution of the model parameters is consistent under both model selection and model averaging when the number of regressors is fixed. Further, we derive tractable expressions for the intrinsic posterior distribution as well as sampling algorithms for both a selected model and model averaging. We compare the intrinsic prior to other mixtures of g-priors and provide details on the consistency properties of modified versions of the Zellner–Siow prior and hyper g-priors. Supplementary materials for this article are available online.