Natural Conjugate Research Articles

Bayesian analysis for reversible Markov chains

We introduce a natural conjugate prior for the transition matrix of a reversible Markov chain. This allows estimation and testing. The prior arises from random walk with reinforcement in the same way the Dirichlet prior arises from Pólya’s urn. We give closed form normalizing constants, a simple method of simulation from the posterior and a characterization along the lines of W. E. Johnson’s characterization of the Dirichlet prior.

Estimators for the Finite Mixture of Rayleigh Model Based on Progressively Censored Data

In this article, based on progressively Type-II censored samples from a heterogeneous population that can be represented by a finite mixture of two-component Rayleigh lifetime model, the problem of estimating the parameters and some lifetime parameters (reliability and hazard functions) are considered. Both Bayesian and maximum likelihood estimators are of interest. A class of natural conjugate prior densities is considered in the Bayesian setting. The Bayes estimators are obtained using both the symmetric (squared error) loss function, and the asymmetric (LINEX and General Entropy) loss functions. It has been seen that the estimators obtained can be easily evaluated for this type of censoring by using suitable numerical methods. Finally, the performance of the estimates have been compared on the basis of their simulated maximum square error via a Monte Carlo simulation study.

Propagation of Population Pharmacokinetic Information Using a Bayesian Approach: Comparison with Meta-Analysis

We investigated the propagation of population pharmacokinetic information across clinical studies by applying Bayesian techniques. The aim was to summarize the population pharmacokinetic estimates of a study in appropriate statistical distributions in order to use them as Bayesian priors in consequent population pharmacokinetic analyses. Various data sets of simulated and real clinical data were fitted with WinBUGS, with and without informative priors. The posterior estimates of fittings with non-informative priors were used to build parametric informative priors and the whole procedure was carried on in a consecutive manner. The posterior distributions of the fittings with informative priors where compared to those of the meta-analysis fittings of the respective combinations of data sets. Good agreement was found, for the simulated and experimental datasets when the populations were exchangeable, with the posterior distribution from the fittings with the prior to be nearly identical to the ones estimated with meta-analysis. However, when populations were not exchangeble an alternative parametric form for the prior, the natural conjugate prior, had to be used in order to have consistent results. In conclusion, the results of a population pharmacokinetic analysis may be summarized in Bayesian prior distributions that can be used consecutively with other analyses. The procedure is an alternative to meta-analysis and gives comparable results. It has the advantage that it is faster than the meta-analysis, due to the large datasets used with the latter and can be performed when the data included in the prior are not actually available.

Conjugate Priors Represent Strong Pre‐Experimental Assumptions

Abstract. It is well known that Jeffreys’ prior is asymptotically least favorable under the entropy risk, i.e. it asymptotically maximizes the mutual information between the sample and the parameter. However, in this paper we show that the prior that minimizes (subject to certain constraints) the mutual information between the sample and the parameter is natural conjugate when the model belongs to a natural exponential family. A conjugate prior can thus be regarded as maximally informative in the sense that it minimizes the weight of the observations on inferences about the parameter; in other words, the expected relative entropy between prior and posterior is minimized when a conjugate prior is used.

Bayesian variants of some classical semiparametric regression techniques

Abstract This paper develops new Bayesian methods for semiparametric inference in the partial linear Normal regression model: y=zβ+f(x)+e where f(.) is an unknown function. These methods draw solely on the Normal linear regression model with natural conjugate prior. Hence, posterior results are available which do not suffer from some problems which plague the existing literature such as computational complexity. Methods for testing parametric regression models against semiparametric alternatives are developed. We discuss how these methods can, at some cost in terms of computational complexity, be extended to other models (e.g. qualitative choice models or those involving censoring or truncation) and provide precise details for a semiparametric probit model. We show how the assumption of Normal errors can easily be relaxed.

Extensions of the conjugate prior through the Kullback–Leibler separators

The conjugate prior for the exponential family, referred to also as the natural conjugate prior, is represented in terms of the Kullback–Leibler separator. This representation permits us to extend the conjugate prior to that for a general family of sampling distributions. Further, by replacing the Kullback–Leibler separator with its dual form, we define another form of a prior, which will be called the mean conjugate prior. Various results on duality between the two conjugate priors are shown. Implications of this approach include richer families of prior distributions induced by a sampling distribution and the empirical Bayes estimation of a high-dimensional mean parameter.

Bayesian Estimation of the Distribution Function of the Poisson Model

AbstractThis paper presents the Bayes estimators of the Poisson distribution function based on complete and truncated data under a natural conjugate prior. Laplace transform of the incomplete gamma function and the Gauss hypergeometric function have been employed in order to overcome the intractability of the integrals. Numerical examples from biosciences are given to illustrate the results. A Monte Carlo study has been carried out to compare Bayes estimators under complete data with the corresponding maximum liklihood estimators.

Form-invariant bivariate weighted models

In this paper the class of continuous bivariate distributions that has form-invariant weighted distribution with weight function [Formula: See Text] is identified. It is shown that the class includes some well known bivariate models. Bayesian inference on the parameters of the class is considered and it is shown that there exist natural conjugate priors for the parameters.

The Contribution of Domestic and External Factors to Emerging Market Devaluation Crises: An Early Warning Systems Approach

This paper uses Bayesian techniques to compare three definitions of optimality for the basic job search model: the standard income-maximizing definition, an approximation to the standard definition, and a simple alternative. The important role of prior choice in these comparisons is illustrated. Using natural conjugate priors to represent hypothetical samples of data, we find that the simple alternative is preferred to the standard definition of optimality. However, using priors constructed from findings in the literature, we are able to find some evidence in favor of the standard definition of optimality.

Testing Optimality in Job Search Models

This paper uses Bayesian techniques to compare three definitions of optimality for the basic job search model: the standard income-maximizing definition, an approximation to the standard definition, and a simple alternative. The important role of prior choice in these comparisons is illustrated. Using natural conjugate priors to represent hypothetical samples of data, we find that the simple alternative is preferred to the standard definition of optimality. However, using priors constructed from findings in the literature, we are able to find some evidence in favor of the standard definition of optimality.

Testing Optimality in Job Search Models

This paper uses Bayesian techniques to compare three definitions of optimality for the basic job search model: the standard income-maximizing definition, an approximation to the standard definition, and a simple alternative. The important role of prior choice in these comparisons is illustrated. Using natural conjugate priors to represent hypothetical samples of data, we find that the simple alternative is preferred to the standard definition of optimality. However, using priors constructed from findings in the literature, we are able to find some evidence in favor of the standard definition of optimality.

A decision model for deteriorating repairable systems

Complex systems are generally repaired rather than replaced after failures. If deterioration in a repairable system is detected, i.e., the successive inter-failure times become stochastically smaller and smaller, then the decision of when to overhaul or discard the system is of fundamental importance. However, such a decision involves many uncertainties, such as the initial status of the system, the degree of deterioration, expected system lifetime, repair cost, accidental cost, etc. which are important factors and need to be evaluated carefully. In this paper, a Bayesian decision theoretic approach is developed. A nonhomogeneous Poisson process with a power law failure intensity function is used to describe the behavior of the deteriorating repairable system. Also, a proposed natural conjugate prior distribution is applied to make the Bayesian decision-making process more effective and efficient.

Bayesian reliability modeling with a new loss function and numerical simulation

The aim the present study is to develop a Bayesian reliability model using a new loss function that has been introduced by Higgins and Tsokos. The underlyng failure model is the exponential and four priors are being used: the inverted gamma , the general uniform, the exponential and the lognormal. The concept of efficienncy along with the notion of integrated mean square are being used as a criterion to analytically and numerically compare our findings with respect to the natural conjugate prior. It is shown that the Bayesian reliability estimates are in general sensitive to the choice of the prior probability distribution.

Non-conjugate Prior Distribution Assessment for Multivariate Normal Sampling

Summary Elicitation methods are proposed for quantifying expert opinion about a multivariate normal sampling model. The natural conjugate prior family imposes a relationship between the mean vector and the covariance matrix that can portray an expert's opinion poorly. Instead we assume that opinions about the mean and the covariance are independent and suggest innovative forms of question which enable the expert to quantify separately his or her opinion about each of these parameters. Prior opinion about the mean vector is modelled by a multivariate normal distribution and about the covariance matrix by both an inverse Wishart distribution and a generalized inverse-Wishart (GIW) distribution. To construct the latter, results are developed that give insight into the GIW parameters and their interrelationships. Certain of the elicitation methods exploit unconditional assessments as fully as possible, since these can reflect an expert's beliefs more accurately than conditional assessments. Methods are illustrated through an example.

Benchmark priors for Bayesian model averaging

In contrast to a posterior analysis given a particular sampling model, posterior model probabilities in the context of model uncertainty are typically rather sensitive to the specification of the prior. In particular, ‘diffuse’ priors on model-specific parameters can lead to quite unexpected consequences. Here we focus on the practically relevant situation where we need to entertain a (large) number of sampling models and we have (or wish to use) little or no subjective prior information. We aim at providing an ‘automatic’ or ‘benchmark’ prior structure that can be used in such cases. We focus on the normal linear regression model with uncertainty in the choice of regressors. We propose a partly non-informative prior structure related to a natural conjugate g-prior specification, where the amount of subjective information requested from the user is limited to the choice of a single scalar hyperparameter g 0 j . The consequences of different choices for g 0 j are examined. We investigate theoretical properties, such as consistency of the implied Bayesian procedure. Links with classical information criteria are provided. More importantly, we examine the finite sample implications of several choices of g 0 j in a simulation study. The use of the MC 3 algorithm of Madigan and York (Int. Stat. Rev. 63 (1995) 215), combined with efficient coding in Fortran, makes it feasible to conduct large simulations. In addition to posterior criteria, we shall also compare the predictive performance of different priors. A classic example concerning the economics of crime will also be provided and contrasted with results in the literature. The main findings of the paper will lead us to propose a ‘benchmark’ prior specification in a linear regression context with model uncertainty.

Bayesian analysis of disturbances variance in the linear regression model under asymmetric loss functions

The present paper obtains a Bayes estimator of the disturbance variance in a linear regression model under two asymmetric loss functions viz., LINEX loss function and SQUAREX loss function, assuming a natural conjugate prior distribution for the parameters. The expression for the Bayes risk of the estimator is also derived.

Independence Structure of Natural Conjugate Densities to Exponential Families and the Gibbs' Sampler

In this paper the independence between a block of natural parameters and the complementary block of mean value parameters holding for densities which are natural conjugate to some regular exponential families is used to design in a convenient way a Gibbs' sampler with block updates. Even when the densities of interest are obtained by conditioning to zero a block of natural parameters in a density conjugate to a larger saturated model, the updates require only the computation of marginal distributions under the uncondi- tional density. For exponential families which are closed under marginalization, including both the zero mean Gaussian family and the cross-classified Bernoulli family such an implementation of the Gibbs' sampler can be seen as an Iterative Proportional Fitting algorithm with random inputs.

Bayesian decision analysis and reliability certification

Abstract Reliability certification is set as a problem of Bayesian Decision Analysis. Uncertainties about the system reliability are quantified by assuming the parameters of the models describing the stochastic behavior of components as random variables. A utility function quantifies the relative value of each possible level of system reliability after having been accepted or the opportunity loss of the same level if the system has been rejected. A decision about accepting or rejecting the system can be made either on the basis of the existing prior assessment of uncertainties or after obtaining further information through testing of the components or the system at a cost. The concepts of value of perfect information, expected value of sample information and the expected net gain of sampling are specialized to the reliability of a multicomponent system to determine the optimum component testing scheme prior to deciding on the system's certification. A component importance ranking is proposed on the basis of the expected value of perfect information about the reliability of each component. The proposed approach is demonstrated on a single component system failing according to a Poisson random process and with natural conjugate prior probability density functions (pdf) for the failure rate and for a multicomponent system under general assumptions.

Improving Viterbi Bayesian predictive classification via sequential bayesian learning in robust speech recognition

In this paper, we extend our proposed Viterbi Bayesian predictive classification (VBPC) algorithm to a new class of prior probability density function (pdf), namely a family of natural conjugate prior pdf's of the complete-data density in continuous density hidden Markov model (CDHMM) and their mixtures. In this way, we can on-line adapt the prior pdf via a sequential Bayesian learning algorithm when some new data are available, so that the performance of VBPC can be continuously improved. Moreover, we also study a sequential Bayesian learning strategy for CDHMM based on a finite mixture approximation of its prior/posterior density which attempts to derive a more accurate prior pdf to describe the unknown mismatches. The experimental results on a speaker-independent recognition task of isolated Japanese digits confirm the viability and the usefulness of the proposed method.

A natural conjugate prior for the nonhomogeneous poisson process with an exponential intensity function

This article presents a natural conjugate prior for the nonhomogeneous Poisson process (NHPP) with an exponential intensity function, for modeling the failure rate of repairable systems. The behavior of the conjugate prior distribution with respect to its parameters is studied, and the use of this prior in Bayesian estimation is compared to two other estimation approaches (the use of independent prior distributions, and the bivariate normal distribution). The use of the conjugate prior proposed here facilitates Bayesian statistical analysis of aging. In particular, the proposed prior allows us to explicitly account for dependence between the initial failure rate and the aging rate. This is a significant improvement over the assumptions made in most prior work (either the assumption that the aging rate is known, or the assumption that the initial failure rate and the aging rate are independent). Monte Carlo simulation shows that Bayesian estimation using the proposed prior generally performs at least as well as Bayesian estimation using independent priors for the initial failure rate and the aging rate,except in the case where the prior distribution underestimates both the initial failure rate and the aging rate.