Posterior Conditional Distributions Research Articles

Bayesian framework for interval-valued data using Jeffreys’ prior and posterior predictive checking methods

Interval-valued data have been widely used in both academic and industry research areas. With the development of computational statistics, the Bayesian methods are growing rapidly. Based on the existing theoretical researches, this paper tries to establish a Bayesian framework for interval-valued data, which contains the determination of Jeffreys’ prior, posterior inferences, and posterior predictive checking methods. First, Jeffreys’ prior for interval-valued data is proposed for the first time. Then, the posterior conditional distribution is derived, and posterior samples are obtained by using Gibbs sampling methods. Based on posterior samples, we obtain the Bayesian point and credible interval estimations for interval-valued data. Finally, we propose a posterior predictive checking method for interval-valued data based on novel test quantities. If the posterior predictive checking method shows that the data do not meet the distribution assumptions, we suggest the use of a data transformation. The effectiveness of our method is demonstrated by simulations and two cardiological data sets.

Convex Nonparanormal Regression

Quantifying uncertainty in predictions or, more generally, estimating the posterior conditional distribution, is a core challenge in machine learning and statistics. We introduce Convex Nonparanormal Regression (CNR), a conditional nonparanormal approach for coping with this task. CNR involves a convex optimization of a posterior defined via a rich dictionary of pre-defined non linear transformations on Gaussians. It can fit an arbitrary conditional distribution, including multimodal and non-symmetric posteriors. For the special but powerful case of a piecewise linear dictionary, we provide a closed form of the posterior mean which can be used for point-wise predictions. Finally, we demonstrate the advantages of CNR over classical competitors using synthetic and real world data.

Bayesian maximum entropy and data fusion for processing qualitative data: theory and application for crowdsourced cropland occurrences in Ethiopia

Categorical data play an important role in a wide variety of spatial applications, while modeling and predicting this type of statistical variable has proved to be complex in many cases. Among other possible approaches, the Bayesian maximum entropy methodology has been developed and advocated for this goal and has been successfully applied in various spatial prediction problems. This approach aims at building a multivariate probability table from bivariate probability functions used as constraints that need to be fulfilled, in order to compute a posterior conditional distribution that accounts for hard or soft information sources. In this paper, our goal is to generalize further the theoretical results in order to account for a much wider type of information source, such as probability inequalities. We first show how the maximum entropy principle can be implemented efficiently using a linear iterative approximation based on a minimum norm criterion, where the minimum norm solution is obtained at each step from simple matrix operations that converges to the requested maximum entropy solution. Based on this result, we show then how the maximum entropy problem can be related to the more general minimum divergence problem, which might involve equality and inequality constraints and which can be solved based on iterated minimum norm solutions. This allows us to account for a much larger panel of information types, where more qualitative information, such as probability inequalities can be used. When combined with a Bayesian data fusion approach, this approach deals with the case of potentially conflicting information that is available. Although the theoretical results presented in this paper can be applied to any study (spatial or non-spatial) involving categorical data in general, the results are illustrated in a spatial context where the goal is to predict at best the occurrence of cultivated land in Ethiopia based on crowdsourced information. The results emphasize the benefit of the methodology, which integrates conflicting information and provides a spatially exhaustive map of these occurrence classes over the whole country.

Bayesian inference of high-dimensional, cluster-structured ordinary differential equation models with applications to brain connectivity studies

We build a new ordinary differential equation (ODE) model for the directional interaction, also called effective connectivity, among brain regions whose activities are measured by intracranial electrocorticography (ECoG) data. In contrast to existing ODE models that focus on effective connectivity among only a few large anatomic brain regions and that rely on strong prior belief of the existence and strength of the connectivity, the proposed high-dimensional ODE model, motivated by statistical considerations, can be used to explore connectivity among multiple small brain regions. The new model, called the modular and indicator-based dynamic directional model (MIDDM), features a cluster structure, which consists of modules of densely connected brain regions, and uses indicators to differentiate significant and void directional interactions among brain regions. We develop a unified Bayesian framework to quantify uncertainty in the assumed ODE model, identify clusters, select strongly connected brain regions, and make statistical comparison between brain networks across different experimental trials. The prior distributions in the Bayesian model for MIDDM parameters are carefully designed such that the ensuing joint posterior distributions for ODE state functions and the MIDDM parameters have well-defined and easy-to-simulate posterior conditional distributions. To further speed up the posterior simulation, we employ parallel computing schemes in Markov chain Monte Carlo steps. We show that the proposed Bayesian approach outperforms an existing optimization-based ODE estimation method. We apply the proposed method to an auditory electrocorticography dataset and evaluate brain auditory network changes across trials and different auditory stimuli.

Estimation of genetic parameters for growth trait of turbot using Bayesian and REML approaches

Bayesian and restricted maximum likelihood (REML) approaches were used to estimate the genetic parameters in a cultured turbot Scophthalmus maximus stock. The data set consisted of harvest body weight from 2 462 progenies (17 months old) from 28 families that were produced through artificial insemination using 39 parent fish. An animal model was applied to partition each weight value into a fixed effect, an additive genetic effect, and a residual effect. The average body weight of each family, which was measured at 110 days post-hatching, was considered as a covariate. For Bayesian analysis, heritability and breeding values were estimated using both the posterior mean and mode from the joint posterior conditional distribution. The results revealed that for additive genetic variance, the posterior mean estimate (σ a 2 =9 320) was highest but with the smallest residual variance, REML estimates (σ a 2 =8 088) came second and the posterior mode estimate (σ a 2 =7 849) was lowest. The corresponding three heritability estimates followed the same trend as additive genetic variance and they were all high. The Pearson correlations between each pair of the three estimates of breeding values were all high, particularly that between the posterior mean and REML estimates (0.996 9). These results reveal that the differences between Bayesian and REML methods in terms of estimation of heritability and breeding values were small. This study provides another feasible method of genetic parameter estimation in selective breeding programs of turbot.

Dynamics of Consumption and Dividends Over the Business Cycle

We examine a trivariate time series model that is subject to a regime switch, where the shifts are governed by an unobserved, two-state variable that follows a Markov process. The analysis is performed in a Bayesian framework developed by Albert and Chib (1993), where the unobserved states are treated as missing data and then analyzed via Gibbs sampling. This approach generates the posterior conditional distribution of all the parameters given the hidden states, and the posterior conditional distribution of the states given the parameters. This allows us to obtain the estimated values of all the parameters of interest.

Estimation of genetic and phenotypic parameters for bacteriological status of the udder, somatic cell score, and milk yield in dairy sheep using a threshold animal model

The objective of this study was to estimate the genetic parameters for infection status (INF), as indicator of mastitis, SCS (i.e., log-transformed SCC), and milk yield (MY), by using a Gibbs sampling algorithm. The data comprised 17,843 test-day records of 2040 ewes. The pedigree file included 2948 animals. A bivariate variance component analysis was performed using the TM software. Fixed effects considered in the analysis were litter size, parity, flock by test-day interaction, year by season of lambing interaction, and stage of lactation; whereas the animal, and the permanent environmental effect within and across lactations were considered as random as well as the error. Flat priors were used for both fixed effects and variance components. Parameters were drawn from the posterior conditional distributions. The posterior means of heritability for MY, SCS and INF were equal to 0.14, 0.09, and 0.09, respectively; whereas the repeatability within lactation was around 0.30 for the three traits, and ranged between 0.29 and 0.41 across lactations. The genetic correlation between INF and SCS was equal to 0.93, suggesting that selection for low SCS would also lead to a reduced incidence of mastitis. On the other hand, the positive and moderate genetic correlation between mastitis and milk yield (0.59) confirms the antagonistic association between udder health and milk yield. Therefore, in breeding programs that emphasize milk yield, the unfavorable genetic correlation between milk yield and mastitis, may result in an increased incidence of the latter.

Particle filtering in high-dimensional chaotic systems

We present an efficient particle filtering algorithm for multiscale systems, which is adapted for simple atmospheric dynamics models that are inherently chaotic. Particle filters represent the posterior conditional distribution of the state variables by a collection of particles, which evolves and adapts recursively as new information becomes available. The difference between the estimated state and the true state of the system constitutes the error in specifying or forecasting the state, which is amplified in chaotic systems that have a number of positive Lyapunov exponents. In this paper, we propose a reduced-order particle filtering algorithm based on the homogenized multiscale filtering framework developed in Imkeller et al. “Dimensional reduction in nonlinear filtering: A homogenization approach,” Ann. Appl. Probab. (to be published). In order to adapt the proposed algorithm to chaotic signals, importance sampling and control theoretic methods are employed for the construction of the proposal density for the particle filter. Finally, we apply the general homogenized particle filtering algorithm developed here to the Lorenz'96 [E. N. Lorenz, “Predictability: A problem partly solved,” in Predictability of Weather and Climate, ECMWF, 2006 (ECMWF, 2006), pp. 40–58] atmospheric model that mimics mid-latitude atmospheric dynamics with microscopic convective processes.

Bayesian analysis of extreme value regression

The article takes up Bayesian inference in extreme value distributions and also considers extreme value regression, which appears relatively uncommon in the regression literature. Numerical methods are organized around Gibbs sampling. It is shown that simple and reliable numerical techniques can be devised by exploiting the particular form of the posterior conditional distributions. The sampling behaviour of the proposed estimators is also explored via Monte Carlo simulation.

Particle Filters in a Multiscale Environment: Homogenized Hybrid Particle Filter

State estimation of random dynamical systems with noisy observations has been an important problem in many areas of science and engineering. Efficient new algorithms to estimate the present and future state of a dynamic signal based upon corrupted, distorted, and possibly partial observations of the signal are required. Since the true state is usually hidden and evolves according to its own dynamics, the objective of this work is to get an optimal estimation of the true state via noisy observations. The theory of filtering provides a recursive procedure for estimating an evolving signal or state from a noisy observation process. We consider a particle filter approach for nonlinear filtering in multiscale dynamical systems. Particle filters represent the posterior conditional distribution of the state variables by a system of particles, which evolves and adapts recursively as new information becomes available. Particle filters suffer from computational inefficiency when applied to high dimensional problems. In practice, large numbers of particles may be required to provide adequate approximations in higher dimensional poblems. In several high dimensional applications, after a sequence of updates, the particle system will often collapse to a single point. With the help of rigorous dimensional reduction methods, particle filters could regain their versatility. Based on our theoretical developments (Park, J. H., Sri Namachchivaya, N., and Sowers, R. B., 2008, “A Problem in Stochastic Averaging of Nonlinear Filters,” Stochastics Dyn., 8(3), pp. 543–560; Park, J. H., Sowers, R. B., and Sri Namachchivaya, N., 2010, “Dimensional Reduction in Nonlinear Filtering,” Nonlinearity, 23(2), pp. 305–324), we devise an efficient particle filter algorithm, which is applicable to high dimensional multiscale nonlinear filtering problems. In this paper, we present the homogenized hybrid particle filtering method that combines homogenization of random dynamical systems, reduced order nonlinear filtering, and particle methods.

Genetic parameters and evaluation of the Tunisian dairy cattle population for milk yield by Bayesian and BLUP analyses

A Markov Chain Monte Carlo Bayesian method and BLUP analyses were used on Tunisian dairy cattle data. Data included 92,106 lactation records collected on 37,536 animals over 19 freshening years, from 1983 to 2001. Each record was partitioned into the fixed effects of herd-year, month of calving, and age-parity, a permanent environmental effect, an additive genetic effect, and a residual effect. Posterior conditional distributions were determined for variance components and model effects. Solutions (BLUE) and posterior means for levels of herd-year, month of calving, and age-parity showed similar patterns. Posterior means of heritability and repeatability were 0.17 ± 18 × 10 − 5 and 0.39 ± 8 × 10 − 5 , respectively. Posterior means of bull's breeding values were compared to bull's BLUP solutions. BLUP solutions were obtained using 0.17 and 0.39, estimated from the data, and 0.25 and 0.40 estimates for heritability and repeatability, respectively. Rank correlations between bull's posterior means and BLUP breeding values were 0.998 and 0.994 using genetic parameters estimated from the data and from the literature, respectively. This correlation coefficient was 0.995 between bull's BLUP solutions using either of the two sets of genetic parameters.

Generalized Nonlinear Modeling With Multivariate Free-Knot Regression Splines

A Bayesian method is presented for the nonparametric modeling of univariate and multivariate non-Gaussian response data. Data-adaptive multivariate regression splines are used where the number and location of the knot points are treated as random. The posterior model space is explored using a reversible-jump Markov chain Monte Carlo sampler. Computational difficulties are partly alleviated by introducing a random residual effect in the model that leaves many of the posterior conditional distributions of the model parameters in standard form. The use of the latent residual effect provides a convenient vehicle for modeling correlation in multivariate response data, and as such our method can be seen to generalize the seemingly unrelated regression model to non-Gaussian data. We illustrate the method on a number of examples, including two previously unpublished datasets relating to the spatial smoothing of multivariate accident data in Texas and the modeling of credit card use across multiple retail sectors.

Spatial prediction of categorical variables: the Bayesian maximum entropy approach

Being a non-linear method based on a rigorous formalism and an efficient processing of various information sources, the Bayesian maximum entropy (BME) approach has proven to be a very powerful method in the context of continuous spatial random fields, providing much more satisfactory estimates than those obtained from traditional linear geostatistics (i.e., the various kriging techniques). This paper aims at presenting an extension of the BME formalism in the context of categorical spatial random fields. In the first part of the paper, the indicator kriging and cokriging methods are briefly presented and discussed. A special emphasis is put on their inherent limitations, both from the theoretical and practical point of view. The second part aims at presenting the theoretical developments of the BME approach for the case of categorical variables. The three-stage procedure is explained and the formulations for obtaining prior joint distributions and computing posterior conditional distributions are given for various typical cases. The last part of the paper consists in a simulation study for assessing the performance of BME over the traditional indicator (co)kriging techniques. The results of these simulations highlight the theoretical limitations of the indicator approach (negative probability estimates, probability distributions that do not sum up to one, etc.) as well as the much better performance of the BME approach. Estimates are very close to the theoretical conditional probabilities, that can be computed according to the stated simulation hypotheses.

Multiple trait genetic analysis of underlying biological variables of production functions

Abstract A Bayesian procedure to analyze performance data from production functions is presented. The method implies the consideration of each parameter of the production function as a different trait. The systematic effects and genetic relationship between animals are taken into account in the adjustment of production for each animal. The method weights all possible sources of information. In this sense, it allows estimation of the parameters of production functions with only one data along the production cycle and to reduce the influence of outliers. Under analysis, marginal posterior distributions of the function parameters, breeding values, systematic effects and (co)variance components were obtained using the Gibbs sampling algorithm. The procedure requires the definition of the joint posterior distribution to be generalized to any production function. From this joint posterior distribution, the full set of posterior conditional distributions needed for the Gibbs sampling algorithm can be easily obtained.