MCMC Sampling Research Articles

Hierarchical Bayesian models for the estimation of unemployment rates in small domains of the italian labour force survey

In this paper we introduce shrinkage estimators for the estimation of the Unemployment rate in small domains of the Italian Labour Force Survey. The proposed estimators are based on Hierarchical Linear Mixed Models and on the borrowing strength on both time series and cross section. Auxiliary information from source external to the Labour Force Survey is not considered. A Hierarchical Bayesian approach is adopted, in which models are solved by means of MCMC sampling algorithms. This allows to measure variability associated to estimators accounting, in a simple way, for all e uncertainty sources. Results highlight how, simple hierarchical models allows for remarkable gain in efficiency with respect to published estimates, and that models with a time series component perform better than those based exclusively on data from the same repetition of the survey.

Bayesian clustering for row effects models

We deal with two-way contingency tables having ordered column categories. We use a row effects model wherein each interaction term is assumed to have a multiplicative form involving a row effect parameter and a fixed column score. We propose a methodology to cluster row effects in order to simplify the interaction structure and to enhance the interpretation of the model. Our method uses a product partition model with a suitable specification of the cohesion function, so that we can carry out our analysis on a collection of models of varying dimensions using a straightforward MCMC sampler. The methodology is illustrated with reference to simulated and real data sets.

Bayesian model selection for logistic regression with misclassified outcomes

We consider the problem of variable selection for logistic regression when the dependent variable is measured imperfectly, under both differential and non-differential misclassification. An MCMC sampling scheme is designed, incorporating uncertainty about which explanatory variables affect the dependent variable and which affect the probability of misclassification. We assume that a small gold standard perfectly measured sample is available to augment the imperfectly measured sample, under the differential misclassification framework. A simulation study illustrates favourable results both in terms of variable selection and parameter estimation. Examples analysing the risk of violence against young women by their partner and the risk of injury in highway motor accidents are considered.

Volatility forecasting using threshold heteroskedastic models of the intra-day range

An effective approach for forecasting return volatility via threshold nonlinear heteroskedastic models of the daily asset price range is provided. The range is defined as the difference between the highest and lowest log intra-day asset price. A general model specification is proposed, allowing the intra-day high–low price range to depend nonlinearly on past information, or an exogenous variable such as US market information. The model captures aspects such as sign or size asymmetry and heteroskedasticity, which are commonly observed in financial markets. The focus is on parameter estimation, inference and volatility forecasting in a Bayesian framework. An MCMC sampling scheme is employed for estimation and shown to work well in simulation experiments. Finally, competing range-based and return-based heteroskedastic models are compared via out-of-sample forecast performance. Applied to six international financial market indices, the range-based threshold heteroskedastic models are well supported by the data in terms of finding significant threshold nonlinearity, diagnostic checking and volatility forecast performance under various volatility proxies.

Learning and Inferring Motion Patterns using Parametric Segmental Switching Linear Dynamic Systems

Switching Linear Dynamic System (SLDS) models are a popular technique for modeling complex nonlinear dynamic systems. An SLDS can describe complex temporal patterns more concisely and accurately than an HMM by using continuous hidden states. However, the use of SLDS models in practical applications is challenging for three reasons. First, exact inference in SLDS models is computationally intractable. Second, the geometric duration model induced in standard SLDSs limits their representational power. Third, standard SLDSs do not provide a principled way to interpret systematic variations governed by higher order parameters. The contributions in this paper address all of these three challenges. First, we present a data-driven MCMC (DD-MCMC) sampling method for approximate inference in SLDSs. We show DD-MCMC provides an efficient method for estimation and learning in SLDS models. Second, we present segmental SLDSs (S-SLDS), where the geometric distributions of the switching state durations are replaced with arbitrary duration models. Third, we extend the standard SLDS model with additional global parameters that can capture systematic temporal and spatial variations. The resulting parametric SLDS model (P-SLDS) uses EM to robustly interpret parametrized motions by incorporating additional global parameters that underly systematic variations of the overall motion. The overall development of the extensions for SLDSs provide a principled framework to interpret complex motions. The framework is applied to the honey bee dance interpretation task in the context of the on-going BioTracking project at the Georgia Institute of Technology. The experimental results suggest that the enhanced models provide an effective framework for a wide range of motion analysis applications.

Predicting RNA secondary structures with pseudoknots by MCMC sampling

The most probable secondary structure of an RNA molecule, given the nucleotide sequence, can be computed efficiently if a stochastic context-free grammar (SCFG) is used as the prior distribution of the secondary structure. The structures of some RNA molecules contain so-called pseudoknots. Allowing all possible configurations of pseudoknots is not compatible with context-free grammar models and makes the search for an optimal secondary structure NP-complete. We suggest a probabilistic model for RNA secondary structures with pseudoknots and present a Markov-chain Monte-Carlo Method for sampling RNA structures according to their posterior distribution for a given sequence. We favor Bayesian sampling over optimization methods in this context, because it makes the uncertainty of RNA structure predictions assessable. We demonstrate the benefit of our method in examples with tmRNA and also with simulated data. McQFold, an implementation of our method, is freely available from http://www.cs.uni-frankfurt.de/~metzler/McQFold.

Bayesian Smoothing with Gaussian Processes Using Fourier Basis Functions in thespectralGPPackage

The spectral representation of stationary Gaussian processes via the Fourier basis provides a computationally efficient specification of spatial surfaces and nonparametric regression functions for use in various statistical models. I describe the representation in detail and introduce the spectralGP package in R for computations. Because of the large number of basis coefficients, some form of shrinkage is necessary; I focus on a natural Bayesian approach via a particular parameterized prior structure that approximates stationary Gaussian processes on a regular grid. I review several models from the literature for data that do not lie on a grid, suggest a simple model modification, and provide example code demonstrating MCMC sampling using the spectralGP package. I describe reasons that mixing can be slow in certain situations and provide some suggestions for MCMC techniques to improve mixing, also with example code, and some general recommendations grounded in experience.

Building Outline Extraction from Digital Elevation Models Using Marked Point Processes

This work presents an automatic algorithm for extracting vectorial land registers from altimetric data in dense urban areas. We focus on elementary shape extraction and propose a method that extracts rectangular buildings. The result is a vectorial land register that can be used, for instance, to perform precise roof shape estimation. Using a spatial point process framework, we model towns as configurations of and unknown number of rectangles. An energy is defined, which takes into account both low level information provided by the altimetry of the scene, and geometric knowledge about the disposition of buildings in towns. Estimation is done by minimizing the energy using simulated annealing. We use an MCMC sampler that is a combination of general Metropolis Hastings Green techniques and the Geyer and Moller algorithm for point process sampling. We define some original proposition kernels, such as birth or death in a neighborhood and define the energy with respect to an inhomogeneous Poisson point process. We present results on real data provided by the IGN (French National Geographic Institute). Results were obtained automatically. These results consist of configurations of rectangles describing a dense urban area.

Inference for Adaptive Time Series Models: Stochastic Volatility and Conditionally Gaussian State Space Form

In this paper we model the Gaussian errors in the standard Gaussian linear state space model as stochastic volatility processes. We show that conventional MCMC algorithms for this class of models are ineffective, but that the problem can be alleviated by reparameterizing the model. Instead of sampling the unobserved variance series directly, we sample in the space of the disturbances, which proves to lower correlation in the sampler and thus increases the quality of the Markov chain. Using our reparameterized MCMC sampler, it is possible to estimate an unobserved factor model for exchange rates between a group of n countries. The underlying n + 1 country-specific currency strength factors and the n + 1 currency volatility factors can be extracted using the new methodology. With the factors, a more detailed image of the events around the 1992 EMS crisis is obtained. We assess the fit of competitive models on the panels of exchange rates with an effective particle filter and find that indeed the factor model is strongly preferred by the data.

Investigating Heterogeneity in Pneumococcal Transmission

The analysis of communicable agent transmission from field data is typically hampered by missing data, dependence between individual trajectories, and sometimes by heterogeneity among competing pathogens. Methods based on data augmentation and Markov chain Monte Carlo sampling have been used to analyze such data in small communities (typically households), with little diversity in pathogens. In this article the approach is extended to analyze the transmission of 15 Streptococcus pneumoniae serotypes in schoolchildren, where hundreds of individual trajectories interact and a substantial portion of trajectories are unobserved. For each child, the data were augmented to describe the detailed time course of S. pneumoniae carriage. The Bayesian hierarchical model ensured consistency between observed and augmented data; described the latent dynamics of S. pneumoniae acquisition and clearance; and specified priors. To investigate heterogeneity among serotypes, a clustering step was introduced to select a parsimonious description of transmission characteristics. The joint posterior distribution of parameters, augmented data, and clusters of serotypes was explored by reversible-jump MCMC sampling. The approach made it possible to make inferences simultaneously on the number of clusters of serotypes and on the transmission characteristics of each cluster.

On the ergodicity properties of some adaptive MCMC algorithms

In this paper we study the ergodicity properties of some adaptive Markov chain Monte Carlo algorithms (MCMC) that have been recently proposed in the literature. We prove that under a set of verifiable conditions, ergodic averages calculated from the output of a so-called adaptive MCMC sampler converge to the required value and can even, under more stringent assumptions, satisfy a central limit theorem. We prove that the conditions required are satisfied for the independent Metropolis–Hastings algorithm and the random walk Metropolis algorithm with symmetric increments. Finally, we propose an application of these results to the case where the proposal distribution of the Metropolis–Hastings update is a mixture of distributions from a curved exponential family.

Model weights for model choice and averaging

A method is suggested to estimate posterior model probabilities and model averaged parameters via MCMC sampling under a Bayesian approach. The estimates use pooled output for J models ( J > 1 ) whereby all models are updated at each iteration. Posterior probabilities are based on averages of continuous weights obtained for each model at each iteration, while samples of averaged parameters are obtained from iteration specific averages that are based on these weights. Parallel sampling of models assists in deriving posterior densities for parameter contrasts between models and in assessing hypotheses regarding model averaged parameters. Four worked examples illustrate application of the approach, two involving fixed effect regression, and two involving random effects.

The Cluster Variation Method for Efficient Linkage Analysis on Extended Pedigrees

BackgroundComputing exact multipoint LOD scores for extended pedigrees rapidly becomes infeasible as the number of markers and untyped individuals increase. When markers are excluded from the computation, significant power may be lost. Therefore accurate approximate methods which take into account all markers are desirable.MethodsWe present a novel method for efficient estimation of LOD scores on extended pedigrees. Our approach is based on the Cluster Variation Method, which deterministically estimates likelihoods by performing exact computations on tractable subsets of variables (clusters) of a Bayesian network. First a distribution over inheritances on the marker loci is approximated with the Cluster Variation Method. Then this distribution is used to estimate the LOD score for each location of the trait locus.ResultsFirst we demonstrate that significant power may be lost if markers are ignored in the multi-point analysis. On a set of pedigrees where exact computation is possible we compare the estimates of the LOD scores obtained with our method to the exact LOD scores. Secondly, we compare our method to a state of the art MCMC sampler. When both methods are given equal computation time, our method is more efficient. Finally, we show that CVM scales to large problem instances.ConclusionWe conclude that the Cluster Variation Method is as accurate as MCMC and generally is more efficient. Our method is a promising alternative to approaches based on MCMC sampling.

SIMMAP: stochastic character mapping of discrete traits on phylogenies.

BackgroundCharacter mapping on phylogenies has played an important, if not critical role, in our understanding of molecular, morphological, and behavioral evolution. Until very recently we have relied on parsimony to infer character changes. Parsimony has a number of serious limitations that are drawbacks to our understanding. Recent statistical methods have been developed that free us from these limitations enabling us to overcome the problems of parsimony by accommodating uncertainty in evolutionary time, ancestral states, and the phylogeny.ResultsSIMMAP has been developed to implement stochastic character mapping that is useful to both molecular evolutionists, systematists, and bioinformaticians. Researchers can address questions about positive selection, patterns of amino acid substitution, character association, and patterns of morphological evolution.ConclusionStochastic character mapping, as implemented in the SIMMAP software, enables users to address questions that require mapping characters onto phylogenies using a probabilistic approach that does not rely on parsimony. Analyses can be performed using a fully Bayesian approach that is not reliant on considering a single topology, set of substitution model parameters, or reconstruction of ancestral states. Uncertainty in these quantities is accommodated by using MCMC samples from their respective posterior distributions.

Estimating Population Parameters using the Structured Serial Coalescent with Bayesian MCMC Inference when some Demes are Hidden

Using the structured serial coalescent with Bayesian MCMC and serial samples, we estimate population size when some demes are not sampled or are hidden, ie ghost demes. It is found that even with the presence of a ghost deme, accurate inference was possible if the parameters are estimated with the true model. However with an incorrect model, estimates were biased and can be positively misleading. We extend these results to the case where there are sequences from the ghost at the last time sample. This case can arise in HIV patients, when some tissue samples and viral sequences only become available after death. When some sequences from the ghost deme are available at the last sampling time, estimation bias is reduced and accurate estimation of parameters associated with the ghost deme is possible despite sampling bias. Migration rates for this case are also shown to be good estimates when migration values are low.

Assessing Approximate Inference for Binary Gaussian Process Classification

Gaussian process priors can be used to define flexible, probabilistic classification models. Unfortunately exact Bayesian inference is analytically intractable and various approximation techniques have been proposed. In this work we review and compare Laplace's method and Expectation Propagation for approximate Bayesian inference in the binary Gaussian process classification model. We present a comprehensive comparison of the approximations, their predictive performance and marginal likelihood estimates to results obtained by MCMC sampling. We explain theoretically and corroborate empirically the advantages of Expectation Propagation compared to Laplace's method.

Bayesian analysis for erosion modelling of sediments in combined sewer systems

Previous research has confirmed that the sediments at the bed of combined sewer systems are the main source of particulate and organic pollution during rain events contributing to combined sewer overflows. However, existing urban stormwater models utilize inappropriate sediment transport formulas initially developed from alluvial hydrodynamics. Recently, a model has been formulated and profoundly assessed based on laboratory experiments to simulate the erosion of sediments in sewer pipes taking into account the increase in strength with depth in the weak layer of deposits. In order to objectively evaluate this model, this paper presents a Bayesian analysis of the model using field data collected in sewer pipes in Paris under known hydraulic conditions. The test has been performed using a MCMC sampling method for calibration and uncertainty assessment. Results demonstrate the capacity of the model to reproduce erosion as a direct response to the increase in bed shear stress. This is due to the model description of the erosional strength in the deposits and to the shape of the measured bed shear stress. However, large uncertainties in some of the model parameters suggest that the model could be over-parameterised and necessitates a large amount of informative data for its calibration.

A tractable probabilistic model for Affymetrix probe-level analysis across multiple chips

Affymetrix GeneChip arrays are currently the most widely used microarray technology. Many summarization methods have been developed to provide gene expression levels from Affymetrix probe-level data. Most of the currently popular methods do not provide a measure of uncertainty for the expression level of each gene. The use of probabilistic models can overcome this limitation. A full hierarchical Bayesian approach requires the use of computationally intensive MCMC methods that are impractical for large datasets. An alternative computationally efficient probabilistic model, mgMOS, uses Gamma distributions to model specific and non-specific binding with a latent variable to capture variations in probe affinity. Although promising, the main limitations of this model are that it does not use information from multiple chips and does not account for specific binding to the mismatch (MM) probes. We extend mgMOS to model the binding affinity of probe-pairs across multiple chips and to capture the effect of specific binding to MM probes. The new model, multi-mgMOS, provides improved accuracy, as demonstrated on some bench-mark datasets and a real time-course dataset, and is much more computationally efficient than a competing hierarchical Bayesian approach that requires MCMC sampling. We demonstrate how the probabilistic model can be used to estimate credibility intervals for expression levels and their log-ratios between conditions. Both mgMOS and the new model multi-mgMOS have been implemented in an R package, which is available at http://www.bioinf.man.ac.uk/resources/puma.

A benchmark methodology for managing uncertainties in urban runoff quality models

In this paper we present a benchmarking methodology, which aims at comparing urban runoff quality models, based on the Bayesian theory. After choosing the different configurations of models to be tested, this methodology uses the Metropolis algorithm, a general MCMC sampling method, to estimate the posterior distributions of the models' parameters. The analysis of these posterior distributions allows a quantitative assessment of the parameters' uncertainties and their interaction structure, and provides information about the sensitivity of the probability distribution of the model output to parameters. The effectiveness and efficiency of this methodology are illustrated in the context of 4 configurations of pollutants' accumulation/erosion models, tested on 4 street subcatchments. Calibration results demonstrate that the Metropolis algorithm produces reliable inferences of parameters thus, helping on the improvement of the mathematical concept of model equations.

Benchmark Estimation for Markov chain Monte Carlo Samples

While studying various features of the posterior distribution of a vector-valued parameter using an MCMC sample, a subsample is often all that is available for analysis. The goal of benchmark estimation is to use the best available information, that is, the full MCMC sample, to improve future estimates made on the basis of the subsample. We discuss a simple approach to do this and provide a theoretical basis for the method. The methodology and benefits of benchmark estimation are illustrated using a well-known example from the literature. We obtain nearly a 90% reduction in MSE with the technique based on a 1-in-10 subsample and show that greater benefits accrue with the thinner subsamples that are often used in practice.