MCMC Sampling Research Articles

Dynamic probabilistic threshold networks to infer signaling pathways from time-course perturbation data.

BackgroundNetwork inference deals with the reconstruction of molecular networks from experimental data. Given N molecular species, the challenge is to find the underlying network. Due to data limitations, this typically is an ill-posed problem, and requires the integration of prior biological knowledge or strong regularization. We here focus on the situation when time-resolved measurements of a system’s response after systematic perturbations are available.ResultsWe present a novel method to infer signaling networks from time-course perturbation data. We utilize dynamic Bayesian networks with probabilistic Boolean threshold functions to describe protein activation. The model posterior distribution is analyzed using evolutionary MCMC sampling and subsequent clustering, resulting in probability distributions over alternative networks. We evaluate our method on simulated data, and study its performance with respect to data set size and levels of noise. We then use our method to study EGF-mediated signaling in the ERBB pathway.ConclusionsDynamic Probabilistic Threshold Networks is a new method to infer signaling networks from time-series perturbation data. It exploits the dynamic response of a system after external perturbation for network reconstruction. On simulated data, we show that the approach outperforms current state of the art methods. On the ERBB data, our approach recovers a significant fraction of the known interactions, and predicts novel mechanisms in the ERBB pathway.Electronic supplementary materialThe online version of this article (doi:10.1186/1471-2105-15-250) contains supplementary material, which is available to authorized users.

Methods for time series analysis of RNA-seq data with application to human Th17 cell differentiation.

Motivation: Gene expression profiling using RNA-seq is a powerful technique for screening RNA species’ landscapes and their dynamics in an unbiased way. While several advanced methods exist for differential expression analysis of RNA-seq data, proper tools to anal.yze RNA-seq time-course have not been proposed.Results: In this study, we use RNA-seq to measure gene expression during the early human T helper 17 (Th17) cell differentiation and T-cell activation (Th0). To quantify Th17-specific gene expression dynamics, we present a novel statistical methodology, DyNB, for analyzing time-course RNA-seq data. We use non-parametric Gaussian processes to model temporal correlation in gene expression and combine that with negative binomial likelihood for the count data. To account for experiment-specific biases in gene expression dynamics, such as differences in cell differentiation efficiencies, we propose a method to rescale the dynamics between replicated measurements. We develop an MCMC sampling method to make inference of differential expression dynamics between conditions. DyNB identifies several known and novel genes involved in Th17 differentiation. Analysis of differentiation efficiencies revealed consistent patterns in gene expression dynamics between different cultures. We use qRT-PCR to validate differential expression and differentiation efficiencies for selected genes. Comparison of the results with those obtained via traditional timepoint-wise analysis shows that time-course analysis together with time rescaling between cultures identifies differentially expressed genes which would not otherwise be detected.Availability: An implementation of the proposed computational methods will be available at http://research.ics.aalto.fi/csb/software/Contact: tarmo.aijo@aalto.fi or harri.lahdesmaki@aalto.fiSupplementary information: Supplementary data are available at Bioinformatics online.

A statistical approach to the inverse problem in magnetoencephalography

Magnetoencephalography (MEG) is an imaging technique used to measure the magnetic field outside the human head produced by the electrical activity inside the brain. The MEG inverse problem, identifying the location of the electrical sources from the magnetic signal measurements, is ill-posed, that is, there are an infinite number of mathematically correct solutions. Common source localization methods assume the source does not vary with time and do not provide estimates of the variability of the fitted model. Here, we reformulate the MEG inverse problem by considering time-varying locations for the sources and their electrical moments and we model their time evolution using a state space model. Based on our predictive model, we investigate the inverse problem by finding the posterior source distribution given the multiple channels of observations at each time rather than fitting fixed source parameters. Our new model is more realistic than common models and allows us to estimate the variation of the strength, orientation and position. We propose two new Monte Carlo methods based on sequential importance sampling. Unlike the usual MCMC sampling scheme, our new methods work in this situation without needing to tune a high-dimensional transition kernel which has a very high cost. The dimensionality of the unknown parameters is extremely large and the size of the data is even larger. We use Parallel Virtual Machine (PVM) to speed up the computation.

Sampling from Dirichlet process mixture models with unknown concentration parameter: mixing issues in large data implementations.

We consider the question of Markov chain Monte Carlo sampling from a general stick-breaking Dirichlet process mixture model, with concentration parameter alpha . This paper introduces a Gibbs sampling algorithm that combines the slice sampling approach of Walker (Communications in Statistics - Simulation and Computation 36:45–54, 2007) and the retrospective sampling approach of Papaspiliopoulos and Roberts (Biometrika 95(1):169–186, 2008). Our general algorithm is implemented as efficient open source C++ software, available as an R package, and is based on a blocking strategy similar to that suggested by Papaspiliopoulos (A note on posterior sampling from Dirichlet mixture models, 2008) and implemented by Yau et al. (Journal of the Royal Statistical Society, Series B (Statistical Methodology) 73:37–57, 2011). We discuss the difficulties of achieving good mixing in MCMC samplers of this nature in large data sets and investigate sensitivity to initialisation. We additionally consider the challenges when an additional layer of hierarchy is added such that joint inference is to be made on alpha . We introduce a new label-switching move and compute the marginal partition posterior to help to surmount these difficulties. Our work is illustrated using a profile regression (Molitor et al. Biostatistics 11(3):484–498, 2010) application, where we demonstrate good mixing behaviour for both synthetic and real examples.Electronic supplementary materialThe online version of this article (doi:10.1007/s11222-014-9471-3) contains supplementary material, which is available to authorized users.

Bayesian Estimation of Population-Level Trends in Measures of Health Status

Improving health worldwide will require rigorous quantification of population-level trends in health status. However, global-level surveys are not available, forcing researchers to rely on fragmentary country-specific data of varying quality. We present a Bayesian model that systematically combines disparate data to make country-, region- and global-level estimates of time trends in important health indicators. The model allows for time and age nonlinearity, and it borrows strength in time, age, covariates, and within and across regional country clusters to make estimates where data are sparse. The Bayesian approach allows us to account for uncertainty from the various aspects of missingness as well as sampling and parameter uncertainty. MCMC sampling allows for inference in a high-dimensional, constrained parameter space, while providing posterior draws that allow straightforward inference on the wide variety of functionals of interest. Here we use blood pressure as an example health metric. High blood pressure is the leading risk factor for cardiovascular disease, the leading cause of death worldwide. The results highlight a risk transition, with decreasing blood pressure in high-income regions and increasing levels in many lower-income regions.

Korean Welfare Panel Data: A Computational Bayesian Method for Ordered Probit Random Effects Models

We introduce a MCMC sampling for a generalized linear normal random effects model with the ordered probit link function based on latent variables from suitable truncated normal distribution. Such models have proven useful in practice and we have observed numerically reasonable results in the estimation of fixed effects when the random effect term is provided. Applications that utilize Korean Welfare Panel Study data can be difficult to model; subsequently, we find that an ordered probit model with the random effects leads to an improved analyses with more accurate and precise inferences.

Comparison of sampling techniques for Bayesian parameter estimation

The posterior probability distribution for a set of model parameters encodes all that the data have to tell us in the context of a given model; it is the fundamental quantity for Bayesian parameter estimation. In order to infer the posterior probability distribution we have to decide how to explore parameter space. Here we compare three prescriptions for how parameter space is navigated, discussing their relative merits. We consider Metropolis-Hasting sampling, nested sampling and affine-invariant ensemble MCMC sampling. We focus on their performance on toy-model Gaussian likelihoods and on a real-world cosmological data set. We outline the sampling algorithms themselves and elaborate on performance diagnostics such as convergence time, scope for parallelisation, dimensional scaling, requisite tunings and suitability for non-Gaussian distributions. We find that nested sampling delivers high-fidelity estimates for posterior statistics at low computational cost, and should be adopted in favour of Metropolis-Hastings in many cases. Affine-invariant MCMC is competitive when computing clusters can be utilised for massive parallelisation. Affine-invariant MCMC and existing extensions to nested sampling naturally probe multi-modal and curving distributions.

A new bivariate exponential distribution for modeling moderately negative dependence

This paper introduces a new bivariate exponential distribution, called the Bivariate Affine-Linear Exponential distribution, to model moderately negative dependent data. The construction and characteristics of the proposed bivariate distribution are presented along with estimation procedures for the model parameters based on maximum likelihood and objective Bayesian analysis. We derive Jeffreys prior and discuss its frequentist properties based on a simulation study and MCMC sampling techniques. A real data set of mercury concentration in largemouth bass from Florida lakes is used to illustrate the methodology.

Time to Consider Time, and Time to Predict?

The purpose of this paper is to illustrate the important but often neglected role which time has in the statistical treatment of causal problems. Another motivation is to provide a demonstration of how the tools offered by Bayesian modeling and inference, which likewise seem to have largely been ignored in the mainstream statistical literature on causality, can be usefully applied in this context. To offer a concrete example, I consider a particular problem, exploring the effect which the type of day care appears to have on the incidence of acute middle ear infections in small children. Existing epidemiological knowledge is used in setting up a stochastic model, which is formulated in terms of a suitably defined marked point process and applying non-parametric specification of model components where such prior knowledge seems vague. In this context, the important role of the concept of local independence is stressed. A combination of Bayesian inferential methods and an application of MCMC sampling in the numerical work are then shown to lead to immediately useful and understandable quantitative results, formulated in terms of predictive distributions of future observables. The approach adopted here differs in several respects from those currently considered standard in causal modeling and inference. The paper ends with some conclusions of a general nature.

A generalizable knowledge framework for semantic indoor mapping based on Markov logic networks and data driven MCMC

In this paper, we propose a generalizable knowledge framework for data abstraction, i.e., finding a compact abstract model for input data using predefined abstract terms. Based on these abstract terms, intelligent autonomous systems, such as a robot, should be able to make inferences according to a specific knowledge base, so that they can better handle the complexity and uncertainty of the real world. We propose to realize this framework by combining Markov logic networks (MLNs) and data driven MCMC sampling, because the former are a powerful tool for modeling uncertain knowledge and the latter provides an efficient way to draw samples from unknown complex distributions. Furthermore, we show in detail how to adapt this framework to a certain task, in particular, semantic robot mapping. Based on MLNs, we formulate task-specific context knowledge as descriptive soft rules. Experiments on real world data and simulated data confirm the usefulness of our framework.

Comments on “Joint Bayesian Model Selection and Estimation of Noisy Sinusoids Via Reversible Jump MCMC”

Reversible jump MCMC (RJ-MCMC) sampling techniques, which allow to jointly tackle model selection and parameter estimation problems in a coherent Bayesian framework, have become increasingly popular in the signal processing literature since the seminal paper of Andrieu and Doucet [“Joint Bayesian model selection and estimation of noisy sinusoids via reversible jump MCMC,” IEEE Trans. Signal Process, vol. 47, no. 10, pp. 2667-2676, 1999]. Crucial to the implementation of any RJ-MCMC sampler is the computation of the so-called Metropolis-Hastings-Green (MHG) ratio, which determines the acceptance probability for the proposed moves. It turns out that the expression of the MHG ratio that was given in the paper of Andrieu and Doucet for “Birth-or-Death” moves is erroneous and has been reproduced in many subsequent papers dealing with RJ-MCMC sampling in the signal processing literature. This note fixes the erroneous expression and briefly discusses its cause and consequences.

MCMC implementation for Bayesian hidden semi-Markov models with illustrative applications

Hidden Markov models (HMMs) are flexible, well-established models useful in a diverse range of applications. However, one potential limitation of such models lies in their inability to explicitly structure the holding times of each hidden state. Hidden semi-Markov models (HSMMs) are more useful in the latter respect as they incorporate additional temporal structure by explicit modelling of the holding times. However, HSMMs have generally received less attention in the literature, mainly due to their intensive computational requirements. Here a Bayesian implementation of HSMMs is presented. Recursive algorithms are proposed in conjunction with Metropolis-Hastings in such a way as to avoid sampling from the distribution of the hidden state sequence in the MCMC sampler. This provides a computationally tractable estimation framework for HSMMs avoiding the limitations associated with the conventional EM algorithm regarding model flexibility. Performance of the proposed implementation is demonstrated through simulation experiments as well as an illustrative application relating to recurrent failures in a network of underground water pipes where random effects are also included into the HSMM to allow for pipe heterogeneity.

Recovering networks from distance data

A fully probabilistic approach to reconstructing Gaussian graphical models from distance data is presented. The main idea is to extend the usual central Wishart model in traditional methods to using a likelihood depending only on pairwise distances, thus being independent of geometric assumptions about the underlying Euclidean space. This extension has two advantages: the model becomes invariant against potential bias terms in the measurements, and can be used in situations which on input use a kernel- or distance matrix, without requiring direct access to the underlying vectors. The latter aspect opens up a huge new application field for Gaussian graphical models, as network reconstruction is now possible from any Mercer kernel, be it on graphs, strings, probabilities or more complex objects. We combine this likelihood with a suitable prior to enable Bayesian network inference. We present an efficient MCMC sampler for this model and discuss the estimation of module networks. Experiments depict the high quality and usefulness of the inferred networks.

Variance Swap Premium under Stochastic Volatility and Self-Exciting Jumps

Abstract Based on the MCMC samples drawn from SV-Jump model, we found evidence that jumps are clustering even if we allow the variance process to be time varying. We introduced self-exciting jump intensity process to capture excited jump intensity after a realized jump. On the other hand, we are able to use the proposed model to price options and variance swap based on the characteristic function. By comparing the MCMC estimation under physical measure and option calibration under risk neutral measure, we are able to distinguish the risk premium from stochastic volatility and jump intensity. The results show that jump intensity is always overpriced but its premium decays fast, which explains the high risk premium after unexpected big movements in the market; however, stochastic volatility is much more persistent but sometime is underpriced especially in high volatility periods which explains the big loss by shorting variance swap in subprime crisis.

Fast MCMC sampling for Markov jump processes and extensions

Markov jump processes (or continuous-time Markov chains) are a simple and important class of continuous-time dynamical systems. In this paper, we tackle the problem of simulating from the posterior...

Efficient MCMC for Binomial Logit Models

This article deals with binomial logit models where the parameters are estimated within a Bayesian framework. Such models arise, for instance, when repeated measurements are available for identical covariate patterns. To perform MCMC sampling, we rewrite the binomial logit model as an augmented model which involves some latent variables called random utilities. It is straightforward, but inefficient, to use the individual random utility model representation based on the binary observations reconstructed from each binomial observation. Alternatively, we present in this article a new method to aggregate the random utilities for each binomial observation. Based on this aggregated representation, we have implemented an independence Metropolis-Hastings sampler, an auxiliary mixture sampler, and a novel hybrid auxiliary mixture sampler. A comparative study on five binomial datasets shows that the new aggregation method leads to a superior sampler in terms of efficiency compared to previously published data augmentation samplers.

Markov chain Monte Carlo for incomplete information discrete games

This paper outlines a Bayesian approach to estimating discrete games of incomplete information. The MCMC routine proposed features two changes to the traditional Metropolis–Hastings algorithm to facilitate the estimation of games. First, we propose a new approach to sample equilibrium probabilities using a probabilistic equilibrium selection rule that allows for the evaluation of the parameter posterior. Second, we propose a differential evolution based MCMC sampler which is capable of handling the unwieldy posterior that only has support on the equilibrium manifold. We also present two applications to demonstrate the feasibility of our proposed methodology.

Asymmetric Volatility Models with Structural Breaks

This article develops an asymmetric volatility model that takes into consideration the structural breaks in the volatility process. Break points and other parameters of the model are estimated using MCMC and Gibbs sampling techniques. Models with different number of break points are compared using the Bayes factor and BIC. We provide a formal test and hence a new procedure for Bayes factor computation to choose between models with different number of breaks. The procedure is illustrated using simulated as well as real data sets. The analysis shows an evidence to the fact that the financial crisis in the market from the first week of September 2008 has caused a significant break in the structure of the return series of two major NYSE indices viz., S & P 500 and Dow Jones. Analysis of the USD/EURO exchange rate data also shows an evidence of structural break around the same time.

Asteroid orbital inversion using a virtual-observation Markov-chain Monte Carlo method

A novel virtual-observation Markov-chain Monte Carlo method (MCMC) is presented for the asteroid orbital inverse problem posed by small to moderate numbers of astrometric observations. In the method, the orbital-element proposal probability density is chosen to mimic the convolution of the a posteriori density by itself: first, random errors are simulated for each observation, resulting in a set of virtual observations; second, least-squares orbital elements are derived for the virtual observations using the Nelder–Mead downhill simplex method; third, repeating the procedure gives a difference between two sets of what can be called virtual least-squares elements; and, fourth, the difference obtained constitutes a symmetric proposal in a random-walk Metropolis–Hastings algorithm, avoiding the explicit computation of the proposal density. In practice, the proposals are based on a large number of pre-computed sets of orbital elements. Virtual-observation MCMC is thus based on the characterization of the phase-space volume of solutions before the actual MCMC sampling. Virtual-observation MCMC is compared to MCMC orbital ranging, a random-walk Metropolis–Hastings algorithm based on sampling with the help of Cartesian positions at two observation dates, in the case of the near-Earth asteroid (85640) 1998 OX4. In the present preliminary comparison, the methods yield similar results for a 9.1-day observational time interval extracted from the full current astrometry of the asteroid. In the future, both of the methods are to be applied to the astrometric observations of the Gaia mission.

Efficient sampling for Bayesian inference of conjunctive Bayesian networks

Cancer development is driven by the accumulation of advantageous mutations and subsequent clonal expansion of cells harbouring these mutations, but the order in which mutations occur remains poorly understood. Advances in genome sequencing and the soon-arriving flood of cancer genome data produced by large cancer sequencing consortia hold the promise to elucidate cancer progression. However, new computational methods are needed to analyse these large datasets. We present a Bayesian inference scheme for Conjunctive Bayesian Networks, a probabilistic graphical model in which mutations accumulate according to partial order constraints and cancer genotypes are observed subject to measurement noise. We develop an efficient MCMC sampling scheme specifically designed to overcome local optima induced by dependency structures. We demonstrate the performance advantage of our sampler over traditional approaches on simulated data and show the advantages of adopting a Bayesian perspective when reanalyzing cancer datasets and comparing our results to previous maximum-likelihood-based approaches. An R package including the sampler and examples is available at http://www.cbg.ethz.ch/software/bayes-cbn. niko.beerenwinkel@bsse.ethz.ch.