Nonparametric Bayesian Statistics Research Articles

Search for epistatically interacting genetic variants that are associated with vasovagal syncope within biallelic combinations

The most common cause of transient loss of consciousness is vasovagal syncope (VVS), which occurs due to hypoperfusion of the brain due to the interruption of vegetative blood circulation control leading to arterial hypotension. It is known that there is a genetic predisposition to VVS, but the data on the role of individual genes are quite inconsistent. Using APSampler software,which based on a Markov chain Monte Carlo technique and Bayesian nonparametric statistics, we identified biallelic combinations associated with VVS and investigated the nature of interaction between their components. We used the previously obtained results of genomic typing of single nucleotide polymorphisms (SNPs) of 5 genes, the products of which are involved in neurohumoral regulation, and 4 SNPs within locus 2q32.1, supplemented with data for new individuals included in the study. The total sample included 175 patients with a confirmed diagnosis of VVS and 200 control individuals without a history of syncope. Eleven pairwise combinations of SNPs of different genes were found to be associated with VVS. Five of these combinations were epistatic, four of which included SNPs at the 2q32.1 locus located within or near noncoding RNA genes. It is suggested that genes of noncoding RNAs localized on chromosome 2 may directly or indirectly (through cascades of interactions) participate in the regulation of the activity of genes forming epistatic combinations with them.

The social readjustment rating scale: Updated and modernised.

The Social Readjustment Rating Scale, originally devised in 1967 by Holmes and Rahe, measures the impact of life events stress. At the time, the SRRS advanced its field of research by standardising the impact of stress with a set of independently derived weights called 'life change units' (LCUs) for 43 life events found to predict illness onset. The scale has been criticised for being outdated, e.g. "Mortgage over $10,000" and biased, e.g. "Wife begin or stop work". The aim of this cross-sectional survey study is to update and improve the SRRS whilst allowing backwards compatibility. We successfully updated the SRRS norms/LCUs using the ratings of 540 predominantly UK adults aged 18 to 84. Moreover, we also updated wording of 12 SRRS items and evaluated the impact of demographics, personal experience and loneliness. Using non-parametric frequentist and Bayesian statistics we found that the updated weights were higher but broadly consistent with those of the original study. Furthermore, changes to item wording did not affect raters' evaluations relative to the original thereby ensuring cross-comparability with the original SRRS. The raters were not unduly influenced by their personal experiences of events nor loneliness. The target sample was UK rather than US-based and was proportionately representative regarding age, sex and ethnicity. Moreover, the age range was broader than the original SRRS. In addition, we modernised item wording, added one optional extra item to the end of the scale to evaluate the readjustment to living alone and identified 3 potential new items proposed by raters. Backwards-compatibility is maintained.

Exact Simulation of Poisson-Dirichlet Distribution and Generalised Gamma Process

Let J_1>J_2>dots be the ranked jumps of a gamma process tau _{alpha } on the time interval [0,alpha ], such that tau _{alpha }=sum _{k=1}^{infty }J_k. In this paper, we design an algorithm that samples from the random vector (J_1, dots , J_N, sum _{k=N+1}^{infty }J_k). Our algorithm provides an analog to the well-established inverse Lévy measure (ILM) algorithm by replacing the numerical inversion of exponential integral with an acceptance-rejection step. This research is motivated by the construction of Dirichlet process prior in Bayesian nonparametric statistics. The prior assigns weight to each atom according to a GEM distribution, and the simulation algorithm enables us to sample from the N largest random weights of the prior. Then we extend the simulation algorithm to a generalised gamma process. The simulation problem of inhomogeneous processes will also be considered. Numerical implementations are provided to illustrate the effectiveness of our algorithms.

Identifying the role of the FACT histone chaperone in epigenetics through single-molecule fluorescence imaging.

Modifications of histones in eukaryotic cells lead to reversible changes in gene expression without altering the genetic code itself. Such epigenetic modifications are necessary for the proper functioning of cellular processes like gene expression, DNA repair, and cell division. Modified histones directly affect chromatin dynamics and promote other regulation factors such as chromatin remodelers, including histone chaperones. Histone chaperones make functional DNA templates that maintain epigenetic information by remodeling the nucleosome during essential life processes, such as transcription and replication, and by avoiding histone aggregation. The facilitates chromatin transcription (FACT) histone chaperone includes two essential proteins, Pob3 and Spt16. Previously, we found that deleting Pob3 suppresses epigenetic inheritance, indicating that FACT plays an important role in epigenetic maintenance. Therefore, we engineered Schizosaccharomyces pombe (fission yeast) cell strains that express fusions of Spt16 or Pob3 to the fluorescent protein PAmCherry and investigated the dynamics of single proteins in the FACT complex in living S. pombe cells to identify the mechanism for this epigenetic inheritance suppression, including measuring changes in interactions during transcription and replication. By analyzing the motion of Spt16-PAmCherry single-molecule trajectories with nonparametric Bayesian statistics, we were able to distinguish three types of single-molecule motion: a fast (free) mobility state, an intermediate (transiently bound) mobility state, and a slow (bound) mobility state. We are assigning these biophysical states to specific biochemical processes by constructing Spt16 and Pob3 mutants with impaired histone protein binding properties, as well as drug-susceptible strains in which we can inhibit transcription and replication to identify changes in the FACT protein mobility. Overall, our live-cell single-molecule experiments reveal how the FACT complex interacts with histones and other chaperones and indicate a mechanism for epigenetic maintenance.

Fully automated analysis of NMR data with Bayesian nonparametric statistical learning.

The characterization of individual peaks in NMR spectra is an important but challenging step in the investigation of complex biomolecular systems. The major challenge arises due to peak identification which, conventionally, occurs prior to peak characterization. Unluckily, the identification of uncharacterized peaks is highly problematic because of noise and, most importantly, substantial overlap with nearby peaks. For this reason, often the analysis of NMR spectra proceeds by first relying on experts’ opinion to identify peaks and subsequent parameter fitting. Mathematically, peak identification is a problem of model selection and, within a nonparametric analysis framework, can be avoided altogether, this way eliminating the need for experts’ input. In this study, we apply Bayesian nonparametric statistics to develop a comprehensive mathematical framework for automated peak identification and characterization. Our approach relies on advanced statistical representations of FIDs and allows for: simultaneous identification and characterization of individual peaks in an NMR spectrum, accurate noise deconvolution, and extensions to multidimensional NMR or incorporation of specialized protocols. We show that our methods can correctly identify and characterize overlapping peaks without user input. We benchmark our analyses against ground truth peak characteristics using synthetic as well as experimental datasets.

Truncated Poisson–Dirichlet approximation for Dirichlet process hierarchical models

The Dirichlet process was introduced by Ferguson in 1973 to use with Bayesian nonparametric inference problems. A lot of work has been done based on the Dirichlet process, making it the most fundamental prior in Bayesian nonparametric statistics. Since the construction of Dirichlet process involves an infinite number of random variables, simulation-based methods are hard to implement, and various finite approximations for the Dirichlet process have been proposed to solve this problem. In this paper, we construct a new random probability measure called the truncated Poisson–Dirichlet process. It sorts the components of a Dirichlet process in descending order according to their random weights, then makes a truncation to obtain a finite approximation for the distribution of the Dirichlet process. Since the approximation is based on a decreasing sequence of random weights, it has a lower truncation error comparing to the existing methods using stick-breaking process. Then we develop a blocked Gibbs sampler based on Hamiltonian Monte Carlo method to explore the posterior of the truncated Poisson–Dirichlet process. This method is illustrated by the normal mean mixture model and Caron–Fox network model. Numerical implementations are provided to demonstrate the effectiveness and performance of our algorithm.

Covariate dependent Beta-GOS process

Covariate-dependent processes have been widely used in Bayesian nonparametric statistics thanks to their flexibility to incorporate covariate information and correlation among process realizations. Unlike most of the existing work that focuses on extensions of exchangeable species sampling processes such as Dirichlet process, a new class of covariate-dependent nonexchangeable priors is proposed by considering the generalization of an nonexchangeable sequence, namely the Beta-GOS model. The proposed prior has an equivalent formulation under a continuous kernel mixture. It also has a latent variable representation that leads to a natural nonexchangeable parallel with the classical dependent Dirichlet process formulation. This prior is further applied in regression and autoregressive models and it is shown that its posterior sampling algorithm enjoys the same computational complexity with that of the Beta-GOS. The excellent numerical performance of the method is demonstrated via simulation and two real data examples.

Foundations of Bayesian Inference for Complex Statistical Models

In this virtual workshop a variety of recent developments in Bayesian high-dimensional and nonparametric statistics were discussed in depth. There were 12 in depth talks delivered via zoom in the afternoons (to allow for US attendance), and several informal evening time meetings on wonder.me, where follow up discussions of the most important mathematical developments took place.

Clustering and the efficient use of cognitive resources

A central component of human intelligence is the ability to make abstractions, to gloss over some details in favor of drawing out higher-order structure. Clustering stimuli together is a classic example of this. However, the crucial question remains of how one should make these abstractions—what details to retain and what to throw away? How many clusters to form? We provide an analysis of how a rational agent with limited cognitive resources should approach this problem, considering not only how well a clustering fits the data but also by how ‘complex’ it is, i.e. how cognitively expensive it is to represent. We show that the solution to this problem provides a way to reinterpret a wide range of psychological models that are based on principles from non-parametric Bayesian statistics. In particular, we show that the Chinese Restaurant Process prior, ubiquitous in rational models of human and animal clustering behavior, can be interpreted as minimizing an intuitive formulation of representational complexity.

Deep Bayesian Unsupervised Lifelong Learning

Lifelong Learning (LL) refers to the ability to continually learn and solve new problems with incremental available information over time while retaining previous knowledge. Much attention has been given lately to Supervised Lifelong Learning (SLL) with a stream of labelled data. In contrast, we focus on resolving challenges in Unsupervised Lifelong Learning (ULL) with streaming unlabelled data when the data distribution and the unknown class labels evolve over time. Bayesian framework is natural to incorporate past knowledge and sequentially update the belief with new data. We develop a fully Bayesian inference framework for ULL with a novel end-to-end Deep Bayesian Unsupervised Lifelong Learning (DBULL) algorithm, which can progressively discover new clusters without forgetting the past with unlabelled data while learning latent representations. To efficiently maintain past knowledge, we develop a novel knowledge preservation mechanism via sufficient statistics of the latent representation for raw data. To detect the potential new clusters on the fly, we develop an automatic cluster discovery and redundancy removal strategy in our inference inspired by Nonparametric Bayesian statistics techniques. We demonstrate the effectiveness of our approach using image and text corpora benchmark datasets in both LL and batch settings.

Nobias: analyzing anomalous diffusion in single-molecule tracks with nonparametric Bayesian inference

Single-molecule imaging techniques and single-particle tracking (SPT) of molecules inside living cells investigate biomolecular dynamics at nanometer spatial resolution and millisecond temporal resolution. However, the complexity of living cells requires rich statistical approaches to deal with heterogeneous dynamics and complex diffusion models. To achieve these goals, we have developed a new SPT analysis framework: NOBIAS (Nonparametric Bayesian Inference for Anomalous Diffusion in Single-Molecule Tracking), which combines nonparametric Bayesian statistics and recurrent neural network (RNN) methods to deal with SPT datasets that contain heterogeneous dynamics with a mixture of unknown number of diffusive states, even within a single trajectory, asymmetrical motion of target biomolecules, and non-Brownian anomalous diffusion.

NOBIAS: Analyzing anomalous diffusion in single-molecule tracks with nonparametric Bayesian inference.

Single particle tracking (SPT) enables the investigation of biomolecular dynamics at a high temporal and spatial resolution in living cells, and the analysis of these SPT datasets can reveal biochemical interactions and mechanisms. Still, how to make the best use of these tracking data for a broad set of experimental conditions remains an analysis challenge in the field. Here, we develop a new SPT analysis framework: NOBIAS (NOnparametric Bayesian Inference for Anomalous Diffusion in Single-Molecule Tracking), which applies nonparametric Bayesian statistics and deep learning approaches to thoroughly analyze SPT datasets. In particular, NOBIAS handles complicated live-cell SPT data for which: the number of diffusive states is unknown, mixtures of different diffusive populations may exist within single trajectories, symmetry cannot be assumed between the x and y directions, and anomalous diffusion is possible. NOBIAS provides the number of diffusive states without manual supervision, it quantifies the dynamics and relative populations of each diffusive state, it provides the transition probabilities between states, and it assesses the anomalous diffusion behavior for each state. We validate the performance of NOBIAS with simulated datasets and apply it to the diffusion of single outer-membrane proteins in Bacteroides thetaiotaomicron. Furthermore, we compare NOBIAS with other SPT analysis methods and find that, in addition to these advantages, NOBIAS is robust and has high computational efficiency and is particularly advantageous due to its ability to treat experimental trajectories with asymmetry and anomalous diffusion.

Stick-Breaking processes, Clumping, and Markov Chain Occupation Laws

We connect the empirical or ‘occupation’ laws of certain discrete space time-inhomogeneous Markov chains, related to simulated annealing, to a novel class of ‘stick-breaking’ processes, a ‘nonexchangeable’ generalization of the Dirichlet process used in nonparametric Bayesian statistics. To make this unexpected correspondence, we examine an intermediate ‘clumped’ structure in both the time-inhomogeneous Markov chains and the stick-breaking processes, perhaps of its own interest, which records the sequence of different states visited and the scaled proportions of time spent on them. By matching the associated intermediate structures, we identify the limits of the empirical measures of the time-inhomogeneous Markov chains as types of stick-breaking processes.

Kullback–Leibler divergence for Bayesian nonparametric model checking

Bayesian nonparametric statistics is an area of considerable research interest. While recently there has been an extensive concentration in developing Bayesian nonparametric procedures for model checking, the use of the Dirichlet process, in its simplest form, along with the Kullback–Leibler divergence is still an open problem. This is mainly attributed to the discreteness property of the Dirichlet process and that the Kullback–Leibler divergence between any discrete distribution and any continuous distribution is infinity. The approach proposed in this paper, which is based on incorporating the Dirichlet process, the Kullback–Leibler divergence and the relative belief ratio, is considered the first concrete solution to this issue. Applying the approach is simple and does not require obtaining a closed form of the relative belief ratio. A Monte Carlo study and real data examples show that the developed approach exhibits excellent performance.

SMAUG: Analyzing single-molecule tracks with nonparametric Bayesian statistics.

Single-molecule fluorescence microscopy probes nanoscale, subcellular biology in real time. Existing methods for analyzing single-particle tracking data provide dynamical information, but can suffer from supervisory biases and high uncertainties. Here, we develop a method for the case of multiple interconverting species undergoing free diffusion and introduce a new approach to analyzing single-molecule trajectories: the Single-Molecule Analysis by Unsupervised Gibbs sampling (SMAUG) algorithm, which uses nonparametric Bayesian statistics to uncover the whole range of information contained within a single-particle trajectory dataset. Even in complex systems where multiple biological states lead to a number of observed mobility states, SMAUG provides the number of mobility states, the average diffusion coefficient of single molecules in that state, the fraction of single molecules in that state, the localization noise, and the probability of transitioning between two different states. In this paper, we provide the theoretical background for the SMAUG analysis and then we validate the method using realistic simulations of single-particle trajectory datasets as well as experiments on a controlled in vitro system. Finally, we demonstrate SMAUG on real experimental systems in both prokaryotes and eukaryotes to measure the motions of the regulatory protein TcpP in Vibrio cholerae and the dynamics of the B-cell receptor antigen response pathway in lymphocytes. Overall, SMAUG provides a mathematically rigorous approach to measuring the real-time dynamics of molecular interactions in living cells.

Two part envelopes for rejection sampling of some completely random measures

This paper shows that rejection sampling with a two-piece Lévy intensity envelope can outperform both the Ferguson–Klass algorithm and previously proposed envelopes for simulating realizations of completely random measures typically used in Bayesian nonparametric statistics.

Diffusion limit for the random walk Metropolis algorithm out of stationarity

The Random Walk Metropolis (RWM) algorithm is a Metropolis–Hastings Markov Chain Monte Carlo algorithm designed to sample from a given target distribution π^N with Lebesgue density on R^N. Like any other Metropolis–Hastings algorithm, RWM constructs a Markov chain by randomly proposing a new position (the “proposal move”), which is then accepted or rejected according to a rule which makes the chain reversible with respect to π^N. When the dimension N is large, a key question is to determine the optimal scaling with N of the proposal variance: if the proposal variance is too large, the algorithm will reject the proposed moves too often; if it is too small, the algorithm will explore the state space too slowly. Determining the optimal scaling of the proposal variance gives a measure of the cost of the algorithm as well. One approach to tackle this issue, which we adopt here, is to derive diffusion limits for the algorithm. Such an approach has been proposed in the seminal papers (Ann. Appl. Probab. 7 (1) (1997) 110–120; J. R. Stat. Soc. Ser. B. Stat. Methodol. 60 (1) (1998) 255–268). In particular, in (Ann. Appl. Probab. 7 (1) (1997) 110–120) the authors derive a diffusion limit for the RWM algorithm under the two following assumptions: (i) the algorithm is started in stationarity; (ii) the target measure π^N is in product form. The present paper considers the situation of practical interest in which both assumptions (i) and (ii) are removed. That is (a) we study the case (which occurs in practice) in which the algorithm is started out of stationarity and (b) we consider target measures which are in non-product form. Roughly speaking, we consider target measures that admit a density with respect to Gaussian; such measures arise in Bayesian nonparametric statistics and in the study of conditioned diffusions. We prove that, out of stationarity, the optimal scaling for the proposal variance is O(N^(−1)), as it is in stationarity. In this optimal scaling, a diffusion limit is obtained and the cost of reaching and exploring the invariant measure scales as O(N). Notice that the optimal scaling in and out of stationatity need not be the same in general, and indeed they differ e.g. in the case of the MALA algorithm (Stoch. Partial Differ. Equ. Anal Comput. 6 (3) (2018) 446–499). More importantly, our diffusion limit is given by a stochastic PDE, coupled to a scalar ordinary differential equation; such an ODE gives a measure of how far from stationarity the process is and can therefore be taken as an indicator of convergence. In this sense, this paper contributes understanding to the old-standing problem of monitoring convergence of MCMC algorithms.

Firing rate estimation using infinite mixture models and its application to neural decoding.

Neural decoding is a framework for reconstructing external stimuli from spike trains recorded by various neural recordings. Kloosterman et al. proposed a new decoding method using marked point processes (Kloosterman F, Layton SP, Chen Z, Wilson MA. J Neurophysiol 111: 217-227, 2014). This method does not require spike sorting and thereby improves decoding accuracy dramatically. In this method, they used kernel density estimation to estimate intensity functions of marked point processes. However, the use of kernel density estimation causes problems such as low decoding accuracy and high computational costs. To overcome these problems, we propose a new decoding method using infinite mixture models to estimate intensity. The proposed method improves decoding performance in terms of accuracy and computational speed. We apply the proposed method to simulation and experimental data to verify its performance.NEW & NOTEWORTHY We propose a new neural decoding method using infinite mixture models and nonparametric Bayesian statistics. The proposed method improves decoding performance in terms of accuracy and computation speed. We have successfully applied the proposed method to position decoding from spike trains recorded in a rat hippocampus.

Bayesian nonparametric statistics: A new toolkit for discovery in cancer research.

Many commonly used statistical methods for data analysis or clinical trial design rely on incorrect assumptions or assume an over-simplified framework that ignores important information. Such statistical practices may lead to incorrect conclusions about treatment effects or clinical trial designs that are impractical or that do not accurately reflect the investigator's goals. Bayesian nonparametric (BNP) models and methods are a very flexible new class of statistical tools that can overcome such limitations. This is because BNP models can accurately approximate any distribution or function and can accommodate a broad range of statistical problems, including density estimation, regression, survival analysis, graphical modeling, neural networks, classification, clustering, population models, forecasting and prediction, spatiotemporal models, and causal inference. This paper describes 3 illustrative applications of BNP methods, including a randomized clinical trial to compare treatments for intraoperative air leaks after pulmonary resection, estimating survival time with different multi-stage chemotherapy regimes for acute leukemia, and evaluating joint effects of targeted treatment and an intermediate biological outcome on progression-free survival time in prostate cancer.

Bayesian Nonparametric Methods for Partially-Observable Reinforcement Learning

Making intelligent decisions from incomplete information is critical in many applications: for example, robots must choose actions based on imperfect sensors, and speech-based interfaces must infer a user's needs from noisy microphone inputs. What makes these tasks hard is that often we do not have a natural representation with which to model the domain and use for choosing actions; we must learn about the domain's properties while simultaneously performing the task. Learning a representation also involves trade-offs between modeling the data that we have seen previously and being able to make predictions about new data. This article explores learning representations of stochastic systems using Bayesian nonparametric statistics. Bayesian nonparametric methods allow the sophistication of a representation to scale gracefully with the complexity in the data. Our main contribution is a careful empirical evaluation of how representations learned using Bayesian nonparametric methods compare to other standard learning approaches, especially in support of planning and control. We show that the Bayesian aspects of the methods result in achieving state-of-the-art performance in decision making with relatively few samples, while the nonparametric aspects often result in fewer computations. These results hold across a variety of different techniques for choosing actions given a representation.