Abstract

We present an architectural design of a library for Bayesian modelling and inference in modern functional programming languages. The novel aspect of our approach are modular implementations of existing state-of-the-art inference algorithms. Our design relies on three inherently functional features: higher-order functions, inductive data-types, and support for either type-classes or an expressive module system. We provide a performant Haskell implementation of this architecture, demonstrating that high-level and modular probabilistic programming can be added as a library in sufficiently expressive languages. We review the core abstractions in this architecture: inference representations, inference transformations, and inference representation transformers. We then implement concrete instances of these abstractions, counterparts to particle filters and Metropolis-Hastings samplers, which form the basic building blocks of our library. By composing these building blocks we obtain state-of-the-art inference algorithms: Resample-Move Sequential Monte Carlo, Particle Marginal Metropolis-Hastings, and Sequential Monte Carlo Squared. We evaluate our implementation against existing probabilistic programming systems and find it is already competitively performant, although we conjecture that existing functional programming optimisation techniques could reduce the overhead associated with the abstractions we use. We show that our modular design enables deterministic testing of inherently stochastic Monte Carlo algorithms. Finally, we demonstrate using OCaml that an expressive module system can also implement our design.

Highlights

  • Probabilistic programs, as used for statistical machine learning and data science, are computer programs with two specific computational effects: one for drawing random variables from probability distributions and one adjusting the relative weight of specific values of a random variable

  • The final transformer we present supports a class of algorithms known as Trace Markov Chain Monte Carlo (MCMC), which mix two ingredients

  • We compare with state-of-the-art probabilistic programming systems Anglican [Wood et al 2014] and WebPPL [Goodman and Stuhlmüller 2014] since they implement similar inference algorithms and their front-end languages are extensions of popular programming languages, Clojure and Javascript respectively

Read more

Summary

Introduction

Probabilistic programs, as used for statistical machine learning and data science, are computer programs with two specific computational effects: one for drawing random variables from probability distributions and one adjusting the relative weight of specific values of a random variable. For a traditional explanation of Bayesian modelling and inference we refer the reader to one of the many excellent textbooks available [Barber 2012; Bishop 2006; MacKay 2003]

Objectives
Results
Conclusion
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call