Contribution to the Discussion of the Paper ‘Geodesic Monte Carlo on Embedded Manifolds’

Abstract

We would like to start by congratulating Byrne and Girolami for writing such a thoughtful and extremely interesting paper. This is in fact a worthy addition to other high impact papers recently published by Professor Girolami’s lab in this field. The common theme of these papers is to use geometrically motivated methods to improve efficiency of sampling algorithms. In their seminal paper, Girolami & Calderhead (2011) proposed a novel Hamiltonian Monte Carlo (HMC) method, called Riemannian manifold HMC, that exploits the Riemannian geometry of the target distribution to improve standard HMC’s efficiency by automatically adapting to the local structure. Although this is a natural and beautiful idea, there are significant computational difficulties, which arise in effectively implementing this algorithm. In contrast, in this current contribution, Byrne and Girolami focused on special probability distributions, which give rise to particularly nice Riemannian geometries. In particular, the examples under consideration described in Section 4 allow for closed-form solutions to the geodesic equation, which can be used to reduce computational cost of geometrically motivated Monte Carlo methods. Although the proposed splitting algorithm is quiet interesting, we initially doubted its impact because Riemannian metrics with closed-form geodesics are extremely rare. However, we are now convinced that this approach will likely see application beyond what is outlined herein. For example, we believe that this approach can be used to improve computational efficiency of sampling algorithms when the parameter space is constrained. The standard HMC algorithm needs to evaluate each proposal to ensure that it is within the boundaries imposed by the constraints. Alternatively, as discussed by Neal (2011), one could modify standard HMC so the sampler bounces back after hitting the boundaries. In Appendix A, Byrne and Girolami discussed this approach for geodesic updates on the simplex. In many cases, a constrained parameter space can be bijectively mapped to a unit ball,