Shadow Magnetic Hamiltonian Monte Carlo

Abstract

Incorporating partial momentum refreshment into Magnetic Hamiltonian Monte Carlo (MHMC) to create Magnetic Hamiltonian Monte Carlo with partial momentum refreshment (PMHMC) has been shown to improve the sampling performance of MHMC significantly. At the same time, sampling from an integrator-dependent shadow or modified target density has been utilised to boost the acceptance rates of Hamiltonian Monte Carlo (HMC), which leads to more efficient sampling as the integrator is better conserved by the shadow Hamiltonian than the true Hamiltonian. Sampling from the shadow Hamiltonian associated with the numerical integrator used in MHMC is yet to be explored in the literature. This work aims to address this gap in the literature by combining the benefits of the non-canonical Hamiltonian dynamics of MHMC with those achieved by targeting the modified Hamiltonian. We first determine the modified Hamiltonian associated with the MHMC integrator and use this to construct a novel method, which we refer to as Shadow Magnetic Hamiltonian Monte Carlo (SMHMC), that leads to better sampling behaviour when compared to MHMC while leaving the target distribution invariant. The new SMHMC method is compared to MHMC and PMHMC across various target posterior distributions, including datasets modeled using Bayesian Neural Networks and Bayesian Logistic Regression models.