Adaptive parameters tuning based on energy-preserving splitting integration for Hamiltonian Monte Carlo Method

Abstract

Splitting schemes, a class of numerical integrators for Hamiltonian problems, offer a favorable alternative to the Störmer–Verlet method in Hamiltonian Monte Carlo (HMC) methodology. However, the performance of HMC is highly sensitive to the adopted step size. In this paper, we propose a novel approach for selecting the step size h for advancing with the method defined by the free parameter b, within the family of one-parameter second order splitting procedures. Our methodology utilizes a designated function hb of the parameter b to determine the step size, i.e. h=hb(b). By appropriately restricting the domain of hb to a suitable interval I, the pairs (b,hb) with b∈I ensure both stability and Hamiltonian preservation when sampling from Gaussian distributions. As a result, our technique never rejects a sample within the HMC process, and this characteristic is the key factor behind its superior performance compared to similar methods recently introduced in other studies. Additionally, we assess the effectiveness of the methods defined by the pairs (b,hb) for general not-Gaussian distribution sampling. In this case we also present a technique based on an adaptive selection of the b parameter for improving the HMC performance. The effectiveness of the proposed approach is evaluated through benchmark examples from literature and experiments involving the Log-Gaussian Cox process and Bayesian Logistic Regression.