Model selection via marginal likelihood estimation by combining thermodynamic integration and gradient matching

Benn Macdonald,Dirk Husmeier

doi:10.1007/s11222-018-9840-4

Abstract

Conducting statistical inference on systems described by ordinary differential equations (ODEs) is a challenging problem. Repeatedly numerically solving the system of equations incurs a high computational cost, making many methods based on explicitly solving the ODEs unsuitable in practice. Gradient matching methods were introduced in order to deal with the computational burden. These methods involve minimising the discrepancy between predicted gradients from the ODEs and those from a smooth interpolant. Work until now on gradient matching methods has focused on parameter inference. This paper considers the problem of model selection. We combine the method of thermodynamic integration to compute the log marginal likelihood with adaptive gradient matching using Gaussian processes, demonstrating that the method is robust and able to outperform BIC and WAIC.

Highlights

Ordinary differential equations (ODEs) are a powerful way of providing an observed system with a mathematical description
We note that there is an alternative version of the thermodynamic integration scheme that we have proposed in Sect. 3, which we describe in Section 9.2 of the supplementary material (SM) and which, naively, appears to be more straightforward
In order to assess the performance of the new scheme outlined in Sect. 3, the method will be tested on two ODE systems and various candidate models of each

Summary

Introduction

Ordinary differential equations (ODEs) are a powerful way of providing an observed system with a mathematical description. N } denotes one of N components (referred to throughout as “species”), xs(u) denotes the concentration of species s as a function of u (typically time or space), θ s is the vector of ODE parameters for species s and x(u) the vector of concentrations of all species of the variable u. Parameter inference can be carried out by solving the system of equations for a given parameter set and minimising the discrepancy between the predicted signals from the ODEs and the data. Robinson (2004) contains a background on methods used for obtaining numerical solutions for ODEs and amongst other topics, discusses the use of Euler’s method and the Runge–Kutta method as ways to do so Since solutions to the ODEs typically do not exist in closed form, explicit solutions of the ODEs need to be computed numerically. Robinson (2004) contains a background on methods used for obtaining numerical solutions for ODEs and amongst other topics, discusses the use of Euler’s method and the Runge–Kutta method as ways to do so

Methods

Results

Conclusion