Optimal experimental design for machine learning using the Fisher information matrix

Abstract

Optimal experimental design (OED) refers to a class of methods for selecting new data collection conditions that minimize the statistical uncertainty in the inferred parameter values of a model. The Fisher information matrix (FIM) gives an estimate of the relative uncertainty in and correlation among the model parameters based on the local curvature of the cost function. FIM-based approaches to OED allow for rapid assessment of many different experimental conditions (e.g., input data type, parameterizations, etc.). In machine learning models, accurate parameter estimates are often not a priority (nor even desirable) as they have no direct physical meaning. Instead, one would like to minimize the uncertainty in the model predictions for several quantities of interest. FIM approaches to OED can be generalized to minimize statistical variance, not in parameters, but in predictions of the quantities of interest. This approach has been applied, for example, to systems biology models of biochemical reaction networks [Transtrum and Qiu, BMC Bioinformatics 13(1), 181 (2012)]. Preliminary application of the FIM to optimize experimental design for source localization in an uncertain ocean environment is a first step towards an efficient machine learning algorithm that produces results with the least uncertainty in the quantities of interest.