Optimal experimental design for machine learning using the Fisher information

Abstract

Optimal experimental design focuses on selecting experiments that minimize the statistical uncertainty in inferred parameter or predictions. In traditional optimizations, the experiment consists of input data, model parameters, and cost function. For machine learning and deep learning, the features, labels, and loss function define the experiment. One tool for optimal experimental design is the Fisher information, which gives an estimate of the relative uncertainty in and correlation among the model parameters based on the local curvature of the cost function. Using the Fisher information allows for rapid assessment of many different experimental conditions. In machine learning, the Fisher information can provide guidance as to which types of input features and labels maximize the gradients in the search space. This approach has been applied, for example, to systems biology models of biochemical reaction networks [Transtrum and Qiu, BMC Bioinformatics 13(1), 181 (2012)]. Preliminary application of the Fisher information to optimize experimental design for source localization in an uncertain ocean environment is a step towards finding an efficient machine learning algorithm that produces results with the least uncertainty in the quantities of interest.