Optimal experimental design with fast neural network surrogate models

Abstract

Designing optimal experiments minimizes the uncertainty of results and maximizes the efficient use of resources. Herein, machine learning surrogate models and the approximate coordinate exchange (ACE) algorithm are used to determine optimal experimental designs (OEDs) over large or arbitrarily restrictive design spaces. OED is particularly salient in materials science, where experiments are expensive and material properties must often be inferred indirectly. The proposed framework is demonstrated by finding optimal experiments with which the hidden constituent properties of composite materials can be most efficiently inferred from observable experimental outcomes. The OED is given by an information-theoretic criterion that maximizes the conditional mutual information between the hidden properties and the expected experimental outcomes. To perform tractable optimization, a neural network is trained as a surrogate model to mimic a physics-based simulation, which can calculate the expected experimental outcome based on a candidate experimental design and sampled constituent properties. The ACE algorithm is used to optimize over large design spaces with many tests and controlled parameters where an exhaustive search would be intractable even with the surrogate model. Using this approach, OEDs that are consistent with those produced by heuristic knowledge and established best practices are found; then optimal designs in larger design spaces where heuristic knowledge is unavailable are examined. All code, data, and trained models to reproduce the work in this paper is available at https://github.com/nasa/OED-with-NN-surrogates.