Optimal Experimental Design Problem Research Articles

Optimal experimental design for linear time invariant state–space models

The linear time invariant state–space model representation is common to systems from several areas ranging from engineering to biochemistry. We address the problem of systematic optimal experimental design for this class of model. We consider two distinct scenarios: (i) steady-state model representations and (ii) dynamic models described by discrete-time representations. We use our approach to construct locally D-optimal designs by incorporating the calculation of the determinant of the Fisher Information Matrix and the parametric sensitivity computation in a Nonlinear Programming formulation. A global optimization solver handles the resulting numerical problem. The Fisher Information Matrix at convergence is used to determine model identifiability. We apply the methodology proposed to find approximate and exact optimal experimental designs for static and dynamic experiments for models representing a biochemical reaction network where the experimental purpose is to estimate kinetic constants.

Robust optimal identification experiment design for multisine excitation

In least costly experiment design, the optimal spectrum of an identification experiment is determined in such a way that the cost of the experiment is minimized under some accuracy constraint on the identified parameter vector. Like all optimal experiment design problems, this optimization problem depends on the unknown true system, which is generally replaced by an initial estimate. One important consequence of this is that we can underestimate the actual cost of the experiment and that the accuracy of the identified model can be lower than desired. Here, based on an a-priori uncertainty set for the true system, we propose a convex optimization approach that allows to prevent these issues from happening. We do this when the to-be-determined spectrum is the one of a multisine signal.

Optimal Experimental Design for Uncertain Systems Based on Coupled Differential Equations

We consider the optimal experimental design problem for an uncertain Kuramoto model, which consists of N interacting oscillators described by coupled ordinary differential equations. The objective is to design experiments that can effectively reduce the uncertainty present in the coupling strengths between the oscillators, thereby minimizing the cost of robust control of the uncertain Kuramoto model. We demonstrate the importance of quantifying the operational impact of the potential experiments in designing optimal experiments.

A Scheme for Simultaneous Optimal Tracking Control and Experiment Design

This paper is about the formulation and numerical solutions of simultaneous optimal experiment design and optimal tracking control problems. Our motivating example is a robot arm that is mounted on a kitchen wall over the hotplates in order to assist humans with cooking. The robot arm can take over simple tasks such as stirring, automatic seasoning, or adding ingredients to a pot following a given recipe. Here, one of the main challenges is that the robot has to learn about the mass, inertial and other properties of the objects it is picking up while satisfying control tasks. Thus, we are facing a classical dual control problem, where system excitation for the purpose of learning has to be trade-off with other objectives such as tracking performance. After reviewing existing approaches, we propose a new formulation which allows to implement a trade-off between experiment design as well as tracking objectives in a systematic way by exploiting recent ideas from the field of economic experiment design. The approach is tested with a dynamic robot arm model performing a simple but illustrative cooking maneuver where learning and control goals are in conflict.

Design and Analysis of Switchback Experiments

Switchback experiments, where a firm sequentially exposes an experimental unit to random treatments, are among the most prevalent designs used in the technology sector, with applications ranging from ride-hailing platforms to online marketplaces. Although practitioners have widely adopted this technique, the derivation of the optimal design has been elusive, hindering practitioners from drawing valid causal conclusions with enough statistical power. We address this limitation by deriving the optimal design of switchback experiments under a range of different assumptions on the order of the carryover effect --- the length of time a treatment persists in impacting the outcome. We cast the optimal experimental design problem as a minimax discrete optimization problem, identify the worst-case adversarial strategy, establish structural results, and solve the reduced problem via a continuous relaxation. For switchback experiments conducted under the optimal design, we provide two approaches for performing inference. The first provides exact randomization based p-values, and the second uses a new finite population central limit theorem to conduct conservative hypothesis tests and build confidence intervals. We further provide theoretical results when the order of the carryover effect is misspecified and provide a data-driven procedure to identify the order of the carryover effect. We conduct extensive simulations to study the empirical properties of our results and conclude with practical suggestions.

Optimal experimental design for multi-response modeling of technological processes

The problem of optimal experimental design for the construction of multi-response models of the technological process is considered. Optimal experimental design can significantly improve the accuracy of model building and their predictive ability.

Optimization Based Multi-View Coverage Path Planning for Autonomous Structure From Motion Recordings

The area of autonomous mapping and autonomous maintenance using unmanned robots has received much attention in recent years. However, usually only single-view coverage is ensured, which is not sufficient if a camera-based three-dimensional reconstruction is desired. If high precision is required for this reconstruction, computations are expensive and not real-time feasible, such that recording and reconstruction procedure must be executed consecutively. This calls for a reliable method for collecting a sufficient set of images during the recording procedure. In this letter, we present the first approach for multi-view coverage path planning, specifically targeted at an autonomous structure from motion data acquisition. For that purpose, we introduce an incremental, computational efficient update formula, that gives an upper bound on the expected reconstruction quality of observed points on the boundary of a structure. Using this quality estimate, we iteratively solve the coverage path-planning problem by greedily selecting a next-best-view as a solution to an optimal experimental design problem. Additional geometric and dynamic constraints ensure the admissibility of the solution. The correctness of our quality estimate and coverage path-planning approach are validated in the physics simulator Gazebo.

A nonlinear multidimensional knapsack problem in the optimal design of mixture experiments

Mixture experiments usually involve various constraints on the proportions of the ingredients of the mixture under study. In this paper, inspired by the fact that the available stock of certain ingredients is often limited, we focus on a new type of constraint, which we refer to as an ingredient availability constraint. This type of constraint substantially complicates the search for optimal designs for mixture experiments. One difficulty, for instance, is that the optimal number of experimental runs is not known a priori. The resulting optimal experimental design problem belongs to the class of nonlinear nonseparable multidimensional knapsack problems. We present a variable neighborhood search algorithm as well as a mixed integer nonlinear programming approach to tackle the problem to identify D- and I-optimal designs for mixture experiments when there is a limited stock of certain ingredients, and we show that the variable neighborhood search algorithm is highly competitive in terms of solution quality and computing time.

Experiment Design Frameworks for Accelerated Discovery of Targeted Materials Across Scales

Over the last decade, there has been a paradigm shift away from labor-intensive and time-consuming materials discovery methods, and materials exploration through informatics approaches is gaining traction at present. Current approaches are typically centered around the idea of achieving this exploration through high-throughput (HT) experimentation/computation. Such approaches, however, do not account for the practicalities of resource constraints which eventually result in bottlenecks at various stage of the workflow. Regardless of how many bottlenecks are eliminated, the fact that ultimately a human must make decisions about what to do with the acquired information implies that HT frameworks face hard limits that will be extremely difficult to overcome. Recently, this problem has been addressed by framing the materials discovery process as an optimal experiment design problem. In this article, we discuss the need for optimal experiment design, the challenges in it’s implementation and finally discuss some successful examples of materials discovery via experiment design.

LPV system identification for control using the local approach

In this paper, we consider the problem of optimal experiment design for LPV system identification with the local approach when the identified model has to be used for LPV control. This local approach for LPV system identification is characterised by a number of local LTI identification experiments performed at constant values of the scheduling variable. In this paper, we determine these constant values of the scheduling variable as well as the input spectra of the corresponding local LTI identification experiments in an optimal way. The optimal LPV identification experiment is here defined as the experiment with the least input energy while guaranteeing that the identified LPV model is sufficiently accurate to yield a satisfactory LPV controller.

Optimal experiment design in nonlinear parameter estimation with exact confidence regions

A model-based optimal experiment design (OED) of nonlinear systems is studied. OED represents a methodology for optimizing the geometry of the parametric joint-confidence regions (CRs), which are obtained in an a posteriori analysis of the least-squares parameter estimates. The optimal design is achieved by using the available (experimental) degrees of freedom such that more informative measurements are obtained. Unlike the commonly used approaches, which base the OED procedure upon the linearized CRs, we explore a path where we explicitly consider the exact CRs in the OED framework. We use a methodology for a finite parametrization of the exact CRs within the OED problem and we introduce a novel approximation technique of the exact CRs using inner- and outer-approximating ellipsoids as a computationally less demanding alternative. The employed techniques give the OED problem as a finite-dimensional mathematical program of bilevel nature. We use two small-scale illustrative case studies to study various OED criteria and compare the resulting optimal designs with the commonly used linearization-based approach. We also assess the performance of two simple heuristic numerical schemes for bilevel optimization within the studied problems.

Sequential Optimal Inference for Experiments With Bayesian Particle Filters

In behavioral experiments, carefully choosing the stimuli is critical for success. Recently, several adaptive Bayesian methods gained popularity by proposing to optimally select the stimulus in each trial based on the results of the preceding trials. However, current methods are computationally expensive and might require a long waiting period between each question. Moreover, they are often tailored to a particular model and a particular objective, such as parameter estimation, prediction or model selection. It is left to the researcher to extend these approaches to other models by providing a suitable Bayesian inference method. We propose to apply the Sequential Monte Carlo (SMC) framework to solve both the inference problem and the optimal experimental design problem. This new method, called Sequential Optimal Inference (SOI) provides gains in computational efficiency and allows for the use of a broad class of complex models and objectives. We demonstrate its validity with simulation studies. An implementation of the method in MATLAB and Python is provided.

Optimal Experiment Design for Magnetic Resonance Fingerprinting: Cramér-Rao Bound Meets Spin Dynamics.

Magnetic resonance (MR) fingerprinting is a new quantitative imaging paradigm, which simultaneously acquires multiple MR tissue parameter maps in a single experiment. In this paper, we present an estimation-theoretic framework to perform experiment design for MR fingerprinting. Specifically, we describe a discrete-time dynamic system to model spin dynamics, and derive an estimation-theoretic bound, i.e., the Cramér-Rao bound, to characterize the signal-to-noise ratio (SNR) efficiency of an MR fingerprinting experiment. We then formulate an optimal experiment design problem, which determines a sequence of acquisition parameters to encode MR tissue parameters with the maximal SNR efficiency, while respecting the physical constraints and other constraints from the image decoding/reconstruction process. We evaluate the performance of the proposed approach with numerical simulations, phantom experiments, and in vivo experiments. We demonstrate that the optimized experiments substantially reduce data acquisition time and/or improve parameter estimation. For example, the optimized experiments achieve about a factor of two improvement in the accuracy of T2 maps, while keeping similar or slightly better accuracy of T1 maps. Finally, as a remarkable observation, we find that the sequence of optimized acquisition parameters appears to be highly structured rather than randomly/pseudo-randomly varying as is prescribed in the conventional MR fingerprinting experiments.

An induced natural selection heuristic for finding optimal Bayesian experimental designs

Bayesian optimal experimental design has immense potential to inform the collection of data so as to subsequently enhance our understanding of a variety of processes. However, a major impediment is the difficulty in evaluating optimal designs for problems with large, or high-dimensional, design spaces. An efficient search heuristic suitable for general optimisation problems, with a particular focus on optimal Bayesian experimental design problems, is proposed. The heuristic evaluates the objective (utility) function at an initial, randomly generated set of input values. At each generation of the algorithm, input values are “accepted” if their corresponding objective (utility) function satisfies some acceptance criteria, and new inputs are sampled about these accepted points. The new algorithm is demonstrated by evaluating the optimal Bayesian experimental designs for the previously considered death, pharmacokinetic and logistic regression models. Comparisons to the current “gold-standard” method are given to demonstrate the proposed algorithm as a computationally-efficient alternative for moderately-large design problems (i.e., up to approximately 40-dimensions).

Optimal Experimental Design for Inverse Problems with State Constraints

In this paper, we address the challenging problem of optimal experimental design (OED) of inverse problems with state constraints. We consider two OED formulations that allow us to reduce the exper...

Parallelizable Real-Time Algorithm for Integrated Experiment Design MPC

This paper proposes a parallelizable real-time algorithm for integrated experiment-design model predictive control (MPC). Integrated experiment design MPC is needed if a system is not observable at a tracking reference and needs to be excited on purpose in order to be able to estimate the system’s states and parameters. The contribution of this paper is a real-time MPC algorithm using two processors. On the first processor an extended Kalman filter (EKF) as well as a parametric certainty-equivalent MPC controller are implemented, which can provide immediate feedback at high sampling rates. On the second processor, optimal experiment design (OED) problems are solved in parallel in order to perturb the certainty-equivalent MPC control loop improving the accuracy of the state estimator at a lower sampling rate. We show that this framework can achieve optimal tradeoffs between OED and control objectives. The approach is applied to a biochemical process in order to illustrate that the proposed controller can achieve superior control performance when compared to certainty-equivalent MPC.

Optimal Experimental Design for Parameter Estimation of an IL-6 Signaling Model

IL-6 signaling plays an important role in inflammatory processes in the body. While a number of models for IL-6 signaling are available, the parameters associated with these models vary from case to case as they are non-trivial to determine. In this study, optimal experimental design is utilized to reduce the parameter uncertainty of an IL-6 signaling model consisting of ordinary differential equations, thereby increasing the accuracy of the estimated parameter values and, potentially, the model itself. The D-optimality criterion, operating on the Fisher information matrix and, separately, on a sensitivity matrix computed from the Morris method, was used as the objective function for the optimal experimental design problem. Optimal input functions for model parameter estimation were identified by solving the optimal experimental design problem, and the resulting input functions were shown to significantly decrease parameter uncertainty in simulated experiments. Interestingly, the determined optimal input functions took on the shape of PRBS signals even though there were no restrictions on their nature. Future work should corroborate these findings by applying the determined optimal experimental design on a real experiment.

A Feedback Optimal Control Algorithm with Optimal Measurement Time Points

Nonlinear model predictive control has been established as a powerful methodology to provide feedback for dynamic processes over the last decades. In practice it is usually combined with parameter and state estimation techniques, which allows to cope with uncertainty on many levels. To reduce the uncertainty it has also been suggested to include optimal experimental design into the sequential process of estimation and control calculation. Most of the focus so far was on dual control approaches, i.e., on using the controls to simultaneously excite the system dynamics (learning) as well as minimizing a given objective (performing). We propose a new algorithm, which sequentially solves robust optimal control, optimal experimental design, state and parameter estimation problems. Thus, we decouple the control and the experimental design problems. This has the advantages that we can analyze the impact of measurement timing (sampling) independently, and is practically relevant for applications with either an ethical limitation on system excitation (e.g., chemotherapy treatment) or the need for fast feedback. The algorithm shows promising results with a 36% reduction of parameter uncertainties for the Lotka-Volterra fishing benchmark example.

Bayesian optimal experimental design for parameter estimation and response predictions in complex dynamical systems

A Bayesian optimal experimental design (OED) framework is revisited and applied to a number of structural dynamics problems. The objective is to optimize the design of the experiment such that the most informative data are obtained for either for parameter estimation or response predictions. The Bayesian OED is based on maximizing the expected utility function taken as the Kullback-Leibler divergence between the prior and posterior distribution of the model parameters. Asymptotic approximations for the multi-dimensional integrals arising in the formulation of the expected utility function are proposed, valid for large number of data and small prediction errors. The OED based on these approximations are shown to be equivalent to the OED based on the robust information entropy introduced in the past for structural dynamics applications. Analytical expressions are developed to point out the effect of the variances of Bayesian Gaussian priors on the optimal design. The design variables may include the location of sensors, location of actuators or characteristics of the excitation such as amplitude variation and frequency content characteristics. A stochastic optimization algorithm is conveniently used to solve the optimization problem in the continuous physical domain of variation of the design variables. The proposed framework is applicable to complex linear and nonlinear dynamical systems. The asymptotic results are compared to the results obtained from accurate but computationally expensive sampling algorithms and are shown to be adequate for experimental design purposes. Two optimal experimental design problems illustrate the proposed methodology: 1) optimal sensor placement for load identification in nonlinear beam models, 2) optimal sensor placement for modal identification of bridges using complex FE models.

Optimal experiment design for magnetic resonance fingerprinting.

Magnetic resonance (MR) fingerprinting is an emerging quantitative MR imaging technique that simultaneously acquires multiple tissue parameters in an efficient experiment. In this work, we present an estimation-theoretic framework to evaluate and design MR fingerprinting experiments. More specifically, we derive the Cramér-Rao bound (CRB), a lower bound on the covariance of any unbiased estimator, to characterize parameter estimation for MR fingerprinting. We then formulate an optimal experiment design problem based on the CRB to choose a set of acquisition parameters (e.g., flip angles and/or repetition times) that maximizes the signal-to-noise ratio efficiency of the resulting experiment. The utility of the proposed approach is validated by numerical studies. Representative results demonstrate that the optimized experiments allow for substantial reduction in the length of an MR fingerprinting acquisition, and substantial improvement in parameter estimation performance.