Optimal Experimental Design Methods Research Articles

Entropy-Based Experimental Design for Optimal Model Discrimination in the Geosciences

Choosing between competing models lies at the heart of scientific work, and is a frequent motivation for experimentation. Optimal experimental design (OD) methods maximize the benefit of experiments towards a specified goal. We advance and demonstrate an OD approach to maximize the information gained towards model selection. We make use of so-called model choice indicators, which are random variables with an expected value equal to Bayesian model weights. Their uncertainty can be measured with Shannon entropy. Since the experimental data are still random variables in the planning phase of an experiment, we use mutual information (the expected reduction in Shannon entropy) to quantify the information gained from a proposed experimental design. For implementation, we use the Preposterior Data Impact Assessor framework (PreDIA), because it is free of the lower-order approximations of mutual information often found in the geosciences. In comparison to other studies in statistics, our framework is not restricted to sequential design or to discrete-valued data, and it can handle measurement errors. As an application example, we optimize an experiment about the transport of contaminants in clay, featuring the problem of choosing between competing isotherms to describe sorption. We compare the results of optimizing towards maximum model discrimination with an alternative OD approach that minimizes the overall predictive uncertainty under model choice uncertainty.

Optimal Experimental Design to Identify the Average Stress‐Strain Response in Short Fiber‐Reinforced Plastics

AbstractWe show how to use optimal experimental design methods for the parameter identification of short fiber reinforced plastic (SFRP) materials. The experimental data is given by computer simulations of representative volume elements (RVE) of the SFRP material. The experiments are designed such that a minimal number of RVE simulations is required and that the model response attains a minimal variance for a class of strains and fiber orientations. (© 2016 Wiley‐VCH Verlag GmbH & Co. KGaA, Weinheim)

A bootstrap‐based method for optimal design of experiments

Bootstrapping can be used for the estimation of parameter variances, and it is straightforward to be implemented but computationally demanding compared with other methods for parameter error estimation. It is not bound to any restrictions such as the distribution of measurement errors. And because of the possible asymmetry of the probability densities of the parameters, the parameter estimation errors acquired by bootstrapping are likely to be more accurate.In this work the feasibility of a bootstrap‐based method for optimal experimental design was evaluated for the Peleg model. The optimal design was performed, based on the Cramér‐Rao lower bound as a benchmark. Afterwards, the optimal design was calculated based on the bootstrap method.It is demonstrated that a bootstrap‐based optimal design of experiments will give comparable results with the Cramér‐Rao lower bound optimal designs, however with slightly different measurement points in time. If the parameter errors obtained from both optimal experimental designs are compared, they deviate for the 2 methods on average by 1.5%.Bootstrapping can be used for problems, which cannot be solved using Cramér‐Rao lower bound because of necessary but invalid assumptions. However, the benefits of the bootstrap method come at the cost of a significant increase in computational effort. Under similar conditions, the computation time for a bootstrap‐based optimal design was 25 minutes compared with 5 seconds when using the Cramér‐Rao lower bound method. As computers get faster and faster over time, the increase in computational demand will probably become less relevant in the future.

Adaptive A-Optimal Experimental Design for Linear Dynamical Systems

While optimal experimental design for static linear inverse problems has been well studied, there is little in the way of experimental design methods for dynamical systems---in particular, dynamic problems where historic data is included in the optimal experimental design method such that the experiment tracks the motion of the model. Thus, we propose an adaptive design method which minimizes the adapted mean squared error of the regularized model estimate, defined by introducing a monitor function which scales the mean squared error according to historic model estimates. We present the adaptive design method for two model estimation formulations based on the classification of the error in the dynamic process. First, we consider the case where there is no error in the dynamical system and formulate the inverse problem as a PDE constrained optimization problem to recover the initial condition. Second, we include the error in the dynamics and formulate the inverse problem as a Kalman smoothing system to rec...

Research on optimization design of conformal cooling channels in hot stamping tool based on response surface methodology and multi-objective optimization

In order to optimize the layout of the conformal cooling channels in hot stamping tools, a response surface methodology and multi-objective optimization technique are proposed. By means of an Optimal Latin Hypercube experimental design method, a design matrix with 17 factors and 50 levels is generated. Three kinds of design variables, the radius Rad of the cooling channel, the distance H from the channel center to tool work surface and the ratio rat of each channel center, are optimized to determine the layout of cooling channels. The average temperature and temperature deviation of work surface are used to evaluate the cooling performance of hot stamping tools. On the basis of the experimental design results, quadratic response surface models are established to describe the relationship between the design variables and the evaluation objectives. The error analysis is performed to ensure the accuracy of response surface models. Then the layout of the conformal cooling channels is optimized in accordance with a multi-objective optimization method to find the Pareto optimal frontier which consists of some optimal combinations of design variables that can lead to an acceptable cooling performance.

Efficient experimental design for uncertainty reduction in gene regulatory networks.

BackgroundAn accurate understanding of interactions among genes plays a major role in developing therapeutic intervention methods. Gene regulatory networks often contain a significant amount of uncertainty. The process of prioritizing biological experiments to reduce the uncertainty of gene regulatory networks is called experimental design. Under such a strategy, the experiments with high priority are suggested to be conducted first.ResultsThe authors have already proposed an optimal experimental design method based upon the objective for modeling gene regulatory networks, such as deriving therapeutic interventions. The experimental design method utilizes the concept of mean objective cost of uncertainty (MOCU). MOCU quantifies the expected increase of cost resulting from uncertainty. The optimal experiment to be conducted first is the one which leads to the minimum expected remaining MOCU subsequent to the experiment. In the process, one must find the optimal intervention for every gene regulatory network compatible with the prior knowledge, which can be prohibitively expensive when the size of the network is large. In this paper, we propose a computationally efficient experimental design method. This method incorporates a network reduction scheme by introducing a novel cost function that takes into account the disruption in the ranking of potential experiments. We then estimate the approximate expected remaining MOCU at a lower computational cost using the reduced networks.ConclusionsSimulation results based on synthetic and real gene regulatory networks show that the proposed approximate method has close performance to that of the optimal method but at lower computational cost. The proposed approximate method also outperforms the random selection policy significantly. A MATLAB software implementing the proposed experimental design method is available at http://gsp.tamu.edu/Publications/supplementary/roozbeh15a/.

Optimal design of tests for heat exchanger fouling identification

Particulate fouling in plate fin heat exchangers of aircraft environmental control systems is a recurring issue in environments rich in foreign object debris. Heat exchanger fouling detection, in terms of quantification of its severity, is critical for aircraft maintenance scheduling and safe operation. In this work, we focus on methods for offline fouling detection during aircraft ground handling, where the allowable variability range of admissible inputs is wider. We explore methods of optimal experimental design to estimate heat exchanger inputs and input trajectories that maximize the identifiability of fouling. In particular, we present a methodology in which D-optimality is used as a criterion for statistically significant inference of heat exchanger fouling in uncertain environments. The optimal tests are designed on the basis of a heat exchanger model of the inherent mass, energy and momentum balances, validated against literature data. The model is then used to infer sensitivities of the heat exchanger outputs with respect to fouling metrics and maximize them by manipulating input trajectories; thus enhancing the accuracy in quantifying the fouling extent. The proposed methodology is evaluated with statistical indices of the confidence in estimating thermal fouling resistance at uncertain operating conditions, explored in a series of case studies.

Adaptive time-lapse optimized survey design for electrical resistivity tomography monitoring

Adaptive optimal experimental design methods use previous data and results to guide the choice and design of future experiments. This paper describes the formulation of an adaptive survey design technique to produce optimal resistivity imaging surveys for time-lapse geoelectrical monitoring experiments. These survey designs are time-dependent and, compared to dipole–dipole or static optimized surveys that do not change over time, focus a greater degree of the image resolution on regions of the subsurface that are actively changing. The adaptive optimization method is validated using a controlled laboratory monitoring experiment comprising a well-defined cylindrical target moving along a trajectory that changes its depth and lateral position. The algorithm is implemented on a standard PC in conjunction with a modified automated multichannel resistivity imaging system. Data acquisition using the adaptive survey designs requires no more time or power than with comparable standard surveys, and the algorithm processing takes place while the system batteries recharge. The results show that adaptively designed optimal surveys yield a quantitative increase in image quality over and above that produced by using standard dipole–dipole or static (time–independent) optimized surveys.

BioPreDyn-bench: a suite of benchmark problems for dynamic modelling in systems biology.

BackgroundDynamic modelling is one of the cornerstones of systems biology. Many research efforts are currently being invested in the development and exploitation of large-scale kinetic models. The associated problems of parameter estimation (model calibration) and optimal experimental design are particularly challenging. The community has already developed many methods and software packages which aim to facilitate these tasks. However, there is a lack of suitable benchmark problems which allow a fair and systematic evaluation and comparison of these contributions.ResultsHere we present BioPreDyn-bench, a set of challenging parameter estimation problems which aspire to serve as reference test cases in this area. This set comprises six problems including medium and large-scale kinetic models of the bacterium E. coli, baker’s yeast S. cerevisiae, the vinegar fly D. melanogaster, Chinese Hamster Ovary cells, and a generic signal transduction network. The level of description includes metabolism, transcription, signal transduction, and development. For each problem we provide (i) a basic description and formulation, (ii) implementations ready-to-run in several formats, (iii) computational results obtained with specific solvers, (iv) a basic analysis and interpretation.ConclusionsThis suite of benchmark problems can be readily used to evaluate and compare parameter estimation methods. Further, it can also be used to build test problems for sensitivity and identifiability analysis, model reduction and optimal experimental design methods. The suite, including codes and documentation, can be freely downloaded from the BioPreDyn-bench website, https://sites.google.com/site/biopredynbenchmarks/.Electronic supplementary materialThe online version of this article (doi:10.1186/s12918-015-0144-4) contains supplementary material, which is available to authorized users.

Cause and cure of sloppiness in ordinary differential equation models.

Data-based mathematical modeling of biochemical reaction networks, e.g., by nonlinear ordinary differential equation (ODE) models, has been successfully applied. In this context, parameter estimation and uncertainty analysis is a major task in order to assess the quality of the description of the system by the model. Recently, a broadened eigenvalue spectrum of the Hessian matrix of the objective function covering orders of magnitudes was observed and has been termed as sloppiness. In this work, we investigate the origin of sloppiness from structures in the sensitivity matrix arising from the properties of the model topology and the experimental design. Furthermore, we present strategies using optimal experimental design methods in order to circumvent the sloppiness issue and present nonsloppy designs for a benchmark model.

Optimal Experimental Design for Probabilistic Model Discrimination Using Polynomial Chaos

Building dynamic models is important in many applications including model-based design, optimization, and control. When multiple hypothesized models have predictions that are consistent with the measurements, experimental design is used to discriminate between the models. This task is particularly challenging for nonlinear systems subject to uncertainties. An optimal experimental design method for model discrimination for polynomial uncertain systems is presented that can be used to discriminate models based on dissimilarity of the probability densities of the model outputs. Generalized polynomial chaos theory in conjunction with Galerkin projection is used to derive an extended set of ordinary differential equations. Simulation of the extended system enables prediction of the propagation of probabilistic uncertainties associated with the model parameters and initial conditions, and to obtain the output probability densities. The simulation of the hypothetical models is embedded in a nonlinear optimization problem to determine an optimal input sequence that maximizes model dissimilarity. The experimental design method is demonstrated using a numerical example.

Application of D‐optimal experimental design method to optimize the formulation of O/W cosmetic emulsions

This study investigates the application of D-optimal mixture experimental design in optimization of O/W cosmetic emulsions. Cetearyl glucoside was used as a natural, biodegradable non-ionic emulsifier in the relatively low concentration (1%), and the mixture of co-emulsifiers (stearic acid, cetyl alcohol, stearyl alcohol and glyceryl stearate) was used to stabilize the formulations. To determine the optimal composition of co-emulsifiers mixture, D-optimal mixture experimental design was used. Prepared emulsions were characterized with rheological measurements, centrifugation test, specific conductivity and pH value measurements. All prepared samples appeared as white and homogenous creams, except for one homogenous and viscous lotion co-stabilized by stearic acid alone. Centrifugation testing revealed some phase separation only in the case of sample co-stabilized using glyceryl stearate alone. The obtained pH values indicated that all samples expressed mild acid value acceptable for cosmetic preparations. Specific conductivity values are attributed to the multiple phases O/W emulsions with high percentages of fixed water. Results of the rheological measurements have shown that the investigated samples exhibited non-Newtonian thixotropic behaviour. To determine the influence of each of the co-emulsifiers on emulsions properties, the obtained results were evaluated by the means of statistical analysis (ANOVA test). On the basis of comparison of statistical parameters for each of the studied responses, mixture reduced quadratic model was selected over the linear model implying that interactions between co-emulsifiers play the significant role in overall influence of co-emulsifiers on emulsions properties. Glyceryl stearate was found to be the dominant co-emulsifier affecting emulsions properties. Interactions between the glyceryl stearate and other co-emulsifiers were also found to significantly influence emulsions properties. These findings are especially important as they can be used for development of the product that meets users' requirements, as represented in the study.

Optimal experimental design for sampling voltage on dendritic trees in the low-SNR regime

Due to the limitations of current voltage sensing techniques, optimal filtering of noisy, undersampled voltage signals on dendritic trees is a key problem in computational cellular neuroscience. These limitations lead to voltage data that is incomplete (in the sense of only capturing a small portion of the full spatiotemporal signal) and often highly noisy. In this paper we use a Kalman filtering framework to develop optimal experimental design methods for voltage sampling. Our approach is to use a simple greedy algorithm with lazy evaluation to minimize the expected square error of the estimated spatiotemporal voltage signal. We take advantage of some particular features of the dendritic filtering problem to efficiently calculate the Kalman estimator's covariance. We test our framework with simulations of real dendritic branching structures and compare the quality of both time-invariant and time-varying sampling schemes. While the benefit of using the experimental design methods was modest in the time-invariant case, improvements of 25-100% over more naïve methods were found when the observation locations were allowed to change with time. We also present a heuristic approximation to the greedy algorithm that is an order of magnitude faster while still providing comparable results.

Optimal Design in Population Kinetic Experiments by Set-Valued Methods

We propose a new method for optimal experimental design of population pharmacometric experiments based on global search methods using interval analysis; all variables and parameters are represented as intervals rather than real numbers. The evaluation of a specific design is based on multiple simulations and parameter estimations. The method requires no prior point estimates for the parameters, since the parameters can incorporate any level of uncertainty. In this respect, it is similar to robust optimal design. Representing sampling times and covariates like doses by intervals gives a direct way of optimizing with rigorous sampling and dose intervals that can be useful in clinical practice. Furthermore, the method works on underdetermined problems for which traditional methods typically fail.

Hessian optimal design for image retrieval

Recently there has been a considerable interest in active learning from the perspective of optimal experimental design (OED). OED selects the most informative samples to minimize the covariance matrix of the parameters, so that the expected prediction error of the parameters, as well as the model output, can be minimized. Most of the existing OED methods are based on either linear regression or Laplacian regularized least squares (LapRLS) models. Although LapRLS has shown a better performance than linear regression, it suffers from the fact that the solution is biased towards a constant and the lack of extrapolating power. In this paper, we propose a novel active learning algorithm called Hessian optimal design (HOD). HOD is based on the second-order Hessian energy for semi-supervised regression which overcomes the drawbacks of Laplacian based methods. Specifically, HOD selects those samples which minimize the parameter covariance matrix of the Hessian regularized regression model. The experimental results on content-based image retrieval have demonstrated the effectiveness of our proposed approach.

Optimal Experimental Design for Discriminating Numerous Model Candidates: The AWDC Criterion

While model-based optimal experimental design (OED) strategies aiming at maximizing the parameter precision are regularly applied in industry and academia, only a little attention has been payed to OED techniques for model discrimination in practical applications. A broader use of these techniques is mainly hindered by two drawbacks: (i) The use of such techniques is desirable in an early model identification phase, where only a little knowledge on the process is available. The known methods, however, rely on good estimates of the parameters of all candidate model structures. (ii) The available methods are tailored to few (ideally two) model candidates and do not work well if numerous candidate structures are taken into account. In this work we propose a novel design criterion for model-based OED for model discrimination in the case of multiple model candidates. The resulting OED method is thus well-suited for designing experiments in an early stage of the model identification process to efficiently reduce the number of model candidates, thereby reducing the overall cost for model identification.

Identification of Reaction Mechanisms with a Dynamic PFR Model

Abstract Abstract In this work, a dynamic model of a catalytic fixed bed reactor (FBR) based on partial differential equations (PDE) is introduced and used for the identification of reaction mechanisms which take place during the oxidation of sulfur dioxide over a vanadium pentoxid catalyst. The measured data is collected from a pilot plant, which uses commercial sized catalyst particles. In order to reduce the experimental effort, a developed framework based on the methods of nonlinear optimal experimental design is applied using a steady state FBR model. The systematic procedure is improved using a dynamic reactor model. This makes the time dependent measurement data valuable for the identification procedure.

Optimal experimental design in an epidermal growth factor receptor signalling and down-regulation model

We apply the methods of optimal experimental design to a differential equation model for epidermal growth factor receptor signalling, trafficking and down-regulation. The model incorporates the role of a recently discovered protein complex made up of the E3 ubiquitin ligase, Cbl, the guanine exchange factor (GEF), Cool-1 (beta -Pix) and the Rho family G protein Cdc42. The complex has been suggested to be important in disrupting receptor down-regulation. We demonstrate that the model interactions can accurately reproduce the experimental observations, that they can be used to make predictions with accompanying uncertainties, and that we can apply ideas of optimal experimental design to suggest new experiments that reduce the uncertainty on unmeasurable components of the system.

Curvature-Based Methods for Designing Optimally Informative Experiments in Multiresponse Nonlinear Dynamic Situations

It is common practice in nonlinear regression situations to use asymptotic linear approximations of the model functions to construct parameter inference regions; such approximations may turn out to be a poor representation of the true underlying surfaces, especially for highly nonlinear situations and small sample sizes. For this reason, experimental designs based on these approximations could well be moderately noninformative. We present a new method for optimal experimental design for improving parametric precision while taking account of curvature in multiresponse nonlinear structured dynamic models. We base the curvature measures in the multiresponse case on the Box-Draper estimation criterion through use of the generalized least-squares model conditioned on the maximum likelihood estimate of the variance-covariance matrix for the responses. Curvature measures commensurate with those found in the literature are used for the generalized least-squares model in the neighborhood of the parameter point estimates. The problem of designing dynamic experiments is cast as an optimal control problem that enables the calculation of a fixed number of optimal sampling points, experiment duration, fixed and variable external control profiles, and initial conditions of a dynamic experiment subject to general constraints on inputs and outputs. We illustrate the experimental design concepts with a relatively simple but pedagogical example of the dynamic modeling of the fermentation of baker's yeast.

Optimal experimental design in stochastic structural dynamics

This paper provides a methodology for optimal prediction of the response of randomly vibrating structures using information from a limited number of measurements. The objective is to optimize the locations of sensors for the purpose of making the most accurate predictions of the response at unmeasured locations in structural systems. The kriging method is used to find the response predictions and the corresponding mean-square errors at unmeasured locations. The mean-square errors in the predictions depend on the locations of sensors and the correlation characteristics of the response evaluated from the model of dynamics and the characteristics of the excitation. The response predictions depend also on the information contained in measurements. The optimal sensor locations are selected to minimize the total mean-square error of the response predictions at unmeasured points. This leads to a complicated non-convex optimization problem in which multiple local and global optima may exist. A hybrid optimization method based on evolution strategies is used to determine a global minimum. The optimal experimental design method presented in the paper is illustrated by designing the optimal sensor locations for an elastic beam and a plate subjected to a class of random stationary loads.