Optimal Experimental Design Methods Research Articles

The optimal experiment? Influence of solution strategies on model-based optimal experimental design

Model-based optimal experimental design (OED) is a well known tool for efficient model development. However, it is not used very often. A few reasons for that are: a lack of understanding on how to work with complex OED methods and a small amount of ready-to-use tools available to directly apply OED methods. In the presented contribution OED and sampling strategies are used to categorize OED formulations as nonlinear programs. Different strategies and their combination are analyzed based on performance and robustness. Depending on the availability of measurements, control flexibility of the experimental setup, and model accuracy some strategies are more efficient than others. Based on the proposed guidelines, engineers will have a better understanding about which NLP formulation should be used for their specific task. The methods described are available to the community as a part of open-source code developed in Python.

PyOED: An Extensible Suite for Data Assimilation and Model-Constrained Optimal Design of Experiments

This paper describes PyOED, a highly extensible scientific package that enables developing and testing model-constrained optimal experimental design (OED) for inverse problems. Specifically, PyOED aims to be a comprehensive Python toolkit for model-constrained OED . The package targets scientists and researchers interested in understanding the details of OED formulations and approaches. It is also meant to enable researchers to experiment with standard and innovative OED technologies with a wide range of test problems (e.g., simulation models). OED, inverse problems (e.g., Bayesian inversion), and data assimilation (DA) are closely related research fields, and their formulations overlap significantly. Thus, PyOED is continuously being expanded with a plethora of Bayesian inversion, DA, and OED methods as well as new scientific simulation models, observation error models, and observation operators. These pieces are added such that they can be permuted to enable testing OED methods in various settings of varying complexities. The PyOED core is completely written in Python and utilizes the inherent object-oriented capabilities; however, the current version of PyOED is meant to be extensible rather than scalable. Specifically, PyOED is developed to “enable rapid development and benchmarking of OED methods with minimal coding effort and to maximize code reutilization.” This paper provides a brief description of the PyOED layout and philosophy and provides a set of exemplary test cases and tutorials to demonstrate the potential of the package.

Bayesian Optimal Experimental Design for Inferring Causal Structure

Inferring the causal structure of a system typically requires interventional data, rather than just observational data. Since interventional experiments can be costly, it is preferable to select interventions that yield the maximum amount of information about a system. We propose a novel Bayesian method for optimal experimental design by sequentially selecting interventions that minimize the expected posterior entropy as rapidly as possible. A key feature is that the method can be implemented by computing simple summaries of the current posterior, avoiding the computationally burdensome task of repeatedly performing posterior inference on hypothetical future datasets drawn from the posterior predictive. After deriving the method in a general setting, we apply it to the problem of inferring causal networks. We present a series of simulation studies, in which we find that the proposed method performs favorably compared to existing alternative methods. Finally, we apply the method to real data from two gene regulatory networks.

An exploratory model-based design of experiments approach to aid parameters identification and reduce model prediction uncertainty

The management of trade-off between experimental design space exploration and information maximization is still an open question in the field of optimal experimental design. In classical optimal experimental design methods, the uncertainty of model prediction throughout the design space is not always assessed after parameter identification and parameters precision maximization do not guarantee that the model prediction variance is minimized in the whole domain of model utilization. To tackle these issues, we propose a novel model-based design of experiments (MBDoE) method that enhances space exploration and reduces model prediction uncertainty by using a mapping of model prediction variance (G-optimality mapping). This explorative MBDoE (eMBDoE) named G-map eMBDoE is tested on two models of increasing complexity and compared against conventional factorial design of experiments, Latin Hypercube (LH) sampling and MBDoE methods. The results show that G-map eMBDoE is more efficient in exploring the experimental design space when compared to a standard MBDoE and outperforms classical design of experiments methods in terms of model prediction uncertainty reduction and parameters precision maximization.

Optimization of aerodynamic drag reduction for vehicles with non-smooth surfaces and research on aerodynamic characteristics under crosswind

Energy consumption and environmental pollution problems have become increasingly prominent in China. The drag reduction of vehicles is an effective method to improve the fuel economy of cars and reduce greenhouse emissions. In this paper, based on the MIRA squareback model, the influence of the shapes, arrangements, layout positions, and dimensions of the non-smooth units on the aerodynamic characteristics of vehicles are investigated through the computational fluid dynamics (CFD) method. The depth, longitudinal spacing, and horizontal spacing are taken as the design variables and the aerodynamic drag coefficient is used as the optimization objective. The optimal Latin hypercube experimental design method is adopted to select sample points, and the response values of each experiment sample point are calculated through the integrated platform in Isight software. According to the response values, the Kriging approximation model is built and validated. Furthermore, the multi-island genetic algorithm is applied to optimize the dimensions of pit non-smooth units. The aerodynamic characteristics of the smooth surface model and the optimized non-smooth surface model under steady crosswind are conducted using numerical simulation. The findings obtained from the investigation will provide a certain reference for the application of non-smooth units in the field of automobiles.

Genetic optimization design for aerodynamic structure of oblique flow compressor radial inlet chamber in compressed air energy storage system

AbstractThe compressor plays a significant role in the compressed air energy storage (CAES) system, and its performance directly determines the overall efficiency of the system and the economy of energy storage. Internal flow losses and outlet distortion due to poor design of its radial inlet chamber (RIC) geometry can degrade the performance of downstream components and reduce compressor unit performance. In this article, a multiobjective optimization design is carried out for the RIC of the oblique flow compressor in CAES system, and a parametric optimization design method of the RIC annular convergent channel meridian profile is developed. The optimal Latin hypercube experimental design (opt LHD) method is used to generate the sample library, and the radial basis function neural network (RBFNN) is used to establish the surrogate model of each design parameter and RIC performance. The multiobjective optimization is executed by the second‐generation non‐dominated sorting genetic algorithm (NSGA‐II). After optimization, the area of the annular convergent channel of the RIC is expanded, the transition of the meridian profile is smoother, the distortion coefficient of the RIC is reduced by 0.58%, and the total pressure loss coefficient is reduced by 16.45%. By using the optimized RIC, the performance of oblique flow compressor has been improved. Compared with the original model, the total pressure ratio and isentropic efficiency of the whole machine at the design point have been increased by 0.6% and 0.32%, respectively.

NLoed: A Python Package for Nonlinear Optimal Experimental Design in Systems Biology.

Modeling in systems and synthetic biology relies on accurate parameter estimates and predictions. Accurate model calibration relies, in turn, on data and on how well suited the available data are to a particular modeling task. Optimal experimental design (OED) techniques can be used to identify experiments and data collection procedures that will most efficiently contribute to a given modeling objective. However, implementation of OED is limited by currently available software tools that are not well suited for the diversity of nonlinear models and non-normal data commonly encountered in biological research. Moreover, existing OED tools do not make use of the state-of-the-art numerical tools, resulting in inefficient computation. Here, we present the NLoed software package and demonstrate its use with in vivo data from an optogenetic system in Escherichia coli. NLoed is an open-source Python library providing convenient access to OED methods, with particular emphasis on experimental design for systems biology research. NLoed supports a wide variety of nonlinear, multi-input/output, and dynamic models and facilitates modeling and design of experiments over a wide variety of data types. To support OED investigations, the NLoed package implements maximum likelihood fitting and diagnostic tools, providing a comprehensive modeling workflow. NLoed offers an accessible, modular, and flexible OED tool set suited to the wide variety of experimental scenarios encountered in systems biology research. We demonstrate NLoed's capabilities by applying it to experimental design for characterization of a bacterial optogenetic system.

Digital Twin Concepts with Uncertainty for Nuclear Power Applications

Digital Twins (DTs) are receiving considerable attention from multiple disciplines. Much of the literature at this time is dedicated to the conceptualization of digital twins, and associated enabling technologies and challenges. In this paper, we consider these propositions for the specific application of nuclear power. Our review finds that the current DT concepts are amenable to nuclear power systems, but benefit from some modifications and enhancements. Further, some areas of the existing modeling and simulation infrastructure around nuclear power systems are adaptable to DT development, while more recent efforts in advanced modeling and simulation are less suitable at this time. For nuclear power applications, DT development should rely first on mechanistic model-based methods to leverage the extensive experience and understanding of these systems. Model-free techniques can then be adopted to selectively, and correctively, augment limitations in the model-based approaches. Challenges to the realization of a DT are also discussed, with some being unique to nuclear engineering, however most are broader. A challenging aspect we discuss in detail for DTs is the incorporation of uncertainty quantification (UQ). Forward UQ enables the propagation of uncertainty from the digital representations to predict behavior of the physical asset. Similarly, inverse UQ allows for the incorporation of data from new measurements obtained from the physical asset back into the DT. Optimization under uncertainty facilitates decision support through the formal methods of optimal experimental design and design optimization that maximize information gain, or performance, of the physical asset in an uncertain environment.

An exchange algorithm for optimal calibration of items in computerized achievement tests

The importance of large scale achievement tests, like national tests in school, eligibility tests for university, or international assessments for evaluation of students, is increasing. Pretesting of questions for the above mentioned tests is done to determine characteristic properties of the questions by adding them to an ordinary achievement test. If computerized tests are used, it has been shown using optimal experimental design methods that it is efficient to assign pretest questions to examinees based on their abilities. The specific distribution of abilities of the available examinees are considered and restricted optimal designs are applied. A new algorithm is developed which builds on an equivalence theorem. It discretizes the design space with the possibility to change the grid adaptively during the run, makes use of an exchange idea and filters computed designs. It is illustrated how the algorithm works through some examples as well as how convergence can be checked. The new algorithm is flexible and can be used even if different models are assumed for different questions.

Accelerated knowledge discovery from omics data by optimal experimental design

How to design experiments that accelerate knowledge discovery on complex biological landscapes remains a tantalizing question. We present an optimal experimental design method (coined OPEX) to identify informative omics experiments using machine learning models for both experimental space exploration and model training. OPEX-guided exploration of Escherichia coli’s populations exposed to biocide and antibiotic combinations lead to more accurate predictive models of gene expression with 44% less data. Analysis of the proposed experiments shows that broad exploration of the experimental space followed by fine-tuning emerges as the optimal strategy. Additionally, analysis of the experimental data reveals 29 cases of cross-stress protection and 4 cases of cross-stress vulnerability. Further validation reveals the central role of chaperones, stress response proteins and transport pumps in cross-stress exposure. This work demonstrates how active learning can be used to guide omics data collection for training predictive models, making evidence-driven decisions and accelerating knowledge discovery in life sciences.

Gaussian Process Based Expected Information Gain Computation for Bayesian Optimal Design.

Optimal experimental design (OED) is of great significance in efficient Bayesian inversion. A popular choice of OED methods is based on maximizing the expected information gain (EIG), where expensive likelihood functions are typically involved. To reduce the computational cost, in this work, a novel double-loop Bayesian Monte Carlo (DLBMC) method is developed to efficiently compute the EIG, and a Bayesian optimization (BO) strategy is proposed to obtain its maximizer only using a small number of samples. For Bayesian Monte Carlo posed on uniform and normal distributions, our analysis provides explicit expressions for the mean estimates and the bounds of their variances. The accuracy and the efficiency of our DLBMC and BO based optimal design are validated and demonstrated with numerical experiments.

Use a sequential gradient-enhanced-Kriging optimal experimental design method to build high-precision approximate model for complex simulation problem

The surrogate model based on Kriging has been widely used to approximate simulation problems of expensive computing. Although the accuracy of the gradient enhanced Kriging (GEK) is often higher than that of ordinary Kriging, designers cannot avoid more time consuming during gradient calculation of GEK. To this end, a sequential gradient-enhanced-Kriging optimal experimental design method with the Gaussian correlation function (GCF) is investigated to approximate complex black-box simulation problems by introducing gradient information of Kriging parameters. Due to the differentiable GCF, the gradient information can be simply evaluated. This characteristic make the proposed method effectively improve the modeling accuracy and efficiency of GEK. As expected, the test results from benchmark functions and the cycloid gear pump simulation show the feasibility, stability and applicability of the proposed method.

A Bayesian Robust Observation Design Approach for Systems with (Large) Parametric Uncertainties

Classical optimal experimental design (OED) methods have not been fully exploited in modeling of complex systems, due to the brittle design results generated based on prior models and computational burden in the optimization scheme. In this work, a novel method for robust experimental design (RED) of combined measurement set selection and sampling time scheduling has been proposed for systems with large parameter uncertainties. A Bayesian design framework is employed, involving Gaussian quadrature formula (GQF) approximation of the expected performance of the posterior distribution over uncertain parameter domain. The robust Bayesian experimental design (BED) has been relaxed to a semi-definite programming (SDP) problem which can be solved as a convex optimization problem. The proposed method has been examined by simulation studies on a lab-scale enzymatic biodiesel production system, with results compared to OED and uniform sampling under two design scenarios.

A novel integrated process–machine approach for designing customized milling machines

This paper provides a novel integrated approach for designing customized milling machines. Static stiffness matching design of milling machine is carried out according to the static stiffness requirement at TCP. Rapid evaluation approach of milling stability of milling machine with position-dependent has been developed. The theoretical model which can describe the relationships between and among milling stability, dynamic stiffness, and natural frequency has been established. Dynamic characteristics’ simulation of milling machine has been carried out based on CAD/CAE co-simulation analysis platform. The structural component mass can be changed by changing the critical dimension, and then, the rule of structural component mass influence on the natural frequency of the milling machine can be obtained by optimal Latin hypercube experimental design method. The lightweight design of milling machine has been achieved according to the rule. The approach improves the rapidity, parameterization, and digitization of customized milling machine design.

Optimal parameter identification of synthetic gene networks using harmony search algorithm.

Computational modeling of engineered gene circuits is an important while challenged task in systems biology. In order to describe and predict the response behaviors of genetic circuits using reliable model parameters, this paper applies an optimal experimental design(OED) method to obtain input signals. In order to obtain informative observations, this study focuses on maximizing Fisher information matrix(FIM)-based optimal criteria and to provide optimal inputs. Furthermore, this paper designs a two-stage optimization with the modified E-optimal criteria and applies harmony search(HS)-based OED algorithm to minimize estimation errors. The proposed optimal identification methodology involves estimation errors and the sample size to pursue a trade-off between estimation accuracy and measurement cost in modeling gene networks. The designed cost function takes two major factors into account, in which experimental costs are proportional to the number of time points. Experiments select two types of synthetic genetic networks to validate the effectiveness of the proposed HS-OED approach. Identification outcomes and analysis indicate the proposed HS-OED method outperforms two candidate OED approaches, with reduced computational effort.

Fast Bayesian experimental design: Laplace-based importance sampling for the expected information gain

In calculating expected information gain in optimal Bayesian experimental design, the computation of the inner loop in the classical double-loop Monte Carlo requires a large number of samples and suffers from underflow if the number of samples is small. These drawbacks can be avoided by using an importance sampling approach. We present a computationally efficient method for optimal Bayesian experimental design that introduces importance sampling based on the Laplace method to the inner loop. We derive the optimal values for the method parameters in which the average computational cost is minimized for a specified error tolerance. We use three numerical examples to demonstrate the computational efficiency of our method compared with the classical double-loop Monte Carlo, and a single-loop Monte Carlo method that uses the Laplace approximation of the return value of the inner loop. The first demonstration example is a scalar problem that is linear in the uncertain parameter. The second example is a nonlinear scalar problem. The third example deals with the optimal sensor placement for an electrical impedance tomography experiment to recover the fiber orientation in laminate composites.

Optimal measurement design for parameter identification in mechanized tunneling

When performing shallow tunnel construction, settlements on the ground surface often cannot be prevented. Anticipating these surface displacements is only possible with profound knowledge of the constitutive parameters of the surrounding soil. Performing inverse analysis on the basis of in situ settlement data is an efficient method for obtaining such information. However, during this process, considering which measurement arrangement can provide the most reliable results is generally neglected. This aspect is addressed in this study by applying the so-called “optimal experimental design” to the mechanized tunneling field. A global sensitivity analysis (GSA) is firstly performed to determine the most relevant model parameters to be identified via back analysis, by employing the considered numerical model and experimental data. Furthermore, the GSA results are utilized to determine where and when measurements should be performed to minimize uncertainty in the identified constitutive parameters. The optimal experimental design (OED) concept is further applied to evaluate the observation set-up efficiency for damage mitigation measures within a representative synthetic example of a tunneling project passing beneath an existing building. Parameter identification based on synthetic noisy experiments is performed to validate the presented method for optimal experimental design. Thus, the soil stiffness and strength parameters are identified according to both an intuitive and the elaborated method, employing the proposed OED strategy and experimental designs, making it possible to assess the feasibility of the OED results.

Bounded-error optimal experimental design via global solution of constrained min–max program

We present an improvement of existing methods for globally solving optimal experimental design (OED) for bounded-error estimation based on a bilevel formulation from Mukkala et al. (2017). The proposed solution method for the min–max program is based on our method for generalized semi-infinite programs (via restriction of the right-hand side). The algorithm employed has the advantage that it guarantees a global solution for the OED assuming the global solution of two subproblems. To obtain a feasible solution only the lower-level problem has to be solved globally. In case of a local solution of the upper-level problem, the solution is still feasible though it is an upper bound of the global solution. The min–max method for OED is illustrated with four examples: two simple chemical reactions, BET-adsorption and a reformulated predator-prey system. The benefits of global methods are shown along with the limitations of state-of-the-art global solvers.

Principles of Experimental Design for Big Data Analysis.

Big Datasets are endemic, but are often notoriously difficult to analyse because of their size, heterogeneity and quality. The purpose of this paper is to open a discourse on the potential for modern decision theoretic optimal experimental design methods, which by their very nature have traditionally been applied prospectively, to improve the analysis of Big Data through retrospective designed sampling in order to answer particular questions of interest. By appealing to a range of examples, it is suggested that this perspective on Big Data modelling and analysis has the potential for wide generality and advantageous inferential and computational properties. We highlight current hurdles and open research questions surrounding efficient computational optimisation in using retrospective designs, and in part this paper is a call to the optimisation and experimental design communities to work together in the field of Big Data analysis.

Bayesian Optimal Sensor Placement for Modal Identification of Civil Infrastructures

A Bayesian optimal experimental design (OED) method is proposed in this work for estimating the best locations of sensors in structures so that the measured data are most informative for estimating reliably the structural modes. The information contained in the data is measured by the Kullback-Leibler (K-L) divergence between the prior and posterior distribution of the model parameters taken in modal identification to be the modal coordinates. The optimal sensor placement that maximizes the expected K-L divergence is shown also to minimize the information entropy of the posterior distribution. Unidentifiability issues observed in existing formulations when the number of sensors is less than the number of identified modes, are resolved using a non-uniform prior in the Bayesian OED. An insightful analysis is presented that demonstrates the effect of the variances of Bayesian priors on the optimal design. For dense mesh finite element models, sensor clustering phenomena are avoided by integrating in the methodology spatially correlated prediction error models. A heuristic forward sequential sensor placement algorithm and a stochastic optimization algorithm are used to solve the optimization problem in the continuous physical domain of variation of the sensor locations. The theoretical developments and algorithms are applied for the optimal sensor placement design along the deck of a 537 m concrete bridge.