Model-based Experimental Design Research Articles

Robust multi-stage model-based design of optimal experiments for nonlinear estimation

We study approaches to the robust model-based design of experiments in the context of maximum-likelihood estimation. These approaches provide robustification of model-based methodologies for the design of optimal experiments by accounting for the effect of the parametric uncertainty. We study the problem of robust optimal design of experiments in the framework of nonlinear least-squares parameter estimation using linearized confidence regions. We investigate several well-known robustification frameworks in this respect and propose a novel methodology based on multi-stage robust optimization. The proposed methodology aims at problems, where the experiments are designed sequentially with a possibility of re-estimation in-between the experiments. The multi-stage formalism aids in identifying experiments that are better conducted in the early phase of experimentation, where parameter knowledge is poor. We demonstrate the findings and effectiveness of the proposed methodology using four case studies of varying complexity.

Model‐based design of experiments for polyether production from bio‐based 1,3‐propanediol

AbstractSequential model‐based design of experiments (MDBOE) accounts for information from previous experiments when selecting conditions for new experiments. In the current study, sequential MBDOE is used to select operating conditions for experiments in a batch‐reactor that produces bio‐based polytrimethylene ether glycol (PO3G). These Bayesian A‐optimal experiments are designed to obtain improved estimates of 70 fundamental‐model parameters, while accounting for industrial data from eight previous runs. Settings are selected for three decision variables: reactor temperature, initial catalyst level, and initial water concentration. If only one new experiment is conducted, it should be run at high temperature, with relatively high concentrations of catalyst and initial water. When two new runs are conducted, one should use an intermediate catalyst concentration. The effectiveness of the proposed MBDOE approach is tested using Monte‐Carlo simulations, revealing that the selected experiments are superior compared to experiments selected randomly from corners of the permissible design space.

Digital Twin Application for Model-Based DoE to Rapidly Identify Ideal Process Conditions for Space-Time Yield Optimization

The fast exploration of a design space and identification of the best process conditions facilitating the highest space-time yield are of great interest for manufacturers. To obtain this information, depending on the design space, a large number of practical experiments must be performed, analyzed, and evaluated. To reduce this experimental effort and increase the process understanding, we evaluated a model-based design of experiments to rapidly identify the optimum process conditions in a design space maximizing space-time yield. From a small initial dataset, hybrid models were implemented and used as digital bioprocess twins, thus obtaining the recommended optimal experiment. In cases where these optimum conditions were not covered by existing data, the experiment was carried out and added to the initial data set, re-training the hybrid model. The procedure was repeated until the model gained certainty about the best process conditions, i.e., no new recommendations. To evaluate this workflow, we utilized different initial data sets and assessed their respective performances. The fastest approach for optimizing the space-time yield in a three-dimensional design space was found with five initial experiments. The digital twin gained certainty after four recommendations, leading to a significantly reduced experimental effort compared to other state-of-the-art approaches. This highlights the benefits of in silico design space exploration for accelerating knowledge-based bioprocess development, and reducing the number of hands-on experiments, time, energy, and raw materials.

Safe model-based design of experiments using Gaussian processes

The construction of kinetic models has become an indispensable step in developing and scale-up of processes in the industry. Model-based design of experiments (MBDoE) has been widely used to improve parameter precision in nonlinear dynamic systems. Such a framework needs to account for both parametric and structural uncertainty, as the physical or safety constraints imposed on the system may well turn out to be violated, leading to unsafe experimental conditions when an optimally designed experiment is performed. In this work, Gaussian processes are utilized in a two-fold manner: 1) to quantify the uncertainty realization of the physical system and calculate the plant-model mismatch, 2) to compute the optimal experimental design while accounting for the parametric uncertainty. The proposed method, Gaussian process-based MBDoE (GP-MBDoE), guarantees the probabilistic satisfaction of the constraints in the context of model-based design of experiments. GP-MBDoE is assisted with the use of adaptive trust regions to facilitate a satisfactory local approximation. The proposed method can allow the design of optimal experiments starting from limited preliminary knowledge of the parameter set, leading to a safe exploration of the parameter space. This method’s performance is demonstrated through illustrative case studies regarding the parameter identification of kinetic models in flow reactors.

Model-Based Design of Experiments for High-Dimensional Inputs Supported by Machine-Learning Methods

Algorithms that compute locally optimal continuous designs often rely on a finite design space or on the repeated solution of difficult non-linear programs. Both approaches require extensive evaluations of the Jacobian Df of the underlying model. These evaluations are a heavy computational burden. Based on the Kiefer-Wolfowitz Equivalence Theorem, we present a novel design of experiments algorithm that computes optimal designs in a continuous design space. For this iterative algorithm, we combine an adaptive Bayes-like sampling scheme with Gaussian process regression to approximate the directional derivative of the design criterion. The approximation allows us to adaptively select new design points on which to evaluate the model. The adaptive selection of the algorithm requires significantly less evaluations of Df and reduces the runtime of the computations. We show the viability of the new algorithm on two examples from chemical engineering.

Two-phase approaches to optimal model-based design of experiments: how many experiments and which ones?

Model-based experimental design is attracting increasing attention in chemical process engineering. Typically, an iterative procedure is pursued: an approximate model is devised, prescribed experiments are then performed and the resulting data is exploited to refine the model. To help to reduce the cost of trial-and-error approaches, strategies for model-based design of experiments suggest experimental points where the expected gain in information for the model is the largest. It requires the resolution of a large nonlinear, generally nonconvex, optimization problem, whose solution may greatly depend on the starting point. We present two discretization strategies that can assist the experimenter in setting the number of relevant experiments and performing an optimal selection, and we compare them against two pattern-based strategies that are independent of the problem. The validity of the approaches is demonstrated on an academic example and two test problems from chemical engineering including a vapor liquid equilibrium and reaction kinetics.

A model-based framework assisting the design of vapor-liquid equilibrium experimental plans

In this paper we propose a framework for Model-based Sequential Optimal Design of Experiments to assist experimenters involved in Vapor-Liquid equilibrium characterization studies to systematically construct thermodynamically consistent models. The approach uses an initial continuous optimal design obtained via semidefinite programming, and then iterates between two stages (i) model fitting using the information available; and (ii) identification of the next experiment, so that the information content in data is maximized. The procedure stops when the number of experiments reaches the maximum for the experimental program or the dissimilarity between the parameter estimates during two consecutive iterations is below a given threshold. This methodology is exemplified with the D-optimal design of isobaric experiments, for characterizing binary mixtures using the NRTL and UNIQUAC thermodynamic models for liquid phase. Significant reductions of the confidence regions for the parameters are achieved compared with experimental plans where the observations are uniformly distributed over the domain.

Accelerating the Development and Transfer of Freeze-Drying Operations for the Manufacturing of Biopharmaceuticals by Model-Based Design of Experiments

In the pharmaceutical industry, freeze-drying (also known as lyophilization) is often used to increase the shelf life of heat-sensitive biopharmaceuticals such as protein-based therapeutic drugs and vaccines. The most time- and energy-consuming step of a lyophilization process is primary drying. When a new system configuration (e.g., a new equipment, product formulation, or primary packaging) is to be tested in process development or when transferring to manufacturing facilities, one needs to characterize heat and mass transfer in the new configuration through a battery of experiments that can be very demanding. In this study, we use a model-based design of experiments (MBDoE) to design optimal experiments for fast primary-drying model parameter estimation. We show that MBDoE allows estimating all key heat- and mass-transfer parameters in a statistically meaningful way using one single, optimally designed experiment. Using data from industrial equipment, we test the effectiveness of the methodology on two typical problems to be faced in pharmaceutical process development and transfer: (i) transfer of product manufacturing between two different pieces of equipment and (ii) change of vial type for a given product in a given equipment. Results demonstrate that the proposed methodology can substantially ease the experimentation, thus accelerating the development of freeze-drying operations.

Using prior parameter knowledge in model‐based design of experiments for pharmaceutical production

AbstractSequential model‐based design of experiments (MBDoE) uses information from previous experiments to select new experimental conditions. Computation of MBDoE objective functions can be impossible due to a noninvertible Fisher information matrix (FIM). Previously, we evaluated a leave‐out (LO) approach that designed experiments by removing problematic model parameters from the design process. Unfortunately, the LO approach can be computationally expensive due to its iterative nature. In this study, we propose a simplified Bayesian approach that makes the FIM invertible by accounting for prior parameter information. We compare the proposed simplified Bayesian approach to the LO approach for sequential A‐optimal design. Results from a pharmaceutical case study show that the proposed approach is superior, on average, for design of experiments. We suggest that simplified Bayesian MBDoE should be combined with a subset‐selection‐based approach for parameter estimation. This combined methodology gave the best results on average for the case study.

Iterative model-based experimental design for efficient uncertainty minimization of chemical mechanisms

The uncertainties of chemical kinetic model parameters induce uncertainties in model predictions. Automatic optimization and uncertainty minimization techniques have been developed to constrain these uncertainties based on sets of experimental target data for quantities of interest. While such methods were frequently used to optimize models for relatively well-studied systems with large numbers of available targets, only few of these experimental data points may be of crucial importance. In addition, for novel fuel candidates such as biofuels and synthetic fuels, the number of available measurements is generally limited. Thus, an important aspect to be explored in this context is the number of experimental data points required to achieve a certain degree of uncertainty reduction, and the determination of optimal experimental conditions for these. To target this question, a model-based experimental design framework based on the criterion of D-optimality is used in the present work to automatically identify these optimal conditions. As an example, the auto-ignition of dimethyl ether is investigated. The majority of experiments with high priority cover the intermediate- and low-temperature regimes, where the employed prior model exhibits the largest prediction uncertainties. It is also found that 90 % of the maximum observed reduction of average prediction uncertainty in ignition delay times can be achieved based on only the ten most informative experiments alone. The results thus demonstrate that few well-selected measurements allow for efficient model uncertainty reduction, and the employed approach provides an effective means of identifying the optimal conditions, which is useful for further experimental investigation. On the other hand, the inclusion of more experiments into the calibration process still provides additional benefit in terms of the posterior uncertainties of a number of important model parameters, which points to the importance of taking such data into account in case of their availability.

Sequential model-based design of experiments for development of mathematical models for thin film deposition using chemical vapor deposition process

Experimental data is required to estimate unknown parameters for a mathematical model describing thin film growth and to optimize the chemical vapor deposition (CVD) process. The most informative experimental data can be obtained by sequential model-based design of experiments (MBDoE), but this requires significant effort to implement compared to factorial design of experiments (DoE). For CVD processes it is not clear which approach is better or more efficient. Here, we compare the effectiveness of sequential A-optimal (MBDoE) with a factorial (DoE) using a case study involving a mechanistic model of thin-film deposition. The sequential A-optimal MBDoE and factorial DoE are compared based on their effectiveness in obtaining accurate parameter estimates and model predictions. Synthetic experimental data is generated from simulations with the true parameters plus noise. The results suggest that the A-optimal MBDoE is slightly better than factorial DoE for estimating model parameters. The parameter estimates from each experimental design approach are used to solve a model-based optimization problem. The results indicate that model-based optimization from the model with parameter estimates obtained by the A-optimal design gives a significantly better estimate of the optimal process conditions than factorial DoE.

Disentangling model complexity in green roof hydrological analysis: A Bayesian perspective

Green Roofs (GRs) have proven to be a sustainable solution to stormwater management in urban areas. To boost their adoption at the large scale, there is a need to develop numerical models, which are accurate, computationally cheap, and as complex as needed to reproduce the hydrological behavior of GRs. Alternative conceptual and mechanistic approaches have been proposed and tested, however the most appropriate level of model complexity for GRs’ analysis is still unknown. To cover this scientific gap, we provide a Bayesian comprehensive perspective of GR hydrological modeling, which includes a statistically rigorous Bayesian comparison of one conceptual and multiple Richards-based mechanistic GR models, and a probabilistic assessment of the information content of different observations. The analysis of the marginal likelihoods reveals that the conceptual and the unimodal van Genuchten – Mualem models are the most appropriate parameterizations, and that further layers of model complexity are not fully supported by the measurements. In addition to that, the estimated Kullback-Leibler divergences suggest that the measured volumetric water content outperforms the measured subsurface outflow and tracer concentrations in terms of informativeness, leading to the lowest model predictive uncertainty for the simulation of water fluxes. The findings of this study represent a first step to clarify the role of model complexity in GRs’ analysis, and open new perspective on GRs’ model-based experimental design.

Predicting Pharmaceutical Particle Size Distributions Using Kernel Mean Embedding.

In the pharmaceutical industry, the transition to continuous manufacturing of solid dosage forms is adopted by more and more companies. For these continuous processes, high-quality process models are needed. In pharmaceutical wet granulation, a unit operation in the ConsiGma-25 continuous powder-to-tablet system (GEA Pharma systems, Collette, Wommelgem, Belgium), the product under study presents itself as a collection of particles that differ in shape and size. The measurement of this collection results in a particle size distribution. However, the theoretical basis to describe the physical phenomena leading to changes in this particle size distribution is lacking. It is essential to understand how the particle size distribution changes as a function of the unit operation’s process settings, as it has a profound effect on the behavior of the fluid bed dryer. Therefore, we suggest a data-driven modeling framework that links the machine settings of the wet granulation unit operation and the output distribution of granules. We do this without making any assumptions on the nature of the distributions under study. A simulation of the granule size distribution could act as a soft sensor when in-line measurements are challenging to perform. The method of this work is a two-step procedure: first, the measured distributions are transformed into a high-dimensional feature space, where the relation between the machine settings and the distributions can be learnt. Second, the inverse transformation is performed, allowing an interpretation of the results in the original measurement space. Further, a comparison is made with previous work, which employs a more mechanistic framework for describing the granules. A reliable prediction of the granule size is vital in the assurance of quality in the production line, and is needed in the assessment of upstream (feeding) and downstream (drying, milling, and tableting) issues. Now that a validated data-driven framework for predicting pharmaceutical particle size distributions is available, it can be applied in settings such as model-based experimental design and, due to its fast computation, there is potential in real-time model predictive control.

Model-based design of transient flow experiments for the identification of kinetic parameters

Rapid and precise estimation of kinetic parameters is facilitated by transient flow experiments designed using model-based design of experiments.

Closed-Loop Model-Based Design of Experiments for Kinetic Model Discrimination and Parameter Estimation: Benzoic Acid Esterification on a Heterogeneous Catalyst

An autonomous reactor platform was developed to rapidly identify a kinetic model for the esterification of benzoic acid with ethanol with the heterogeneous Amberlyst-15 catalyst. A five-step methodology for kinetic studies was employed to systematically reduce the number of experiments required to identify a practical kinetic model. This included (i) initial screening using traditional factorial designed steady-state experiments, (ii) proposing and testing candidate kinetic models, (iii) performing an identifiability analysis to reject models whose model parameters cannot be estimated for a given experimental budget, (iv) performing online Model-Based Design of Experiments (MBDoE) for model discrimination to identify the best model from a list of candidates, and (v) performing online MBDoE for improving parameter precision for the chosen model. This methodology combined with the reactor platform, which conducted all kinetic experiments unattended, reduces the number of experiments and time required to identify kinetic models, significantly increasing lab productivity.

Model-based experimental design for nonlinear dynamical systems with unknown state delay and continuous state inequalities

This paper studied the model-based design of experiment (MBDoE) for nonlinear dynamical systems with unknown state delay and continuous state inequalities (SDCSI). The design problem is formulated as a time-delay optimal control problem of a dynamic system governed by augmented sensitivity-state equations (ASSE). Consider the co-existence of control delay and multiple state delays in the ASSE, each continuous state inequality constraint is approximated by an integral constraint using a constraint-handling technique based on local smoothing approximation and constraint transcription, which guaranteed the satisfaction of state inequality constraints during the experiment. In addition, an efficient numerical procedure is derived to determine the gradients of the objective function and constraints, which involved integrating an auxiliary impulsive time-delay system backward in time. The procedure is combined with standard gradient-based optimization methods to solve the MBDoE-SDCSI problem. The performance of the proposed scheme is illustrated through the experimental design of a fed-batch biomass fermentation process.

Implementation and Application of Model-Based Design of Experiments in a Flowsheet Simulator

The benefit of model-based process simulation and optimization to enhance and improve process development depends on the reliability of the underlying model. Its selection and the values of the mod...

A new generation of predictive models: The added value of hybrid models for manufacturing processes of therapeutic proteins.

Due to the lack of complete understanding of metabolic networks and reaction pathways, establishing a universal mechanistic model for mammalian cell culture processes remains a challenge. Contrarily, data-driven approaches for modeling these processes lack extrapolation capabilities. Hybrid modeling is a technique that exploits the synergy between the two modeling methods. Although mammalian cell cultures are among the most relevant processes in biotechnology and indeed looks ideal for hybrid modeling, their application has only been proposed but never developed in the literature. This study provides a quantitative assessment of the improvement brought by hybrid models with respect to the state-of-the-art statistical predictive models in the context of therapeutic protein production. This is illustrated using a dataset obtained from a 3.5 L fed-batch experiment. With the goal to robustly define the process design space, hybrid models reveal a superior capability to predict the time evolution of different process variables using only the initial and process conditions in comparison to the statistical models. Hybrid models not only feature more accurate prediction results but also demonstrate better robustness and extrapolation capabilities. For the future application, this study highlights the added value of hybrid modeling for model-based process optimization and design of experiments.

Monte Carlo Simulations for the Analysis of Non-linear Parameter Confidence Intervals in Optimal Experimental Design.

Especially in biomanufacturing, methods to design optimal experiments are a valuable technique to fully exploit the potential of the emerging technical possibilities that are driving experimental miniaturization and parallelization. The general objective is to reduce the experimental effort while maximizing the information content of an experiment, speeding up knowledge gain in R&D. The approach of model-based design of experiments (known as MBDoE) utilizes the information of an underlying mathematical model describing the system of interest. A common method to predict the accuracy of the parameter estimates uses the Fisher information matrix to approximate the 90% confidence intervals of the estimates. However, for highly non-linear models, this method might lead to wrong conclusions. In such cases, Monte Carlo sampling gives a more accurate insight into the parameter's estimate probability distribution and should be exploited to assess the reliability of the approximations made through the Fisher information matrix. We first introduce the model-based optimal experimental design for parameter estimation including parameter identification and validation by means of a simple non-linear Michaelis-Menten kinetic and show why Monte Carlo simulations give a more accurate depiction of the parameter uncertainty. Secondly, we propose a very robust and simple method to find optimal experimental designs using Monte Carlo simulations. Although computational expensive, the method is easy to implement and parallelize. This article focuses on practical examples of bioprocess engineering but is generally applicable in other fields.

Influence of the error description on model-based design of experiments

Measurement noise and errors are inherent to all experimental measurements, thus it is necessary to consider them while optimally designing any experiment. The scope of this paper is to determine the influence of various error descriptions in the measurement data considering different approaches for the model-based design of experiments. The considered types of error are: random and systematic. The random error is assumed to be normally distributed wita finite variance. The systematic error is represented in two ways: either as a relative factor to the “exact” data or as a drift related to measurement time. Combinations of these types of error form the used error descriptions. Basically, two methods of finding the optimal experimental design are used: one is based on the Fisher Information Matrix (FIM) and the other one is using mean-squared reconstruction errors (MSE). Based on an application example which is a cantilever beam, a slender civil structure, consisting of two different materials, implemented by the Young’s modulus, the optimal designs for three sensor locations are calculated for 28 different error combinations which differ in error type and noise level. The results for the best sensor setups are displayed and discussed depending on the method used for the design of the experiment and the error description.The evaluation of the two approaches shows that the FIM does not take systematic errors into account and also it is not depending on the random noise level, whereas the MSE approach is considering systematic errors, but to its disadvantage, it is computationally much more costly.