Sequential Experimental Design Research Articles

On Inadequacy of Sequential Design of Experiments for Performance-Driven Surrogate Modeling of Antenna Input Characteristics

Design of contemporary antennas necessarily involves electromagnetic (EM) simulation tools. Their employment is imperative to ensure evaluation reliability but also to carry out the design process itself, especially, the adjustment of antenna dimensions. For the latter, traditionally used parameter sweeping is more and more often replaced by rigorous numerical optimization, which entails considerable computational expenses, sometimes prohibitive. A potentially attractive way of expediting the simulation-based design procedures is the replacement of expensive EM analysis by fast surrogate models (or metamodels). Unfortunately, due to the curse of dimensionality and considerable nonlinearity of antenna characteristics, applicability of conventional modeling methods is limited to structures described by small numbers of parameters within narrow ranges thereof. A recently proposed nested kriging technique works around these issues by allocating the surrogate model domain within the regions containing designs that are of high quality with respect to the selected performance figures. This paper investigates whether sequential design of experiments (DoE) is capable of enhancing the modeling accuracy over one-shot space-filling data sampling originally implemented in the nested kriging framework. Numerical verification carried out for two microstrip antennas indicates that no noticeable benefits can be achieved, which contradicts the common-sense expectations. This result can be explained by a particular geometry of the confined domain of the performance-driven surrogate. As this set consists of nearly-optimum designs, the average nonlinearity of the antenna responses therein is almost location independent, therefore optimum training data allocation should be close to uniform. This is indeed corroborated by our experiments.

Experimental Design to Improve Cell Growth and Ethanol Production in Syngas Fermentation by Clostridium carboxidivorans

Fermentation of gases from biomass gasification, named syngas, is an important alternative process to obtain biofuels. Sequential experimental designs were used to increase cell growth and ethanol production during syngas fermentation by Clostridium carboxidivorans. Based on ATCC (American Type Culture Collection) 2713 medium composition, it was possible to propose a best medium composition for cell growth, herein called TYA (Tryptone-Yeast extract-Arginine) medium and another one for ethanol production herein called TPYGarg (Tryptone-Peptone-Yeast extract-Glucose-Arginine) medium. In comparison to ATCC® 2713 medium, TYA increased cell growth by 77%, reducing 47% in cost and TPYGarg increased ethanol production more than four-times, and the cost was reduced by 31%. In 72 h of syngas fermentation in TPYGarg medium, 1.75-g/L of cells, 2.28 g/L of ethanol, and 0.74 g/L of butanol were achieved, increasing productivity for syngas fermentation.

Bundling, pricing schemes and extra features preferences for mobility as a service: Sequential portfolio choice experiment

Mobility as a Service (MaaS) represents an innovative solution expected to induce people abandoning their cars in favor of more sustainable ways of travelling. However, a precondition for the successful implementation and diffusion of MaaS is public acceptance. In this paper, the latent demand for MaaS is estimated using a choice model based on a stated preference survey, conducted in the Netherlands. Specifically, a sequential experimental design approach based on portfolio choice is developed. It consists of two steps aimed to investigate (i) individual intention to subscribe to MaaS and preferences for bundle configurations, (ii) willingness to pay for extra features of the service. This article reports results of the analysis of the first step of the experiment. Two different mixed logit models are estimated to capture unobserved heterogeneity in individual’s preferences. First, a binary mixed logit model is formulated to estimate the effect of service attributes, social influence, socio-demographics and transportation-related characteristics on the decision to subscribe. Then, a mixed logit model is formulated to analyze the effects of transportation mode pricing schemes, and cross effects between transportation modes and individual characteristics on the choice which transportation modes to include in the subscription. The estimation results indicate that, overall, respondents are not yet inclined to subscribe to this new service. The service attribute characteristics, especially the price of the monthly subscription, and the social influence variables have an important effect on the subscription intention. The results also show that public transportation is the most preferred transportation mode, revealing that it may play a key role in MaaS platforms. The decision to subscribe and the choice which transportation modes to include in the bundle, appear to be significantly related to socio-demographic profiles and individuals’ transportation-related characteristics. The findings can be used as a valuable source of information for transportation planners and policy makers.

Efficient use of multiple information sources in material design

We present a general framework for the design/optimization of materials that is capable of accounting for multiple information sources available to the materials designer. We demonstrate the framework through the microstructure-based design of multi-phase microstructures. Specifically, we seek to maximize the strength normalized strain-hardening rate of a dual-phase ferritic/martensitic steel through a multi-information source Bayesian optimal design strategy. We assume that we have multiple sources of information with varying degrees of fidelity as well as cost. The available information from all sources is fused through a reification approach and then a sequential experimental design is carried out. The experimental design seeks not only to identify the most promising region in the materials design space relative to the objective at hand, but also to identify the source of information that should be used to query this point in the decision space. The selection criterion for the source used, accounts for the discrepancy between the source and the ‘ground truth’ as well as its cost. It is shown that when there is a hard constraint on the budget available to carry out the optimization, accounting for the cost of querying individual sources is essential.

The experimental design approach to the optimization of a simple quick easy cheap effective rugged and safe method for the analysis of phytoestrogens in complex soy-based food.

The present paper deals with the multivariate optimization of an extraction-purification strategy for the determination of phytoestrogens (daidzein, genistein, coumestrol, formononetin, and biochanin A) in soy-based meat substitutes by high performance liquid chromatography with tandem mass spectrometry. For a reliable quantitation of these new concerning compounds in such a complex matrix, recovery and matrix effect must be carefully evaluated. Therefore, two sequential experimental designs were used to optimize the sample-pretreatment of soy-based burgers: the chosen technique was the quick, easy, cheap, effective, rugged and safe methodology, which does not require any particular facility or instrumentation. Thanks to the first screening design (Plackett-Burman), the significant factors influencing the studied responses were identified and further investigated through a response surface design (Box-Behnken). The optimal values of the variables (volume of extraction solvent mix/sample mass ratio and two clean-up sorbents) led to quantitative recoveries (97-104%) and low ion suppression (matrix effect 60-93%) for all analytes. This optimized method was characterized by low detection limits (0.2-1.5ng/g) and excellent intraday precision (RSD 2-4%). It was applied to the determination of the considered compounds in several soy-burgers from the Italian market, detecting low ng/g levels (up to 40ng/g) of coumestrol, formononetin, and biochanin A, and high concentrations (7.9-78µg/g) of genistein and daidzein.

A sequential experimental design for multivariate sensitivity analysis using polynomial chaos expansion

ABSTRACT Multivariate output sensitivity analysis has gained much attention when the output of the computational model is a vector. A preferable strategy to deal with the multivariate output issue is the covariance decomposition approach based on the polynomial chaos expansion (PCE) metamodel. However, since the PCE construction depends on the quality of experimental design to some extent, the selection of design points is significant in determining the accuracy of the sensitivity estimator. In this article, a PCE-based sequential experimental design is proposed to estimate the multivariate output sensitivity index. In this method, the optimal design point is sequentially selected to minimize the determinant of covariance matrix of the sensitivity estimator. To validate the performance of the proposed method, several numerical examples are presented, which show that the sequential design approach performs better than other prevalent methods in terms of accuracy and robustness.

Optimal experimental design for joint reflection-transmission ultrasound breast imaging: From ray- to wave-based methods.

Ultrasound computed tomography (USCT) is an emerging modality to image the acoustic properties of the breast tissue for cancer diagnosis. With the need of improving the diagnostic accuracy of USCT, while maintaining the cost low, recent research is mainly focused on improving (1) the reconstruction methods and (2) the acquisition systems. D-optimal sequential experimental design (D-SOED) offers a method to integrate these aspects into a common systematic framework. The transducer configuration is optimized to minimize the uncertainties in the estimated model parameters, and to reduce the time to solution by identifying redundancies in the data. This work presents a formulation to jointly optimize the experiment for transmission and reflection data and, in particular, to estimate the speed of sound and reflectivity of the tissue using either ray-based or wave-based imaging methods. Uncertainties in the parameters can be quantified by extracting properties of the posterior covariance operator, which is analytically computed by linearizing the forward problem with respect to the prior knowledge about parameters. D-SOED is first introduced by an illustrative toy example, and then applied to real data. This shows that the time to solution can be substantially reduced, without altering the final image, by selecting the most informative measurements.

Tool for Recovering after Meteorological Events Using a Real-Time REM and IoT Management Platform

This paper is the design of a Radio Environment Map (REM) with a real-time tool to sense the radiofrequency spectrum and optimally places with Surrogate Modelling and Sequential Experimental Design tools a total of 72 SDR sensors in the selected area, using LoRa and/or NB-IoT technologies for networking. It permits the regulatory body to check the correct use of the assigned spectrum and constitutes a communication alternative in case of a catastrophic event, such as a hurricane or an earthquake, where radio and TV broadcasting play an important role in keeping people informed after such meteorological event. The radiobroadcast services use large antennas and high towers, making them vulnerable to such events. Regardless of the chosen technology, the IoT monitoring network will be more robust, since it uses small antennas and lower towers, and often a given area is covered by multiple base stations. The tool can be used to deploy new services in the nonserved area (e.g., 4G in the 700 MHz band at a lower cost or using TVWS techniques to provide communications and internet connection) and optimal interference management.

Efficient Use of Multiple Information Sources in Material Design

We present a general framework for the design/optimization of materials that is capable of accounting for multiple information sources available to the materials designer. We demonstrate the framework through the microstructure-based design of multi-phase microstructures. Specifically, we seek to maximize the strength normalized strain-hardening rate of a dual-phase ferritic/martensitic steel through a multi-information source Bayesian optimal design strategy. We assume that we have multiple sources of information with varying degrees of fidelity as well as cost. The available information from all sources is fused through a reification approach and then a sequential experimental design is carried out. The experimental design seeks not only to identify the most promising region in the materials design space relative to the objective at hand, but also to identify the source of information that should be used to query this point in the decision space. The selection criterion for the source used accounts for the discrepancy between the source and the 'ground truth' as well as its cost. It is shown that when there is a hard constraint on the budget available to carry out the optimization, accounting for the cost of querying individual sources is essential.

Exact group sequential designs for two-arm experiments with Poisson distributed outcome variables

We describe and compare two methods for the group sequential design of two-arm experiments with Poisson distributed data, which are based on a normal approximation and exact calculations respectively. A framework to determine near-optimal stopping boundaries is also presented. Using this framework, for a considered example, we demonstrate that a group sequential design could reduce the expected sample size under the null hypothesis by as much as 44% compared to a fixed sample approach. We conclude with a discussion of the advantages and disadvantages of the two presented procedures.

Bayesian Sequential Design of Experiments for Extraction of Single-Crystal Material Properties from Spherical Indentation Measurements on Polycrystalline Samples

This paper explores a formal strategy for the extraction of single-crystal elastic constants of cubic polycrystalline metals based on the optimal design of spherical indentation experiments. Recent work has formulated the extraction of elastic constants using a two-step protocol. The first step established a surrogate model to capture the dependence of the indentation elastic modulus on the grain (i.e., crystal lattice) orientation and values of single-crystal elastic constants. The second step involved the use of Markov chain Monte Carlo to establish distributions on the elastic constants consistent with experimentally measured indentation moduli in grains of different orientations. The present study refines the second step in order to sequentially identify specific grains that should be indented to provide the most utility in improving the estimated distributions on the elastic constants. The proposed workflow is demonstrated for the extraction of elastic constants for an as-cast cubic polycrystalline Fe-3%-Si sample.

Optimised Geophysical Survey Design for CO2 Monitoring – A Synthetic Study

In this work, we evaluate strategies for design of fit-for-purpose geophysical survey layouts that allow acquisition of the minimum necessary amount of data while maintaining a sufficient level of resolution or accuracy of the subsurface parameters to be recovered. The objective is to determine the optimal survey layout for CO2 monitoring based on our, potentially limited, knowledge of the subsurface prior to data acquisition. The proposed method is based on the analysis of the eigenvalue spectrum of the data misfit Hessian. The spectrum provides a simple and intuitive means to evaluate and compare the quality of given survey designs. Examples from seismic and EM monitoring are included. We perform synthetic studies and assess whether a sequential experimental design can be used to determine if only a small amount of data is sufficient to achieve comparable resolution to when the comprehensive dataset is used. Comparisons between random and optimal data subset selection are performed. The results suggest that the method is quite robust with respect to the starting model used for the analysis. The method offers a powerful means to acquire enough information at a minimal cost and analyse the pattern of the optimal acquisition design for a given tomographic method.

Active Anomaly Detection in Heterogeneous Processes

An active inference problem of detecting anomalies among heterogeneous processes is considered. At each time, a subset of processes can be probed. The objective is to design a sequential probing strategy that dynamically determines which processes to observe at each time and when to terminate the search so that the expected detection time is minimized under a constraint on the probability of misclassifying any process. This problem falls into the general setting of sequential design of experiments pioneered by Chernoff in 1959, in which a randomized strategy, referred to as the Chernoff test, was proposed and shown to be asymptotically optimal as the error probability approaches zero. For the problem considered in this paper, a low-complexity deterministic test is shown to enjoy the same asymptotic optimality while offering significantly better performance in the finite regime and faster convergence to the optimal rate function, especially when the number of processes is large. The computational complexity of the proposed test is also of a significantly lower order.

Improved Calculation Method for Insulation-based Fire Resistance of Composite Slabs

Abstract Floor slabs play a critical role in the fire resistance of buildings, not only by maintaining structural stability and integrity, but also by providing thermal insulation to limit the rise in temperature of floors above a fire. Composite slabs, consisting of concrete topping on steel decking, are common in steel building construction, but the profiled geometry of the decking makes the analysis of heat transfer in composite slabs more complex than for flat slabs. A method for calculating the insulation-based fire resistance of composite slabs with profiled steel decking is provided in Annex D of Eurocode 4 (EC4). However, the applicability of the EC4 calculation method is limited to a range of commonly used slab geometries from the 1990s, which is narrower than the range used in current practice. In addition, the EC4 calculation method assumes a specific value of moisture content for the concrete, and different values of moisture content can significantly affect the fire resistance, as shown in this study. This paper proposes an improved algebraic expression for estimation of the insulation-based fire resistance of composite slabs that explicitly accounts for moisture content and is applicable to an extended range of slab geometries. The proposed expression is developed based on computed values of fire resistance obtained from a validated finite element modeling approach. A set of 54 composite slab configurations are selected for analysis using a sequential experimental design. The accuracy of the proposed method is verified against numerical results for an additional set of 32 slab configurations and is also validated against experimental data. Comparisons of the proposed calculation method with the results of the verification analyses show deviations of less than 15 min in all cases for the insulation-based fire resistance of the composite slabs.

Improved calculation method for insulation-based fire resistance of composite slabs

Floor slabs play a critical role in the fire resistance of buildings, not only by maintaining structural stability and integrity, but also by providing thermal insulation to limit the rise in temperature of floors above a fire. Composite slabs, consisting of concrete topping on steel decking, are common in steel building construction, but the profiled geometry of the decking makes the analysis of heat transfer in composite slabs more complex than for flat slabs. A method for calculating the insulation-based fire resistance of composite slabs with profiled steel decking is provided in Annex D of Eurocode 4 (EC4). However, the applicability of the EC4 calculation method is limited to a range of commonly used slab geometries from the 1990s, which is narrower than the range used in current practice. In addition, the EC4 calculation method assumes a specific value of moisture content for the concrete, and different values of moisture content can significantly affect the fire resistance, as shown in this study. This paper proposes an improved algebraic expression for estimation of the insulation-based fire resistance of composite slabs that explicitly accounts for moisture content and is applicable to an extended range of slab geometries. The proposed expression is developed based on computed values of fire resistance obtained from a validated finite element modeling approach. A set of 54 composite slab configurations are selected for analysis using a sequential experimental design. The accuracy of the proposed method is verified against numerical results for an additional set of 32 slab configurations and is also validated against experimental data. Comparisons of the proposed calculation method with the results of the verification analyses show deviations of less than 15 min in all cases for the insulation-based fire resistance of the composite slabs.

Optimal Bayesian experiment design for nonlinear dynamic systems with chance constraints

The optimal design of experiments is crucial for maximizing the information content of data across a wide-range of experimental goals. This paper presents a Bayesian approach to optimal experiment design (OED) for parameter inference in constrained, dynamic, and nonlinear systems under noisy, incomplete, and indirect measurements. Bayesian OED maximizes an expected utility objective, which accounts for prior and posterior uncertainty in the model parameters from an information-theoretic standpoint. Due to the complicated form of the expected utility, it must be estimated using sample-based methods and, in particular, a nested Monte Carlo estimator that is expensive to evaluate using the full dynamic model. We propose a novel surrogate model based on arbitrary polynomial chaos (aPC), which readily applies to any type of prior distribution. The aPC expansions are constructed locally at each design visited during the iterative optimization procedure. The main cost in aPC, which is the determination of the expansion coefficients, is minimized by estimating these coefficients from only a minimal set of dynamic model evaluations. Although sample-based estimators can also be applied to the chance constraints, this leads to a potentially large number of binary variables, such that a smooth moment-based approximation is preferred in this work. Numerical simulations indicate that the proposed surrogate can significantly lower the computational cost of the Bayesian OED, while guaranteeing the original chance constraints are satisfied without noticeably increasing the average time to find a solution. As such, this methodology appears to have the potential to pave the way for real-time or sequential dynamic experiment design in a fully Bayesian setting.

Sequential infection experiments for quantifying innate and adaptive immunity during influenza infection.

Laboratory models are often used to understand the interaction of related pathogens via host immunity. For example, recent experiments where ferrets were exposed to two influenza strains within a short period of time have shown how the effects of cross-immunity vary with the time between exposures and the specific strains used. On the other hand, studies of the workings of different arms of the immune response, and their relative importance, typically use experiments involving a single infection. However, inferring the relative importance of different immune components from this type of data is challenging. Using simulations and mathematical modelling, here we investigate whether the sequential infection experiment design can be used not only to determine immune components contributing to cross-protection, but also to gain insight into the immune response during a single infection. We show that virological data from sequential infection experiments can be used to accurately extract the timing and extent of cross-protection. Moreover, the broad immune components responsible for such cross-protection can be determined. Such data can also be used to infer the timing and strength of some immune components in controlling a primary infection, even in the absence of serological data. By contrast, single infection data cannot be used to reliably recover this information. Hence, sequential infection data enhances our understanding of the mechanisms underlying the control and resolution of infection, and generates new insight into how previous exposure influences the time course of a subsequent infection.

Efficient sequential experimental design for surrogate modeling of nested codes

In this paper we consider two nested computer codes, with the first code output as one of the second code inputs. A predictor of this nested code is obtained by coupling the Gaussian predictors of the two codes. This predictor is non Gaussian and computing its statistical moments can be cumbersome. Sequential designs aiming at improving the accuracy of the nested predictor are proposed. One of the criteria allows to choose which code to launch by taking into account the computational costs of the two codes. Finally, two adaptations of the non Gaussian predictor are proposed in order to compute the prediction mean and variance rapidly or exactly.

Sequential Model-Based A-Optimal Design of Experiments When the Fisher Information Matrix Is Noninvertible

The sequential model-based optimal design of experiments (e.g., A-, D-, and E-optimal design) is a well-known technique for selecting experimental conditions that lead to informative data for obtai...

An experimental design framework for Markovian gene regulatory networks under stationary control policy

BackgroundA fundamental problem for translational genomics is to find optimal therapies based on gene regulatory intervention. Dynamic intervention involves a control policy that optimally reduces a cost function based on phenotype by externally altering the state of the network over time. When a gene regulatory network (GRN) model is fully known, the problem is addressed using classical dynamic programming based on the Markov chain associated with the network. When the network is uncertain, a Bayesian framework can be applied, where policy optimality is with respect to both the dynamical objective and the uncertainty, as characterized by a prior distribution. In the presence of uncertainty, it is of great practical interest to develop an experimental design strategy and thereby select experiments that optimally reduce a measure of uncertainty.ResultsIn this paper, we employ mean objective cost of uncertainty (MOCU), which quantifies uncertainty based on the degree to which uncertainty degrades the operational objective, that being the cost owing to undesirable phenotypes. We assume that a number of conditional probabilities characterizing regulatory relationships among genes are unknown in the Markovian GRN. In sum, there is a prior distribution which can be updated to a posterior distribution by observing a regulatory trajectory, and an optimal control policy, known as an “intrinsically Bayesian robust” (IBR) policy. To obtain a better IBR policy, we select an experiment that minimizes the MOCU remaining after applying its output to the network. At this point, we can either stop and find the resulting IBR policy or proceed to determine more unknown conditional probabilities via regulatory observation and find the IBR policy from the resulting posterior distribution. For sequential experimental design this entire process is iterated. Owing to the computational complexity of experimental design, which requires computation of many potential IBR policies, we implement an approximate method utilizing mean first passage times (MFPTs) – but only in experimental design, the final policy being an IBR policy.ConclusionsComprehensive performance analysis based on extensive simulations on synthetic and real GRNs demonstrate the efficacy of the proposed method, including the accuracy and computational advantage of the approximate MFPT-based design.