Experimental Design Framework Research Articles

An optimal experimental design framework for fast kinetic model identification based on artificial neural networks

The development of mathematical models to describe reaction kinetics is crucial in process design, control, and optimization. However, distinguishing between different candidate kinetic models presents a non-trivial challenge. Recent works on this topic introduced an approach that employs artificial neural networks (ANNs) to identify kinetic models. In this paper, the ANNs-based model identification approach is expanded by introducing an optimal experimental design procedure. The performance of the method is evaluated through a case study related to the identification of kinetics in a batch reaction system, where different combinations of experimental design variables and noise level on the measurements are compared to assess their impact on kinetic model identification. The proposed experimental design methodology effectively reduces the number of required experiments while enhancing the artificial neural network's ability to accurately identify the appropriate set of equations defining the kinetic model structure.

Using Behavioral Nudges and Incentives to Increase Affordability of Transit Systems through Donations

This study relies on an experimental design framework to identify the behavioral nudges and incentives that can facilitate altruism for a transportation-related cause of the members of a university community. Our findings reveal the overall willingness to support transportation options for low-income community members either through financial contributions or, in the case of some of the students, donating their infrequently used student transit passes. We also found the positive effect of recognition on donating for students and male car commuters, as well as higher chances for a sunk cost of a contribution to increase the likelihood of a donation.

Experimental design using the Taguchi method for the development of conductive textiles used in flexible thermoelectric generators

This paper presents several aspects of the robust experimental design methodology using the Taguchi method to develop electrically conductive textiles. These conductive textiles will be used to make thermoelectric generators based on the Seebeck effect. Since the experimental development involves the use of the magnetron sputtering method with more than three variables, the Taguchi method was selected to observe how different parameters (5 independent variables such as argon flow, power, pressure, deposition surface area and metal type used for solid targets) influence the mean and variance of the process performance defined by the thickness of the metal film deposited (dependent variable). In the experimental design framework, the analysis using the Taguchi method, followed by optimisation, helped select the optimal experiments from the set of possible experiments. This methodology reduced the number of experiments by 21–42% and minimised resource consumption (e.g., metal targets, argon, energy).

Four principles for improved statistical ecology

Abstract Increasing attention has been drawn to the misuse of statistical methods over recent years, with particular concern about the prevalence of practices such as poor experimental design, cherry picking and inadequate reporting. These failures are largely unintentional and no more common in ecology than in other scientific disciplines, with many of them easily remedied given the right guidance. Originating from a discussion at the 2020 International Statistical Ecology Conference, we show how ecologists can build their research following four guiding principles for impactful statistical research practices: (1) define a focussed research question, then plan sampling and analysis to answer it; (2) develop a model that accounts for the distribution and dependence of your data; (3) emphasise effect sizes to replace statistical significance with ecological relevance; and (4) report your methods and findings in sufficient detail so that your research is valid and reproducible. These principles provide a framework for experimental design and reporting that guards against unsound practices. Starting with a well‐defined research question allows researchers to create an efficient study to answer it, and guards against poor research practices that lead to poor estimation of the direction, magnitude, and uncertainty of ecological relationships, and to poor replicability. Correct and appropriate statistical models give sound conclusions. Good reporting practices and a focus on ecological relevance make results impactful and replicable. Illustrated with two examples—an experiment to study the impact of disturbance on upland wetlands, and an observational study on blue tit colouring—this paper explains the rationale for the selection and use of effective statistical practices and provides practical guidance for ecologists seeking to improve their use of statistical methods.

Sequential Designs for Filling Output Spaces

Space-filling designs are commonly used in computer experiments to fill the space of inputs so that the input–output relationship can be accurately estimated. However, in certain applications such as inverse design or feature-based modeling, the aim is to fill the response or feature space. In this article, we propose a new experimental design framework that aims to sequentially fill the space of the outputs (responses or features). Several examples are given to show the advantages of the proposed method over the traditional input space-filling designs.

Systematic review of the performance evaluation of clinicians with or without the aid of machine learning clinical decision support system.

For the adoption of machine learning clinical decision support systems (ML-CDSS) it is critical to understand the performance aid of the ML-CDSS. However, it is not trivial, how the performance aid should be evaluated. To design reliable performance evaluation study, both the knowledge from the practical framework of experimental study design and the understanding of domain specific design factors are required. The aim of this review study was to form a practical framework and identify key design factors for experimental design in evaluating the performance of clinicians with or without the aid of ML-CDSS. The study was based on published ML-CDSS performance evaluation studies. We systematically searched articles published between January 2016 and December 2022. From the articles we collected a set of design factors. Only the articles comparing the performance of clinicians with or without the aid of ML-CDSS using experimental study methods were considered. The identified key design factors for the practical framework of ML-CDSS experimental study design were performance measures, user interface, ground truth data and the selection of samples and participants. In addition, we identified the importance of randomization, crossover design and training and practice rounds. Previous studies had shortcomings in the rationale and documentation of choices regarding the number of participants and the duration of the experiment. The design factors of ML-CDSS experimental study are interdependent and all factors must be considered in individual choices. The online version contains supplementary material available at 10.1007/s12553-023-00763-1.

Blockchain-Based Business Process Management (BPM) for Finance: The Case of Credit and Claim Requests

Because of the competitive economy, organizations today seek to rationalize, innovate, and adapt to changing environments and circumstances as part of business process improvement efforts. The strength of blockchain technology lies in its usage as an apt technology to enhance the efficiency and effectiveness of business processes; furthermore, it prevents the use of erroneous or obsolete data and allows sharing of confidential data securely. The use of superior technology in the execution and automation of business processes brings opportunities to rethink the specific process itself as well. Business processes modeling and verification are essential to control and assure organizational evolution, therefore, the aim of this paper is three-fold: firstly, to provide business process management patterns in finance, based on blockchain, specifically for the loan-application process in the banking industry and claim process in the insurance industry that could be used and customized by companies; secondly, to critically analyze challenges and opportunities from the introduction of such approach for companies, and thirdly, to outline how companies can implement the loan business process as a web service. Partner companies (a bank and an insurance company) formulated the potential requirements for M2P along with the application of blockchain technology. An experimental design framework was established that gave the necessary services to model the requirements, check the models, and operationalize the models. The applied research methodologies are as follows: design science research paradigm and software case study, model-to-programming (M2P) of business processes, and utilization of patterns of workflow and blockchain.

A Fast and Scalable Computational Framework for Large-Scale High-Dimensional Bayesian Optimal Experimental Design

We develop a fast and scalable computational framework to solve Bayesian optimal experimental design problems governed by partial differential equations (PDEs) with application to optimal sensor placement by maximizing expected information gain (EIG). Such problems are particularly challenging due to the curse of dimensionality for high-dimensional parameters and the expensive solution of large-scale PDEs. To address these challenges, we exploit two fundamental properties: (1) the low-rank structure of the Jacobian of the parameter-to-observable map, to extract the intrinsically low-dimensional data-informed subspace, and (2) a series of approximations of the EIG that reduce the number of PDE solves while retaining high correlation with the true EIG. Based on these properties, we propose an efficient offline-online decomposition for the optimization problem. The offline stage dominates the cost and entails precomputing all components that require PDE solves. The online stage optimizes sensor placement and does not require any PDE solves. For the online stage, we propose a new greedy algorithm that first places an initial set of sensors using leverage scores and then swaps the selected sensors with other candidates until certain convergence criteria are met, which we call a swapping greedy algorithm. We demonstrate the efficiency and scalability of the proposed method by both linear and nonlinear inverse problems. In particular, we show that the number of required PDE solves is small, independent of the parameter dimension, and only weakly dependent on the data dimension for both problems.

Rational Design of a Polyurethane Foam.

Polyurethane (PU) foams are exceptionally versatile due to the nature of PU bond formation and the large variety of polymeric backbones and formulation components such as catalysts and surfactants. This versatility introduces a challenge, namely a near unlimited number of variables for formulating foams. In addition to this, PU foam development requires expert knowledge, not only in polyurethane chemistry but also in the art of evaluating the resulting foams. In this work, we demonstrate that a rational experimental design framework in conjunction with a design of experiments (DoE) approach reduces both the number of experiments required to understand the formulation space and reduces the need for tacit knowledge from a PU expert. We focus on an in-depth example where a catalyst and two surfactants of a known formulation are set as factors and foam physical properties are set as responses. An iterative DoE approach is used to generate a set of foams with substantially different cell morphology and hydrodynamic behaviour. We demonstrate that with 23 screening formulations and 16 final formulations, foam physical properties can be modelled from catalyst and surfactant loadings. This approach also allows for the exploration of relationships between the cell morphology of PU foam and its hydrodynamic behaviour.

Comparing Well and Geophysical Data for Temperature Monitoring Within a Bayesian Experimental Design Framework

AbstractTemperature logs are an important tool in the geothermal industry. Temperature measurements from boreholes are used for exploration, system design, and monitoring. The number of observations, however, is not always sufficient to fully determine the temperature field or explore the entire parameter space of interest. Drilling in the best locations is still difficult and expensive. It is therefore critical to optimize the number and location of boreholes. Due to its higher spatial resolution and lower cost, four‐dimensional (4D) temperature field monitoring via time‐lapse Electrical Resistivity Tomography has been investigated as a potential alternative. We use Bayesian Evidential Learning (BEL), a Monte Carlo‐based training approach, to optimize the design of a 4D temperature field monitoring experiment. We demonstrate how BEL can take into account various data source combinations (temperature logs combined with geophysical data) in the Bayesian optimal experimental design (BOED). To determine the optimal data source combination, we use the Root Mean Squared Error of the predicted target in the low dimensional latent space where BEL is solving the prediction problem. The parameter estimates are accurate enough to use in BOED. Furthermore, the method is not limited to monitoring temperature fields and can be applied to other similar experimental design problems. The method is computationally efficient and requires little training data. For the considered optimal design problem, a training set of only 200 samples and a test set of 50 samples is sufficient.

An integrated experimental design framework for optimizing solute transport monitoring locations in heterogeneous sedimentary media

This study develops an integrated framework to guide the monitoring network optimization and duration selection for solute transport in heterogeneous sand tank experiments. The method is designed based on entropy and data worth analysis. Numerical models are applied to approach prior observation datasets and to support optimization analysis. Several candidate monitoring locations are synthetically assumed in numerical models. Entropy analysis considers local scale heterogeneity in experiment and identifies stable monitoring locations through extracting maximum information and minimizing optimization redundancies. Data worth analysis quantifies the potential of observation data to reduce the uncertainty of key parameters and selects the monitoring locations with higher data worth. Final monitoring network comprises of optimized monitoring locations obtained based on entropy and data worth analysis. A lab-scale tracer experiment is presented to explore the applicability of the proposed framework. Results show that the optimized monitoring network can accurately characterize the distribution of contaminant plumes in 3D domains and provides estimation of key flow and transport parameters (e.g., hydraulic conductivity and dispersivity). With the extension of experiment time, the total information of monitoring network is maximized, while the uncertainty of key parameters is minimized. The recommended experimental duration is the time by which both joint entropy and parameter variation coefficients are stabilized. Our developed methodology can be used as a flexible and powerful tool to design more complex transport experiments at different spatiotemporal scales.

CAMERA: A method for cost-aware, adaptive, multifidelity, efficient reliability analysis

Estimating probability of failure in aerospace systems is a critical requirement for flight certification and qualification. Failure probability estimation involves resolving tails of probability distributions, and Monte Carlo sampling methods are intractable when expensive high-fidelity simulations have to be queried. We propose a method to use models of multiple fidelities that trade accuracy for computational efficiency. Specifically, we propose the use of multifidelity Gaussian process models to efficiently fuse models at multiple fidelity, thereby offering a cheap surrogate model that emulates the original model at all fidelities. Furthermore, we propose a novel sequential acquisition function based experiment design framework that can automatically select samples from appropriate fidelity models to make predictions about quantities of interest at the highest fidelity. We use our proposed approach in an importance sampling setting and demonstrate our method on the failure level set and probability estimation on synthetic test functions and two real-world applications, namely, the reliability analysis of a gas turbine engine blade using a finite element method and a transonic aerodynamic wing test case using Reynolds-averaged Navier-Stokes equations. We show that our method predicts the failure boundary and probability more accurately and at a fraction of the computational cost compared with using just a single expensive high-fidelity model. Finally, we show that our sequential approach is guaranteed to asymptotically converge to the true failure boundary with high probability.

OptEx: An integrated framework for experimental design and combustion kinetic model optimization

Computational fluid dynamics (CFD) simulation in the design of combustion devices imposes increased demand on combustion kinetic models with acceptable uncertainties. Model optimization is often utilized to constrain the model parameters with experimental data to reduce the prediction uncertainties. Since it is unaffordable to conduct experiments under all the concerned conditions, experimental design approaches are proposed to find the most valuable experiments to be conducted. An integrated computational framework, OptEx (Optimal Experiments), is proposed to facilitate applications of experimental design, data clustering, and model optimization with optimal experimental data. Specifically, this framework integrates the functions of dimension reduction, global sensitivity analysis, forward uncertainty quantification, model-analysis-based experimental design, and model optimization. The share of data and surrogate models between different modules significantly improves the computational efficiencies of model analysis, experimental design and model optimization. Two case studies of a methanol system are utilized to demonstrate the functionalities of OptEx. First, experimental designs are performed based on sensitivity entropy and surrogate model similarity analysis to find out informative while independent experiments. Second, experimental data clustering with OptEx is demonstrated by grouping massive experiments and selecting optimal experiments for each group. For both cases, a small size of optimal experimental datasets are utilized for model optimization, yielding an optimized model with smaller uncertainty bounds. Meanwhile, the input parameter uncertainties are significantly reduced and parameter correlations are identified via the joint probability distributions of optimized parameters.

A Local Search Framework for Experimental Design

We present a local search framework to design and analyze both combinatorial algorithms and rounding algorithms for experimental design problems. This framework provides a unifying approach to match and improve all known results in D/A/E-design and to obtain new results in previously unknown settings. For combinatorial algorithms, we provide a new analysis of the classical Fedorov's exchange method. We prove that this simple local search algorithm works well as long as there exists an almost optimal solution with good condition number. Moreover, we design a new combinatorial local search algorithm for E-design using the regret minimization framework. For rounding algorithms, we provide a unified randomized exchange algorithm to match and improve previous results for D/A/E-design. Furthermore, the algorithm works in the more general setting to approximately satisfy multiple knapsack constraints, which can be used for weighted experimental design and for incorporating fairness constraints into experimental design.

Experimental design for estimating the power potential of the Wairakei geothermal field, New Zealand: A retrospective analysis

Uncertainty quantification of power (MWe) prediction from a geothermal numerical model is time-consuming. Existing techniques and workflows are still being tested. Our study focuses on using the Experimental Design (ED) and Response Surface Methodology (RSM) framework as an alternative method to facilitate probabilistic MWe estimation. This study involves two types of Experimental Design: the two-level Full Factorial and Plackett Burman, and the investigation is divided into two parts. The first part investigated the difference in the prediction of the Wairakei geothermal field power potential using the polynomial model built from the 64 simulation runs based on the two-level Full Factorial design and the 12 simulation runs based on the Plackett-Burman configuration. The second section compared the MWe predictions using the Plackett-Burman design and the volumetric stored heat method. The simulated MWe potential of Wairakei for 50 years is comparable for both the two-level Full Factorial and Plackett-Burman designs, indicating the latter as a practical design for building a proxy model of a numerical model. Furthermore, the MWe prediction of the polynomial model created based on the ED Framework closely resembles the actual production capacity of Wairakei compared to the volumetric stored heat method. The results based on the proxy modelling further indicate that the 353 MWe production generation could be sustainable at least for another 40 years.

CAMDLES: CFD-DEM Simulation of Microbial Communities in Spaceflight and Artificial Microgravity.

We present CAMDLES (CFD-DEM Artificial Microgravity Developments for Living Ecosystem Simulation), an extension of CFDEM®Coupling to model biological flows, growth, and mass transfer in artificial microgravity devices. For microbes that accompany humans into space, microgravity-induced alterations in the fluid environment are likely to be a major factor in the microbial experience of spaceflight. Computational modeling is needed to investigate how well ground-based microgravity simulation methods replicate that experience. CAMDLES incorporates agent-based modeling to study inter-species metabolite transport within microbial communities in rotating wall vessel bioreactors (RWVs). Preexisting CFD modeling of RWVs has not yet incorporated growth; CAMDLES employs the simultaneous modeling of biological, chemical, and mechanical processes in a micro-scale rotating reference frame environment. Simulation mass transfer calculations were correlated with Monod dynamic parameters to predict relative growth rates between artificial microgravity, spaceflight microgravity, and 1 g conditions. By simulating a microbial model community of metabolically cooperative strains of Escherichia coli and Salmonella enterica, we found that the greatest difference between microgravity and an RWV or 1 g gravity was when species colocalized in dense aggregates. We also investigated the influence of other features of the system on growth, such as spatial distribution, product yields, and diffusivity. Our simulation provides a basis for future laboratory experiments using this community for investigation in artificial microgravity and spaceflight microgravity. More broadly, our development of these models creates a framework for novel hypothesis generation and design of biological experiments with RWVs, coupling the effects of RWV size, rotation rate, and mass transport directly to bacterial growth in microbial communities.

Probabilistic geothermal resource assessment using experimental design and response surface methodology: The Leyte geothermal production field

Estimating power potential (MWe) with higher accuracy remains a necessity and a challenge confronting the geothermal industry. Reservoir models can predict MWe capacity probabilistically for resource management and decision-making use, but the process remains laborious to implement. This work involves applying the Experimental Design (ED) and Response Surface Methodology (RSM) framework to a calibrated natural-state reservoir model for predicting MWe probabilistically, quantifying uncertainties in the prediction and identifying the key driving factors.This study has chosen six uncertain parameters and implemented the Plackett-Burman design to build twelve versions of the Leyte reservoir model for uncertainty quantification. These uncertain parameters are permeability in the x, y and z-direction, porosity, reinjection enthalpy (RI Enthalpy), and the fraction of reinjection (% RI). Results revealed that all of these parameters except RI Enthalpy are significant predictors of MWe and that the Leyte Geothermal Production Field (LGPF) has an indicative capacity of 670 MWe (P90), 730 MWe (P50) and 790 MWe (P10) if it operates for 50 years.

Adaptive wideband sensing scheme based on Bayesian experimental design

In this paper, we investigate the problem of wideband sensing for cognitive radio. Due to resource constraint, a cognitive node targets at finding a given number of spectrum holes, which is studied under a novel wideband sensing architecture. Specifically, the cognitive node can dynamically focus its sensing measurements on different portions of contiguous channels. An adaptive wideband sensing scheme is proposed under the standard Bayesian experimental design framework via three steps: (1) define a Bayesian cost that incorporates the numbers of desired spectrum holes, (2) design hypothesis testing rule to detect access opportunities, and (3) identify a scheme to adapt the measurements during the sensing process. In order to facilitate implementation, we further propose an approximated Bayesian experiment design scheme to reduce the computational complexity. The performance gain of our wideband sensing scheme is demonstrated via numerical simulation.

An empirical analysis of surface-level methane emission from anthropogenic sources in India

This study presents an empirical analysis of surface-level methane emission from anthropogenic sources in two major Indian cities. Three categories of sources were considered- (1) fossil fuel exploration and energy production (2) agriculture and livestock farming and (3) waste processing and disposal. In each city, specific sources were identified and mapped to the nearest methane monitoring station. As these sources were active for varying periods, treatment and control periods were designed by partitioning the observation period based on the activity of sources. This experimental design framework was used to formulate and test hypotheses on contribution of these sources to methane emission. Given the limitations and statistical properties (non-normal residuals and heteroskedasticity) of the data a robust alternative to classical analysis of variance (ANOVA) was used to test three hypotheses. Tests confirmed that these sources contributed significantly to methane emission. Specifically, emission from MSW disposal facilities was substantial as evident from the data in Delhi. In Bengaluru, aggregate emission from following sources was found significant- (1) a diesel-based power plant and a landfill (2) a sewage treatment plant and a landfill. This empirical work compliments previous studies on greenhouse gas emissions, which were based on climate models and satellite remote sensing data. This analysis of natural experiment ends with recommendations to manage methane emission from anthropogenic sources.

Framework of Experimental Design and Data Mining in Multi-agent Simulation

Multi-agent based simulation (MABS) is an important approach for studying complex systems. The Agent-based model often contains many parameters, these parameters are usually not independent, with differences in their range, and may be subjected to constraints. How to use MABS investigating complex systems effectively is still a challenge. The common tasks of MABS include: summarizing the macroscopic patterns of the system, identifying key factors, establishing a meta-model, and optimization. We proposed a framework of experimental design and data mining for MABS. In the framework, method of experimental design is used to generate experiment points in the parameter space, then generate simulation data, and finally using data mining techniques to analyze data. With this framework, we could explore and analyze complex system iteratively. Using central composite discrepancy (CCD) as measure of uniformity, we designed an algorithm of experimental design in which parameters could meet any constraints. We discussed the relationship between tasks of complex system simulation and data mining, such as using cluster analysis to classify the macro patterns of the system, and using CART, PCA, ICA and other dimensionality reduction methods to identify key factors, using linear regression, stepwise regression, SVM, neural network, etc. to build the meta-model of the system. This framework integrates MABS with experimental design and data mining to provide a reference for complex system exploration and analysis.