Optimal Experimental Design Research Articles

Stimulus Selection in a Q-learning Model Using Fisher Information and Monte Carlo Simulation

Reinforcement learning models have been extensively studied for decision-making tasks with reward feedback. However, in designing an experiment to collect data for Q-learning models, the quantitative effect of a presented stimulus on the estimation precision of participant parameters has generally not been considered. That is, the lack of a mathematical framework has prevented researchers from designing an optimal experiment. To tackle this problem, this study analytically derives the Fisher information. Furthermore, this study formulates a stochastic representation of the Q-learning model, which is one of the most commonly applied reinforcement learning models. With this derivation, a two-step procedure is proposed to select the optimal stimuli in terms of estimation precision, in which low-cost Fisher information evaluation and more detailed finite-sample Monte Carlo simulation are combined. The simulation studies show that reward probability reversal leads to a high estimation precision for the learning rate parameter. By contrast, for the inverse temperature parameter, a larger difference in reward probability between options leads to higher estimation precision. These results reveal that the optimal experimental design is dependent on which trait parameters of the Q-learning model are of interest to researchers. Further, it is found that the use of undesirable stimuli in terms of trait parameter precision leads to a large bias in the correlation coefficient estimate. Based on the results, the approaches to designing experiments in the Q-learning model are discussed.

An Offline-Online Decomposition Method for Efficient Linear Bayesian Goal-Oriented Optimal Experimental Design: Application to Optimal Sensor Placement

Bayesian optimal experimental design (OED) plays an important role in minimizing model uncertainty with limited experimental data in a Bayesian framework. In many applications, rather than minimizing the uncertainty in the inference of model parameters, one seeks to minimize the uncertainty of a model-dependent quantity of interest (QoI). This is known as goal-oriented OED (GOOED). Here, we consider GOOED for linear Bayesian inverse problems governed by large-scale models represented by partial differential equations (PDE) that are computationally expensive to solve. In particular, we consider optimal sensor placement by maximizing an expected information gain (EIG) for the QoI. We develop an efficient method to solve such problems by deriving a new formulation of the goal-oriented EIG. Based on this formulation we propose an offline-online decomposition scheme that achieves significant computational reduction by computing all of the PDE-dependent quantities in an offline stage just once, and optimizing the sensor locations in an online stage without solving any PDEs. Moreover, in the offline stage we need only to compute low-rank approximations of two Hessian-related operators. The computational cost of these low-rank approximations, measured by the number of PDE solves, does not depend on the parameter or data dimensions for a large class of elliptic, parabolic, and sufficiently dissipative hyperbolic inverse problem that exhibit dimension-independent rapid spectra decay. We carry out detailed error analysis for the approximate goal-oriented EIG due to the low-rank approximations of the two operators. Furthermore, in the online stage we extend a swapping greedy method to optimize the sensor locations developed in our recent work that is demonstrated to be more efficient than a standard greedy method. We conduct a numerical experiment for a contaminant transport inverse problem with an infinite-dimensional parameter field to demonstrate the efficiency, accuracy, and both data- and parameter-dimension independence of the proposed algorithm.

The ENBIS‐21 quality and reliability engineering international special issue

The ENBIS-21 Quality and Reliability Engineering International Special Issue is related to the contributions to the 21st Annual Conference of the European Network for Business and Industrial Statistics (ENBIS) held online from 13 to 15 September, 2021. The topics covered by the 12 papers included in this special issue fall within the scope of ENBIS, whose ultimate goal is to bridge the gap between methodology and application in the field of business and industrial statistics and foster networking between statisticians in academic and commercial settings. It becomes evident from the papers on this issue that classical methods of industrial statistics are still having a wide range of applications and expansions towards complex and big data structures as well as they are increasingly cross-fertilized by artificial intelligence and machine learning perspectives and approaches. The issue is opened by the contribution of Yang et al.1 in the Statistical Process Monitoring (SPM) of data in the form of tensors. The authors develop a hierarchical in-situ process monitoring approach for anomaly detection, which is applied to thermal video streams acquired during a metal additive manufacturing process for highly complex parts. A more traditional SPM approach based on generalized likelihood ratio (GLR) control charts is elaborated by Rizzo and Di Bucchianico2 to set up tailor-made monitoring for high-purity (high-quality) processes and deals with issues arising in GLR control charts for discrete processes. Along the line of developing statistical methods for complex data structures, Gril et al.3 set up a tensor-on-tensor regression model able to dynamically predict repetitive human motions helpful in avoiding unwanted collisions at the interface with robots in industrial assembly tasks. Chuquin et al.4 present a straightforward extension of the K-means clustering to spatially correlated functional data, with an application in the analysis of the normalized difference vegetation index in a large region of Ecuador. From a more general perspective, Vanacore et al.5 address the problem of assessing prediction performance in the multi-class classification of imbalanced data sets. In the optimal Design of Experiments (DoE) Pesce et al.6 offer new results that are oriented to bias reduction in large datasets and protection against confounders. Whereas, Kirchhoff et al.7 use DoE as the initial solution for the optimization of shift planning in high-bay warehouse operations based on Gaussian process models with low-rank correlation kernels. Risk assessment and management is the topic of the following three papers. Testik and Unlu8 compare traditional and fuzzy failure modes and effect analysis to rank risks in different areas of testing and calibration laboratories. Meynaoui et al.9 deal with an accidental scenario in a sodium-cooled fast nuclear reactor by means of a global sensitivity analysis of the input parameter distribution used in numerical simulator outputs to model physical phenomena. Lai and Zwetsloot10 offer a data-driven ensemble ranking system of product quality, which is validated in the identification of high-risk manufacturers in a solar industry case study. The final two papers are devoted to the Reliability and Maintenance of repairable systems. El-Aroui and Gaudoin11 propose a general goodness-of-fit test for imperfect repair models that has the favorable property of being asymptotically distribution-free for renewal and power law processes. Fouladirad et al.12 develop a new bounded transformed gamma process model to describe and predict degradation phenomena by relaxing the typical assumption that the degradation level can grow indefinitely, which is often unrealistic for technological units. The proposed model is also applied to a set of wear measures of cylinder liners that equip a diesel engine for marine propulsion. We are deeply grateful to the journal staff for their professional assistance in the editorial process, as well as to the anonymous referees for the time and effort spent to provide careful and constructive comments, and to all the authors, who made revisions on a very tight timescale.

Optimal designs for detecting and characterizing hormesis in toxicological tests

Toxicological tests are experiments that show the effects of a toxic on organisms, ecosystems, etc. This study focuses on tests in the aquatic environment, in which the test involving Ceriodaphnia Dubia organism stands out. The literature indicates that in two out of every three experiments carried out with this organism, there is hormesis. This study applies optimal experimental design theory to a linear quadratic model with a Poisson distribution for the response, in order to obtain designs that allow efficient detection and characterization of hormesis. To this end, a variety of utility functions are used, including the dose for the zero equivalent point, the area under the curve, the dose at which maximum response is reached or the dose at which there is a given relative inhibition with respect to the control or the maximum. A study of cross efficiencies of the calculated designs shows the importance of correctly defining the goal of the experiment, in order to obtain the most appropriate design.

Comparison of Full Factorial and Optimal Experimental Design for Perceptual Evaluation of Audiovisual Quality

Comparison of Full Factorial and Optimal Experimental Design for Perceptual Evaluation of Audiovisual Quality

Application of Response Surface Methodology (RSM) for Optimizing Turbidity of Paper Recycling Wastewater Using Microwave Technology

Aims: The aim of the study was to use Response Surface Methodology (RSM) to find optimal experimental design for wastewater treatments from office paper recycling. In this way, interactive effects of treatment factors were evaluated, including microwave power (MW) and durations with centrifuge time while turbidity of wastewater was chosen as the dependent output variable or an optimal response.
 Methodology: The RSM approach was utilized for optimization of the process parameters and identifying the optimal conditions for the removal of turbidity in paper recycling wastewater. In this regard, a three-factor RSM were selected, using MW irradiation power (Watts), durations (seconds) and centrifuge time (min). Statistical analysis of variance (ANOVA) was carried out to identify the adequacy of the developed model. In this case, specially prepared office papers (one-side laser printed) were subjected to standard paper recycling procedure for obtaining wastewater at laboratory conditions. The experimentally derived RSM model was validated using range of statistical parameters.
 Conclusions: The study revealed that under the RSM optimized conditions, a marked reduction in the turbidity of wastewater was observed for both groups studied. The R2, R2(adj) and R2(pred) values were indicates, the developed model was significant which revealed a well agreement between the experimental data and proposed model. In this approach, the R2=99.710% and lack-of-fit value were found to be 0.111 (p>0.05), which shows the model and the data consisted to each other. The lowest turbidity value was found with 150.000 Watts and 60.000 seconds in MW conditions with 15.000 minutes of centrifugation time, experimentally. With employing these variables, the turbidity value of 6.65 NTU was determined. However, the highest turbidity value of 18.013 NTU was found with MW power of 200.00 Watts with 40.000 seconds of durations and 1.591 minute of centrifugation time. It is important to note that with using optimized parameters, the turbidity value of 1.43 NTU was calculated while 1.47 NTU was found with experimentally.

An integrated approach to fast model-based process design: Integrating superstructure optimization under uncertainties and optimal design of experiments

During the early-stage design of chemical production processes many decisions have to be made on the basis of incomplete knowledge about the underlying chemical and physical phenomena. Therefore, optimization-based approaches are often applied only in a later stage when more knowledge has been generated. In this work, an integrated approach to fast, model and optimization based process design for the selection of reactors and separation networks is discussed. The approach is based on superstructure optimization under uncertainties about the parameters of the models using a two-stage scenario-based approach. Usually, due to the uncertainties, structural decisions cannot be made in the initial stage as different structures are superior for different scenarios of the uncertain parameters. In order to arrive at such decisions, the models need to be refined based on experimental studies. We combine design of experiments with optimization under uncertainty to optimize the experiments such that the information obtained about those parameters that are critical with respect to taking structural design decisions is maximized. We apply the methodology to the development of a process for the hydroaminomethylation of 1-decene in a thermomorphic solvent system. It is shown that the integrated approach can help to significantly reduce the number of required experiments. Using the integrated approach, only 7 experiments need to be performed while more than 16 experiments following a full-factorial experimental design do not provide the same reduction of the design space.

Pursuit of the Ultimate Regression Model for Samarium(III), Europium(III), and LiCl Using Laser-Induced Fluorescence, Design of Experiments, and a Genetic Algorithm for Feature Selection.

Laser-induced fluorescence spectroscopy, Raman scattering, and partial least squares regression models were optimized for the quantification of samarium (0-150 μg mL-1), europium (0-75 μg mL-1), and lithium chloride (0.1-12 M) with a transformational preprocessing strategy. Selecting combinations of preprocessing methods to optimize the prediction performance of regression models is frequently a major bottleneck for chemometric analysis. Here, we propose an optimization tool using an innovative combination of optimal experimental designs for selecting preprocessing transformation and a genetic algorithm (GA) for feature selection. A D-optimal design containing 26 samples (i.e., combinations of preprocessing strategies) and a user-defined design (576 samples) did not statistically lower the root mean square error of the prediction (RMSEP). The greatest improvement in prediction performance was achieved when a GA was used for feature selection. This feature selection greatly lowered RMSEP statistics by an average of 53%, resulting in the top models with percent RMSEP values of 0.91, 3.5, and 2.1% for Sm(III), Eu(III), and LiCl, respectively. These results indicate that preprocessing corrections (e.g., scatter, scaling, noise, and baseline) alone cannot realize the optimal regression model; feature selection is a more crucial aspect to consider. This unique approach provides a powerful tool for approaching the true optimum prediction performance and can be applied to numerous fields of spectroscopy and chemometrics to rapidly construct models.

Autonomous kinetic model identification using optimal experimental design and retrospective data analysis: methane complete oxidation as a case study

An uncertainty-aware autonomous flow reactor platform was developed by combining automation and feedback optimization. The platform was applied to identify appropriate kinetic models online for a gas–solid catalytic reaction.

Optimising chemical concentration and injection volume in the ASP process: a combined approach incorporating experiments and the optimal experimental design

Optimising chemical concentration and injection volume in the ASP process: a combined approach incorporating experiments and the optimal experimental design

Physics-Informed Optimal Experiment Design of Calendar Aging Tests and Sensitivity Analysis for SEI Parameters Estimation in Lithium-ion Batteries

This paper focuses on sensitivity analysis and optimal experiment design for calendar aging tests to find SEI (solid-electrolyte interface) degradation parameters. This parameterization facilitates predicting the Li-ion battery end-of-life. To this end, a physics-based model of SEI layer is used to develop a closed form solution for loss of Lithium inventory during calendar aging. This model is shown to match experimental data. Further, the developed model is used to find the sensitivity of LLI (loss of Lithium inventory) to SEI parameters in diffusion-limited and kinetically limited SEI patterns.Through this sensitivity analysis, we show that in the kinetically limited case, performing calendar aging tests at high SOCs gives the most accurate results. For the diffusion limited case, we use optimal experiment design techniques to find the optimal calendar aging SOCs. It is also shown that, these optimal SOC values result in higher accuracy estimations compared to sub-optimal SOCs.

Optimising chemical concentration and injection volume in the ASP process: a combined approach incorporating experiments and the optimal experimental design

Optimising chemical concentration and injection volume in the ASP process: a combined approach incorporating experiments and the optimal experimental design

Experimental Design for Overparameterized Learning with Application to Single Shot Deep Active Learning.

The impressive performance exhibited by modern machine learning models hinges on the ability to train such models on a very large amounts of labeled data. However, since access to large volumes of labeled data is often limited or expensive, it is desirable to alleviate this bottleneck by carefully curating the training set. Optimal experimental design is a well-established paradigm for selecting data point to be labeled so to maximally inform the learning process. Unfortunately, classical theory on optimal experimental design focuses on selecting examples in order to learn underparameterized (and thus, non-interpolative) models, while modern machine learning models such as deep neural networks are overparameterized, and oftentimes are trained to be interpolative. As such, classical experimental design methods are not applicable in many modern learning setups. Indeed, the predictive performance of underparameterized models tends to be variance dominated, so classical experimental design focuses on variance reduction, while the predictive performance of overparameterized models can also be, as is shown in this paper, bias dominated or of mixed nature. In this paper we propose a design strategy that is well suited for overparameterized regression and interpolation, and we demonstrate the applicability of our method in the context of deep learning by proposing a new algorithm for single shot deep active learning.

MAXIMIZATION OF COMPRESSIVE STRENGTH IN TIG WELDING OF MILD STEEL USING RESPONSE SURFACE METHODOLOGY

Welding defects influence the desired properties of welded joints giving fabrication experts a common problem of not being able to produce weld structures with optimal strength and quality. In this study, fatigue was minimized using artificial intelligence such as the Response Surface Methodology An optimal design of experiment was developed which was used as a guiding plan to conduct the experiment., thereafter a second order polynomial l model was developed which was used to minimize the fatigue with very significant statistical results. The result shows that the quadratic model was the most suitable for minimizing the fatigue response with a P-value < 0.05

From Optimal Design of Experiment to Safe System Identification in Type 2 Diabetes

Model-based design of experiment (MBDoE) provides a framework to collect informative data for system identification. However, a parametric and structural mismatch between the design model and the underlying physical system can lead to hazardous experiments in safety critical systems. In this work, we present a method to safely improve system identification based on insights from a model-based optimal experimental design. From a visual inspection of a MBDoE, we select an approximated output curve fulfilling system constraints as a reference for the physical system. To avoid open-loop implementation of the MBDoE, we use our approximated reference together with a reference-tracking controller to collect experimental data in closed-loop. In this type 2 diabetes (T2D) case study, the proposed design method is safe and provides informative experimental data for system identification.

A Dual Control Approach to Solve Exploration Versus Exploitation Trade-Offs in the Design of Personalized Physical Exercise Sessions

Using biofeedback in medical therapies has proven to be effective for adapting patient behaviors while keeping the patients engaged and motivated in an exercise session. This paper considers general problems in personalized exercise sessions where the input is opportune biofeedback and the session goal is to maximize a particular exercise effect. Due to the individual differences between patients and their physiological signals, however, personalized patient models also need to be identified. With the two objectives: 1) maximize a training effect with minimal control effort, and 2) identify the individualized patient model, we have a typical exploration vs. exploration trade-off. Control problems of this form are called dual control problems. In this paper, we formulate a dual control problem for a personalized exercise session and test the approach against classical optimal control and optimal experimental design approaches in an illustrative example of performing Kegel exercises where the control and identification goals conflict with each other.

Optimal Experimental Designs for Hypothesis Testing with Multiple Factors: Maximizing Power for the Biological Sciences

Optimal Experimental Designs for Hypothesis Testing with Multiple Factors: Maximizing Power for the Biological Sciences

Testing three coping strategies for time pressure in categorizations and similarity judgments

This article compares three psychological mechanisms to make multi-attribute inferences under time pressure in the domains of categorization and similarity judgments. Specifically, we test if people under time pressure attend to fewer object features (attention focus), if they respond less precisely (lower choice sensitivity), or if they simplify a psychological similarity function (simplified similarity). The simpler psychological similarity considers the number of matching features but ignores the actual feature value differences. We conducted three experiments (two of them preregistered) in which we manipulated time pressure: one was a categorization task, which was designed based on optimal experimental design principles, and the other two involved a similarity judgment task. Computational cognitive modeling following an exemplar-similarity framework showed that the behavior of most participants under time pressure is in line with a lower choice sensitivity, this means less precise response selection, especially when people make similarity judgments. We find that the variability of participants’ behavior increases with time pressure, to a point where participants are unlikely to make inferences anymore but instead start choosing readily available response options repeatedly. These findings are consistent with related research in other cognitive domains, such as risky choices, and add to growing evidence that time pressure and other forms of cognitive load do not necessarily alter core cognitive processes themselves but rather affect the precision of response selection.

Optimizing multiplexed imaging experimental design through tissue spatial segregation estimation

Recent advances in multiplexed imaging methods allow simultaneous detection of dozens of proteins and hundreds of RNAs, enabling deep spatial characterization of both healthy and diseased tissues. Parameters for the design of optimal multiplex imaging studies, especially those estimating how much area has to be imaged to capture all cell phenotype clusters, are lacking. Here, using a spatial transcriptomic atlas of healthy and tumor human tissues, we developed a statistical framework that determines the number and area of fields of view necessary to accurately identify all cell phenotypes that are part of a tissue. Using this strategy on imaging mass cytometry data, we identified a measurement of tissue spatial segregation that enables optimal experimental design. This strategy will enable an improved design of multiplexed imaging studies.

Optimal experimental design and estimation for q-space trajectory imaging.

Tensor-valued diffusion encoding facilitates data analysis by q-space trajectory imaging. By modeling the diffusion signal of heterogeneous tissues with a diffusion tensor distribution (DTD) and modulating the encoding tensor shape, this novel approach allows disentangling variations in diffusivity from microscopic anisotropy, orientation dispersion, and mixtures of multiple isotropic diffusivities. To facilitate the estimation of the DTD parameters, a parsimonious acquisition scheme coupled with an accurate and precise estimation of the DTD is needed. In this work, we create two precision-optimized acquisition schemes: one that maximizes the precision of the raw DTD parameters, and another that maximizes the precision of the scalar measures derived from the DTD. The improved precision of these schemes compared to a naïve sampling scheme is demonstrated in both simulations and real data. Furthermore, we show that the weighted linear least squares (WLLS) estimator that uses the squared reciprocal of the noisy signal as weights can be biased, whereas the iteratively WLLS estimator with the squared reciprocal of the predicted signal as weights outperforms the conventional unweighted linear LS and nonlinear LS estimators in terms of accuracy and precision. Finally, we show that the use of appropriate constraints can considerably increase the precision of the estimator with only a limited decrease in accuracy.