Optimal Designs For Nonlinear Models Research Articles

Exact Optimal Designs of Experiments for Factorial Models via Mixed-Integer Semidefinite Programming

The systematic design of exact optimal designs of experiments is typically challenging, as it results in nonconvex optimization problems. The literature on the computation of model-based exact optimal designs of experiments via mathematical programming, when the covariates are categorical variables, is still scarce. We propose mixed-integer semidefinite programming formulations, to find exact D-, A- and I-optimal designs for linear models, and locally optimal designs for nonlinear models when the design domain is a finite set of points. The strategy requires: (i) the generation of a set of candidate treatments; (ii) the formulation of the optimal design problem as a mixed-integer semidefinite program; and (iii) its solution, employing appropriate solvers. For comparison, we use semidefinite programming-based formulations to find equivalent approximate optimal designs. We demonstrate the application of the algorithm with various models, considering both unconstrained and constrained setups. Equivalent approximate optimal designs are used for comparison.

Optimal design under complete class with ancillary functions

AbstractNonlinear models are challenging in optimal designs due to their complexity and lack of canonical forms. The complete class strategy provides a unified framework for studying optimal designs for nonlinear models. However, the current strategy does not apply to many models under this framework. In this article, we propose a tool called ancillary functions as an extension to the complete class strategy. We also provide results on minimally supported designs with proper conditions. We demonstrate this tool with two‐parameter dose–response models, which include the Beta‐Poisson model, the complementary log–log model, and the skewed logit model. The results of this article add to the previous complete class framework and make the minimally supported design available for more nonlinear models that were previously not feasible.

Competitive Swarm Optimizer with Mutated Agents for Finding Optimal Designs for Nonlinear Regression Models with Multiple Interacting Factors.

This paper proposes a novel enhancement for Competitive Swarm Optimizer (CSO) by mutating loser particles (agents) from the swarm to increase the swarm diversity and improve space exploration capability, namely Competitive Swarm Optimizer with Mutated Agents (CSO-MA). The selection mechanism is carried out so that it does not retard the search if agents are exploring in promising areas. Simulation results show that CSO-MA has a better exploration-exploitation balance than CSO and generally outperforms CSO, which is one of the state-of-the-art metaheuristic algorithms for optimization. We show additionally that it also generally outperforms swarm based types of algorithms and an exemplary and popular non-swarm based algorithm called Cuckoo search, without requiring a lot more CPU time. We apply CSO-MA to find a c-optimal approximate design for a high-dimensional optimal design problem when other swarm algorithms were not able to. As applications, we use the CSO-MA to search various optimal designs for a series of high-dimensional statistical models. The proposed CSO-MA algorithm is a general-purpose optimizing tool and can be directly amended to find other types of optimal designs for nonlinear models, including optimal exact designs under a convex or non-convex criterion.

A Metaheuristic Adaptive Cubature Based Algorithm to Find Bayesian Optimal Designs for Nonlinear Models

Finding Bayesian optimal designs for nonlinear models is a difficult task because the optimality criterion typically requires us to evaluate complex integrals before we perform a constrained optimization. We propose a hybridized method where we combine an adaptive multidimensional integration algorithm and a metaheuristic algorithm called imperialist competitive algorithm to find Bayesian optimal designs. We apply our numerical method to a few challenging design problems to demonstrate its efficiency. They include finding D-optimal designs for an item response model commonly used in education, Bayesian optimal designs for survival models, and Bayesian optimal designs for a four-parameter sigmoid Emax dose response model. Supplementary materials for this article are available online and they contain an R package for implementing the proposed algorithm and codes for reproducing all the results in this article.

Optimal Designs for Nonlinear Models with Random Block Effects

Optimal Designs for Nonlinear Models with Random Block Effects

An algorithm based on semidefinite programming for finding minimax optimal designs

An algorithm based on a delayed constraint generation method for solving semi-infinite programs for constructing minimax optimal designs for nonlinear models is proposed. The outer optimization level of the minimax optimization problem is solved using a semidefinite programming based approach that requires the design space be discretized. A nonlinear programming solver is then used to solve the inner program to determine the combination of the parameters that yields the worst-case value of the design criterion. The proposed algorithm is applied to find minimax optimal designs for the logistic model, the flexible 4-parameter Hill homoscedastic model and the general nth order consecutive reaction model, and shows that it (i) produces designs that compare well with minimax D−optimal designs obtained from semi-infinite programming method in the literature; (ii) can be applied to semidefinite representable optimality criteria, that include the common A−,E−,G−,I− and D-optimality criteria; (iii) can tackle design problems with arbitrary linear constraints on the weights; and (iv) is fast and relatively easy to use.

Assessing robustness of designs for random effects parameters for nonlinear mixed-effects models

Optimal designs for nonlinear models are dependent on the choice of parameter values. Various methods have been proposed to provide designs that are robust to uncertainty in the prior choice of parameter values. These methods are generally based on estimating the expectation of the determinant (or a transformation of the determinant) of the information matrix over the prior distribution of the parameter values. For high dimensional models this can be computationally challenging. For nonlinear mixed-effects models the question arises as to the importance of accounting for uncertainty in the prior value of the variances of the random effects parameters. In this work we explore the influence of the variance of the random effects parameters on the optimal design. We find that the method for approximating the expectation and variance of the likelihood is of potential importance for considering the influence of random effects. The most common approximation to the likelihood, based on a first-order Taylor series approximation, yields designs that are relatively insensitive to the prior value of the variance of the random effects parameters and under these conditions it appears to be sufficient to consider uncertainty on the fixed-effects parameters only.

Standardized maximim D-optimal designs for enzyme kinetic inhibition models

Locally optimal designs for nonlinear models require a single set of nominal values for the unknown parameters. An alternative is the maximin approach that allows the user to specify a range of values for each parameter of interest. However, the maximin approach is difficult because we first have to determine the locally optimal design for each set of nominal values before maximin types of optimal designs can be found via a nested optimization process. We show that particle swarm optimization (PSO) techniques can solve such complex optimization problems effectively. We demonstrate numerical results from PSO can help find, for the first time, formulae for standardized maximin D-optimal designs for nonlinear model with 3 or 4 parameters on the compact and nonnegative design space. Additionally, we show locally and standardized maximin D-optimal designs for inhibition models are not necessarily supported at a minimum number of points. To facilitate use of such designs, we create a web-based tool for practitioners to find tailor-made locally and standardized maximin optimal designs.

Special issue in honor of Kathryn Chaloner

We are honored to be guest editors for this special issue of Applied Stochastic Models in Business and Industry, dedicated to Kathryn Chaloner's life and achievements. Contributions from her colleagues and friends in this volume are devoted to Kathryn's early research interest in Bayesian optimal design. She was a pioneer in this field and had a broad impact on the subsequent literature on experimental design. Kathryn was internationally known for her research in Bayesian Statistics, which included design of experiments, outlier detection, prior elicitation, and clinical trials, and for her advancements in the study of HIV/AIDS. She was an accomplished researcher, teacher, and mentor. She actively encouraged women and under-represented minority students to join the field of biostatistics and provided mentorship, support, and encouragement. The special issue contains seven papers. The first three papers present some of the recent work in Bayesian optimal designs for nonlinear models, binary responses, and experiments with adversarial components. This is followed by two articles on Bayesian designs in clinical trials. The last two articles of the special issue involve engineering design problems. The paper by Giovagnoli considers the binary response models of Chaloner and Larntz (1989, Journal of Statistical Planning and Inference, 21:191–208) and develops an adaptive version of A-optimal Bayesian designs that are revised as additional data become available during the experiment. Adaptive Bayesian compound designs is also discussed as another extension. Konstantinou and Dette develop approximate Bayesian D-optimal designs for nonlinear regression models that involve covariates that cannot be observed directly. Applications to specific nonlinear models including the exponential regression model are considered and characterizations of the D-optimal saturated designs are presented. Optimal sample size selection for experiments is considered by De Santis and Gubbiotti in an adversarial setting where the decision makers have different prior opinions while having common utility functions and data. Using a Bayesian decision approach with a quadratic loss function, the authors present results for the one-parameter exponential family and investigate effect of the priors on the optimal sample size. Müller, Xu, and Thall discuss advantages and limitations of Bayesian decision theoretic approach in design of clinical trials. In doing so, the authors present a specific case study and point out the limitations such as the computational difficulties involved in solving sequential problems as well as selection of utility functions capturing different stakeholder interests. The article by Ventz, Parmigiani, and Trippa also consider limitations of the Bayesian framework in design of clinical trials and propose a strategy to alleviate these. The proposed strategy involves combining Bayesian designs with frequentist metrics such as error rates, confidence intervals, etc. that are commonly used by medical community and regulatory agencies. The authors show that this can be achieved by an inclusion of frequentist constraints into the Bayesian formulation. The constrained decision theoretic framework is illustrated via several applications and computational algorithms are presented for implementing the proposed approach. Nakamura, Seepaul, Kadane, and Reeja-Jayan discuss design of experiments in materials science. The authors consider use of a batch-sequential experiment and a regression model to determine optimal levels of input variables for synthesis of ceramic materials. It is illustrated that the optimal settings produce more reliable results than other experiments. The final article by Polson and Soyer considers Bayesian design of accelerated life tests for reliability assessment. An augmented probability simulation (APS) approach using Lindley's (1976, Annals of Statistics, 4:1–10) conjugate utility functions is proposed to compute optimal designs in an efficient manner. The proposed approach is illustrated using single and multiple point designs. Use of particle based methods for APS is also discussed as an alternative to Markov chain Monte Carlo methods.

Adaptive grid semidefinite programming for finding optimal designs

We find optimal designs for linear models using a novel algorithm that iteratively combines a semidefinite programming (SDP) approach with adaptive grid techniques. The proposed algorithm is also adapted to find locally optimal designs for nonlinear models. The search space is first discretized, and SDP is applied to find the optimal design based on the initial grid. The points in the next grid set are points that maximize the dispersion function of the SDP-generated optimal design using nonlinear programming. The procedure is repeated until a user-specified stopping rule is reached. The proposed algorithm is broadly applicable, and we demonstrate its flexibility using (i) models with one or more variables and (ii) differentiable design criteria, such as A-, D-optimality, and non-differentiable criterion like E-optimality, including the mathematically more challenging case when the minimum eigenvalue of the information matrix of the optimal design has geometric multiplicity larger than 1. Our algorithm is computationally efficient because it is based on mathematical programming tools and so optimality is assured at each stage; it also exploits the convexity of the problems whenever possible. Using several linear and nonlinear models with one or more factors, we show the proposed algorithm can efficiently find optimal designs.

Application of imperialist competitive algorithm to find minimax and standardized maximin optimal designs

Finding optimal designs for nonlinear models is complicated because the design criterion depends on the model parameters. If a plausible region for these parameters is available, a minimax optimal design may be used to remove this dependency by minimizing the maximum inefficiency that may arise due to misspecification in the parameters. Minimax optimal designs are often analytically intractable and are notoriously difficult to find, even numerically. A population-based evolutionary algorithm called imperialist competitive algorithm (ICA) is applied to find minimax or nearly minimax D-optimal designs for nonlinear models. The usefulness of the algorithm is also demonstrated by showing it can hybridize with a local search to find optimal designs under a more complicated criterion, such as standardized maximin optimality.

Finding Bayesian Optimal Designs for Nonlinear Models: A Semidefinite Programming-Based Approach.

This paper uses semidefinite programming (SDP) to construct Bayesian optimal design for nonlinear regression models. The setup here extends the formulation of the optimal designs problem as an SDP problem from linear to nonlinear models. Gaussian quadrature formulas (GQF) are used to compute the expectation in the Bayesian design criterion, such as D-, A- or E-optimality. As an illustrative example, we demonstrate the approach using the power-logistic model and compare results in the literature. Additionally, we investigate how the optimal design is impacted by different discretising schemes for the design space, different amounts of uncertainty in the parameter values, different choices of GQF and different prior distributions for the vector of model parameters, including normal priors with and without correlated components. Further applications to find Bayesian D-optimal designs with two regressors for a logistic model and a two-variable generalised linear model with a gamma distributed response are discussed, and some limitations of our approach are noted.

On Optimal Designs for Nonlinear Models: A General and Efficient Algorithm

Finding optimal designs for nonlinear models is challenging in general. Although some recent results allow us to focus on a simple subclass of designs for most problems, deriving a specific optimal design still mainly depends on numerical approaches. There is need for a general and efficient algorithm that is more broadly applicable than the current state-of-the-art methods. We present a new algorithm that can be used to find optimal designs with respect to a broad class of optimality criteria, when the model parameters or functions thereof are of interest, and for both locally optimal and multistage design strategies. We prove convergence to the optimal design, and show in various examples that the new algorithm outperforms the current state-of-the-art algorithms.

A semi-infinite programming based algorithm for finding minimax optimal designs for nonlinear models

Minimax optimal experimental designs are notoriously difficult to study largely because the optimality criterion is not differentiable and there is no effective algorithm for generating them. We ap...

Identifying locally optimal designs for nonlinear models: A simple extension with profound consequences

We extend the approach in [Ann. Statist. 38 (2010) 2499–2524] for identifying locally optimal designs for nonlinear models. Conceptually the extension is relatively simple, but the consequences in terms of applications are profound. As we will demonstrate, we can obtain results for locally optimal designs under many optimality criteria and for a larger class of models than has been done hitherto. In many cases the results lead to optimal designs with the minimal number of support points.

On the de la Garza phenomenon

Deriving optimal designs for nonlinear models is, in general, challenging. One crucial step is to determine the number of support points needed. Current tools handle this on a case-by-case basis. Each combination of model, optimality criterion and objective requires its own proof. The celebrated de la Garza Phenomenon states that under a (p−1)th-degree polynomial regression model, any optimal design can be based on at most p design points, the minimum number of support points such that all parameters are estimable. Does this conclusion also hold for nonlinear models? If the answer is yes, it would be relatively easy to derive any optimal design, analytically or numerically. In this paper, a novel approach is developed to address this question. Using this new approach, it can be easily shown that the de la Garza phenomenon exists for many commonly studied nonlinear models, such as the Emax model, exponential model, three- and four-parameter log-linear models, Emax-PK1 model, as well as many classical polynomial regression models. The proposed approach unifies and extends many well-known results in the optimal design literature. It has four advantages over current tools: (i) it can be applied to many forms of nonlinear models; to continuous or discrete data; to data with homogeneous or nonhomogeneous errors; (ii) it can be applied to any design region; (iii) it can be applied to multiple-stage optimal design and (iv) it can be easily implemented.

Support points of locally optimal designs for nonlinear models with two parameters

We propose a new approach for identifying the support points of a locally optimal design when the model is a nonlinear model. In contrast to the commonly used geometric approach, we use an approach based on algebraic tools. Considerations are restricted to models with two parameters, and the general results are applied to often used special cases, including logistic, probit, double exponential and double reciprocal models for binary data, a loglinear Poisson regression model for count data, and the Michaelis-Menten model. The approach, which is also of value for multi-stage experiments, works both with constrained and unconstrained design regions and is relatively easy to implement.

On representing maximin efficient designs by Taylor series

This paper considers exponential and rational regression models that are nonlinear in some parameters. Recently, locally D-optimal designs for such models were investigated in [Melas, V. B., 2005. On the functional approach to optimal designs for nonlinear models. J. Statist. Plann. Inference 132, 93–116] based upon a functional approach. In this article a similar method is applied to construct maximin efficient D-optimal designs. This approach allows one to represent the support points of the designs by Taylor series, which gives us the opportunity to construct the designs by hand using tables of the coefficients of the series. Such tables are provided here for models with two nonlinear parameters. Furthermore, the recurrent formulas for constructing the tables for arbitrary numbers of parameters are introduced.

On the functional approach to optimal designs for nonlinear models

This paper concerns locally optimal experimental designs for nonlinear regression models. It is based on the functional approach introduced in (Ser. Statist. 9 (1978) 45). In this approach, locally optimal design points and weights are studied as implicitly given functions of the nonlinear parameters included in the model. Representing these functions in a Taylor series enables an analytical solution of the optimal design problem for many nonlinear models. A wide class of such models is introduced here. It includes, in particular, three parameters logistic distribution, hyperexponential and rational models. For these models we construct the analytical solution and use it for studying the efficiency of locally optimal designs. As a criterion of optimality the well-known D-criterion is considered.

Sequential Construction of Multiple-Objective Optimal Designs

We propose a sequential approach for constructing multiple-objective locally optimal designs for nonlinear models. The technique used here is a general one and we demonstrate the added benefits of using a multiple-objective design over a single-objective design with examples from biomedical studies.