Sequential Sampling Scheme Research Articles

A novel sequential exploration-exploitation sampling strategy for global metamodeling

Sampling strategy has direct impact on the accuracy of metamodel. In this paper, we propose a novel sequential exploration-exploitation sampling strategy for global metamodeling. A new criterion is proposed to focus on characteristic of output space. Space-filling requirement is treated as a constraint to avoid clustered sample points. The methodology developed in this paper is compared to existing methods using an analytic numerical case with two inputs. The results indicate that the proposed approach achieves the more desired accuracy of metamodel than the other approaches.

The use of Thompson sampling to increase estimation precision.

In this article, we consider a sequential sampling scheme for efficient estimation of the difference between the means of two independent treatments when the population variances are unequal across groups. The sampling scheme proposed is based on a solution to bandit problems called Thompson sampling. While this approach is most often used to maximize the cumulative payoff over competing treatments, we show that the same method can also be used to balance exploration and exploitation when the aim of the experimenter is to efficiently increase estimation precision. We introduce this novel design optimization method and, by simulation, show its effectiveness.

Radial basis function-based shape optimization of centrifugal impeller using sequential sampling

Multi-objective optimization algorithms become more appropriate in general aerodynamic design problems, despite the number of objective functions evaluated by computational fluid dynamics (CFD) calculations. This paper presents a multi-objective aerodynamic shape optimization (ASO) procedure implemented to minimize the optimization cycle time. Its main ideas consist in replacing some of CFD evaluations by those provided by an approximate meta-model known as the radial basis functions (RBFs). In order to minimize the size of training samples when constructing the RBF meta-model, a sequential sampling strategy based on an iterative refinement of training samples database is considered. When compared with one-stage sampling, the sequential sampling has allowed more reduction in the computational time and a better accuracy in the approximation of the objectives by RBF meta-model. The MASSOUD “multidisciplinary aero/struc shape optimization using deformation” and Bspline techniques are combined to reduce the number of control points for the shape deformation or optimization variables. Once the database composed of a set of geometries is constructed and CFD is used to evaluate the objectives for each element of the database, RBF is used to approximate the objectives with respect to the optimization variables, and finally the NSGA-II “Non-Dominated Sorting Genetic Algorithm” is applied to produce the pareto optimal fronts. An application aimed to optimize the NASA low speed centrifugal compressor (LSCC) resulted in an optimal design within a reasonable computational time and produced the Pareto fronts of trade-offs enhancing the baseline performance.

Multiple crossing sequential fixed-size confidence region methodologies for a multivariate normal mean vector

The asymptotically efficient and asymptotically consistent purely sequential procedure of Mukhopadhyay and Al-Mousawi (1986) is customarily used to construct a confidence region R for the mean vector μ of Np(μ,σ2H). This procedure does not have the exact consistency property. Hp×p is assumed known and positive definite with σ2 unknown. The maximum diameter of R and the confidence coefficient are prefixed.A purely sequential sampling strategy is proposed allowing sampling until sample size crosses the boundary multiple times. We ascertain asymptotic efficiency and asymptotic consistency properties (Theorem 3.1). Its ability to nearly achieve required coverage probability without significant over-sampling is demonstrated with simulations. A truncation technique plus fine-tuning of the multiple crossing rule are proposed to increase practicality. Two real data illustrations are highlighted.

Multi-point objective-oriented sequential sampling strategy for constrained robust design

Metamodelling techniques are widely used to approximate system responses of expensive simulation models. In association with the use of metamodels, objective-oriented sequential sampling methods have been demonstrated to be effective in balancing the need for searching an optimal solution versus reducing the metamodelling uncertainty. However, existing infilling criteria are developed for deterministic problems and restricted to one sampling point in one iteration. To exploit the use of multiple samples and identify the true robust solution in fewer iterations, a multi-point objective-oriented sequential sampling strategy is proposed for constrained robust design problems. In this article, earlier development of objective-oriented sequential sampling strategy for unconstrained robust design is first extended to constrained problems. Next, a double-loop multi-point sequential sampling strategy is developed. The proposed methods are validated using two mathematical examples followed by a highly nonlinear automotive crashworthiness design example. The results show that the proposed method can mitigate the effect of both metamodelling uncertainty and design uncertainty, and identify the robust design solution more efficiently than the single-point sequential sampling approach.

Auto-generation of passive scalable macromodels for microwave components using scattered sequential sampling

This paper presents a method for automatic construction of stable and passive scalable macromodels for parameterized frequency responses. The method requires very little prior knowledge to build the scalable macromodels thereby considerably reducing the burden on the designers. The proposed method uses an efficient scattered sequential sampling strategy with as few expensive simulations as possible to generate accurate macromodels for the system using state-of-the-art scalable macromodeling methods. The scalable macromodels can be used as a replacement model for the actual simulator in overall design processes. Pertinent numerical results validate the proposed sequential sampling strategy.

Value-Based Global Optimization

In this paper, a value-based global optimization (VGO) algorithm is introduced. The algorithm uses kriging-like surrogate models and a sequential sampling strategy based on value of information (VoI) to optimize an objective characterized by multiple analysis models with different accuracies. VGO builds on two main contributions. The first contribution is a novel surrogate modeling method that accommodates data from any number of different analysis models with varying accuracy and cost. Rather than interpolating, it fits a model to the data, giving more weight to more accurate data. The second contribution is the use of VoI as a new metric for guiding the sequential sampling process for global optimization. Based on information about the cost and accuracy of each available model, predictions from the current surrogate model are used to determine where to sample next and with what level of accuracy. The cost of further analysis is explicitly taken into account during the optimization process, and no further analysis occurs if the expected value of the new information is negative. In this paper, we present the details of the VGO algorithm and, using a suite of randomly generated test cases, compare its performance with the performance of the efficient global optimization (EGO) algorithm (Jones, D. R., Matthias, S., and Welch, W. J., 1998, “Efficient Global Optimization of Expensive Black-Box Functions,” J. Global Optim., 13(4), pp. 455–492). Results indicate that the VGO algorithm performs better than EGO in terms of overall expected utility—on average, the same quality solution is achieved at a lower cost, or a better solution is achieved at the same cost.

Estimators for the Horvitz-Thompson Statistic Based on Some Posterior Distributions

The knowledge of first-order inclusion probabilities characterizing a sampling scheme is essential in design-based estimation of finite population totals. Sometimes the scheme is so complex that these probabilities cannot be computed exactly. Instead, both inclusion probabilities and corresponding sampling weights are simulated. One empirical Horvitz-Thompson estimator for a population total using simulation-based range-preserving estimates of sampling weights is obtained by applying the restricted maximum likelihood principle directly to each inclusion probability. The assumption of a prior distribution and the assessment of resulting posterior for a weight lead to two other estimators. One of them is the posterior mean estimator of the Horvitz-Thompson statistic. In a simulation involving Polish agricultural census data and a sequential fixed-cost sampling scheme, this estimator has attractive properties also from a frequentist point of view.

Unbalanced and Partial Group Sequential Methods for Normal Responses in Clinical Trials

Group sequential methods for a two-treatment clinical trial with normal responses are discussed. First we consider the case where the sample sizes for two treatments are possibly unequal between the treatments due to an unequal randomization. Then we discuss group sequential design in the context of a historical control study, that is, under the partial sequential sampling scheme, in which the samples on one treatment, say control, are available at the outset, and the samples on the other treatment, say experimental, are obtained in the group sequential way. We discuss the cases of known and unknown variance for both unbalanced and partial group sequential setups. All of the procedures are discussed with numerical studies.

A Maximum Confidence Enhancement Based Sequential Sampling Scheme for Simulation-Based Design

A maximum confidence enhancement (MCE)-based sequential sampling approach is developed for reliability-based design optimization (RBDO) using surrogate models. The developed approach employs the ordinary Kriging method for surrogate model development and defines a cumulative confidence level (CCL) measure to quantify the accuracy of reliability estimation when Monte Carlo simulation is used based on the developed surrogate model. To improve the computational efficiency, an MCE-based sequential sampling scheme is developed to successively select sample points for surrogate model updating based on the defined CCL measure, in which a sample point that produces the largest CCL improvement will be selected. To integrate the MCE-based sequential sampling approach with RBDO, a new sensitivity analysis approach is developed, enabling smooth design sensitivity information to be accurately estimated based upon the constructed surrogate model without incurring any extra computational costs, thus greatly enhancing the efficiency and robustness of the design process. Two case studies are used to demonstrate the efficacy of the developed approach.

The Sample Complexity of Search Over Multiple Populations

This paper studies the sample complexity of searching over multiple populations. We consider a large number of populations, each corresponding to either distribution P0 or P1. The goal of the search problem studied here is to find one population corresponding to distribution P1 with as few samples as possible. The main contribution is to quantify the number of samples needed to correctly find one such population. We consider two general approaches: non-adaptive sampling methods, which sample each population a predetermined number of times until a population following P1 is found, and adaptive sampling methods, which employ sequential sampling schemes for each population. We first derive a lower bound on the number of samples required by any sampling scheme. We then consider an adaptive procedure consisting of a series of sequential probability ratio tests, and show it comes within a constant factor of the lower bound. We give explicit expressions for this constant when samples of the populations follow Gaussian and Bernoulli distributions. An alternative adaptive scheme is discussed which does not require full knowledge of P1, and comes within a constant factor of the optimal scheme. For comparison, a lower bound on the sampling requirements of any non-adaptive scheme is presented.

Reduced-order model tracking and interpolation to solve PDE-based Bayesian inverse problems

This work presents a computationally efficient probabilistic framework that enables the identification of model parameters from noisy measurements of the response. We consider transient PDE-based models, where the parameters correspond to physical properties. An efficient and reliable procedure for estimation of those unknown parameters is pursued. The proposed framework uses a Bayesian approach, an efficient sequential Monte Carlo sampling scheme, and adaptive reduced-order models (ROMs). The Bayesian approach has several advantages including the ability to provide not only point estimates of the quantities of interest, but also measures of credibility and robustness concerning those estimates. The associated sequential Monte Carlo method sampling scheme is embarrassingly parallelizable, as well as efficient in terms of the number of calls to the forward solver (e.g. a finite element code) used in evaluating the likelihood function. We propose to use a ROM adaptation procedure where projection-based ROMs are seen as points on a certain Riemannian manifold and are ‘tracked’ and interpolated during the sampling process using a database of precomputed ROMs. This approach ensures that an appropriate ROM is used for the likelihood evaluation in every region of the parameter space, thus leading to computational savings while conserving sufficient accuracy. Using numerical examples, we illustrate the capabilities of the proposed framework, and show that it leads to quality estimates with a quantified predictive uncertainty.

Objective-Oriented Sequential Sampling for Simulation Based Robust Design Considering Multiple Sources of Uncertainty

Sequential sampling strategies have been developed for managing complexity when using computationally expensive computer simulations in engineering design. However, much of the literature has focused on objective-oriented sequential sampling methods for deterministic optimization. These methods cannot be directly applied to robust design, which must account for uncontrollable variations in certain input variables (i.e., noise variables). Obtaining a robust design that is insensitive to variations in the noise variables is more challenging. Even though methods exist for sequential sampling in design under uncertainty, the majority of the existing literature does not systematically take into account the interpolation uncertainty that results from limitations on the number of simulation runs, the effect of which is inherently more severe than in deterministic design. In this paper, we develop a systematic objective-oriented sequential sampling approach to robust design with consideration of both noise variable uncertainty and interpolation uncertainty. The method uses Gaussian processes to model the costly simulator and quantify the interpolation uncertainty within a robust design objective. We examine several criteria, including our own proposed criteria, for sampling the design and noise variables and provide insight into their performance behaviors. We show that for both of the examples considered in this paper the proposed sequential algorithm is more efficient in finding the robust design solution than a one-shot space filling design.

EasyABC: performing efficient approximate Bayesian computation sampling schemes using R

Summary Approximate Bayesian computation (ABC), a type of likelihood‐free inference, is a family of statistical techniques to perform parameter estimation and model selection. It is increasingly used in ecology and evolution, where the models used can be too complex to be handled with standard likelihood techniques. The essence of ABC techniques is to compare simulation outputs to observed data, in order to select the parameter values of the simulations which best fit the data. ABC techniques are thus computationally demanding. This constitutes a key limitation to their implementation. We introduce the R package ‘EasyABC’ that enables one to launch a series of simulations from the R platform and to retrieve the simulation outputs in an appropriate format for post‐processing. The ‘EasyABC’ package further implements several efficient parameter sampling schemes to speed up the ABC procedure: on top of the standard prior sampling, it implements various algorithms to perform sequential (ABC‐sequential) and Markov chain Monte Carlo (ABC‐MCMC) sampling schemes. The package functions can furthermore make use of parallel computing. The R package ‘EasyABC’ complements the package ‘abc’ which enables various post‐processing of simulation outputs. ‘EasyABC’ makes several state‐of‐the‐art ABC implementations available to the large community of R users in the fields of ecology and evolution. It is a freely available R package under the GPL license, and it can be downloaded at http://cran.r-project.org/web/packages/EasyABC/index.html.

Exact Group Sequential Methods for Estimating a Binomial Proportion

We first review existing sequential methods for estimating a binomial proportion. Afterward, we propose a new family of group sequential sampling schemes for estimating a binomial proportion with prescribed margin of error and confidence level. In particular, we establish the uniform controllability of coverage probability and the asymptotic optimality for such a family of sampling schemes. Our theoretical results establish the possibility that the parameters of this family of sampling schemes can be determined so that the prescribed level of confidence is guaranteed with little waste of samples. Analytic bounds for the cumulative distribution functions and expectations of sample numbers are derived. Moreover, we discuss the inherent connection of various sampling schemes. Numerical issues are addressed for improving the accuracy and efficiency of computation. Computational experiments are conducted for comparing sampling schemes. Illustrative examples are given for applications in clinical trials.

Adaptive virtual support vector machine for reliability analysis of high-dimensional problems

In this paper, an efficient classification methodology is developed for reliability analysis while maintaining an accuracy level similar to or better than existing response surface methods. The sampling-based reliability analysis requires only the classification information--a success or a failure--but the response surface methods provide function values on the domain as their output, which requires more computational effort. The problem is even more challenging when dealing with high-dimensional problems due to the curse of dimensionality. In the newly proposed virtual support vector machine (VSVM), virtual samples are generated near the limit state function by using an approximation method. The function values are used for approximations of virtual samples to improve accuracy of the resulting VSVM decision function. By introducing the virtual samples, VSVM can overcome the deficiency in existing classification methods where only classification values are used as their input. The universal Kriging method is used to obtain virtual samples to improve the accuracy of the decision function for highly nonlinear problems. A sequential sampling strategy that chooses new samples near the limit state function is integrated with VSVM to improve the accuracy. Examples show the proposed adaptive VSVM yields better efficiency in terms of modeling and response evaluation time and the number of required samples while maintaining similar level or better accuracy, especially for high-dimensional problems.

A Sequential Sampling Strategy to Improve Reliability-Based Design Optimization With Implicit Constraint Functions

Reliability-based design optimization (RBDO) has a probabilistic constraint that is used for evaluating the reliability or safety of the system. In modern engineering design, this task is often performed by a computer simulation tool such as finite element method (FEM). This type of computer simulation or computer experiment can be treated a black box, as its analytical function is implicit. This paper presents an efficient sampling strategy on learning the probabilistic constraint function under the design optimization framework. The method is a sequential experimentation around the approximate most probable point (MPP) at each step of optimization process. Our method is compared with the methods of MPP-based sampling, lifted surrogate function, and nonsequential random sampling. We demonstrate it through examples.

Some partially sequential nonparametric tests for detecting linear trend

In the present study, we develop two nonparametric partially sequential tests for detecting possible presence of linear trend among the incoming series of observations. We assume that a sample of fixed size is available a priori from some unknown univariate continuous population and there is no sign of trend among these historical observations. Our proposed tests can be viewed as the sequential type tests for monitoring structural changes. We use partial sequential sampling schemes based on usual ranks as well as on sequential ranks. We provide detailed discussion on asymptotic studies related to the proposed tests. We compare the two tests under various situations. We also present some numerical results based on simulation studies. Proposed tests are extremely important in profit making in volatile market through Margin Trading. We illustrate the mechanism with a detailed analysis of a stock price data.

Usage of a pair of [formula omitted]-paths in Bayesian estimation of a unimodal density

This paper aims at illustrating the importance of using S -paths in Bayesian estimation of a unimodal density on the real line. A class of species sampling mixture models containing random densities that are unimodal and not necessarily symmetric is considered. A novel and explicit characterization of the posterior distribution expressible as a finite mixture over pairs of two dependent S -paths is derived, resulting in closed-form and tractable Bayes estimators for both the density and the mode as finite sums over the pairs. These results are statistically important as they are proved to be Rao–Blackwell improvements over existing results expressible in terms of partitions, and thus can be estimated with less variability. Extending an effective and newly-developed sequential importance sampling (SIS) scheme for sampling one S -path at a time, an SIS scheme is proposed to approximate the density estimates or any other posterior quantities of the model that are expressible in terms of two S -paths. Simulation results are reported to demonstrate practicality of our methodology and its effectiveness over an existing class of non-iterative algorithms that are based on sampling partitions. Indeed, the latter commonly-used algorithms, widely believed to be feasible, are shown to be ineffective and unreliable, implying that there exists hardly any practical non-iterative algorithm in this context. This prompts the essentiality of a practically useful algorithm for the problem.

Sequential D-optimal designs for generalized linear mixed models

We discuss the construction of D-optimal sequential designs for the analysis of longitudinal data or repeated measurements using generalized linear mixed models (GLMMs). We investigate the performance of the design through a simulation study, which indicates that the proposed design can be very successful in improving the efficiency of the ML estimators in GLMMs relative to some common competitors. Our simulations also suggest that the usual normal-theory inference procedures remain valid under the sequential sampling schemes. We also present an example using real data obtained from a clinical study.