Expected Improvement Function Research Articles

Parametric Dynamic Simulation and Bayesian Design Optimization of a Front-Loading Washing Machine

Abstract Objective This paper presents the dynamic modeling and design optimization of a fully parametrized front-loading washing machine. Methods A thorough mathematical analysis is performed to capture the effect of nine design variables (including rotation speed, spring stiffness, weight balancer mass density, damping coefficient, spring and damper angles) on vibrational characteristics of the washing machine. The parametric simulation reveals a complex relationship between the vibrational dynamics and design variables, entailing a computationally efficient optimization approach. A Bayesian Optimization (BO) framework is developed, in which a Gaussian process model and Genetic Algorithm (GA) are utilized along with carefully selected acquisition functions to enable adaptive sampling and search for optimal design values. The objective function is to minimize the maximum amplitude of the tub displacement inside the washing machine body. Two case studies are performed to consider a different number of design variables and choices of various infill methods. Results Results show that the fully parametrized front-loading washing machine model is applicable for parametric simulation and design optimization. The proposed BO is able to successfully find a set of design variable values corresponding to the lowest maximum displacement for vibration reduction. A comparison analysis is also carried out to exhibit the difference in optimization convergence and computational cost caused by the choice of acquisition functions. Conclusion It is shown that the fully parametrized washing machine model yields a lower maximum displacement than the model with fewer design variables. Of the two acquisition functions studied for optimization, the Expected Improvement function converges more rapidly than the Lower Bound criterion.

Hybrid uncertainty propagation based on multi-fidelity surrogate model

There always exist multiple uncertainties including random uncertainty, interval uncertainty, and fuzzy uncertainty in engineering structures. In the presence of hybrid uncertainties, the hybrid uncertainty propagation analysis can be a challenging problem, which suffers from the computational burden of double-loop procedure when numerical simulation techniques are employed. In this work, a novel method for efficient hybrid uncertainty propagation analysis with the three types of uncertainties is proposed. Generally, multi-fidelity surrogate models, such as Co-Kriging, can greatly improve the computational efficiency by leveraging information from a low-fidelity model to build a high-fidelity approximate model. However, the traditional multi-fidelity surrogate model methods always calculate the hybrid uncertainty propagation result by combining with several numerical simulation techniques. This process can introduce post-processing errors unless unlimited number of samples are used, which is impossible in engineering application. In order to address this issue, the analytical solutions of the output mean and output variance are derived based on the Co-Kriging, and the resulting mean and variance are both random variables. Moreover, a new adaptive framework is established to strengthen the estimation accuracy of the hybrid uncertainty propagation result, by combining the augmented expected improvement function and the derived mean random variable. Several applications are introduced to demonstrate the effectiveness of the proposed method for solving hybrid uncertainty propagation problems.

An Adaptive Sampling Method Based on Expected Improvement Function and Residual Gradient in PINNs

An Adaptive Sampling Method Based on Expected Improvement Function and Residual Gradient in PINNs

Gradient-Enhanced Kriging for High-Dimensional Bayesian Optimization with Linear Embedding

This paper explores the application of gradient-enhanced (GE) kriging for Bayesian optimization (BO) problems with a high-dimensional parameter space. We utilize the active subspace method to embed the original parameter space in a low-dimensional subspace. The active subspace is detected by analyzing the spectrum of the empirical second-moment matrix of the gradients of the response function. By mapping the training data onto their respective subspace, the objective function and the constraint functions are efficiently approximated with low-dimensional GE-kriging models. In each cycle of the BO procedure, a new point is found by maximizing the constrained expected improvement function within a low-dimensional polytope, and it is mapped back to the original space for model evaluation. In this way, the computational costs are significantly reduced when compared with standard GE-kriging. We illustrate and assess the proposed approach with numerical experiments ranging from academic benchmark problems to aerodynamic engineering applications. The experimental results show that the proposed method is promising for optimizing high-dimensional expensive objective functions, especially for problems that exhibit a clear low-dimensional active subspace.

Multi-Objective Bayesian Optimization Supported by an Expected Pareto Distance Change

Abstract The solution to global (a posteriori) multi-objective optimization problems traditionally relies on population-based algorithms, which are very effective in generating a Pareto front. Unfortunately, due to the high number of function evaluations, these methods are of limited use in problems that involve expensive black-box functions. In recent years, multi-objective Bayesian optimization has emerged as a powerful alternative; however, in many applications, these methods fail to generate a diverse and well-spread Pareto front. To address this limitation, our work introduces a novel acquisition function (AF) for multi-objective Bayesian optimization that produces more informative acquisition landscapes. The proposed AF comprises two terms, namely, a distance-based metric and a diversity index. The distance-based metric, referred to as the expected Pareto distance change, promotes the evaluation of high-performing designs and repels low-performing design zones. The diversity term prevents the evaluation of designs that are similar to the ones contained in the current sampling plan. The proposed AF is studied using seven analytical problems and in the design optimization of sandwich composite armors for blast mitigation, which involves expensive finite element simulations. The results show that the proposed AF generates high-quality Pareto sets outperforming well-established methods such as the Euclidean-based expected improvement function. The proposed AF is also compared with respect to a recently proposed multi-objective approach. The difference in their performance is problem dependent.

An improved active Kriging method for reliability analysis combining expected improvement and U learning functions

The reliability assessment of structures with multiple failure modes and small failure probability is challenging due to the time-consuming simulations required. Active learning Kriging methods for structural reliability with multiple failure modes have shown high computational efficiency and accuracy. However, selecting the appropriate sample and its failure mode to update the Kriging models remains a key problem. In this paper, we propose a new learning function and stopping criterion to further improve the efficiency of structural system reliability analysis. Firstly, we propose a new learning function that combines the expected improvement function and the U learning function. This function selects the most suitable samples, balancing the degree of expected improvement of samples to the limit state surface and the degree of misclassification probability of samples. Secondly, we propose a new stopping criterion that considers both the accurate construction of limit state surfaces and the probability of accurately predicting the signs of samples. This criterion avoids premature or late termination of the active learning process. Thirdly, the sequential MCS simulation method is employed in the active learning process to efficiently evaluate small failure probability problems. By analyzing four examples, we verify the accuracy and efficiency of the proposed structural reliability analysis method.

Reliability analysis method of coupling optimal importance sampling density and multi-fidelity Kriging model

The commonly used reliability analysis approaches for Kriging-based models are usually conducted based on high-fidelity Kriging models. However, high-fidelity surrogate models are commonly costly. Therefore, in order to balance the calculation expense and calculation time of the surrogate model, this paper proposes a multi-fidelity Kriging model reliability analysis approach with coupled optimal important sampling density (OISD+MFK). First, the MEI learning function is proposed considering the training sample distance, model computation cost, expected improvement function, and model relevance. Second, a dynamic stopping condition is proposed that takes into account the failure probability estimation error. Finally, the optimal importance sampling density is incorporated into the reliability analysis process, which can effectively reduce failure probability estimation error. The results of the study show that the approach proposed in this paper can reduce the calculation cost while outputting relatively accurate failure probability evaluation results.

Bayesian Optimization Based on Student’s T Process for Microstrip Antenna Design

Bayesian Optimization (BO) is an efficient global optimization algorithm, which is widely used in the field of engineering design. The probabilistic surrogate model and acquisition function are the two keys to the algorithm. Building an efficient probabilistic surrogate model and designing a collection function with excellent exploring capabilities can improve the performance of BO algorithm, allowing it to find the optimal value of the objective function with fewer iterations. Due to the characteristics of small samples and non-parametric derivation of the Gaussian Process (GP), traditional BO algorithms usually use the GP as a surrogate model. Compared with the GP, the Student’s T Process (STP) retains the excellent properties of GP, and has more flexible posterior variance and stronger robustness. In this paper, STP is used as the surrogate model in BO algorithm, the hyperparameters of the model are optimized by STP, and the estimation strategy function (EST) is improved based on the posterior output of the optimized STP, thus realizing the improved BO algorithm based on the STP. To verify the performance of the proposed algorithm, numerical experiments are designed to compare the performances of the traditional BO algorithm, which includes the lower confidence bound function (LCB) and EST as acquisition function respectively and GP as the surrogate model, and the proposed BO algorithm with STP as the surrogate model and LCB, expected improvement function (EI), expected regret minimization function (ERM) as acquisition function respectively. The results show that the proposed algorithm in this paper performs well when finding the global minimum of multimodal functions. Based on the developed algorithm in this paper, the resonant frequency of printed dipole antenna and E-shaped antenna is modeled and optimized, which further confirms the good design ability and design accuracy of the BO algorithm proposed in this paper.

An Adaptive Parallel EI Infilling Strategy Extended by Non-Parametric PMC Sampling Scheme for Efficient Global Optimization

This paper proposes an adaptive parallel Expected Improvement (EI) sampling scheme applied to Efficient Global Optimization, called non-parametric Population Monte Carlo sampling (NPMS) scheme. The NPMS scheme introduces the Population Monte Carlo (PMC) method and the Density-Based Spatial Clustering (DBSCAN) method to address the problem of candidate points aggregation in the parallel EI strategy. In the first stage, samples are uniformly generated from EI function and converge to high EI value sub-domains by PMC iterative succession. A non-parametric sampling method is used to aid PMC convergence and provides valuable sub-domains ranges for the scheme. In the second stage, learning from current potential information, the DBSCAN method is used to cluster the samples and converge to candidate points. During NPMS benchmarking, we verified the feasibility and analyzed the impact of parameters on the scheme. Compared to original EI strategy, NPMS improved EGO minimum results by 14.6% and reduced candidate points by 15.8%. In addition, four test groups are used to compare the NPMS with five other EI-based strategies. Results show that the NPMS scheme has advantages in finding results and saving costs. Mainly, in large input space test group, NPMS saves 34.9% optimization costs and can find better results than parallel strategies. NPMS scheme adaptive sampling provides stable optimization results in the poor prior information, which is expected to provide efficient sampling solutions in more complex problems.

An active-learning reliability method based on support vector regression and cross validation

To reduce the high computational cost of the probability density evolution method (PDEM) for structural reliability analysis, an active-learning reliability method that combines the support vector regression (SVR) and the PDEM is proposed, which is abbreviated as the ASVR-PDEM. First, the empirical probability distribution of the SVR is proposed based on the leave-one-out cross-validation (LOOCV) strategy. On this basis, two different learning functions are proposed according to the concept of the PDEM-oriented expected improvement function, which accommodate to any type of metamodels in essence. Then, the pseudo-interpolation replacement is devised to avoid the prediction error of the SVR at the representative points in the region of interest and, thus, maintains the precision of failure probability. Moreover, the influences of four key ingredients on the proposed method are investigated, namely the type of metamodels, the form of learning functions, the deployment of the pseudo-interpolation replacement and the training scheme in the LOOCV strategy. Three numerical examples are investigated to show the feasibility of the ASVR-PDEM. Comparisons are made against several conventional reliability methods. The results highlight the superiority of the ASVR-PDEM in terms of computational accuracy, efficiency and computational time, especially in the case of the time-consuming dynamic reliability problems.

A kriging-assisted bi-objective constrained global optimization algorithm for expensive constrained optimization problems

Computationally expensive constrained optimization problems are challenging owing to their high complexity and computational cost. To solve these problems efficiently, a kriging-assisted bi-objective constrained global optimization (BOCGO) algorithm is developed, where three phases with three bi-objective subproblems are performed. In phase I, the constraints are searched locally and globally to find the feasible region. Once a feasible region has been located, the two terms of the constrained expected improvement function are utilized to exploit and explore the feasible region in phase II. As the kriging models are accurate enough in the concerned region, a local search is processed to improve the optimal solution in phase III. The capability of the BOCGO algorithm is demonstrated by comparison with two classical and two state-of-the-art algorithms on 20 problems and an engineering simulation problem. The results show that the BOCGO algorithm performs better in more than three-fifths of problems, illustrating its effectiveness and robustness.

Robustness under parameter and problem domain alterations of Bayesian optimization methods for chemical reactions

The related problems of chemical reaction optimization and reaction scope search concern the discovery of reaction pathways and conditions that provide the best percentage yield of a target product. The space of possible reaction pathways or conditions is too large to search in full, so identifying a globally optimal set of conditions must instead draw on mathematical methods to identify areas of the space that should be investigated. An intriguing contribution to this area of research is the recent development of the Experimental Design for Bayesian optimization (EDBO) optimizer [1]. Bayesian optimization works by building an approximation to the true function to be optimized based on a small set of simulations, and selecting the next point (or points) to be tested based on an acquisition function reflecting the value of different points within the input space. In this work, we evaluated the robustness of the EDBO optimizer under several changes to its specification. We investigated the effect on the performance of the optimizer of altering the acquisition function and batch size, applied the method to other existing reaction yield data sets, and considered its performance in the new problem domain of molecular power conversion efficiency in photovoltaic cells. Our results indicated that the EDBO optimizer broadly performs well under these changes; of particular note is the competitive performance of the computationally cheaper acquisition function Thompson Sampling when compared to the original Expected Improvement function, and some concerns around the method’s performance for “incomplete” input domains.

Ensemble of metamodels-assisted probability density evolution method for structural reliability analysis

An active-learning reliability method called the AEM-PDEM is proposed that combines adaptive ensemble of metamodels (EM) and the probability density evolution method (PDEM). Three critical aspects are addressed in this method. First, the ensemble of three diverse metamodels, i.e., the polynomial chaos Kriging (PCK), the low-rank approximation (LRA) and the support vector regression (SVR), is built by weighted combination according to their global error measures, which enables to provide both predicted value and variance. Second, the EM predictions at the training samples are replaced by the true computational model responses, so as to secure the accuracy of failure probability estimate. Third, according to the PDEM-oriented expected improvement function (PEIF), a multi-point enrichment process is developed based on the EM and the three component metamodels. Then, three numerical examples are investigated and comparisons are made between the AEM-PDEM and other existing reliability methods. Results demonstrate that, in comparison with the existing APCK-PDEM, the AEM-PDEM needs roughly 85-95% of the number of computational model evaluations. More importantly, it only requires approximately 30-45% of the number of iterations during the active-learning process. As a result, it just consumes nearly 35-50% of computational time of the APCK-PDEM, especially in high-dimensional dynamic problems and practical complex engineering problems.

Multi-objective surrogate model-based optimization of a small aircraft engine air-intake duct

Aviation industry is constantly striving for more efficient design processes in respect to optimal time, human and computational resources utilization. This implies a need for application of an approximation techniques enabling for fast responses generation with maintained level of results quality. This study focuses on an advancement of aerodynamic shape optimization process of a small aircraft engine intake system by introduction of a surrogate modelling step into the design loop. The multi-objective metamodel assisted optimization is carried out in order to reduce pressure losses along the engine intake duct and increase flow homogeneity at the engine compressor intake plane. Latin Hypercube Design method is utilized in order to sample the design space. A set of initial objective function evaluations is generated with application of Reynolds-averaged Navier–Stokes solver. The ensemble of samples is further used to train a Kriging-based surrogate model. The Efficient Global Optimization algorithm basing on the Expected Improvement function is employed to gradually increase the metamodel prediction quality by usage of sequential sampling technique. Finally, the optimal point predicted by the Kriging surrogate is validated against the high-fidelity model with usage of the Computational Fluid Dynamics code. The paper presents an application of the abovementioned methodology to the design process of the I-31T aircraft turboprop engine intake system. Proposed Kriging-based optimization workflow is utilized in order to reduce pressure losses and improve flow homogeneity in the engine air-intake duct.

Reliability analysis using adaptive Polynomial-Chaos Kriging and probability density evolution method

An efficient reliability method that combines adaptive Polynomial-Chaos Kriging (PC-Kriging) and probability density evolution method (PDEM) is developed, which is abbreviated as the APCK-PDEM. First, according to the relative contributions of different representative points to the failure probability calculated by the PDEM, the notation of region of interest (ROI) is proposed, in which the representative points make critical contributions to the resultant failure probability. Then, three key aspects involved in the proposed APCK-PDEM are addressed: (a) A new learning function called PDEM-oriented expected improvement function (PEIF) is devised to cater for the demand of PDEM on the PC-Kriging accuracy; (b) A pertinent convergence criterion is defined in terms of the bound of failure probability estimated by the APCK-PDEM; (c) Since the true value of the boundary of ROI is unknown in the PEIF, an iterative determination scheme of this metric is performed at each iteration during the adaptive sampling process. Three examples are studied to showcase the performance of APCK-PDEM, and comparisons are made against other existing reliability methods. Numerical analyses and results show that the APCK-PDEM gains satisfactory estimation accuracy and high computational efficiency.

Multidisciplinary design for structural integrity using a collaborative optimization method based on adaptive surrogate modelling

As one of the multidisciplinary design optimization methods, the collaborative optimization (CO) can enjoy a high degree of discipline autonomy. It has received extensive attention and been used in the modern engineering design process. However, the original CO is low optimization accuracy and efficiency for the design of high-dimensional nonlinearity systems because of the involved compatibility constraints. To solve this problem, an enhanced CO method based on the adaptive surrogate model is proposed in this study. The strategy of the proposed method includes the following steps. Firstly, the traditional surrogate models of system objective and performance function are constructed by the Latin hypercube sampling and the Kriging method. Then, the expected improvement function and the self-cycling optimization strategy are utilized to modify the traditional surrogate models into the corresponding adaptive surrogate models. Finally, the original objective and constraints at the system level and disciplinary level are replaced by their adaptive surrogate models, respectively. Consequently, the formulation of CO based on the adaptive surrogate model can be obtained. In the proposed method, the accuracy of the non-linear region of the response surface can be enhanced efficiently. An engineering structure design problem is introduced to show the effectiveness of the proposed method.

Multilevel Design of Sandwich Composite Armors for Blast Mitigation using Bayesian Optimization and Non-Uniform Rational B-Splines

<div class="section abstract"><div class="htmlview paragraph">In regions at war, the increasing use of improvised explosive devices (IEDs) is the main threat against military vehicles. Large cabin”s penetrations and high gross accelerations are primary threats against the occupants” survivability. The occupants” survivability under an IED event largely depends on the design of the vehicle armor. Under a blast load, a vehicle armor should maintain its structural integrity while providing low cabin penetrations and low gross accelerations. This investigation employs Bayesian global optimization (BGO) and non-uniform rational B-splines (NURBS) to design sandwich composite armors that simultaneously mitigate the cabin”s penetrations and the reaction force at the armor”s supports. The armors are made of four layers: steel, carbon fiber reinforced polymer (CFRP), aluminum honeycomb, and CFRP. BGO is a methodology to solve optimization problems that require the evaluation of expensive black-box functions such as the finite element (FE) simulations of the vehicle armor under a blast event. BGO has two main components: the surrogate model of the black-box function and the acquisition function that guides the optimization. In this study, the surrogate models are Gaussian processes and the acquisition function is the multi-objective expected improvement function. NURBS generate the armor”s shape. The numerical examples show three alternatives to optimize the armor at two levels: (1) thicknesses of the sandwich”s layers and (2) the armor”s shape. The three design alternatives differ in the number of optimized levels and the optimization approach (sequential or simultaneous). The results show that the simultaneous optimization of the thicknesses of the sandwich”s layers and the armor”s shape is the most effective approach to design vehicle armors for blast mitigation.</div></div>

HALK: A hybrid active-learning Kriging approach and its applications for structural reliability analysis

The paper presents a hybrid active-learning approach for structural reliability analysis via adaptive Kriging surrogate models. The quasi first-order reliability method is first proposed for characteristic truncation point of a structural performance function. This is used to define a truncation boundary via the joint probability distribution function of input random variables. To reduce simulation time for new training samples, a U-function based criterion is further implemented to refine the candidate sample set. Since the reliability-based expected improvement function and U functions are combined together to evaluate new training samples, it finalizes a hybrid active-learning Kriging (HALK) to develop adaptive surrogate models for the structural reliability analysis. Several numerical examples are presented to demonstrate potential applications of the proposed HALK algorithm. Compared to benchmark results provided by the brutal force Monte-Carlo simulation method, the effectiveness of the HALK approach has been justified by dealing with various structural reliability problems.

An effective Kriging-based approximation for structural reliability analysis with random and interval variables

Aleatory and epistemic uncertainties usually coexist within a mechanistic model, which motivates the hybrid structural reliability analysis considering random and interval variables in this paper. An introduction of the interval variable requires one to recursively evaluate embedded optimizations for the extremum of a performance function. The corresponding structural reliability analysis, hence, becomes a rather computationally intensive task. In this paper, physical characteristics for potential optima of the interval variable are first derived based on the Karush-Kuhn-Tucker condition, which is further programmed as a simulation procedure to pair qualified candidate samples. Then, an outer truncation boundary provided by the first-order reliability method is used to link the size of a truncation domain with the targeted failure probability, whereas the U function is acted as a refinement criterion to remove inner samples for an increased learning efficiency. Given new samples detected by the revised reliability-based expected improvement function, an adaptive Kriging surrogate model is determined to tackle the hybrid structural reliability analysis. Several numerical examples in the literature are presented to demonstrate applications of this proposed algorithm. Compared to benchmark results provided by the brute-force Monte Carlo simulation, the high accuracy and efficiency of this proposed approach have justified its potentials for the hybrid structural reliability analysis.

A novel kriging based active learning method for structural reliability analysis

In the reliability analysis of engineering structures, there are usually implict and highly nonlinear performance function problems, which leads to the time-consuming computations. In this paper, a novel Kriging based reliability analysis method combined with the improved efficient global optimization (IEGO) and a secondary point selection strategy is proposed. Based on the IEGO algorithm, the expected improvement function is redefining, which will focus on the points both with large variance and near the limit state surface. Moreover, a secondary point selection strategy is raised to find the point with larger expected improvement and closed to the limit state surface, which can further improve the efficiency of the active learning process. Five examples indicates that the raised method has satisfactory global and local search capability, and can evaluate the probability of failure efficiently.