Expensive Constrained Optimization Problems Research Articles

Two-layer surrogate-assisted collaborative framework for expensive constrained optimization problems involving mixed integer variables

Many surrogate-assisted evolutionary algorithms (SAEAs) have been shown exceptional abilities in solving expensive constrained optimization problems (ECOPs). However, few of them are designed for ECOPs involving mixed integer variables (ECOPs-MI). Therefore, a two-layer surrogate-assisted collaborative framework called TLSACF is designed, in which the radial basis function (RBF) and Gaussian Process (GP)-based mixed-integer collaborative frameworks are combined effectively. In the RBF-based collaborative framework, the combination between RBF-assisted prescreening and RBF-based local search achieves effective population update by balancing the exploration and exploitation on high potential parental information. For the GP-based collaborative framework, a variable type-based cooperative mutation is utilized to obtain potential candidate individuals by assigning targeted mutation for different variables, and a hypervolume-based complete expected improvement matrix function is derived to prescreen these candidate solutions from the perspective of balancing the optimization on constraints and the objective for the effective search on disconnected sub-feasible regions caused by integer variables. Experimental results on ten classical problems show that TLSACF performs excellent in solving ECOPs-MI.

Population state-driven surrogate-assisted differential evolution for expensive constrained optimization problems with mixed-integer variables

Many surrogate-assisted evolutionary algorithms (SAEAs) have been shown excellent search performance in solving expensive constrained optimization problems (ECOPs) with continuous variables, but few of them focus on ECOPs with mixed-integer variables (ECOPs-MI). Hence, a population state-driven surrogate-assisted differential evolution algorithm (PSSADE) is proposed for solving ECOPs-MI, in which the adaptive population update mechanism (APUM) and the collaborative framework of global and local surrogate-assisted search (CFGLS) are combined effectively. In CFGLS, a probability-driven mixed-integer mutation (PMIU) is incorporated into the classical global DE/rand/2 and local DE/best/2 for improving the diversity and potentials of candidate solutions, respectively, and the collaborative framework further integrates both the superiority of global and local mutation for the purpose of achieving a good balance between exploration and exploitation. Moreover, the current population is adaptively reselected based on the efficient non-dominated sorting technique in APUM when the population distribution is too dense. Empirical studies on 10 benchmark problems and 2 numerical engineering cases demonstrate that the PSSADE shows a more competitive performance than the existing state-of-the-art algorithms. More importantly, PSSADE provides excellent performance in the design of infrared stealth material film.

Surrogate-assisted differential evolution using manifold learning-based sampling for high- dimensional expensive constrained optimization problems

To address the challenges of high-dimensional constrained optimization problems with expensive simulation models, a Surrogate-Assisted Differential Evolution using Manifold Learning-based Sampling (SADE-MLS) is proposed. In SADE-MLS, differential evolution operators are executed to generate numerous high-dimensional candidate points. To alleviate the curse of dimensionality, a Manifold Learning-based Sampling (MLS) mechanism is developed to explore the high-dimensional design space effectively. In MLS, the intrinsic dimensionality of the candidate points is determined by a maximum likelihood estimator. Then, the candidate points are mapped into a low-dimensional space using the dimensionality reduction technique, which can avoid significant information loss during dimensionality reduction. Thus, Kriging surrogates are constructed in the low-dimensional space to predict the responses of the mapped candidate points. The candidate points with high constrained expected improvement values are selected for global exploration. Moreover, the local search process assisted by radial basis function and differential evolution is performed to exploit the design space efficiently. Several numerical benchmarks are tested to compare SADE-MLS with other algorithms. Finally, SADE-MLS is successfully applied to a solid rocket motor multidisciplinary optimization problem and a re-entry vehicle aerodynamic optimization problem, with the total impulse and lift to drag ratio being increased by 32.7% and 35.5%, respectively. The optimization results demonstrate the practicality and effectiveness of the proposed method in real engineering practices.

A Kriging-assisted Double Population Differential Evolution for Mixed-Integer Expensive Constrained Optimization Problems with Mixed Constraints

Many surrogate-assisted evolutionary algorithms (SAEA) with outstanding performance have been developed to handle Expensive Constrained Optimization Problems (ECOPs). But most of them are limited to solving ECOPs with continuous variables and inequality constraints. Therefore, a Kriging-assisted Double Population Differential Evolution (KDPDE) is proposed to deal with mixed-integer ECOPs with inequality and equality constraints. In particular, promising regions near the feasible region are created by Integer restriction Relaxation-based Double Population (IRDP) search framework, and then an Expected Improvement-based Classification local Search (EICS) is adopted to guide the infeasible solutions in the promising region into the feasible region. In order to improve the robustness of the algorithm, the widely distributed elite solutions are utilized by Elite solutions Retention-based Multi-directional Exploration (ERME) for diverse exploration, and the repetition rate information of individuals in the population is used by Population Diversity Maintenance Operation (PDMO) to adaptively avoid the population from falling into a local region. Therefore, KDPDE is capable of balancing the performance between convergence and robustness for mixed-integer ECOPS with mixed constraints. Experimental studies on several benchmark problems and a real-world application example demonstrate that KDPDE has excellent performance on solving such kind of problems under a limited computational budget.

A Hybrid Regressor and Classifier-Assisted Evolutionary Algorithm for Expensive Optimization With Incomplete Constraint Information

Surrogate-assisted evolutionary algorithms (SAEAs) have become a popular tool to solve expensive optimization problems and have been gradually used to deal with expensive constraints. To handle inequality expensive constraints, existing SAEAs need both the information of constraint violation and satisfaction to construct surrogate models for constraints. However, many problems only feedback whether the candidate solution is feasible or how much degree it violates constraints. There is no detailed information of how much degree the candidate satisfies constraints. The performance of most existing SAEAs decreases a lot in solving expensive constrained optimization problems (ECOPs) with such incomplete constraint information. To solve the problem, this article proposes a hybrid regressor and classifier-assisted evolutionary algorithm (HRCEA). HRCEA adopts a radial basis function regression model to approximate the degree of constraint violation. In order to make a more credible prediction, a logistic regression classifier (LRC) is constructed for the offspring rectification. The classifier works in cooperation with the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\alpha$</tex-math> </inline-formula> -cut strategy, in which the high confidence level can significantly improve the prediction reliability. Besides, the LRC is built based on the boundary training data selection strategy, which is devised to select samples around feasible boundaries. This strategy is helpful for the LRC to fit the local feasibility structure. Extensive experiments on commonly used benchmark functions in CEC2006 and CEC 2010 demonstrate that HRCEA has satisfactory performance in found results and execution efficiency when solving ECOPs with incomplete constraint information. Furthermore, HRCEA is utilized to solve ceramic formula design optimization problem, which shows its promising application in real-world optimization problems.

Distributed and Expensive Evolutionary Constrained Optimization With On-Demand Evaluation

Expensive optimization problems (EOPs) are common in industry and surrogate-assisted evolutionary algorithms (SAEAs) have been developed for solving them. However, many EOPs have not only expensive objective, but also expensive constraints, which are evaluated through distributed ways. We define this kind of EOPs as distributed expensive constrained optimization problems (DECOPs). The distributed characteristic of DECOPs leads to the asynchronous evaluation of both objective and constraints. Though some researchers have studied the asynchronous evaluation of objectives, the asynchronous evaluation of constraints has not gained much attention. Therefore, this paper gives a formal formulation of DECOPs and proposes a distributed evolutionary constrained optimization algorithm with on-demand evaluation (DEAOE). DEAOE can adaptively evolve different constraints in an asynchronous way through the on-demand evaluation strategy. The on-demand evaluation works from two aspects to improve the population convergence and diversity. From the aspect of individual selection, a joint sample selection strategy is adopted to determine which candidates are promising. From the aspect of constraint selection, an infeasible-first evaluation strategy is devised to judge which constraints need to be further evolved. Extensive experiments and analyses on benchmark functions and engineering problems demonstrate that DEAOE has better performance and higher efficiency compared to centralized state-of-the-art SAEAs.

A surrogate-assisted differential evolution for expensive constrained optimization problems involving mixed-integer variables

Many Surrogate-Assisted Evolutionary Algorithms (SAEAs) have been developed for expensive constrained optimization problems (ECOPs) with continuous variables. However, there exist some ECOPs that contain mixed-integer variables in real world engineering applications. The mixed-integer variables make the feasible regions of ECOPs exhibit complex disjoint characteristics, which brings two challenges for the existing methods, i.e., the classical surrogate-assisted pre-screening may easily fall into a local optimum and the population diversity tends to reduce with the influence of integer variables. For tackling these problems, a surrogate-assisted differential evolution for solving ECOPs involving mixed-integer variables (SADE-MI) is proposed. Firstly, the mutation strategy DE/best/2 is used to balance the local and global characteristics of the candidate solutions. Secondly, an adaptive pre-screening operation where the information of the population is effectively employed to construct the distance constraints is designed to avoid excessive exploitation in local regions. Finally, by selecting diversely-located individuals and limiting the number of individuals with the same integer variables, a population diversity maintenance operation is introduced to strengthen global exploration. Experimental studies on some benchmark problems and engineering optimization cases demonstrate that SADE-MI outperforms other classical algorithms for solving ECOPs with mixed-integer variables.

A general framework of surrogate-assisted evolutionary algorithms for solving computationally expensive constrained optimization problems

The objective and constraints of expensive constrained optimization problems (ECOPs) are often evaluated using simulations with different computational costs. However, the existing algorithms always assume that both the objective and constraints can be simultaneously obtained after one expensive simulation, i.e., the most time-consuming one among the evaluations, based on parallel computing. This will affect the optimization efficiency since it is not necessary to call all the simulations of objective and constraints for each solution. Therefore, a general framework of surrogate-assisted evolutionary algorithms (GF-SAEAs) is proposed to adaptively arrange search strategies based on actual simulation cost differences. Specifically, all constraints are classified into several constraint levels by effectively quantifying the computational cost differences of objective and constraints, and a level-by-level feasible region-driven local search strategy is designed to locate potential sub-feasible regions for each constraint level. Then three different search mechanisms are employed to explore and exploit these located regions. Additionally, an adaptive population regeneration strategy is utilized to restart the algorithm and prevent premature convergence. In summary, GF-SAEAs can maintain a good balance between feasibility and convergence. Experimental studies on benchmark problems and two real-world cases show that GF-SAEAs performs better than other state-of-the-art algorithms.

A kriging-assisted bi-objective constrained global optimization algorithm for expensive constrained optimization problems

Computationally expensive constrained optimization problems are challenging owing to their high complexity and computational cost. To solve these problems efficiently, a kriging-assisted bi-objective constrained global optimization (BOCGO) algorithm is developed, where three phases with three bi-objective subproblems are performed. In phase I, the constraints are searched locally and globally to find the feasible region. Once a feasible region has been located, the two terms of the constrained expected improvement function are utilized to exploit and explore the feasible region in phase II. As the kriging models are accurate enough in the concerned region, a local search is processed to improve the optimal solution in phase III. The capability of the BOCGO algorithm is demonstrated by comparison with two classical and two state-of-the-art algorithms on 20 problems and an engineering simulation problem. The results show that the BOCGO algorithm performs better in more than three-fifths of problems, illustrating its effectiveness and robustness.

An enhanced variable-fidelity optimization approach for constrained optimization problems and its parallelization

In this paper, a variable-fidelity constrained lower confidence bound (VF-CLCB) criterion is presented for computationally expensive constrained optimization problems (COPs) with two levels of fidelity. In VF-CLCB, the hierarchical Kriging model is adopted to model the objective and inequality constraints. Two infill sampling functions are developed based on the objective and the constraints, respectively, and an adaptive selection strategy is set to select the elite sample points. Moreover, based on the VF-CLCB criterion, a parallel optimization method noted as PVF-CLCB is subsequently developed to accelerate the optimization process. In PVF-CLCB, a VF influence function is defined to approximately evaluate the estimation error of the hierarchical Kriging models, based on which multiple promising points can be determined at each iteration. In addition, an allocation strategy is proposed to distribute the computation resources between the objective- and constraint-oriented functions properly. Lastly, the proposed VF-CLCB and PVF-CLCB approaches are compared with the alternative methods on 12 benchmark numerical cases, and their significant superiority in solving computationally expensive COPs is verified. Furthermore, the proposed methods are employed to optimize the global stability of the stiffened cylindrical shell, and the optimum structure is yielded.

A Surrogate-Assisted Evolutionary Algorithm for Space Component Thermal Layout Optimization

In space engineering, the electronic component layout has a very important impact on the centroid stability and heat dissipation of devices. However, the expensive thermodynamic simulations in the component thermal layout optimization problems bring great challenges to the current optimization algorithms. To reduce the cost, a surrogate-assisted evolutionary algorithm with restart strategy is proposed in this work. The algorithm is consisted of the local search and global search. A restart strategy is designed to make the local search jump out of the local optimum promptly and speed up the population convergence. The proposed algorithm is compared with two state-of-the-art algorithms on the CEC2006, CEC2010, and CEC2017 benchmark problems. The experiment results show that the proposed algorithm has a high convergence speed and excellent ability to find the optimum in the expensive constrained optimization problems under the very limited computation budget. The proposed algorithm is also applied to solve an electronic component layout optimization problem. The final results demonstrate the good performance of the proposed algorithm, which is of great significance to the component layout optimization.

Bayesian Optimization Via Barrier Functions

Hybrid optimization methods that combine statistical modeling with mathematical programming have become a popular solution for Bayesian optimization (BO) because they can better leverage both the efficient local search properties of the numerical method and the global search properties of the statistical model. These methods seek to create a sequential design strategy for efficiently optimizing expensive black-box functions when gradient information is not readily available. In this article, we propose a novel BO strategy that combines response surface modeling with barrier methods to efficiently solve expensive constrained optimization problems in computer modeling. At the heart of all BO algorithms is an acquisition function for effectively guiding the search. Our hybrid algorithm is guided by a novel acquisition function that tries to decrease the objective function as much as possible while simultaneously trying to ensure that the boundary of the constraint space is never crossed. Illustrations highlighting the success of our method are provided, including a real-world computer model optimization experiment from hydrology. Supplementary files for this article are available online.

A parallel constrained lower confidence bounding approach for computationally expensive constrained optimization problems

Computationally expensive constrained optimization problems (COPs) are very common in real-world engineering designs. This work proposes a parallel constrained lower confidence bounding (PCLCB) approach to solve challenging optimization problems with computationally expensive objective and constraints. In the PCLCB approach, a two-phase constrained lower confidence bounding (CLCB) criterion is proposed to adaptively allocate infill sample points. In the first phase where no feasible solution exists, a lower confidence bounding (LCB) function of constraints is developed to search for feasible points and improve the accuracies of the Kriging models. After a feasible point is found, the CLCB criterion focuses on local search in the second phase to improve the obtained optimum. In this phase, the search is implemented in a limited region where points may be feasible. Furthermore, an enhanced Kriging believer strategy is presented to parallelize the CLCB criterion to generate a batch of candidate points at each iteration. The PCLCB approach is tested on sixteen numerical benchmark cases and compared with the state-of-the-art alternative methods. Results indicate that the performance of PCLCB is highly competitive to the recently published algorithms in terms of efficiency and effectiveness. To verify the capability of PCLCB in solving real-world engineering problems, the lightweight design of micro-aerial vehicle (MAV) fuselage is presented. The weight of the MAV fuselage obtained by the proposed approach is on average 3.5% less than that of the other alternative methods.

A Comparative Study of Infill Sampling Criteria for Computationally Expensive Constrained Optimization Problems

Engineering optimization problems often involve computationally expensive black-box simulations of underlying physical phenomena. This paper compares the performance of four constrained optimization algorithms relying on a Gaussian process model and an infill sampling criterion under the framework of Bayesian optimization. The four infill sampling criteria include expected feasible improvement (EFI), constrained expected improvement (CEI), stepwise uncertainty reduction (SUR), and augmented Lagrangian (AL). Numerical tests were rigorously performed on a benchmark set consisting of nine constrained optimization problems with features commonly found in engineering, as well as a constrained structural engineering design optimization problem. Based upon several measures including statistical analysis, our results suggest that, overall, the EFI and CEI algorithms are significantly more efficient and robust than the other two methods, in the sense of providing the most improvement within a very limited number of objective and constraint function evaluations, and also in the number of trials for which a feasible solution could be located.

A parallel constrained efficient global optimization algorithm for expensive constrained optimization problems

ABSTRACT The Constrained Expected Improvement (CEI) criterion used in the so-called Constrained Efficient Global Optimization (C-EGO) algorithm is one of the most famous infill criteria for expensive constrained optimization problems. However, the standard CEI criterion selects only one point to evaluate in each cycle, which is time consuming when parallel computing architecture is available. This work proposes a new Parallel Constrained EGO (PC-EGO) algorithm to extend the C-EGO algorithm to parallel computing. The proposed PC-EGO algorithm is tested on sixteen analytical problems as well as one real-world engineering problem. The experiment results show that the proposed PC-EGO algorithm converges significantly faster and finds better solutions on the test problems compared to the standard C-EGO algorithm. Moreover, when compared to another state-of-the-art parallel constrained EGO algorithm, the proposed PC-EGO algorithm shows more efficient and robust performance.

Surrogate-assisted classification-collaboration differential evolution for expensive constrained optimization problems

Expensive Constrained Optimization Problems (ECOPs) widely exist in various scientific and industrial applications. Surrogate-Assisted Evolutionary Algorithms (SAEAs) have recently exhibited great ability in solving these expensive optimization problems. This paper proposes a Surrogate-Assisted Classification-Collaboration Differential Evolution (SACCDE) algorithm for ECOPs with inequality constraints. In SACCDE, the current population is classified into two subpopulations based on certain feasibility rules, and a classification-collaboration mutation operation is designed to generate multiple promising mutant solutions by not only using promising information in good solutions but also fully exploiting potential information hidden in bad solutions. Afterwards, the surrogate is utilized to identify the most promising offspring solution for accelerating the convergence speed. Furthermore, considering that the population diversity may decrease due to the excessive incorporation of greedy information brought by the classified solutions, a global search framework that can adaptively adjust the classification-collaboration mutation operation based on the iterative information is introduced for achieving an effective global search. Therefore, the proposed algorithm can strike a well balance between local and global search. The experimental results of SACCDE and other state-of-the-art algorithms demonstrate that the performance of SACCDE is highly competitive.

An adaptive multi-team perturbation-guiding Jaya algorithm for optimization and its applications

This study proposes an adaptive multi-team perturbation-guiding Jaya (AMTPG-Jaya) algorithm which uses multiple teams to explore the search space. The proposed algorithm adapts the number of teams to explore the search space based on the convergence to the optimum. Furthermore, each team uses the same set of the population, and there is a different perturbation or movement equation for each team. As each team has a different perturbation scheme, the set of the moves to new positions by each team is unique. The moving equation of the worst performing team will be updated by the superiority of solutions produced by each team. The superiority of the solutions for each team is calculated based on the fitness value and boundary violations of solutions. The proposed algorithm is examined using computationally expensive constrained optimization problems taken from the CEC-2017 technical report. Computational test results have demonstrated the effectiveness of the AMTPG-Jaya algorithm when compared to the other well-known approaches. Also, a multi-objective optimization is carried out on a solar dish Stirling engine system to find the optimal thermo-economic parameters to maximize dimensionless power and thermal efficiency. The computational results revealed that the AMTPG-Jaya algorithm results are superior to those achieved by the other algorithms presented in this work.

Global and Local Surrogate-Assisted Differential Evolution for Expensive Constrained Optimization Problems With Inequality Constraints.

For expensive constrained optimization problems (ECOPs), the computation of objective function and constraints is very time-consuming. This paper proposes a novel global and local surrogate-assisted differential evolution (DE) for solving ECOPs with inequality constraints. The proposed method consists of two main phases: 1) global surrogate-assisted phase and 2) local surrogate-assisted phase. In the global surrogate-assisted phase, DE serves as the search engine to produce multiple trial vectors. Afterward, the generalized regression neural network is used to evaluate these trial vectors. In order to select the best candidate from these trial vectors, two rules are combined. The first is the feasibility rule, which at first guides the population toward the feasible region, and then toward the optimal solution. In addition, the second rule puts more emphasis on the solution with the highest predicted uncertainty, and thus alleviates the inaccuracy of the surrogates. In the local surrogate-assisted phase, the interior point method coupled with radial basis function is utilized to refine each individual in the population. During the evolution, the global surrogate-assisted phase has the capability to promptly locate the promising region and the local surrogate-assisted phase is able to speed up the convergence. Therefore, by combining these two important elements, the number of fitness evaluations can be reduced remarkably. The proposed method has been tested on numerous benchmark test functions from three test suites and two real-world cases. The experimental results demonstrate that the performance of the proposed method is better than that of other state-of-the-art methods.

Constrained global optimization via a DIRECT-type constraint-handling technique and an adaptive metamodeling strategy

Recent metamodel-based global optimization algorithms are very promising for box-constrained expensive optimization problems. However, few of them can tackle constrained optimization problems. This article presents an improved constrained optimization algorithm, called eDIRECT-C, for expensive constrained optimization problems. In the eDIRECT-C algorithm, we present a novel DIRECT-type constraint-handling technique that separately handles feasible and infeasible cells. This technique has no user-defined parameter and is beneficial for exploring the undetected feasible regions and boundary of feasible regions. We also employ an adaptive metamodeling strategy to build appropriate metamodel types for objective and constraints respectively. This strategy yields more accurate predictions and therefore significantly speeds up the convergence. To assess the performance of eDIRECT-C, we compare it with some state-of-the-art metamodel-based constrained optimization algorithms and the original DIRECT algorithm on 13 benchmark problems and 4 engineering examples. The comparative results imply that the proposed algorithm is very promising for constrained problems in terms of the convergence speed, quality of final solutions and success rate.

Sequential RBF surrogate-based efficient optimization method for engineering design problems with expensive black-box functions

As a promising technique, surrogate-based design and optimization(SBDO) has been widely used in modern engineering design optimizations. Currently, static surrogate-based optimization methods have been successfully applied to expensive optimization problems. However, due to the low efficiency and poor flexibility, static surrogate-based optimization methods are difficult to efficiently solve practical engineering cases. At the aim of enhancing efficiency, a novel surrogate-based efficient optimization method is developed by using sequential radial basis function(SEO-SRBF). Moreover, augmented Lagrangian multiplier method is adopted to solve the problems involving expensive constraints. In order to study the performance of SEO-SRBF, several numerical benchmark functions and engineering problems are solved by SEO-SRBF and other well-known surrogate-based optimization methods including EGO, MPS, and IARSM. The optimal solutions, number of function evaluations, and algorithm execution time are recorded for comparison. The comparison results demonstrate that SEO-SRBF shows satisfactory performance in both optimization efficiency and global convergence capability. The CPU time required for running SEO-SRBF is dramatically less than that of other algorithms. In the torque arm optimization case using FEA simulation, SEO-SRBF further reduces 21% of the material volume compared with the solution from static-RBF subject to the stress constraint. This study provides the efficient strategy to solve expensive constrained optimization problems.