Expensive Problems Research Articles

A Surrogate-Assisted Differential Evolution With Knowledge Transfer for Expensive Incremental Optimization Problems

A Surrogate-Assisted Differential Evolution With Knowledge Transfer for Expensive Incremental Optimization Problems

A parallel variable-fidelity algorithm for efficient constrained multi-objective aerodynamic design optimization

Modern aerodynamic design optimization aims to discover optimal configurations using computational fluid dynamics under complex flow conditions, which is a typical expensive multi-objective optimization problem. The multi-objective evolutionary algorithm based on decomposition (MOEA/D) combined with efficient global optimization is a promising method but requires enhanced efficiency and faces limitations in its application to multi-objective aerodynamic design optimization (MOADO). To address the issues, an efficient parallel MOEA/D assisted with variable-fidelity optimization (VFO) is proposed for solving MOADO, called the MOEA/D-VFO algorithm. Variable-fidelity surrogates are built for objectives and constraints, achieving higher accuracy using fewer high-fidelity samples and a great number of low-fidelity samples. By retaining more good candidates, the sub-optimization problems defined by decomposing original objectives are capable of discovering more favorable samples using MOEA/D, which prompts optimization convergence. A constraint-handling strategy is developed by incorporating the probability of feasibility functions in the sub-optimizations. The selection of new samples for parallel evaluation is improved by filtering out poor candidates and selecting effective promising samples, which improves the feasibility and diversity of solved Pareto solutions. A Pareto front (PF) can be efficiently found in a single optimization run. The proposed approach is demonstrated by four analytical test functions and verified by two aerodynamic design optimizations of airfoils with and without constraints, respectively. The results indicate that the MOEA/D-VFO approach can greatly improve optimization efficiency and obtain the PF satisfying constraints within an affordable computational budget.

Formulating approximation error as noise in surrogate-assisted multi-objective evolutionary algorithm

Many real multi-objective optimization problems with 20-50 decision variables often have only a small number of function evaluations available, because of their heavy time/money burden. Therefore, surrogate models are often utilized as alternatives for expensive function evaluations. However, the approximation error of the surrogate model is inevitable compared to the real function evaluation. The approximation error has a similar impact on the algorithm as noise, i.e., different optimization stages suffer from various impacts. Therefore, the current optimization stage can be indirectly detected via measuring the impact of the noise formulated by the approximation error. In addition, the rising dimension of the search space leads to an increase in the approximation errors of the surrogate models, which poses a huge challenge for existing surrogate-assisted multi-objective evolutionary algorithms. In this work, we propose a stage-adaptive surrogate-assisted multi-objective evolutionary algorithm to solve the medium-scale optimization problems. In the proposed algorithm, the ensemble model consisting of the latest and historical models is used as the surrogate model, on the basis of which a set of potential candidates can be discovered. Then, a stage-adaptive infill sampling strategy selects the most suitable sampling strategy by analyzing the demand of the current optimization stage on convergence, diversity, model accuracy to sample from the candidates. As for the current optimization stage, it is detected by a noise impact indicator, where the approximation errors of surrogate models are formulated as noise. The experimental results on a series of medium-scale expensive test problems demonstrate the superiority of the proposed algorithm over six state-of-the-art compared algorithms.

A Q-learning driven competitive surrogate assisted evolutionary optimizer with multiple oriented mutation operators for expensive problems

Surrogate-assisted evolutionary algorithms (SAEAs) prevail in the solution of computationally expensive optimization problems. However, with the growth of problem scale and complexity, the high-dimensional problem space greatly restricts their effectiveness and applicability. This paper proposes a Q-learning driven competitive surrogate-assisted evolutionary optimizer with multiple oriented mutation operators (CSEO-MOMO) to improve the prediction effectiveness and accuracy of SAEAs in different scenarios of optimization problems. CSEO-MOMO constructs isomorphic models with different complexities during iteration. The Q-learning method is employed to pick the proper model for prediction based on the state feedback of the iterative population. To explore the search space more comprehensively and increase the probability of finding a better solution, two competitive sub-populations coevolve with personalized mutation operators, of which different oriented optimal tractive base vectors are nominated to guide the mutation direction. In addition, a tailored infilling strategy concerning both the performance of the iterative individuals and their spatial distribution is used to winnow out the individuals with great potential for real evaluation. A suit of high-dimensional CEC benchmarks of different properties and an openly shared topology optimization problem are used for the effectiveness and efficiency verification of CSEO-MOMO. The experimental results reveal that CSEO-MOMO trumps seven excellent SAEAs in solving high-dimensional complex problems and possesses better accuracy and robustness under various fitness landscape scenarios. (The relevant MATLAB code of CSEO-MOMO is publicly available in the first author’s GitHub repository: https://github.com/qinna-zhu/CSEO-MOMO.)

Association of Early Dexmedetomidine Utilization With Clinical Outcomes After Moderate-Severe Traumatic Brain Injury: A Retrospective Cohort Study.

Traumatic brain injury (TBI) is an expensive and common public health problem. Management of TBI oftentimes includes sedation to facilitate mechanical ventilation (MV) for airway protection. Dexmedetomidine has emerged as a potential candidate for improved patient outcomes when used for early sedation after TBI due to its potential modulation of autonomic dysfunction. We examined early sedation patterns, as well as the association of dexmedetomidine exposure with clinical and functional outcomes among mechanically ventilated patients with moderate-severe TBI (msTBI) in the United States. We conducted a retrospective cohort study using data from the Premier dataset and identified a cohort of critically ill adult patients with msTBI who required MV from January 2016 to June 2020. msTBI was defined by head-neck abbreviated injury scale (AIS) values of 3 (serious), 4 (severe), and 5 (critical). We described early continuous sedative utilization patterns. Using propensity-matched models, we examined the association of early dexmedetomidine exposure (within 2 days of intensive care unit [ICU] admission) with the primary outcome of hospital mortality and the following secondary outcomes: hospital length of stay (LOS), days on MV, vasopressor use after the first 2 days of admission, hemodialysis (HD) after the first 2 days of admission, hospital costs, and discharge disposition. All medications, treatments, and procedures were identified using date-stamped hospital charge codes. The study population included 19,751 subjects who required MV within 2 days of ICU admission. The patients were majority male and white. From 2016 to 2020, the annual percent utilization of dexmedetomidine increased from 4.05% to 8.60%. After propensity score matching, early dexmedetomidine exposure was associated with reduced odds of hospital mortality (odds ratio [OR], 0.59; 95% confidence interval [CI], 0.47-0.74; P < .0001), increased risk for liberation from MV (hazard ratio [HR], 1.20; 95% CI, 1.09-1.33; P = .0003), and reduced LOS (HR, 1.11; 95% CI, 1.01-1.22; P = .033). Exposure to early dexmedetomidine was not associated with odds of HD (OR, 1.14; 95% CI, 0.73-1.78; P = .56), vasopressor utilization (OR, 1.10; 95% CI, 0.78-1.55; P = .60), or increased hospital costs (relative cost ratio, 1.98; 95% CI, 0.93-1.03; P = .66). Dexmedetomidine is being utilized increasingly as a sedative for mechanically ventilated patients with msTBI. Early dexmedetomidine exposure may lead to improved patient outcomes in this population.

Surrogate-assisted evolutionary framework with an ensemble of teaching-learning and differential evolution for expensive optimization

Surrogate-Assisted Evolutionary Algorithms (SAEAs) integrate Evolutionary Algorithms (EAs) with surrogate models to reduce the actual number of expensive function evaluations and have been widely used in solving Expensive Optimization Problems (EOPs). However, because of the insufficient diversity of new solutions, SAEAs often fall into local optima and result in evolutionary stagnation when dealing with complex problems. To improve the diversity of new solutions, this paper introduces a Surrogate-Assisted evolutionary Framework with an ensemble of Teaching-learning and Differential evolution (SAF-TD), which successfully integrates the strengths of both EAs to produce more diverse and effective solutions. The main contributions are summarized as follows: First, a novel framework that effectively integrates two distinct EAs using a radial basis function surrogate model and strategic sampling is introduced. Second, a mutation strategy with a supervisory mechanism is proposed to enhance mutation effectiveness and avoid stagnation. Third, a volatility index derived from dimensional improvements is incorporated into parameter control to address fitness-dependent weaknesses. Experiments on five expensive optimization functions and the CEC2017 test suite across multiple dimensions were conducted to evaluate the performance of SAF-TD. The results demonstrate that SAF-TD is highly competitive compared with state-of-the-art SAEAs and exhibits excellent performance in solving EOPs.

An optimization algorithm combining local exploitation and global exploration for computationally expensive problems

An optimization algorithm combining local exploitation and global exploration for computationally expensive problems

A surrogate-assisted evolutionary algorithm with dual restricted Boltzmann machines and reinforcement learning-based adaptive strategy selection

To improve the effectiveness of surrogate-assisted evolutionary algorithms (SAEAs) in solving high-dimensional expensive optimization problems with multi-polar and multi-variable coupling properties, a new approach called DRBM-ASRL is proposed. This approach leverages restricted Boltzmann machines (RBMs) for feature learning and reinforcement learning for adaptive strategy selection. DRBM-ASRL integrates four search strategies based on three heterogeneous surrogate modeling approaches, each catering to different preferences. Two of these strategies focus on generative sampling in the subspaces with varying dimensions, while the other two aim to explore the local and global landscapes in the high-dimensional source space. This allows for more effective tradeoffs between exploration and exploitation in the solution space. Reinforcement learning is employed to adaptively prioritize the search strategies during optimization , based on the online feedback information from the optimal solution. In addition, to enhance the representation of potentially optimal samples in the solution space, two task-driven RBMs are separately trained to construct a feature subspace and reconstruct the features of the source space. DRBM-ASRL has been evaluated on various high-dimensional benchmarks ranging from 50 to 200 dimensions, as well as 14 CEC 2013 complex benchmark problems with 100 dimensions and a power system problem with 118 dimensions. Experimental results demonstrate its superior convergence performance and optimization efficiency compared to eight state-of-the-art SAEAs.

A data-driven co-evolutionary exploration algorithm for computationally expensive constrained multi-objective problems

Surrogate-assisted multi-objective optimization algorithms have attracted widespread attention due to their outstanding performance in computationally expensive real-world problems. However, there is relatively little research about multi-objective optimization with complex and expensive constraints. Hence, a data-driven co-evolutionary exploration (DDCEE) algorithm is presented in this paper for the above-mentioned problems, where Radial Basis Functions are utilized to train dynamically updated surrogate models for each objective and constraint. Specifically, a data-driven co-evolutionary exploration framework is proposed to fully utilize and mine the potential available information of RBF models, and RBF models are constantly updated to guide co-evolutionary in discovering valuable feasible regions and achieving global optimization. In co-evolutionary exploration, one population focuses on exploring the entire space without considering constraints, while the other population focuses on exploring feasible regions and collaborating by sharing their respective offspring. Reference vectors are introduced in co-evolutionary exploration to divide the objective space into several sub-regions for further selection. Furthermore, an adaptive selection of promising samples strategy is presented to reasonably utilize the information of solutions with good convergence and enhance the convergence and diversity of the Pareto front. After comprehensive experiments on constrained multi/many-objective benchmark cases and an engineering application problem, DDCEE shows more stable and impressive performance when compared with five state-of-art algorithms.

Rational Kriging

This article proposes a new kriging that has a rational form. It is shown that the generalized least squares estimator of the mean from rational kriging is much more well behaved than that of ordinary kriging. Parameter estimation and uncertainty quantification for rational kriging are proposed using a Gaussian process framework. A generalized version of rational kriging is also proposed, which includes ordinary and rational kriging as special cases. Extensive simulations carried out over a wide class of functions show that the generalized rational kriging performs on par or better than both ordinary and rational kriging in terms of prediction and uncertainty quantification. The only extra step needed for generalized rational kriging over ordinary kriging is the computation of Perron eigenvector of an augmented correlation matrix which can be computed in near linear time and therefore, its overall computational complexity is no more than that of ordinary kriging. The potential applications of the new kriging methods in the emulation of computationally expensive models and model calibration problems are illustrated with real and simulated examples. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

A multi-strategy surrogate-assisted social learning particle swarm optimization for expensive optimization and applications

Evolutionary algorithms (EAs) require extensive fitness evaluations, which constitutes a barrier to solving computationally complex problems. In contrast, surrogate-assisted evolutionary algorithms (SAEAs) have the potential to solve complex expensive optimization problems. This paper proposes a surrogate-assisted social learning particle swarm optimization (SASLPSO) to handle expensive optimization problems. An adaptive local surrogate (ALS) strategy introduced in SASLPSO is introduced to accurately fit the landscape near the global optimum. ALS consists of two layers of surrogate, the part of the particles closest to the optimal particle and the closest to the optimal particle among the particles that have been eliminated historically. The proposed SASLPSO effectively combines global surrogate (GS) and adaptive local surrogate (ALS) to balance global exploration and local exploitation. Furthermore, a novel random group-based pre-screening (RGBPS) strategy is proposed to screen promising particles for real function evaluation. The proposed SASLPSO is compared with four other state-of-the-art SAEAs on 30D, 50D, and 100D benchmark functions. In addition, the significance of the SASLPSO algorithm was also verified using the Wilcoxon rank test. The test results on the benchmark function show that the SASLPSO algorithm performs better than other comparison algorithms, especially when dealing with high-dimensional benchmark functions. To further validate the effectiveness of the SASLPSO algorithm in solving expensive optimization problems, it was also applied to feature selection problems and real-world engineering problems. In addition, in the real application of node deployment in 3D wireless sensor networks, the highest coverage rate of the SASLPSO algorithm can reach 99.96%, confirming its performance advantages in solving real application problems. Finally, in the application of network intrusion detection, SASLPSO has shown more advantages in multiple metrics, proving its versatility.

An ensemble dual model assisted MOEA/D for tackling medium scale expensive multiobjective optimization

Many surrogate-assisted evolutionary algorithms (SAEAs) have been developed to address expensive multi-objective optimization problems (EMOPs). However, existing research primarily focuses on low-dimensional EMOPs. In this article, we propose an ensemble dual model-assisted multi-objective evolutionary algorithm based on decomposition to tackle medium-scale EMOPs. The proposed approach includes two key insights. First, a new ensemble strategy is proposed to improve both the prediction ability and computation efficiency of surrogates in tackling medium-scale EMOPs. Second, we develop an improved computing resource allocation strategy and an operator pool to enhance convergence capabilities. The improved computing resource allocation strategy gives more computing resources to the subproblems with poor fitness values, while the operator pool is used to generate promising offspring set in higher dimensions. Experimental results from three sets of expensive multi-objective test suites demonstrate that our proposed algorithm significantly outperforms seven compared SAEAs.

Three-Dimensional-Scanning of Pipe Inner Walls Based on Line Laser.

In this study, an innovative laser 3D-scanning technology is proposed to scan pipe inner walls in order to solve the problems of the exorbitant expenses and operational complexities of the current equipment for the 3D data acquisition of the pipe inner wall, and the difficulty of both the efficiency and accuracy of traditional light stripe-center extraction methods. The core of this technology is the monocular-structured light 3D scanner, the image processing strategy based on tracking speckles, and the improved gray barycenter method. The experimental results demonstrate a 52% reduction in the average standard error of the improved gray barycenter method when compared to the traditional gray barycenter method, along with an 83% decrease in the operation time when compared to the Steger method. In addition, the size data of the inner wall of the pipe obtained using this technology is accurate, and the average deviation of the inner diameter and length of the pipe is less than 0.13 mm and 0.41 mm, respectively. In general, it not only reduces the cost, but also ensures high efficiency and high precision, providing a new and efficient method for the 3D data acquisition of the inner wall of the pipe.

Accelerating surrogate assisted evolutionary algorithms for expensive multi-objective optimization via explainable machine learning

A series of surrogate-assisted evolutionary algorithms (SAEAs) have been proposed to handle expensive multi-objective optimization problems (EMOPs). However, the surrogate of these SAEAs is underutilized to a large extent, which limits the search efficiency of these algorithms. To be specific, existing algorithms do not sufficiently exploit the estimated solution quality information from the surrogate models during offspring generation. To address this issue, this paper proposes an SAEA framework named EXO-SAEA (EXplanation Operator based Surrogate-Assisted Evolutionary Algorithm). First, it divides the current population into two populations according to the a priori knowledge from the surrogate model. Then, for each solution in the first population, EXO-SAEA employs the SHapley Additive exPlanations (SHAP) model to estimate the contribution of each decision variable to the fitness values. After that, the Shapley values are then normalized for the offspring generation of the first population, while the second population uses generic GA operators. Two representative surrogate-assisted evolutionary algorithms are used to instantiate the proposed framework. Experimental results on the synthetic benchmark problems and three real-world problems involving six state-of-the-art algorithms demonstrate the effectiveness of the proposed framework.

Population state-driven surrogate-assisted differential evolution for expensive constrained optimization problems with mixed-integer variables

Many surrogate-assisted evolutionary algorithms (SAEAs) have been shown excellent search performance in solving expensive constrained optimization problems (ECOPs) with continuous variables, but few of them focus on ECOPs with mixed-integer variables (ECOPs-MI). Hence, a population state-driven surrogate-assisted differential evolution algorithm (PSSADE) is proposed for solving ECOPs-MI, in which the adaptive population update mechanism (APUM) and the collaborative framework of global and local surrogate-assisted search (CFGLS) are combined effectively. In CFGLS, a probability-driven mixed-integer mutation (PMIU) is incorporated into the classical global DE/rand/2 and local DE/best/2 for improving the diversity and potentials of candidate solutions, respectively, and the collaborative framework further integrates both the superiority of global and local mutation for the purpose of achieving a good balance between exploration and exploitation. Moreover, the current population is adaptively reselected based on the efficient non-dominated sorting technique in APUM when the population distribution is too dense. Empirical studies on 10 benchmark problems and 2 numerical engineering cases demonstrate that the PSSADE shows a more competitive performance than the existing state-of-the-art algorithms. More importantly, PSSADE provides excellent performance in the design of infrared stealth material film.

Surrogate-assisted evolutionary algorithms for expensive combinatorial optimization: a survey

As potent approaches for addressing computationally expensive optimization problems, surrogate-assisted evolutionary algorithms (SAEAs) have garnered increasing attention. Prevailing endeavors in evolutionary computation predominantly concentrate on expensive continuous optimization problems, with a notable scarcity of investigations directed toward expensive combinatorial optimization problems (ECOPs). Nevertheless, numerous ECOPs persist in practical applications. The widespread prevalence of such problems starkly contrasts the limited development of relevant research. Motivated by this disparity, this paper conducts a comprehensive survey on SAEAs tailored to address ECOPs. This survey comprises two primary segments. The first segment synthesizes prevalent global, local, hybrid, and learning search strategies, elucidating their respective strengths and weaknesses. Subsequently, the second segment furnishes an overview of surrogate-based evaluation technologies, delving into three pivotal facets: model selection, construction, and management. The paper also discusses several potential future directions for SAEAs with a focus towards expensive combinatorial optimization.

Expert knowledge data-driven based actor–critic reinforcement learning framework to solve computationally expensive unit commitment problems with uncertain wind energy

With the expansion of power grid, unaffordable computational cost and time will pose serious challenges of time-efficient scheduling in unit commitment problem (UCP). However, existing optimization methods, i.e., mathematical programming methods and meta-heuristic algorithms, are powerless and time-consuming to handle computationally expensive UCP (CEUCP). Thus, reinforcement learning methods with strong inference and time-saving performances are motivated to solve the computationally expensive challenges in tackling CEUCPs. In this paper, a novel expert knowledge data-driven based actor–critic (AC) reinforcement learning methodology is proposed for solving CEUCPs. Specifically, in the proposed AC reinforcement learning methodology, expert knowledge, data-driven surrogate model, and improved meta-heuristic algorithm are integrated for further performance enhancement. Firstly, a novel action selection mechanism (based on the expert knowledge of thermal units characteristic) is integrated into AC to improve the efficiency of network training. Secondly, an improved extreme learning machine (ELM) data-driven surrogate model is proposed to build reward function in AC. In detail, original cost function in reward is replaced by a lightweight ELM model. Shape distance is integrated into ELM for enhancing accuracy. Finally, original marine predators algorithm (MPA) is improved for obtaining optimal dispatching decisions and rewards of AC method quickly and correctly. Original search pattern is replaced by quantum based representation for boosting convergence. The excellent performances of the proposed AC framework are verified by simulations of 10-units, 100-units, and 100-units with wind energy test systems.

Enhancing surrogate-assisted evolutionary optimization for medium-scale expensive problems: a two-stage approach with unsupervised feature learning and Q-learning

Enhancing surrogate-assisted evolutionary optimization for medium-scale expensive problems: a two-stage approach with unsupervised feature learning and Q-learning

Surrogate-assisted multi-objective evolutionary feature selection of generation-based fixed evolution control for time series forecasting with LSTM networks

Surrogate-assisted multi-objective evolutionary algorithms are powerful techniques to solve computationally expensive multi-objective optimization problems. In this paper, we propose a direct fitness replacement method with generation-based fixed evolution control to implement a multi-objective evolutionary algorithm that uses a surrogate model for wrapper-type feature selection, where long short-term memory is established as the learning algorithm. The importance of the work and its benefits lie in the need to reduce the excessive computational time required by conventional wrapper-type feature selection methods based on multi-objective evolutionary algorithms and LSTM networks, maintaining or improving the predictive capacity of the models. We analyze the use of incremental learning to update the surrogate model, in comparison with the conventional non-incremental learning approach. We applied these methods in real-life time series forecasting of air quality, indoor temperature in a smart building and oil temperature in electricity transformers. Multi-step ahead predictions of the forecast models obtained with different meta-learners of the surrogate model were compared by using the Diebold–Mariano statistical test on a multi-criteria performance metric. The proposed method outperformed other approaches for feature selection including, among others, methods based on surrogate-assisted multi-objective evolutionary algorithms developed by the authors in previous research, other surrogate-assisted deterministic methods for feature selection and the conventional wrapper-type feature selection method based on LSTM, improving the prediction on test dataset by 23.98%, 34.61% and 13.77%, respectively.

Surrogate information transfer and fusion in high-dimensional expensive optimization problems

The Kriging surrogate model is less frequently employed in high-dimensional expensive problems than is the radial basis function (RBF) model. This discrepancy is attributed to the challenge of hyperparameter tuning within the Kriging covariance function, which leads to relatively worse predictive performance. However, the Kriging model still plays a crucial role in providing the predicted uncertainty for new point infill that cannot be replaced by the RBF model. To leverage the advantages of both models, a surrogate information transfer and fusion algorithm is presented. Surrogate information transfer introduces global sensitivity information from the constructed RBF model to the hyperparameter optimization of the Kriging model, reducing the dimensions of hyperparameter tuning and improving its approximation performance. Surrogate information fusion combines the RBF and Kriging models, with the RBF model providing more accurate predictions of unknown points for exploitation, while the Kriging model provides predicted uncertainty for swarm updating and new point infilling to ensure exploration of the solution space. Compared with several state-of-the-art algorithms, the proposed algorithm is evaluated on eight benchmark problems and a in real-world optimization case. The experimental results demonstrate the effectiveness of the surrogate information transfer and fusion methods and the significant superiority of the proposed algorithm over the compared algorithms.