Small Number Of Function Evaluations Research Articles

Model Identification of E. coli Cultivation Process Applying Hybrid Crow Search Algorithm

Cultivation process (CP) modeling and optimization are ambitious tasks due to the nonlinear nature of the models and interdependent parameters. The identification procedures for such models are challenging. Metaheuristic algorithms exhibit promising performance for such complex problems since a near-optimal solution can be found in an acceptable time. The present research explores a new hybrid metaheuristic algorithm built upon the good exploration of the genetic algorithm (GA) and the exploitation of the crow search algorithm (CSA). The efficiency of the proposed GA-CSA hybrid is studied with the model parameter identification procedure of the E. coli BL21(DE3)pPhyt109 fed-batch cultivation process. The results are compared with those of the pure GA and pure CSA applied to the same problem. A comparison with two deterministic algorithms, i.e., sequential quadratic programming (SQP) and the Quasi-Newton (Q-N) method, is also provided. A more accurate model is obtained by the GA-CSA hybrid with fewer computational resources. Although SQP and Q-N find a solution for a smaller number of function evaluations, the resulting models are not as accurate as the models generated by the three metaheuristic algorithms. The InterCriteria analysis, a mathematical approach to revealing certain relations between given criteria, and a series of statistical tests are employed to prove that there is a statistically significant difference between the results of the three stochastic algorithms. The obtained mathematical models are then successfully verified with a different set of experimental data, in which, again, the closest one is the GA-CSA model. The GA-CSA hybrid proposed in this paper is proven to be successful in the collaborative hybridization of GA and CSA with outstanding performance.

An adaptive Gaussian process based manifold transfer learning to expensive dynamic multi-objective optimization

Expensive dynamic multi-objective optimization problems (EDMOPs) is one kind of DMOPs where the objectives change over time and the function evaluations commonly involve computationally intensive simulations or costly physical experiments. Hence, the key to solve EDMOPs is to quickly and accurately track the time-varying Pareto optimal fronts under the limit of small number of function evaluations, in which how to augment enough training data to build informative surrogate models and manage the models during the search process. To overcome the issue, we propose a transfer learning based surrogate assisted evolutionary algorithm (TrSA-DMOEA) to efficiently solve EDMOPs. Specifically, when a change occurs, we propose a knee point-based manifold transfer learning method based on geodesic flow kernel, which exploits the knowledge from previous high-quality knee solutions to augment the training data for building Gaussian process models, thereby improving the computational complexity and the quality of solutions. Moreover, to efficiently find the optima with limited budget of function evaluations, a novel surrogate-assisted mechanism based on an adaptive acquisition function is introduced, which achieves a balance between convergence and diversity by adaptively adjusting the weights of the angle-penalized distance and average uncertainty at different search stages. By comparing with state-of-the-art algorithms on widely used test problems, the experimental results demonstrate that the proposed method outperforms others and is able to efficiently solve EDMOPs.

A data augmentation based Kriging-assisted reference vector guided evolutionary algorithm for expensive dynamic multi-objective optimization

Expensive dynamic multi-objective optimization problems (EXDMOPs) involve multiple objective functions changing over time steps. In this kind of problems, only a small number of function evaluations can be allowed in each time step. The challenge of EXDMOP is how to quickly and accurately track the changing optimal solutions with only a small number of function evaluations. Although EXDMOP can be solved as a series of independent expensive multi-objective optimization problems or as an ordinary dynamic multi-objective optimization problems, these two methods either ignore the connection between different time steps or degrade the performance significantly due to the requirement for a large number of function evaluations. In this paper, we incorporate two proposed strategies named training data augmentation and sampling pool expansion to the Kriging-assisted reference vector guided evolutionary algorithm to construct a novel algorithm for solving EXDMOPs. The training data augmentation strategy tries to look for the previous similar time step, and adds the solutions evaluated by the objective functions at that time step to current time step. The accuracies of the Kriging models are improved in this way. The sampling pool expansion strategy sends the predicted optimal solutions to the competition of the function evaluations assignment at each time step, where the predicted optimal solutions are generated by the improved population prediction strategy. We set the ablation experiment for the proposed algorithm and compare it with four dynamic multi-objective evolutionary algorithms on various of problems. The experiment results validate the effectiveness of the proposed two strategies. It also indicates that the proposed algorithm is promising for solving EXDMOPs.

Expensive Multiobjective Evolutionary Optimization Assisted by Dominance Prediction

We propose a new surrogate-assisted evolutionary algorithm for expensive multiobjective optimization. Two classification-based surrogate models are used, which can predict the Pareto dominance relation and <inline-formula> <tex-math notation="LaTeX">$\theta $ </tex-math></inline-formula>-dominance relation between two solutions, respectively. To make such surrogates as accurate as possible, we formulate dominance prediction as an imbalanced classification problem and address this problem using deep learning techniques. Furthermore, to integrate the surrogates based on dominance prediction with multiobjective evolutionary optimization, we develop a two-stage preselection strategy. This strategy aims to select a promising solution to be evaluated among those produced by genetic operations, taking proper account of the balance between convergence and diversity. We conduct an empirical study on a number of well-known multiobjective and many-objective benchmark problems, over a relatively small number of function evaluations. Our experimental results demonstrate the superiority of the proposed algorithm compared with several representative surrogate-assisted algorithms.

A Kriging-Assisted Two-Archive Evolutionary Algorithm for Expensive Many-Objective Optimization

Only a small number of function evaluations can be afforded in many real-world multiobjective optimization problems (MOPs) where the function evaluations are economically/computationally expensive. Such problems pose great challenges to most existing multiobjective evolutionary algorithms (EAs), which require a large number of function evaluations for optimization. Surrogate-assisted EAs (SAEAs) have been employed to solve expensive MOPs. Specifically, a certain number of expensive function evaluations are used to build computationally cheap surrogate models for assisting the optimization process without conducting expensive function evaluations. The infill sampling criteria in most existing SAEAs take all requirements on convergence, diversity, and model uncertainty into account, which is, however, not the most efficient in exploiting the limited computational budget. Thus, this article proposes a Kriging-assisted two-archive EA for expensive many-objective optimization. The proposed algorithm uses one influential point-insensitive model to approximate each objective function. Moreover, an adaptive infill criterion that identifies the most important requirement on convergence, diversity, or uncertainty is proposed to determine an appropriate sampling strategy for reevaluations using the expensive objective functions. The experimental results on a set of expensive multi/many-objective test problems have demonstrated its superiority over five state-of-the-art SAEAs.

Iterated Local Search and Other Algorithms for Buffered Two-Machine Permutation Flow Shops with Constant Processing Times on One Machine.

The two-machine permutation flow shop scheduling problem with buffer is studied for the special case that all processing times on one of the two machines are equal to a constant c. This case is interesting because it occurs in various applications, for example, when one machine is a packing machine or when materials have to be transported. Different types of buffers and buffer usage are considered. It is shown that all considered buffer flow shop problems remain NP-hard for the makespan criterion even with the restriction to equal processing times on one machine. However, the special case where the constant c is larger or smaller than all processing times on the other machine is shown to be polynomially solvable by presenting an algorithm (2BF-OPT) that calculates optimal schedules in O(nlogn) steps. Two heuristics for solving the NP-hard flow shop problems are proposed: (i) a modification of the commonly used NEH heuristic (mNEH) and (ii) an Iterated Local Search heuristic (2BF-ILS) that uses the mNEH heuristic for computing its initial solution. It is shown experimentally that the proposed 2BF-ILS heuristic obtains better results than two state-of-the-art algorithms for buffered flow shop problems from the literature and an Ant Colony Optimization algorithm. In addition, it is shown experimentally that 2BF-ILS obtains the same solution quality as the standard NEH heuristic, however, with a smaller number of function evaluations.

New Adaptive Surrogate-Based Approach Combined Swarm Optimizer Assisted Less Tuning Cost of Dynamic Production-Inventory Control System

Today, most manufacturing control systems are complex and expensive, so they are limited to employ a small number of function evaluations for optimal design. Yet, looking for optimization methods with the less-computational cost is an open issue in engineering control systems. This paper aims to propose an effective adaptive optimization approach by integrating Kriging surrogate and Particle Swarm Optimization (PSO). In this method, a novel iterative adaptive approach is utilized using two sets of training samples including initial training and adaptive sample points. The initial training points are designed by space-filling design, while the adaptive points are generated using a new jackknife resampling approach. The proposed approach can effectively convergence towards the global optimal point using a small number of function evaluations. The efficiency and applicability of the proposed algorithm are evaluated using the optimal design of the fractional-order PID (FOPID) controller for some benchmark transfer functions. Then, the introduced approach is applied for tuning the parameters and the sensitivity analysis of the FOPID controller for a dynamic production-inventory control system. The results are in good agreement with the results reported in the literature, while the proposed approach is executed with a lower computational burden.

Verification of FPA and PSO algorithms for rule curve extraction and optimization of single- and multi-reservoir systems' operations considering their specific purposes

Abstract The optimal operation of reservoirs is known as a complex issue in water resources management, which requires consideration of numerous variables (such as downstream water demand and power generation). For this optimization, researchers have used evolutionary and meta-heuristic algorithms, which are generally inspired by nature. These algorithms have been developed to achieve optimal/near-optimal solutions by a smaller number of function evaluations with less calculation time. In this research, the flower pollination algorithm (FPA) was used to optimize: (1) Aidoghmoush single-reservoir system operation for agricultural water supply, (2) Bazoft single-reservoir system operation for hydropower generation, (3) multi-reservoir system operation of Karun 5, Karun 4, and Bazoft, and (4) Bazoft single-reservoir system for rule curve extraction. To demonstrate the effectiveness of the FPA, it was first applied to solve the mathematical test functions, and then used to determine optimal operations of the reservoir systems with the purposes of downstream water supply and hydropower generation. In addition, the FPA was compared with the particle swarm optimization (PSO) algorithm and the non-linear programming (NLP) method. The results for the Aidoghmoush single-reservoir system showed that the best FPA solution was similar to the NLP solution, while the best PSO solution was about 0.2% different from the NLP solution. The best values of the objective function of the PSO were approximately 3.5 times, 28%, and 43% worse than those of the FPA for the Bazoft single-reservoir system for hydropower generation, the multi-reservoir system, and the Bazoft single-reservoir system for rule curve extraction, respectively. The FPA outperformed the PSO in finding the optimal solutions. Overall, FPA is one of the new evolutionary algorithms, which is capable of determining better (closer to the ideal solution) objective functions, decreasing the calculation time, simplifying the problem, and providing better solutions for decision makers.

Diversity driven multi-parent evolutionary algorithm with adaptive non-uniform mutation

ABSTRACT Any evolutionary algorithm tends to end up in a local optimum. A new approach based on an evolutionary algorithm named as Diversity Driven Multi-Parent Evolutionary Algorithm with Adaptive non-uniform mutation is presented. In the proposed algorithm, Non-uniform mutation is used to maintain diversity in the explored solutions. Fitness variance, which signifies solution space aggregation, is used to detect the premature convergence of the population to a local optimum. The term multi-parent is used in the context of more than two parents participating in crossover operation. After multi-parent selection for cross-over to generate new solutions, the non-uniform adaptive mutation is performed, which in turn is triggered by the diminishing value of fitness variance of candidate solutions and pushes solutions out of local optimum. Hence, it can be said that the algorithm is driven by the diversity of the population and overcomes the tendency of evolutionary algorithms to stuck in local optimum. The performance of this algorithm is tested on 23 basic benchmarks, CEC05 functions, and CEC17 functions. As CEC17 benchmark functions include constraint problems, a constraint-handling technique is proposed based on the fuzzy set theory. In the proposed constrained handling strategy, constraint violation is also taken as another objective along with the main objective. The decision to accept or discard the solution is based on the fuzzy set theory. The values of constraint violation and objective function are calculated and fuzzified by calculating membership values by considering the main objective and constraint violation as triangular fuzzy functions. The best solutions are selected based on cardinal priority ranking. The obtained results from the proposed algorithm are compared with the results available in the literature. The result indicates that this algorithm is competitive, even with a smaller number of function evaluations.

A fast surrogate-assisted particle swarm optimization algorithm for computationally expensive problems

Although many surrogate-assisted evolutionary algorithms (SAEAs) have been proposed to solve computationally expensive problems, they usually need to consume plenty of expensive evaluations to obtain an acceptable solution. In this paper, we proposed a fast surrogate-assisted particle swarm optimization (FSAPSO) algorithm to solve medium scaled computationally expensive problems through a small number of function evaluations (FEs). Two criteria are applied in tandem to select candidates for exact evaluations. The performance-based criterion is used to exploit the current global best and accelerate the convergence rate, while the uncertainty-based criterion is used to enhance the exploration of the algorithm. The distance-based uncertainty criterion in SAEAs does not consider the fitness landscape of different problems. Therefore, we developed a criterion to estimate uncertainty by considering the distance and fitness value information simultaneously. This criterion can make up for the disadvantage of the conventional distance-based uncertainty criterion by considering the fitness landscape of a problem. In addition, it can be applied in any surrogate-assisted evolutionary algorithm irrespective of the used surrogate model. Twenty-three benchmark functions widely adopted in the literature and a 10-dimension propeller design problem are used to test the proposed approach. Experimental results demonstrate the superiority of the proposed FSAPSO algorithm over seven state-of-the-art algorithms.

Probability-Adaptive Kriging in n-Ball (PAK-Bn) for reliability analysis

Complexity of today’s engineering systems inevitably makes the computational simulation of their performance challenging and time-consuming. Since structural reliability analysis methods generally repeat such computational simulations, it is essential to reduce the number of function evaluations required to achieve reliable estimates. In research efforts to fulfill this aim, adaptive Kriging methods have gained significant interest because of desirable properties and accuracy of the surrogate model. A new adaptive Kriging approach proposed in this paper improves the efficiency of reliability analysis by incorporating the probabilistic density of the random variable space into the adaptive procedure of identifying the surrogate limit-state surface. In addition, samples distributed uniformly inside the n-ball domain are used as the candidate points to enrich the experimental design, and the best candidate for simulation is determined in terms of influence on the failure probability estimation. The efficiency and accuracy of the proposed Probability-Adaptive Kriging in n-Ball (PAK-Bn) method are demonstrated by several reliability examples characterized by highly non-linear limit-state functions, small failure probability and multiple design points. The results confirm that the method facilitates convergence to the failure probability with a smaller number of function evaluations. The supporting source codes are available for download at https://github.com/Jungh0Kim/PAK-Bn.

Proceso de optimización en el diseño de sistema de calefacción solar pasivo

La presente investigación plantea definir la solución óptima de un sistema de captación solar pasiva adherible a las construcciones existentes, con el fin de mejorar las condiciones de confort higrotérmico interior, mediante una metodología de trabajo paramétrico, con estudios de caso en el clima del altiplano cundiboyacense en Bogotá, Colombia.&#x0D; Para encontrar la solución óptima para el área de estudio, se integraron simulaciones térmicas dinámicas empleando EnergyPlus, con un proceso automatizado de análisis de optimización con metamodelos usando el algoritmo RBFOpt, el cual utiliza técnicas avanzadas de aprendizaje automático “Machine Learning” para encontrar buenas soluciones con un pequeño número de evaluaciones de funciones.&#x0D; Las primeras conclusiones de la investigación evidencia las variables más importantes para el buen diseño de los sistemas de captación solar, son principalmente el ancho y alto del sistema de captación solar, pero no tanto su profundidad, por lo cual el Muro Trombe es ideal para el clima trabajado. Además, es posible garantizar condiciones de confort al interior de los espacios de uso nocturno sin importar su orientación, un factor favorable para los efectos de la investigación, considerando que las viviendas ya construidas en la zona de trabajo tienen todo tipo de orientaciones.

Iterative Most Probable Point Search Method for Problems With a Mixture of Random and Interval Variables

Abstract To represent input variability accurately, an input distribution model for random variables should be constructed using many data. However, for certain input variables, engineers may have only their intervals, which represent input uncertainty. In practical engineering applications, both random and interval variables could exist at the same time. To consider both input variability and uncertainty, inverse reliability analysis should be carried out considering both random and interval variables—mixed variables—and their mathematical correlation in a performance measure. In this paper, an iterative most probable point (MPP) search method has been developed for the mixed-variable problem. The update procedures for MPP search are developed considering the features of mixed variables in the inverse reliability analysis. MPP search for random and interval variables proceed simultaneously to consider the mathematical correlation. An interpolation method is introduced to find a better candidate MPP without additional function evaluations. Mixed-variable design optimization (MVDO) has been formulated to obtain cost-effective and reliable design in the presence of mixed variables. In addition, the design sensitivity of a probabilistic constraint has been developed for an effective and efficient MVDO procedure. Using numerical examples, it is found that the developed MPP search method finds an accurate MPP more efficiently than the generic optimization method does. In addition, it is verified that the developed method enables the MVDO process with a small number of function evaluations.

Approximation and sampling of multivariate probability distributions in the tensor train decomposition

General multivariate distributions are notoriously expensive to sample from, particularly the high-dimensional posterior distributions in PDE-constrained inverse problems. This paper develops a sampler for arbitrary continuous multivariate distributions that is based on low-rank surrogates in the tensor train format, a methodology that has been exploited for many years for scalable, high-dimensional density function approximation in quantum physics and chemistry. We build upon recent developments of the cross approximation algorithms in linear algebra to construct a tensor train approximation to the target probability density function using a small number of function evaluations. For sufficiently smooth distributions, the storage required for accurate tensor train approximations is moderate, scaling linearly with dimension. In turn, the structure of the tensor train surrogate allows sampling by an efficient conditional distribution method since marginal distributions are computable with linear complexity in dimension. Expected values of non-smooth quantities of interest, with respect to the surrogate distribution, can be estimated using transformed independent uniformly-random seeds that provide Monte Carlo quadrature or transformed points from a quasi-Monte Carlo lattice to give more efficient quasi-Monte Carlo quadrature. Unbiased estimates may be calculated by correcting the transformed random seeds using a Metropolis–Hastings accept/reject step, while the quasi-Monte Carlo quadrature may be corrected either by a control-variate strategy or by importance weighting. We show that the error in the tensor train approximation propagates linearly into the Metropolis–Hastings rejection rate and the integrated autocorrelation time of the resulting Markov chain; thus, the integrated autocorrelation time may be made arbitrarily close to 1, implying that, asymptotic in sample size, the cost per effectively independent sample is one target density evaluation plus the cheap tensor train surrogate proposal that has linear cost with dimension. These methods are demonstrated in three computed examples: fitting failure time of shock absorbers; a PDE-constrained inverse diffusion problem; and sampling from the Rosenbrock distribution. The delayed rejection adaptive Metropolis (DRAM) algorithm is used as a benchmark. In all computed examples, the importance weight-corrected quasi-Monte Carlo quadrature performs best and is more efficient than DRAM by orders of magnitude across a wide range of approximation accuracies and sample sizes. Indeed, all the methods developed here significantly outperform DRAM in all computed examples.

Handling expensive multi-objective optimization problems with a cluster-based neighborhood regression model

This paper gives attention to multi-objective optimization in scenarios where objective function evaluation is expensive, that is, expensive multi-objective optimization. We firstly propose a cluster-based neighborhood regression model, which incorporates the linear regression technique to predict the descent direction and generate new potential offspring. Combining this model with the classical decomposition-based multi-objective optimization framework, we propose an efficient and effective algorithm for tackling computationally expensive multi-objective optimization problems. As opposed to the conventional approach of replacing the original time-consuming objective functions with the approximated ones obtained by surrogate model, the proposed algorithm incorporates the proposed regression model to serve as an operator producing higher-quality offspring so that the algorithm requires fewer iterations to reach a given solution quality. The proposed algorithm is compared with several state-of-the-art surrogate-assisted algorithms on a variety of well-known benchmark problems. Empirical results demonstrate that the proposed algorithm outperforms or is competitive with other peer algorithms, and has the ability to keep a good trade-off between solution quality and running time within a fairly small number of function evaluations. In particular, our proposed algorithm shows obvious superiority in terms of the computational time used for the algorithm components, and can obtain acceptable solutions for expensive problems with high efficiency.

An enhanced time evolutionary optimization for solving engineering design problems

Time evolutionary optimization (TEO) is a novel population-based meta-heuristic optimization algorithm, inspired by natural selection and evolution of creatures over time. Time and the environment are two main factors of evolution at TEO. In this paper, enhanced time evolutionary optimization (ETEO) is presented. ETEO is the new version of TEO which modifies time evolutionary factor and applied population clustering. Population clustering amplified environmental factor to increase the efficiency of ETEO. For this purpose, a memory is used to save some best designs and ETEO can escape from local optimal points. The algorithm was validated by solving several constraint benchmarks and engineering design problems. The comparison results between the proposed algorithm and other metaheuristic methods contain TEO, indicate the ETEO is competitive with them, and in some cases superior to, other available heuristic methods in terms of the efficiency, faster convergence rate, robustness of finding final solution and requires a smaller number of function evaluations for solving constrained engineering problems.

Sensitivity analysis via Karhunen–Loève expansion of a random field model: Estimation of Sobol’ indices and experimental design

We use the Karhunen–Loève expansion of a random-field model to construct a tensorised Bayesian linear model from which Sobol’ sensitivity indices can be estimated straightforwardly. The method combines the advantages of models built from families of orthonormal functions, which facilitate computations, and Gaussian-process models, which offer a lot of flexibility. The posterior distribution of the indices can be derived, and its normal approximation can be used to design experiments especially adapted to their estimation. Implementation details are provided, and values of tuning parameters are indicated that yield precise estimation from a small number of function evaluations. Several illustrative examples are included that show the good performance of the method, in particular in comparison with estimation based on polynomial chaos expansion.

Automatic feature engineering for regression models with machine learning: An evolutionary computation and statistics hybrid

Symbolic Regression (SR) is a well-studied task in Evolutionary Computation (EC), where adequate free-form mathematical models must be automatically discovered from observed data. Statisticians, engineers, and general data scientists still prefer traditional regression methods over EC methods because of the solid mathematical foundations, the interpretability of the models, and the lack of randomness, even though such deterministic methods tend to provide lower quality prediction than stochastic EC methods. On the other hand, while EC solutions can be big and uninterpretable, they can be created with less bias, finding high-quality solutions that would be avoided by human researchers. Another interesting possibility is using EC methods to perform automatic feature engineering for a deterministic regression method instead of evolving a single model; this may lead to smaller solutions that can be easy to understand. In this contribution, we evaluate an approach called Kaizen Programming (KP) to develop a hybrid method employing EC and Statistics. While the EC method builds the features, the statistical method efficiently builds the models, which are also used to provide the importance of the features; thus, features are improved over the iterations resulting in better models. Here we examine a large set of benchmark SR problems known from the EC literature. Our experiments show that KP outperforms traditional Genetic Programming - a popular EC method for SR - and also shows improvements over other methods, including other hybrids and well-known statistical and Machine Learning (ML) ones. More in line with ML than EC approaches, KP is able to provide high-quality solutions while requiring only a small number of function evaluations.

A new PSO-based algorithm for multi-objective optimization with continuous and discrete design variables

This paper presents a new multi-objective optimization algorithm called FC-MOPSO for optimal design of engineering problems with a small number of function evaluations. The proposed algorithm expan...

Surrogate-Based MOEA/D for Electric Motor Design With Scarce Function Evaluations

This paper proposes a surrogate-assisted multiobjective evolutionary algorithm based on decomposition (sMOEA/D) for the design of electric motors. The idea is to improve the surrogate gradually during the optimization. Simulation results show that the proposed method is competitive with state-of-the-art multiobjective optimization algorithms needing only a small number of function evaluations.