Number Of Function Evaluations Research Articles

Extension of K-nearest neighbors and introduction of an applicable prediction criterion for a novel Monte Carlo simulation-based method in structural reliability

Efficiency is an important issue in structural reliability analysis. In this paper, a novel method is proposed to increase the efficiency of Monte Carlo Simulation by means of a machine learning algorithm. Using K-nearest neighbors method, the safe or failure region of many generated samples can be predicted without evaluating the limit state function. The proposed method represents three novel parts as its main core. In the first part of the proposed method, the classical K-nearest neighbors algorithm has been extended to a new fuzzy level such that the proposed method could predict even if all K neighbors are not in the same region. In previous similar methods, there is a prerequisite that K neighbors must be all in the failure region or all in the safe region. This extension to the fuzzy level is carried out by assigning a specific failure weight to each neighbor. In the second part of the proposed method, an innovative diagnostic criterion has been defined to check whether the region of a sample could be predicted or the sample has to be evaluated in the limit state function. The criterion describes the relative position of the newly generated sample with respect to the neighbors. This newly defined criterion is an effective alternative for the laborious action of drawing the convex hull around K neighbors, which is done as a time-consuming part in previous similar methods. Promotion of the accuracy or efficiency in the proposed method could be set using the adjusting parameter of the represented criterion. Since the implementation of the algorithm for each individual sample needs much time, in the third part, the proposed method introduces a technique to perform function evaluations or predictions in clusters. This part effectively reduces the computational burden, i.e. the number of function evaluations and run time. The performance of the proposed method, applying three novel and newly represented parts, has been verified using a number of mathematical and structural examples.

Hybridization of Social Group Optimization and Differential Evolution Algorithm for Solving Speed Reducer Design Problem

Objectives: The objective of this study is to present a hybrid approach named SGO-DE, which combines the Social Group Optimization (SGO) algorithm and differential evolution (DE), aiming to balance exploration and exploitation capacities to improve the accuracy of the optimization algorithm in finding optimal solution for speed reducer design problem. This hybrid approach is simulated for a speed reducer mechanical engineering design problem and the results are compared to several other state-of-the-art optimization algorithms. Method: To improve the exploration and exploitation of SGO, in its acquiring phase Differential Evolution (DE) is introduced. The individual candidate solutions derived from the Improving phase of SGO tries to acquire better values using DE. This helps in striking a better balance between exploration and exploitation, there by achieving improved optimal values and not getting trapped in local optima. The performance of the SGO-DE method is then evaluated and compared to other optimization algorithms through experimentation on the speed reducer design challenge. Findings: The findings of this study indicate that the SGO-DE hybrid approach outperforms other state-of-the-art algorithms by a significant margin in terms of optimization results. The numbers of function evaluations (FEs) significantly go low as less as 6000 compared to other state-of-the algorithms. The comparison demonstrates the efficacy of the SGO-DE method in enhancing solution quality and speeding up execution. Novelty: The novelty of this study lies in the development a hybrid approach of SGO and DE which is efficient in achieving competitive performance in less numbers of function evaluation in speed reducer design problem. This hybridization can strike a better balance between exploration and exploitations. Keywords: SGO, DE, Hybridization, Nature-inspired, Optimization algorithm

A three-term conjugate gradient descent method with some applications

The stationary point of optimization problems can be obtained via conjugate gradient (CG) methods without the second derivative. Many researchers have used this method to solve applications in various fields, such as neural networks and image restoration. In this study, we construct a three-term CG method that fulfills convergence analysis and a descent property. Next, in the second term, we employ a Hestenses-Stiefel CG formula with some restrictions to be positive. The third term includes a negative gradient used as a search direction multiplied by an accelerating expression. We also provide some numerical results collected using a strong Wolfe line search with different sigma values over 166 optimization functions from the CUTEr library. The result shows the proposed approach is far more efficient than alternative prevalent CG methods regarding central processing unit (CPU) time, number of iterations, number of function evaluations, and gradient evaluations. Moreover, we present some applications for the proposed three-term search direction in image restoration, and we compare the results with well-known CG methods with respect to the number of iterations, CPU time, as well as root-mean-square error (RMSE). Finally, we present three applications in regression analysis, image restoration, and electrical engineering.

Approximation in the extended functional tensor train format

This work proposes the extended functional tensor train (EFTT) format for compressing and working with multivariate functions on tensor product domains. Our compression algorithm combines tensorized Chebyshev interpolation with a low-rank approximation algorithm that is entirely based on function evaluations. Compared to existing methods based on the functional tensor train format, the adaptivity of our approach often results in reducing the required storage, sometimes considerably, while achieving the same accuracy. In particular, we reduce the number of function evaluations required to achieve a prescribed accuracy by up to over 96%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$96\\%$$\\end{document} compared to the algorithm from Gorodetsky et al. (Comput. Methods Appl. Mech. Eng. 347, 59–84 2019).

Restricted global optimization for QAOA

The Quantum Approximate Optimization Algorithm (QAOA) has emerged as a promising variational quantum algorithm for addressing NP-hard combinatorial optimization problems. However, a significant limitation lies in optimizing its classical parameters, which is in itself an NP-hard problem. To circumvent this obstacle, initialization heuristics, enhanced problem encodings and beneficial problem scalings have been proposed. While such strategies further improve QAOA’s performance, their remaining problem is the sole utilization of local optimizers. We show that local optimization methods are inherently inadequate within the complex cost landscape of QAOA. Instead, global optimization techniques greatly improve QAOA’s performance across diverse problem instances. While global optimization generally requires high numbers of function evaluations, we demonstrate how restricted global optimizers still show better performance without requiring an exceeding amount of function evaluations.

Boosting scalability for large-scale multiobjective optimization via transfer weights

Large-scale multiobjective optimization problems (LSMOPs), which optimize multiple conflicting objectives with hundreds or even thousands of decision variables, demand increasing computational resources to assure satisfactory performance as the decision variables increase. Multiobjective evolutionary algorithms are naturally scalable, but as the dimension increases, the conflict between analyzing enough solutions and the limited number of function evaluations has hindered further improvements in scalability. In this paper, we first define the scalability of multiobjective evolutionary algorithms and design an indicator to quantitatively measure the scalability. Second, to boost scalability when solving LSMOPs, we propose a scalable multiobjective optimization algorithm by transferring weights between solutions to reduce dependency on ever-increasing computational resources as the problem dimension increases. The proposed framework entails constructing a latent decision space to determine evolutionary weights for chosen representative solutions. These computed weights are then transferred to the remaining solutions, enabling evolutionary optimization to proceed without requiring additional evaluations, even as the dimensionality increases. By utilizing the knowledge gained from the source solutions, each solution is customized with an evolutionary weight scheme that not only preserves computational resources but also enhances optimization performance, thereby boosting scalability. We have conducted experiments on LSMOPs to verify the effectiveness and scalability of the proposed algorithm. The proposed method outperforms selected state-of-the-art algorithms and gains scalability boosts in situations where the dimensions increase while the function evaluation remains constant.

Nonzero Degree-Based Multiobjective Cooperative Coevolutionary for Block Sparse Recovery

Block sparse signals have the characteristics of nonzero values concentrated as blocks. Therefore, block sparse recovery requires recovering the original signal from the observed signal and the measurement matrix, where the recovered error should be small and the signal satisfies the block sparsity. In this paper, block sparse recovery is solved as a multiobjective problem (MOP) and the recovery error, sparsity, and the block number of the recovered signal are considered as the conflicting objectives. Furthermore, the dimensionality of real block sparse signals is often too large, which increases the difficulty of recovery. Cooperative coevolutionary (CC) can effectively alleviate the curse of dimensionality in block sparse recovery. In addition, block sparsity causes zero and nonzero variables in the signal to form natural clusters, which is similar to the decomposition and cooperation of subproblems in CC. Therefore, CC is used in the algorithm to solve the block sparse MOP and decomposes the original problem into different subproblems. Each subproblem has a defined nonzero degree that can be combined with the proposed adaptive operator to adaptively adjust the way of generating new solutions of subproblem, which encourages the complete solution to have block sparsity. For the increased number of function evaluations (FEs) caused by the decomposition, a new list-inquiry evaluation is designed to avoid repeated evaluation. Finally, the experimental results on simulated data and real data have demonstrated the effectiveness of the proposed algorithm.

Accelerating sampling-based tolerance–cost optimization by adaptive surrogate models

ABSTRACT High numbers of function evaluations are inevitable to guarantee the reliability and optimality of sampling-based tolerance–cost optimization. Despite using different countermeasures to increase its efficiency, unreasonably long computation times and unreliable results currently hinder its profitable application. Motivated to overcome this shortcoming, this article presents a novel strategy harmonizing metaheuristic optimization and surrogate modelling. It is based on the idea of adaptive surrogate model-based optimization substituting the tolerance analysis subroutine with a surrogate model, which is iteratively re-modelled with intermediate optimization results to improve its accuracy continuously in potential optima regions. On the one hand, a directed intensification of promising solutions and, on the other hand, an accelerated exploration of the search space is achieved. Optimization studies prove its positive effect on overall efficiency, where a case study with multiple nonlinear assembly response functions and geometrical tolerances serves as a use case.

Hybrid Modified Accelerated Gradient Method for Optimization Processes

This research reveals a hybrid variant of the modified accelerated gradient method. We prove that derived iteration is linearly convergent on the set of uniformly convex functions. Performance profiles of the introduced hybrid method were numerically compared with its non-hybrid version. The analyzed characteristics were CPU time, the number of iterations and the number of function evaluations. The results of the numerical experiments show a better performance in favor of the derived hybrid accelerated model compared with its forerunner.

Emulation-based adaptive differential evolution: fast and auto-tunable approach for moderately expensive optimization problems

In the field of expensive optimization, numerous papers have proposed surrogate-assisted evolutionary algorithms (SAEAs) for a few thousand or even hundreds of function evaluations. However, in reality, low-cost simulations suffice for a lot of real-world problems, in which the number of function evaluations is moderately restricted, e.g., to several thousands. In such moderately restricted scenario, SAEAs become unnecessarily time-consuming and tend to struggle with premature convergence. In addition, tuning the SAEA parameters becomes impractical under the restricted budgets of function evaluations—in some cases, inadequate configuration may degrade performance instead. In this context, this paper presents a fast and auto-tunable evolutionary algorithm for solving moderately restricted expensive optimization problems. The presented algorithm is a variant of adaptive differential evolution (DE) algorithms, and is called emulation-based adaptive DE or EBADE. The primary aim of EBADE is to emulate the principle of sample-efficient optimization, such as that in SAEAs, by adaptively tuning the DE parameter configurations. Specifically, similar to Expected Improvement-based sampling, EBADE identifies parameter configurations that may produce expected-to-improve solutions, without using function evaluations. Further, EBADE incepts a multi-population mechanism and assigns a parameter configuration to each subpopulation to estimate the effectiveness of parameter configurations with multiple samples carefully. This subpopulation-based adaptation can help improve the selection accuracy of promising parameter configurations, even when using an expected-to-improve indicator with high uncertainty, by validating with respect to multiple samples. The experimental results demonstrate that EBADE outperforms modern adaptive DEs and is highly competitive compared to SAEAs with a much shorter runtime.

Optimizing parameters in swarm intelligence using reinforcement learning: An application of Proximal Policy Optimization to the iSOMA algorithm

This paper presents a new algorithm for optimizing parameters in swarm algorithm using reinforcement learning. The algorithm, called iSOMA-RL, is based on the iSOMA algorithm, a population-based optimization algorithm that mimics the competition-cooperation behavior of creatures to find the optimal solution. By using reinforcement learning, iSOMA-RL can dynamically and continuously optimize parameters, which can play a crucial role in determining the performance of the algorithm but are often difficult to determine. The reinforcement learning technique used is the state-of-the-art Proximal Policy Optimization (PPO), which has been successful in many areas. The algorithm was compared to the original iSOMA algorithm and other algorithms from the SOMA family, showing better performance with only constant increase in computational complexity depending on number of function evaluations. Also we examine different sets of parameters to optimize and different reward functions. We also did comparison to widely used and state-of-the-art algorithms to illustrate improvement in performance over the original iSOMA algorithm.

Wind Power Bidding Based on an Ensemble Differential Evolution Algorithm with a Problem-Specific Constraint-Handling Technique

The intermittent nature of wind power generation induces great challenges for power bidding in the electricity market. The deployment of battery energy storage can improve flexibility for power bidding. This paper investigates an optimal power bidding strategy for a wind–storage hybrid power plant in the day-ahead electricity market. To handle the challenges of the uncertainties of wind power generation and electricity prices, the optimal bidding problem is formulated as a risk-aware scenario-based stochastic programming, in which a number of scenarios are generated using a copula-based approach to represent the uncertainties. These scenarios consider the temporal correlation of wind power generation and electricity prices between consecutive time intervals. In the stochastic programming, a more practical but nonlinear battery operation cost function is considered, which leads to a nonlinear constrained optimization problem. To solve the nonlinear constrained optimization problem, an ensemble differential evolution (EDE) algorithm is proposed, which makes use of the merits of an ensemble of mutant operators to generate mutant vectors. Moreover, a problem-specific constraint-handling technique is developed. To validate the effectiveness of the proposed EDE algorithm, it is compared with state-of-the-art DE-based algorithms for constrained optimization problems, including a constrained composite DE (C2oDE) algorithm and a novel DE (NDE) algorithm. The experimental results demonstrate that the EDE algorithm is much more reliable and much faster in finding a better bidding strategy against benchmarking algorithms. More precisely, the average values of the success rate are 0.893, 0.667, and 0.96 for C2oDE, NDE, and EDE, respectively. Compared to C2oDE and NDE, the average value of the mean number of function evaluations to succeed with EDE is reduced by 76% and 59%, respectively.

A four-stage multiderivative explicit Runge-Kutta method for the solution of first order ordinary differential equations

The development of numerical methods for the solution of initial value problems in ordinary differential equation have turned out to be a very rapid research area in recent decades due to the difficulties encountered in finding solutions to some mathematical models composed into differential equations from real life situations. Researchers have in recent times, used higher derivatives in the derivation of numerical methods to produce totally new ways of solving these equations. In this article, a new Runge-Kutta type methods with reduced number of function evaluations in the increment function is constructed, analyzed and implemented. This proposed method border on the use of higher derivatives up to the second derivative in the ki terms of Runge-Kutta method in order to achieve a higher order of accuracy. The qualitative features: local truncation error, consistency, convergence and stability of the new method were investigated and established. Numerical examples were also performed on some initial value problems to confirm the accuracy of the new method and compared with some existing methods of which the numerical results show that the new method competes favorably.

The modification conjugate coefficient for conjugate gradient method to solving unconstrained optimization problems

In this manuscript, a novel conjugate coefficient is derived through the utilization of equality conjugate gradient direction in conjunction with a modified quasi-Newton (QN) direction. The demonstrated results establish that this innovative approach ensures both sufficient descent and global convergence, meeting the criteria of the Wolf line search condition and inexact line search. Through numerical experiments, we employ the proposed method on a diverse set of established test functions, clearly demonstrating its enhanced efficiency. A comparative analysis is conducted against the FR CG method using the Dolan-More performance profile based on the number of function evaluations (NOF), number of iterations (NOI), and CPU time.

Efficient Surrogate Modeling Method for Evolutionary Algorithm to Solve Bilevel Optimization Problems.

The purpose of this study was to develop an evolutionary algorithm (EA) with bilevel surrogate modeling, called BL-SAEA, for tackling bilevel optimization problems (BLOPs), in which an upper level problem is to be solved subject to the optimality of a corresponding lower level problem. The motivation of this article is that the extensive lower level optimization required by each upper level solution consumes too many function evaluations, leading to poor optimization performance of EAs. To this end, during the upper level optimization, the BL-SAEA builds an upper level surrogate model to select several promising upper level solutions for the lower level optimization. Because only a small number of upper level solutions require the lower level optimization, the number of function evaluations can be considerably reduced. During the lower level optimization, the BL-SAEA constructs multiple lower level surrogate models to initialize the population of the lower level optimization, thus further decreasing the number of function evaluations. Experimental results on two widely used benchmarks and two real-world BLOPs demonstrate the superiority of our proposed algorithm over six state-of-the-art algorithms in terms of effectiveness and efficiency.

GPU-accelerated non-dominated sorting genetic algorithm III for maximizing protein production

<abstract><p>Maximizing protein expression levels poses a major challenge in bioengineering. To increase protein expression levels, numerous factors, including codon bias, codon context bias, hidden stop codons, homologous recombination, suitable guanine-cytosine ratio, and hairpin loop structure, are crucial and quantified by six objective functions: CAI, CPB, HSC, HD, GC3, and SL. Optimizing these six objectives simultaneously constitutes a multi-objective optimization problem, aiming to identify the favorable Pareto solutions rather than a singular optimal solution. However, achieving satisfactory solutions requires numerous cycles and solutions, thus leading to a large number of functional evaluations. While there are frameworks for multi-objective optimization problems, they often lack efficient support for objective function computation in protein encoding. In this paper, we proposed a method to design a set of coding sequences (CDSs) based on non-dominated sorting genetic algorithm III (NSGA-III), accelerated using NVIDIA graphical processing units (GPUs). Experimental results indicated that our method is 15,454 times faster than the Pymoo framework and is evaluated using 100 solutions and 100 cycles. Since our GPU implementation facilitated the use of larger solutions and more cycles, we were able to design a superior set of CDSs by increasing solutions to 400 and cycles to 12,800. In addition, our NSGA-III-based method consistently surpassed the NSGA-II approach when the number of cycles exceeded 3200 by utilizing 100 solutions. Finally, we observed that a gradual reduction of the mutation probability as the number of cycles increased yielded better quality results than maintaining a fixed mutation probability.</p></abstract>

Efficient sampling techniques for surrogate-based optimization with thermoelastic application

In many real-life models, the objective function may be very expensive to calculate or sometimes a black-box function. Therefore, a surrogate model is required to reduce the cost of the calculations. To build a robust surrogate model, a good sampling technique is needed. This paper suggests two new sampling techniques for surrogate-based optimization, depending on four random walk steps, which in turn give a good chance for building an accurate approximation for the original function. In addition to that, one of these new sampling techniques has a new feature that was not used before. The main advantage of our two suggested sampling techniques is that there is no optimization subproblem will be solved for the next sample point, which in turn saves CPU time. Furthermore, we applied our two sampling techniques to a 1D thermal shock problem for a half-space in the context of the fractional order theory of thermoelasticity to reduce the number of function evaluations and consequently CPU time. Laplace transform techniques are used to obtain the solution of the system in the frequency domain. Applying our algorithms to the few points obtained by using Laplace’s inverse, we can reach the optimal solution for all functions filed. The predictions of our techniques are discussed and compared with the classical method. 2D and 3D graphs are plotted for our numerical results. The temperature, displacement, and stress distributions are computed and represented graphically for different times and several fractional parameters.

Model Identification of E. coli Cultivation Process Applying Hybrid Crow Search Algorithm

Cultivation process (CP) modeling and optimization are ambitious tasks due to the nonlinear nature of the models and interdependent parameters. The identification procedures for such models are challenging. Metaheuristic algorithms exhibit promising performance for such complex problems since a near-optimal solution can be found in an acceptable time. The present research explores a new hybrid metaheuristic algorithm built upon the good exploration of the genetic algorithm (GA) and the exploitation of the crow search algorithm (CSA). The efficiency of the proposed GA-CSA hybrid is studied with the model parameter identification procedure of the E. coli BL21(DE3)pPhyt109 fed-batch cultivation process. The results are compared with those of the pure GA and pure CSA applied to the same problem. A comparison with two deterministic algorithms, i.e., sequential quadratic programming (SQP) and the Quasi-Newton (Q-N) method, is also provided. A more accurate model is obtained by the GA-CSA hybrid with fewer computational resources. Although SQP and Q-N find a solution for a smaller number of function evaluations, the resulting models are not as accurate as the models generated by the three metaheuristic algorithms. The InterCriteria analysis, a mathematical approach to revealing certain relations between given criteria, and a series of statistical tests are employed to prove that there is a statistically significant difference between the results of the three stochastic algorithms. The obtained mathematical models are then successfully verified with a different set of experimental data, in which, again, the closest one is the GA-CSA model. The GA-CSA hybrid proposed in this paper is proven to be successful in the collaborative hybridization of GA and CSA with outstanding performance.

Diagonal Partitioning Strategy Using Bisection of Rectangles and a Novel Sampling Scheme

In this paper, we consider a global optimization problem where the objective function is assumed to be Lipschitz-continuous with an unknown Lipschitz constant. Building upon the recently introduced BIRECT (BIsection of RECTangles) algorithm, we propose a new diagonal partitioning and sampling scheme. Our framework, named BIRECT-V (V for vertices), combines bisection with the sampling of two points. In the initial hyper-rectangle, these points are located at 1/3 and 1 along the main diagonal. Unlike most DIRECT-type algorithms, where evaluating the objective function at vertices is not suitable for bisection, our strategy, when combined with bisection, provides more comprehensive information about the objective function. However, the creation of new sampling points may coincide with existing ones at shared vertices, resulting in additional evaluations of the objective function and increasing the number of function evaluations per iteration. To overcome this issue, we propose modifying the original optimization domain to obtain a good approximation of the global solution. Experimental investigations demonstrate that this modification positively impacts the performance of the BIRECT-V algorithm. Our proposal shows promise as a global optimization algorithm compared to the original BIRECT and two popular DIRECT-type algorithms on a set of test problems. It particularly excels at high-dimensional problems

P-AK-MCS: Parallel AK-MCS method for structural reliability analysis

In recent years, the active learning reliability method that combines the Kriging model and Monte Carlo simulation (AK-MCS) has emerged as a promising approach due to its computational efficiency and accuracy. However, the commonly used learning functions, such as the expected feasibility function (EFF), U function, H function, and expected risk function (ERF), can only select one training point at each iteration which is time-wasteful when parallel computing is available. Therefore, this paper proposes a parallel active learning Kriging strategy, namely P-AK-MCS, for structural reliability analysis. By introducing an influence function that reflects the impact of the added point on the original learning function, four parallel learning functions are constructed: pseudo-U (PU) function, pseudo-EFF (PEFF), pseudo-H (PH) function, and pseudo-ERF (PERF). These functions aim to identify multiple training points at each iteration without requiring additional functional evaluations. The effectiveness of the proposed method is validated using four examples. The results demonstrate that compared to the standard AK-MCS, the proposed P-AK-MCS significantly reduces the number of computation loops and greatly decreases computational costs. Moreover, the total number of functional evaluations required is similar to that of the standard AK-MCS and remains insensitive to the number of multiple training points.