Number Of Objective Function Calls Research Articles

Hermite least squares optimization: a modification of BOBYQA for optimization with limited derivative information

Derivative-free optimization tackles problems, where the derivatives of the objective function are unknown. However, in practical optimization problems, the derivatives of the objective function are often not available with respect to all optimization variables, but for some. In this work we propose the Hermite least squares optimization method: an optimization method, specialized for the case that some partial derivatives of the objective function are available and others are not. The main goal is to reduce the number of objective function calls compared to state of the art derivative-free solvers, while the convergence properties are maintained. The Hermite least squares method is a modification of Powell’s derivative-free BOBYQA algorithm. But instead of (underdetermined) interpolation for building the quadratic subproblem in each iteration, the training data is enriched with first and—if possible—second order derivatives and then least squares regression is used. Proofs for global convergence are discussed and numerical results are presented. Further, the applicability is verified for a realistic test case in the context of yield optimization. Numerical tests show that the Hermite least squares approach outperforms classic BOBYQA if half or more partial derivatives are available. In addition, it achieves more robustness and thus better performance in case of noisy objective functions.

Dynamic Machine Learning Global Optimization Algorithm and Its Application to Aerodynamics

A dynamic extreme gradient boosting (XGBoost) and MaxLIPO trust region parallel global optimization algorithm is proposed in this paper, and it is applied to the turbomachinery blade aerodynamic optimization coupled with an in-house graphics processing unit (GPU) heterogeneous accelerated compressible flow solver, AeroWhale. The algorithm combines an accurate machine learning regression model, an efficient nongradient optimization method with no hyperparameters, a dynamic update regression strategy, and double convergence criteria to achieve high optimization accuracy and efficiency. The optimization results on the test function indicate that the number of objective function calls is less than 2% of that required by a traditional genetic algorithm, which greatly reduces the optimization time. The dynamic XGBoost model ensures that the regression model accuracy near the optimum is relatively high, which is attributed to the update strategy. The error between the optimal value identified by the proposed algorithm and the theoretical value is only 0.52% after several objective function calls. Finally, the aerodynamic optimization algorithm is applied to the LS89 high-pressure turbine, and the total pressure loss is reduced by 13.16%. The sensitivity of each optimization feature to the objective function is determined, showing that the blade suction surface control point near the trailing edge has the greatest impact on aerodynamic performance.

Adaptive Ranking-Based Constraint Handling for Explicitly Constrained Black-Box Optimization.

We propose a novel constraint-handling technique for the covariance matrix adaptation evolution strategy (CMA-ES). The proposed technique is aimed at solving explicitly constrained black-box continuous optimization problems, in which the explicit constraint is a constraint whereby the computational time for the constraint violation and its (numerical) gradient are negligible compared to that for the objective function. This method is designed to realize two invariance properties: invariance to the affine transformation of the search space, and invariance to the increasing transformation of the objective and constraint functions. The CMA-ES is designed to possess these properties for handling difficulties that appear in black-box optimization problems, such as non-separability, ill-conditioning, ruggedness, and the different orders of magnitude in the objective. The proposed constraint-handling technique (CHT), known as ARCH, modifies the underlying CMA-ES only in terms of the ranking of the candidate solutions. It employs a repair operator and an adaptive ranking aggregation strategy to compute the ranking. We developed test problems to evaluate the effects of the invariance properties, and performed experiments to empirically verify the invariance of the algorithm. We compared the proposed method with other CHTs on the CEC 2006 constrained optimization benchmark suite to demonstrate its efficacy. Empirical studies reveal that ARCH is able to exploit the explicitness of the constraint functions effectively, sometimes even more efficiently than an existing box-constraint handling technique on box-constrained problems, while exhibiting the invariance properties. Moreover, ARCH overwhelmingly outperforms CHTs by not exploiting the explicit constraints in terms of the number of objective function calls.

Multi-level evolution strategies for high-resolution black-box control

This paper introduces a multi-level (m-lev) mechanism into Evolution Strategies (ESs) in order to address a class of global optimization problems that could benefit from fine discretization of their decision variables. Such problems arise in engineering and scientific applications, which possess a multi-resolution control nature, and thus may be formulated either by means of low-resolution variants (providing coarser approximations with presumably lower accuracy for the general problem) or by high-resolution controls. A particular scientific application concerns practical Quantum Control (QC) problems, whose targeted optimal controls may be discretized to increasingly higher resolution, which in turn carries the potential to obtain better control yields. However, state-of-the-art derivative-free optimization heuristics for high-resolution formulations nominally call for an impractically large number of objective function calls. Therefore, an effective algorithmic treatment for such problems is needed. We introduce a framework with an automated scheme to facilitate guided-search over increasingly finer levels of control resolution for the optimization problem, whose on-the-fly learned parameters require careful adaptation. We instantiate the proposed m-lev self-adaptive ES framework by two specific strategies, namely the classical elitist single-child (1+1)-ES and the non-elitist multi-child derandomized $$(\mu _W,\lambda )$$ -sep-CMA-ES. We first show that the approach is suitable by simulation-based optimization of QC systems which were heretofore viewed as too complex to address. We also present a laboratory proof-of-concept for the proposed approach on a basic experimental QC system objective.

A novel multi-objective quantum particle swarm algorithm for suspension optimization

In this paper, a novel multi-objective archive-based Quantum Particle Optimizer (MOQPSO) is proposed for solving suspension optimization problems. The algorithm has been adapted from the well-known single objective QPSO by substantial modifications in the core equations and implementation of new multi-objective mechanisms. The novel algorithm MOQPSO and the long-established NSGA-II and COGA-II (Compressed-Objective Genetic Algorithm with Convergence Detection) are compared. Two situations are considered in this paper: a simple half-car suspension model and a bus suspension model. The numerical model of the bus allows complex dynamic interactions not considered in previous studies. The suitability of the solution is evaluated based on vibration-related ISO Standards, and the efficiency of the proposed algorithm is tested by dominance comparison. For a specifically chosen Pareto front solution found by MOQPSO in the second case, the passengers and driver accelerations attenuated about 50% and 33%, respectively, regarding non-optimal suspension parameters. All solutions found by NSGA-II are dominated by those found by MOQPSO, which presented a Pareto front noticeably wider for the same number of objective function calls.

A Benchmark TEAM Problem for Multi-Objective Pareto Optimization in Magnetics: The Time-Harmonic Regime

The study presented in this article reformulates and generalizes the TEAM benchmark, originally proposed for multi-objective optimization of magnetic devices under dc conditions, now extending it to the ac regime. A solution is furnished which has enabled an extensive search and reliable estimation of the shape of the Pareto front. Field uniformity and losses are considered with reference to a class of power inductors. It is argued that the benchmark provides a challenging target for new algorithms, especially those involving numerical modeling based on finite-element codes, where the number of objective function calls needs to be minimized for practical designs.

Multivariable Electromagnetic Optimization Design Exploiting Hybrid Kriging

Electromagnetic optimization utilizing finite element or similar numerical techniques always carries a heavy burden of computation; therefore, an efficient optimizer reducing the number of objective function calls to find the optimum accurately is essential. Kriging as a regression model is able to locate the optimum; meanwhile, it also can produce the response surface of objective function only based on the spatial correlation of limited information. However, the storage of correlation matrices generated by a kriging optimizer, along with the iterative optimization process, is an issue in the context of the electromagnetic optimization problems with many design variables and multiple optimal objectives. The proposed hybrid kriging methodology incorporating a direct algorithm is able to maintain memory occupation of the optimizer as a nearly constant level. It performs efficiently and reliably while coping with the large-scale multivariable optimization tasks. The feasibility and efficiency of the proposed hybrid kriging are verified via a classic analytic function and the multivariable benchmark TEAM Workshop problem 22.

An Effective Metaheuristic for Bi-objective Feature Selection in Two-Class Classification Problem

Feature selection is known as a very useful technique in machine learning practice as it may result in the development of more straightforward models with better accuracy. Traditionally, feature selection is considered as a single-objective problem, however, it can be easily formulated in terms of two objectives. The solving of such problems requires the application of appropriate multi-objective optimization methods that do not always offer equally good solutions even under the same conditions. This paper focuses on the development of a metaheuristic optimization approach for bi-objective feature selection problem in two-class classification. We consider the solving of this problem in terms of minimization of both misclassification error and feature subset size. For solving the considered problem, an adaptation of the Multi-Objective Adaptive Memory Programming (MOAMP) metaheuristic based on the tabu search strategy is proposed. Our MOAMP adaption has been utilized to obtain the sets of most relevant features for two real classification problems with two classes. Finally, using popular Pareto front quality indicators, the obtained results have been compared with the sets of non-dominated solutions derived by the well-known NSGA2 algorithm. The conducted research allows concluding about the ability of the MOAMP adaptation to get a better efficient frontier for the same number of objective function calls.

AN EVOLUTIONARY ALGORITHM FOR FAST INTENSITY BASED IMAGE MATCHING BETWEEN OPTICAL AND SAR SATELLITE IMAGERY

Abstract. This paper presents a hybrid evolutionary algorithm for fast intensity based matching between satellite imagery from SAR and very high-resolution (VHR) optical sensor systems. The precise and accurate co-registration of image time series and images of different sensors is a key task in multi-sensor image processing scenarios. The necessary preprocessing step of image matching and tie-point detection is divided into a search problem and a similarity measurement. Within this paper we evaluate the use of an evolutionary search strategy for establishing the spatial correspondence between satellite imagery of optical and radar sensors. The aim of the proposed algorithm is to decrease the computational costs during the search process by formulating the search as an optimization problem. Based upon the canonical evolutionary algorithm, the proposed algorithm is adapted for SAR/optical imagery intensity based matching. Extensions are drawn using techniques like hybridization (e.g. local search) and others to lower the number of objective function calls and refine the result. The algorithm significantely decreases the computational costs whilst finding the optimal solution in a reliable way.

A Benchmark TEAM Problem for Multi-Objective Pareto Optimization of Electromagnetic Devices

This paper proposes a new benchmark for multi-objective optimization. A solution is furnished which has enabled an extensive search and reliable estimation of the shape of the Pareto front. Field uniformity and sensitivity are considered in the context of robust design. It is argued that the benchmark will provide a challenging target for new algorithms, especially those involving numerical modeling using finite-element codes where the number of objective function calls needs to be minimized for practical design processes.

Modified Particle Swarm Optimization Algorithm for Multi-Objective Optimization Design of Hybrid Journal Bearings

The objective of design optimization is to determine the design that minimizes the objective function by changing design variables and satisfying design constraints. During multi-objective optimization, which has been widely applied to improve bearing designs, designers must consider several design criteria or objective functions simultaneously. The particle swarm optimization (PSO) method is known for its simple implementation and high efficiency in solving multifactor but single-objective optimization problems. This paper introduces a new multi-objective algorithm (MOA) based on the PSO and Pareto methods that can greatly reduce the number of objective function calls when a suitable swarm size is set.

Strategies for balancing exploration and exploitation in electromagnetic optimisation

PurposeElectromagnetic design utilising finite element or similar numerical methods is computationally expensive, thus efficient algorithms reducing the number of objective function calls to locate the optimum are sought. The balance between exploration and exploitation may be achieved using a reinforcement learning approach, as demonstrated previously. However, in practical design problems, in addition to finding the global optimum efficiently, information about the robustness of the solution may also be important. In this paper, the aim is to discuss the suitability of different search algorithms and to present their fitness to solve the optimization problem in conjunction with providing enough information on the robustness of the solution.Design/methodology/approachTwo novel strategies enhanced by the surrogate model based weighted expected improvement approach are discussed. The algorithms are tested using a two‐variable test function. The emphasis of these strategies is on accurate approximation of the shape of the objective function to accomplish a robust design.FindingsThe two novel strategies aim to pursue the optimal value of weights for exploration and exploitation throughout the iterative process for better prediction of the shape of the objective function.Originality/valueIt is argued that the proposed strategies based on adaptively tuning weights perform better in predicting the shape of the objective function. Good accuracy of predicting the shape of the objective function is crucial for achieving a robust design.

Inverse multi-objective robust evolutionary design

In this paper, we present an Inverse Multi-Objective Robust Evolutionary (IMORE) design methodology that handles the presence of uncertainty without making assumptions about the uncertainty structure. We model the clustering of uncertain events in families of nested sets using a multi-level optimization search. To reduce the high computational costs of the proposed methodology we proposed schemes for (1) adapting the step-size in estimating the uncertainty, and (2) trimming down the number of calls to the objective function in the nested search. Both offline and online adaptation strategies are considered in conjunction with the IMORE design algorithm. Design of Experiments (DOE) approaches further reduce the number of objective function calls in the online adaptive IMORE algorithm. Empirical studies conducted on a series of test functions having diverse complexities show that the proposed algorithms converge to a set of Pareto-optimal design solutions with non-dominated nominal and robustness performances efficiently.