Gradient-free Algorithms Research Articles

Decentralized Gradient-Free Methods for Stochastic Non-smooth Non-convex Optimization

We consider decentralized gradient-free optimization of minimizing Lipschitz continuous functions that satisfy neither smoothness nor convexity assumption. We propose two novel gradient-free algorithms, the Decentralized Gradient-Free Method (DGFM) and its variant, the Decentralized Gradient-Free Method+ (DGFM+). Based on the techniques of randomized smoothing and gradient tracking, DGFM requires the computation of the zeroth-order oracle of a single sample in each iteration, making it less demanding in terms of computational resources for individual computing nodes. Theoretically, DGFM achieves a complexity of O(d^(3/2)δ^(-1)ε^(-4)) for obtaining a (δ,ε)-Goldstein stationary point. DGFM+, an advanced version of DGFM, incorporates variance reduction to further improve the convergence behavior. It samples a mini-batch at each iteration and periodically draws a larger batch of data, which improves the complexity to O(d^(3/2)δ^(-1)ε^(-3)). Moreover, experimental results underscore the empirical advantages of our proposed algorithms when applied to real-world datasets.

Gradient-Free Algorithms for Solving Stochastic Saddle Optimization Problems with the Polyak–Łojasiewicz Condition

Gradient-Free Algorithms for Solving Stochastic Saddle Optimization Problems with the Polyak–Łojasiewicz Condition

Reservoir Production Management With Bayesian Optimization: Achieving Robust Results in a Fraction of the Time

Summary In well control (production) optimization, the computational cost of conducting a full-physics flow simulation on a 3D, rich grid-based model poses a significant challenge. This challenge is exacerbated in a robust optimization (RO) setting, where flow simulation must be repeated for numerous geological realizations, rendering RO impractical for many field-scale cases. In this paper, we introduce and discuss a new optimization workflow that addresses this issue by providing computational efficiency, i.e., achieving a near-global optimum of the predefined objective function with minimal forward model (flow-simulation) evaluations. In this workflow, referred to as “Bayesian optimization (BO),” the objective function for samples of decision (control) variables is first computed using a proper design experiment. Then, given the samples, a Gaussian process regression (GPR) is trained to mimic the surface of the objective function as a surrogate model. While balancing the dilemma to select the next control variable between high mean, low uncertainty (exploitation) and low mean, high uncertainty (exploration), a new control variable is selected, and flow simulation is run for this new point. Later, the GPR is updated, given the output of the flow simulation. This process continues sequentially until the termination criteria are satisfied. To validate the workflow and obtain a better insight into the detailed steps, we first optimized a 1D problem. The workflow is then implemented for a 3D synthetic reservoir model to perform RO in a realistic field scenario (8-dimensional and 45-dimensional optimization problems). The workflow is compared with two other commonly used gradient-free algorithms in the literature: particle swarm optimization (PSO) and genetic algorithm (GA). The main contributions are (1) developing a new optimization workflow to address the computational cost of flow simulation in RO, (2) demonstrating the effectiveness of the workflow on a 3D grid-based model, (3) investigating the robustness of the workflow against randomness in initiation samples and discussing the results, and (4) comparing the workflow with other optimization algorithms, showing that it achieves same near-optimal results while requiring only a fraction of the computational time.

Inverse Design of Linear and Nonlinear Cylindrical Metamaterial Rod

Continuum elastic metamaterials can be used to reduce system-generated 2nd harmonics and they should also pass the fundamental harmonics during nonlinear ultrasonic testing. Linear metamaterials are designed based on the bandgap structure of the periodic layered materials showing contrast in acoustic impedances. Design parameters such as widths of the periodic layered elastic materials are calculated by a parametric sweep by solving eigenvalue problems, and the transfer matrix method. Most of the design studies assume infinitely long periodic elastic layers, but in practice length of metamaterial is limited by the reduction in fundamental amplitude of input short pulse. Though the linear metamaterials are designed for nonlinear ultrasonic applications, considering geometric and material nonlinearity of the layered elastic materials which contributes to harmonic scattering, the sensitivity of widths of layered materials to amplitudes of the 2nd harmonics, along with the linear interference is the most realistic modeling approach. In this study, linear and nonlinear metamaterials are designed using shape optimization techniques by solving transient finite element studies considering realistic geometric and material models. Gradient-free algorithms such as co-ordinate search and Nelder Mead are used during optimization. Effective design approaches are proposed and demonstrated to control longitudinal modes in a cylindrical rod.

Query-Efficient Black-Box Adversarial Attacks Guided by a Transfer-Based Prior.

Adversarial attacks have been extensively studied in recent years since they can identify the vulnerability of deep learning models before deployed. In this paper, we consider the black-box adversarial setting, where the adversary needs to craft adversarial examples without access to the gradients of a target model. Previous methods attempted to approximate the true gradient either by using the transfer gradient of a surrogate white-box model or based on the feedback of model queries. However, the existing methods inevitably suffer from low attack success rates or poor query efficiency since it is difficult to estimate the gradient in a high-dimensional input space with limited information. To address these problems and improve black-box attacks, we propose two prior-guided random gradient-free (PRGF) algorithms based on biased sampling and gradient averaging, respectively. Our methods can take the advantage of a transfer-based prior given by the gradient of a surrogate model and the query information simultaneously. Through theoretical analyses, the transfer-based prior is appropriately integrated with model queries by an optimal coefficient in each method. Extensive experiments demonstrate that, in comparison with the alternative state-of-the-arts, both of our methods require much fewer queries to attack black-box models with higher success rates.

Adaptive search space decomposition method for pre- and post-buckling analyses of space truss structures

The paper proposes a novel adaptive search space decomposition method and a novel gradient-free optimization-based formulation for the pre- and post-buckling analyses of space truss structures. Space trusses are often employed in structural engineering to build large steel constructions, such as bridges and domes, whose structural response is characterized by large displacements. Therefore, these structures are vulnerable to progressive collapses due to local or global buckling effects, leading to sudden failures. The method proposed in this paper allows the analysis of the load-equilibrium path of truss structures to permanent and variable loading, including stable and unstable equilibrium stages and explicitly considering geometric nonlinearities. The goal of this work is to determine these equilibrium stages via optimization of the Lagrangian kinematic parameters of the system, determining the global equilibrium. However, this optimization problem is non-trivial due to the undefined parameter domain and the sensitivity and interaction among the Lagrangian parameters. Therefore, we propose to formulate this problem as a nonlinear, multimodal, unconstrained, continuous optimization problem and develop a novel adaptive search space decomposition method, which progressively and adaptively re-defines the search domain (hypersphere) to evaluate the equilibrium of the system using a gradient-free optimization algorithm. We tackle three benchmark problems and evaluate a medium-sized test representing a real structural problem in this paper. The results are compared to those available in the literature regarding displacement–load curves and deformed configurations. The accuracy and robustness of the adopted methodology show a high potential for gradient-free algorithms to analyze space truss structures.

Multi-objective macrogeometry optimization of gears: Comparison between sequential quadratic programing and genetic algorithm

Deciding a gear pair’s macrogeometry parameters requires consideration of the factors such as cost of production, gear strength, and noise contributing parameters. A practical engineering problem is encountered when best possible gear geometry is needed for a fixed center distance to achieve the conflicting objectives. Further, constraints of practical utility on input parameters need to be taken into consideration for deciding the design space. This work represents comprehensive compilation of formulations of these often-conflicting objectives and constraints as non-linear functions and selection of an appropriate optimization technique to achieve a feasible design through a case study of a helical gear pair in high load carrying capacity mesh. For comparison of the quality of results and computational efficiency between gradient based and gradient free algorithms, Sequential Quadratic Programming and Genetic algorithm were used. The differences in the formulations of the objective functions for two algorithms were analyzed and addressed. Quadratic mutation rates and random crossovers were used to prevent premature convergence of Genetic Algorithm. Significant possible improvement in the base design was observed with both the algorithms.

Inverse design of direction independent metamaterials to control longitudinal and transverse waves

Direction-independent elastic metamaterials that can control longitudinal and transverse elastic waves independently for nonlinear ultrasonic testing are designed using an inverse design approach by defining the time-dependent inverse problem. Forward finite element wave propagation models are solved inversely as multiple multi-objective design optimization problems using gradient-free algorithms. Suppressing the 2nd harmonics (2f) generated due to instrumentation during nonlinear ultrasonic testing is obtained by keeping the first objective as reducing only second harmonics without concentrating on the amplitude of the 1st harmonics, which is a necessary condition for ultrasonic testing, whereas almost all the inversely designed metamaterials take care of this condition due to intelligent selection of material combinations. The second objective is defined as reducing error due to the difference between a pulse with 1st (f) and 2nd (2f) harmonics and a delayed pulse with 1st harmonics (f), as well as maximizing the power of the output pulse by defining min-max multi-objective optimization problems.

Gradient-Free Optimization in Thermoacoustics: Application to a Low-Order Model

Abstract Machine learning and automatized routines for parameter optimization have experienced a surge in development in the past years, mostly caused by the increasing availability of computing capacity. Gradient-free optimization can avoid cumbersome theoretical studies as input parameters are purely adapted based on output data. As no knowledge about the objective function is provided to the algorithms, this approach might reveal unconventional solutions to complex problems that were out of scope of classical solution strategies. In this study, the potential of these optimization methods on thermoacoustic problems is examined. The optimization algorithms are applied to a generic low-order thermoacoustic can-combustor model with several fuel injectors at different locations. We use three optimization algorithms – the well established downhill simplex method, the recently proposed explorative gradient method, and an evolutionary algorithm – to find optimal fuel distributions across the fuel lines while maintaining the amount of consumed fuel constant. The objective is to have minimal pulsation amplitudes. We compare the results and efficiency of the gradient-free algorithms. Additionally, we employ model-based linear stability analysis to calculate the growth rates of the dominant thermoacoustic modes. This allows us to highlight general and thermoacoustic-specific features of the optimization methods and results. The findings of this study show the potential of gradient-free optimization methods on combustor design for tackling thermoacoustic problems, and motivate further research in this direction.

An Improved Parameter Dimensionality Reduction Approach Based on a Fast Marching Method for Automatic History Matching

History matching is a critical step in reservoir numerical simulation algorithms. It is typically hindered by difficulties associated with the high-dimensionality of the problem and the gradient calculation approach. Here, a multi-step solving method is proposed by which, first, a Fast marching method (FMM) is used to calculate the pressure propagation time and determine the single-well sensitive area. Second, a mathematical model for history matching is implemented using a Bayesian framework. Third, an effective decomposition strategy is adopted for parameter dimensionality reduction. Finally, a localization matrix is constructed based on the single-well sensitive area data to modify the gradient of the objective function. This method has been verified through a water drive conceptual example and a real field case. The results have shown that the proposed method can generate more accurate gradient information and predictions compared to the traditional analytical gradient methods and other gradient-free algorithms.

A scalable species-based genetic algorithm for reinforcement learning problems

Abstract Reinforcement Learning (RL) methods often rely on gradient estimates to learn an optimal policy for control problems. These expensive computations result in long training times, a poor rate of convergence, and sample inefficiency when applied to real-world problems with a large state and action space. Evolutionary Computation (EC)-based techniques offer a gradient-free apparatus to train a deep neural network for RL problems. In this work, we leverage the benefits of EC and propose a novel variant of genetic algorithm called SP-GA which utilizes a species-inspired weight initialization strategy and trains a population of deep neural networks, each estimating the Q-function for the RL problem. Efficient encoding of a neural network that utilizes less memory is also proposed which provides an intuitive mechanism to apply Gaussian mutations and single-point crossover. The results on Atari 2600 games outline comparable performance with gradient-based algorithms like Deep Q-Network (DQN), Asynchronous Advantage Actor Critic (A3C), and gradient-free algorithms like Evolution Strategy (ES) and simple Genetic Algorithm (GA) while requiring far fewer hyperparameters to train. The algorithm also improved certain Key Performance Indicators (KPIs) when applied to a Remote Electrical Tilt (RET) optimization task in the telecommunication domain.

A new wake model and comparison of eight algorithms for layout optimization of wind farms in complex terrain

Layout optimization of wind farms constitutes an important and challenging task in complex terrain. This is especially due to the complex interactions of the boundary layer flows in complex terrain and wind turbine wakes, which renders wake modelling in complex terrain difficult. This study tackles this challenge with a new engineering wake model, which is developed by superposing a Gaussian shape wake model on top of the background flow field, assuming that the centerlines of wind turbine wakes follow the streamlines of the background flow field. The model is found to predict wind turbine wakes in complex terrain with good accuracy and at the same time it is computationally cheap to run for optimization applications. Comparisons with high fidelity simulations and field measurements for a real wind farm with 25 turbines in complex terrain demonstrate its effectiveness. A systematic comparison of eight optimization algorithms, which includes two gradient-based and six gradient-free algorithms, is also carried out for the layout optimization problem in complex terrain. To accelerate the optimization process, a double-stage approach is proposed, which optimizes the objective function first neglecting wake effects and then, in the second stage, including them. While all the tested optimization algorithms can improve the original wind farm layout, random search, local search, and pattern search are found to be the top three algorithms in terms of optimization results and computational cost.

Aerothermal optimization of a ribbed U-bend cooling channel using the adjoint method

Aerothermal optimization is a powerful technique for automating the design of turbine internal cooling passages, where both pressure loss and heat transfer are considered. Existing optimization studies commonly adopt gradient-free algorithms, which can handle only a few design variables. However, to enhance heat transfer, internal cooling designs use complex geometries consisting of ribbed serpentine channels, which need to be parameterized by using a large number of design variables. To address this need, we develop herein an approach for aerothermal optimization that uses a gradient-based optimizer in conjunction with a discrete adjoint method to efficiently compute the required gradients with respect to numerous design variables. We apply this approach to the design of a ribbed U-bend channel, which is representative of a section of turbine internal cooling passages. The objective function combines heat transfer and pressure loss as a weighted sum. We find the Pareto front for these two objectives by running five optimizations with different weights. We consider both a rib-free and a ribbed U-bend configuration. For the rib-free configuration, we use 113 design variables to parameterize the U-bend shape. We compare our optimization results with those from gradient-free methods and demonstrate that the proposed method leads to lower pressure loss while enhancing heat transfer. For the ribbed configuration, we use 146 design variables and allow the ribs to change their arrangement independently (shape, height, pitch, and angle). Each rib adopts a requisite arrangement to balance heat transfer and aerodynamics, depending on the local flow conditions. Optimizing the U-bend shape is shown to be more effective for improving overall heat transfer than optimizing the rib arrangement. However, optimizing ribs is more effective for improving local heat transfer. The results demonstrate that the proposed optimization framework has the potential to handle general turbine heat transfer designs, not only for internal cooling but also for other design problems, such as film cooling and jet impingement cooling.

Безградієнтні алгоритми керування для задач динамічної оптимізації

Безградієнтні алгоритми керування для задач динамічної оптимізації

Uniform Decomposition and Positive-Gradient Differential Evolution for Multi-Objective Design of Wind Turbine Blade

Convergence performance and optimization efficiency are two critical issues in the application of commonly used evolution algorithms in multi-objective design of wind turbines. A gradient-based multi-objective evolution algorithm is proposed for wind turbine blade design, based on uniform decomposition and positive-gradient differential evolution. In the uniform decomposition, uniformly distributed reference vectors are established in the objective space to maintain population diversity so that the population aggregations, which are commonly observed for wind turbine blade design using gradient-free algorithms, are minimized. The positive-gradient differential evolution is introduced for population evolution to increase optimization efficiency by guiding the evolutionary process and significantly reducing searching ranges of each individual. Two-, three- and four-objective optimizations of 1.5 MW wind turbine blades reveal that the proposed algorithm can deliver uniformly distributed optimal solutions in an efficient way, and has advantages over gradient-free algorithms in terms of convergence performance and optimization efficiency. These advantages increase with the optimization dimension, and the proposed algorithm is more suitable for optimizations of small size populations, thus remarkably enhancing the design efficiency.

On the influence of optimization algorithm and initial design on wing aerodynamic shape optimization

Aerodynamic shape optimization is a useful tool in wing design, but the impact of the choice of optimization algorithm and the multimodality of the design space in wing design optimization is still poorly understood. To address this, we benchmark both gradient-based and gradient-free optimization algorithms for computational fluid dynamics based aerodynamic shape optimization problems based on the Common Research Model wing geometry. The aerodynamic model solves the Reynolds-averaged Navier–Stokes equations with a Spalart–Allmaras turbulence model. The drag coefficient is minimized subject to lift, pitching moment, and geometry constraints, with up to 720 shape variables and 11 twist variables for two mesh sizes. We benchmark six gradient-based and three gradient-free algorithms by comparing both the accuracy of the optima and the computational cost. Most of the optimizers reach similar optima, but the gradient-based methods converge to more accurate solutions at a much lower computational cost. Since multimodality and nonsmoothness of the design space are common arguments for the use of gradient-free methods, we investigate these issues by solving the same optimization problem starting from a series of randomly generated initial geometries, as well as a wing based on the NACA 0012 airfoil with zero twist and constant thickness-to-chord ratio. All the optimizations consistently converge to practically identical results, where the differences in drag are within 0.05%, and the shapes and pressure distributions are very similar. Our overall conclusion is that the design space for wing design optimization with a fixed planform is largely convex, with a very small flat region that is multimodal because of numerical errors. However, this region is so small, and the differences in drag so minor, that the design space can be considered unimodal for all practical purposes.

Variable metric random pursuit

We consider unconstrained randomized optimization of smooth convex objective functions in the gradient-free setting. We analyze Random Pursuit (RP) algorithms with fixed (F-RP) and variable metric (V-RP). The algorithms only use zeroth-order information about the objective function and compute an approximate solution by repeated optimization over randomly chosen one-dimensional subspaces. The distribution of search directions is dictated by the chosen metric. Variable Metric RP uses novel variants of a randomized zeroth-order Hessian approximation scheme recently introduced by Leventhal and Lewis (Optimization 60(3):329---345, 2011. doi:10.1080/02331930903100141). We here present (1) a refined analysis of the expected single step progress of RP algorithms and their global convergence on (strictly) convex functions and (2) novel convergence bounds for V-RP on strongly convex functions. We also quantify how well the employed metric needs to match the local geometry of the function in order for the RP algorithms to converge with the best possible rate. Our theoretical results are accompanied by numerical experiments, comparing V-RP with the derivative-free schemes CMA-ES, Implicit Filtering, Nelder---Mead, NEWUOA, Pattern-Search and Nesterov's gradient-free algorithms.

A review of closed-loop reservoir management

The closed-loop reservoir management technique enables a dynamic and real-time optimal production schedule under the existing reservoir conditions to be achieved by adjusting the injection and production strategies. This is one of the most effective ways to exploit limited oil reserves more economically and efficiently. There are two steps in closed-loop reservoir management: automatic history matching and reservoir production optimization. Both of the steps are large-scale complicated optimization problems. This paper gives a general review of the two basic techniques in closed-loop reservoir management; summarizes the applications of gradient-based algorithms, gradient-free algorithms, and artificial intelligence algorithms; analyzes the characteristics and application conditions of these optimization methods; and finally discusses the emphases and directions of future research on both automatic history matching and reservoir production optimization.

Selected Engineering Applications of Gradient Free Optimisation Using Cuckoo Search and Proper Orthogonal Decomposition

This paper discusses some engineering applications of gradient free optimisation techniques. This is achieved using the development of the cuckoo search algorithm as a case study. The motivations behind using gradient free algorithms are discussed and illustrated using two specific practical examples. The first involves aerofoil shape optimisation, where it is shown that a modified cuckoo search algorithm performs well when applied both to aerofoil inverse design and aerofoil shape optimisation. This example is then used to discuss the use of reduced order modeling to decrease the computational cost of the optimisation process. We discuss which reduced order modeling techniques based on proper orthogonal decomposition are suitable for these optimisation applications. The second example is that of co-volume mesh optimisation, where it is shown that a modified cuckoo search can significantly outperform alternative non-optimisation and gradient based techniques. We conclude by discussing a number of remaining difficulties which may deter engineers from using gradient free techniques, and suggest ways in which these may be alleviated.

Optimization of Convex Functions with Random Pursuit

We consider unconstrained randomized optimization of convex objective functions. We analyze the Random Pursuit algorithm, which iteratively computes an approximate solution to the optimization problem by repeated optimization over a randomly chosen one-dimensional subspace. This randomized method only uses zeroth-order information about the objective function and does not need any problem-specific parametrization. We prove convergence and give convergence rates for smooth objectives assuming that the one-dimensional optimization can be solved exactly or approximately by an oracle. A convenient property of Random Pursuit is its invariance under strictly monotone transformations of the objective function. It thus enjoys identical convergence behavior on a wider function class. To support the theoretical results we present extensive numerical performance results of Random Pursuit, two gradient-free algorithms recently proposed by Nesterov, and a classical adaptive step size random search scheme. We also pres...