Gradient-free Optimization Method Research Articles

Multi-level delumping strategy for thermal enhanced oil recovery simulations at low pressure

We present a multi-level delumping method suitable for thermal enhanced oil recovery processes. At low pressures, the temperature variable is the most critical factor impacting the displacement process through viscosity reduction and evaporation/condensation effects. Hydrocarbon components are vaporized under high temperatures, move downstream in the gas phase and condense back to the liquid phase. That process is governed by the K-values of the components, evaporating out of the liquid phase sequentially with increasing temperatures. To reduce the computational cost, it is standard practice to reduce the number of (pseudo-) 6 components used in thermal reservoir simulation. Depending on the number and type of hydrocarbon pseudo-components retained in the simulations, we may not be able to capture the correct displacement due to large errors in the lumped phase behavior (flash) computations. We address that problem through a multi-level method: we use data obtained from a short simulation using the most detailed fluid description available, and leverage that information to guide a delumping process. We use temperature as a proxy variable for composition, and select reference temperatures. We extract the corresponding reference compositions from the detailed run and use them to extend the lumped pseudo-components to an approximate detailed composition. We compute the phase mole fractions as well as the gas compressibility factor. We test our method using six heavy oil samples, and under two different recovery processes: hot nitrogen injection and in-situ combustion (air injection and exothermic oxidation reactions). The average error on the liquid mole fraction is reduced by 4–12 times (depending on the oil samples) compared to the flash using pseudo-components, and the maximum error by 6–48 times. We illustrate that the method is amenable to manually adding more information about the physics of some oil samples. We also discuss how to efficiently pick the reference temperatures. For uniformly sampled temperatures (between a minimum and maximum temperature), we conduct a sensitivity study which led us to use six temperatures. We ran both local (Pattern Search, PS) and global (Particle Swarm Optimization, PSO) gradient-free optimization methods. PS is able to find the closest local minimum to the uniform set, giving a limited improvement of 6.5%. The known increased cost for PSO is worth the investment in at least one of the cases we considered, leading to a 67% improvement.

Convergence of a first-order consensus-based global optimization algorithm

Global optimization of a non-convex objective function often appears in large-scale machine learning and artificial intelligence applications. Recently, consensus-based optimization (CBO) methods have been introduced as one of the gradient-free optimization methods. In this paper, we provide a convergence analysis for the first-order CBO method in [J. A. Carrillo, S. Jin, L. Li and Y. Zhu, A consensus-based global optimization method for high dimensional machine learning problems, https://arxiv.org/abs/1909.09249v1 ]. Prior to this work, the convergence study was carried out for CBO methods on corresponding mean-field limit, a Fokker–Planck equation, which does not imply the convergence of the CBO method per se. Based on the consensus estimate directly on the first-order CBO model, we provide a convergence analysis of the first-order CBO method [J. A. Carrillo, S. Jin, L. Li and Y. Zhu, A consensus-based global optimization method for high dimensional machine learning problems, https://arxiv.org/abs/1909.09249v1 ] without resorting to the corresponding mean-field model. Our convergence analysis consists of two steps. In the first step, we show that the CBO model exhibits a global consensus time asymptotically for any initial data, and in the second step, we provide a sufficient condition on system parameters — which is dimension independent — and initial data which guarantee that the converged consensus state lies in a small neighborhood of the global minimum almost surely.

Shape optimization by conventional and extended isogeometric boundary element method with PSO for two-dimensional Helmholtz acoustic problems

In this paper, a new approach is developed for applications of shape optimization on the two-dimensional time harmonic wave propagation (Helmholtz equation) in acoustic problems. The particle swarm optimization (PSO) algorithm - a gradient-free optimization method avoiding the sensitivity analysis - is coupled with two boundary element methods (BEM) and isogeometric analysis (IGA). The first method is the conventional isogeometric boundary element method (IGABEM). The second method is the eXtended IGABEM (XIBEM) enriched with the partition-of-unity expansion using a set of plane waves. In both methods, the computational domain is parameterized and the unknown solution is approximated using non-uniform rational B-splines basis functions (NURBS).In the optimization models, the advantage of IGA is the feature of representing the three models; i.e. shape design/analysis/optimization, using a set of control points, which also represent control variables and optimization parameters, making communication between the three models easy and straightforward.A numerical example is considered for the duct problem to validate the presented techniques against the analytical solution. Furthermore, two different applications for various frequencies are studied; the vertical noise barrier and the horn problems, and the obtained results are compared against previously published numerical methods using sensitivity analysis and genetic algorithms to verify the efficiency of the proposed approaches.

A holistic review on artificial intelligence techniques for well placement optimization problem

Well placement optimization is one of the major challenging factors in the field development process of oil and gas industry. The objective function of well placement optimization is considered as high dimensional, discontinuous and multi-model. Over the last decade, both gradient-based and gradient-free optimization methods have been implemented to tackle this problem. Nature-inspired gradient-free optimization algorithms like particle swarm optimization, genetic algorithm, covariance matrix adaptation evolution strategy and differential evolution have been utilized in this area. These optimization techniques are implemented as stand-alone or as hybrid form to maximize the economic factors. In this paper, several nature-inspired metaheuristic optimization techniques and their application to maximize the economic factors are reviewed. Newly developed optimization algorithms are very efficient and favorable when compared to other established optimization algorithms and in all cases, it has been noticed that hybridization of two or more algorithms works better than stand-alone algorithms. Furthermore, none of the single optimization techniques can be established as being universally superior which aligns with no free lunch theorem. For future endeavor, combining optimization methods and exploiting multiple optimization processes for faster convergence and developing efficient proxy model is expected.

Fine-tuning Deep RL with Gradient-Free Optimization

Abstract Deep reinforcement learning makes it possible to train control policies that map high-dimensional observations to actions. These methods typically use gradient-based optimization techniques to enable relatively efficient learning, but are notoriously sensitive to hyperparameter choices and do not have good convergence properties. Gradient-free optimization methods, such as evolutionary strategies, can offer a more stable alternative but tend to be much less sample efficient. In this work we propose a combination, using the relative strengths of both. We start with a gradient-based initial training phase, which is used to quickly learn both a state representation and an initial policy. This phase is followed by a gradient-free optimization of only the final action selection parameters. This enables the policy to improve in a stable manner to a performance level not obtained by gradient-based optimization alone, using many fewer trials than methods using only gradient-free optimization. We demonstrate the effectiveness of the method on two Atari games, a continuous control benchmark and the CarRacing-v0 benchmark. On the latter we surpass the best previously reported score while using significantly fewer episodes.

Study of Geologically Consistent History Matching Peculiarities by Means of Gradient-Free Optimization Methods

This paper presents some results and distinctive features of the algorithm for automated history matching of 3D reservoir flow models using gradient-free methods. Widely used Nelder-Mead method was chosen for optimization. In order to preserve geological consistency of a model within the history matching process, control parameters are comprised of “porosity-to-permeability” relation parameters, anisotropic (semi)variogram parameters, and reservoir properties at pilot points. The control parameters are used at each iteration of the history matching process to generate porosity and permeability distributions based on static well data and perform flow simulations. Test studies were conducted for a specially developed synthetic model representing a inhomogeneous five-spot well pattern element. Oil flow rates at production wells and water flow rate at the injection well, as well as bottomhole pressures of all wells were chosen as measured dynamic data in the objective function. The influence of initial guess of parameter values and relative weights in the objective function on the quality of reconstruction of the ‘true’ solution and convergence rate were evaluated. Problems of gradient-free optimization methods application and specific features of residuals normalization in the objective function, as well as influence of pilot points are discussed.

Aerodynamic Optimal Design for Horizontal Axis Wind Turbine Airfoil Using Integrated Optimization Method

In this work, a new approach for aerodynamic optimization of horizontal axis wind turbine (HAWT) airfoil is presented. This technique combines commercial computational fluid dynamics (CFD) codes with differential evolution (DE), a reliable gradient-free global optimization method. During the optimization process, commercial CFD codes are used to evaluate aerodynamic characteristics of HAWT airfoil and an improved DE algorithm is utilized to find the optimal airfoil design. The objective of this research is to maximize the aerodynamic coefficients of HAWT airfoil at the design angle of attack (AOA) with specific ambient environment. The airfoil shape is modeled by control points which their coordinates are design variables. The reliability of CFD codes is validated by comparing the analytical results of a typical HAWT airfoil with its experimental data. Finally, the optimal design of wind turbine airfoil is evaluated about aerodynamic performance in comparison with existing airfoils and some discussions are performed.

Accelerated Gradient-Free Optimization Methods with a Non-Euclidean Proximal Operator

We propose an accelerated gradient-free method with a non-Euclidean proximal operator associated with the p-norm (1 ⩽ p ⩽ 2). We obtain estimates for the rate of convergence of the method under low noise arising in the calculation of the function value. We present the results of computational experiments.

A novel multi-objective Dynamic Programming optimization method: Performance management of a solar thermal power plant as a case study

Due to the intermittent nature of solar irradiance, solar power plants are usually equipped with energy storage systems. Suitable charge and discharge management of the storage systems can considerably help increase the reliability and profitability of the solar systems. In this regard, various optimization approaches, with their own strengths and limitations, have been employed by literature. Dynamic Programming (DP) is one of the fittest approaches for the wide range of engineering problems which exhibit the properties of overlapping sub-problems. DP is a simple, gradient-free, efficient, and deterministic optimization method that guarantees the optimal solution. However, this method has not been developed for multi-objective problems. This study develops a multi-objective DP method and employs it for the performance management of a solar power plant equipped with thermal energy storage system. “Daily electricity generation” and “daily revenue obtained from selling electricity” are considered to be the objective functions. The superiority of the developed method is shown through a comparison with one of the most commonly used multi-objective optimization approaches, NSGA-II. This comparison indicates that the multi-objective DP attains 3.0%–7.5% greater values of electricity generation and 3.1%–12.6% higher values of revenue than NSGA-II, for the different levels of solar radiation.

A method for reducing multi-modality in the wind farm layout optimization problem

This paper presents a process using an approach related to continuation optimization methods for reducing multi-modality in the wind farm layout optimization problem, referred to as Wake Expansion Continuation (WEC). The reduction in multi-modality is achieved by starting with an increased wake spread, while maintaining normal velocity deficits at the center of the wakes, and then reducing the wake spread for each of a series of optimization runs until the standard wake spread is used. Two optimization cases were tested, one with 16 turbines and one with 38 turbines, by optimizing from 200 different starting positions with a gradient-based method, a gradient-free method, and a gradient-based method using WEC. Results using WEC show a 4% mean optimized annual energy production (AEP) improvement compared to gradient-based optimization without WEC for both test cases. A gradient-free algorithm had a mean optimized AEP that was 1% higher than WEC for the 16-turbine case, but WEC resulted in a 10% higher mean optimized AEP compared to a gradient-free optimization method for the 38-turbine problem. These results are specific to the test cases and may not be generally representative.

Gradient-based Regularization Parameter Selection for Problems With Nonsmooth Penalty Functions

ABSTRACTIn high-dimensional and/or nonparametric regression problems, regularization (or penalization) is used to control model complexity and induce desired structure. Each penalty has a weight parameter that indicates how strongly the structure corresponding to that penalty should be enforced. Typically, the parameters are chosen to minimize the error on a separate validation set using a simple grid search or a gradient-free optimization method. It is more efficient to tune parameters if the gradient can be determined, but this is often difficult for problems with nonsmooth penalty functions. Here, we show that for many penalized regression problems, the validation loss is actually smooth almost-everywhere with respect to the penalty parameters. We can, therefore, apply a modified gradient descent algorithm to tune parameters. Through simulation studies on example regression problems, we find that increasing the number of penalty parameters and tuning them using our method can decrease the generalization error.

Application of fast analytical approach and AI optimization techniques to hydraulic fracture stage placement in shale gas reservoirs

In the last decades, natural gas from unconventional reservoirs has become a major portion of total gas supply due to advances in horizontal well drilling and multi-stage hydraulic fracturing as well as reduction of operational costs and capital expenditures. However, hydraulic fracturing technique is a still costly and resource intensive production strategy that requires optimal planning to conform to the best safety practices and to obtain the highest returns on investments. Thus, the proposed fast and reliable hydraulic fracture (HF) placement optimization technique based on physics and analytical equations is the key tool to balance between gas production costs and anticipated revenues.In this study, we develop an analytical model in which the modified Wattenbarger slab model with the pseudo-pressure approach are integrated into the Net Present Value (NPV) as the objective function. We consider four decision variables including number of HF stages, HF spacing, HF half-length, and wellbore spacing and use three stochastic gradient-free optimization methods (i.e., genetic algorithm (GA), differential evolution (DE), and particle swarm optimization (PSO)) to optimize the objective function on a synthetic shale gas reservoir model with the Barnett Shale properties. To verify the accuracy of the obtained optimal solutions, we conduct four trials for each stochastic optimization method with 100 generations and the population size of 20. The results show that the best overall value of the NPV found by PSO are 1.7% and 7.6% higher than those obtained by DE and GA, respectively. Moreover, PSO has the fastest convergence rate (in 50 generations), saves at least 10% of the computational time in comparison to those required by other methods, and results in the same optimal solution in all trials. Finally, considering bilinear flow at the early stages of the production period, nonlinear flow at the late production time, and gravitational effects in the analytical model are still open areas for future research in this field.

Adjoint-Based Two-Step Optimization Method Using Proper Orthogonal Decomposition and Domain Decomposition

A modified two-step method for aerodynamic shape optimization is proposed, where proper orthogonal decomposition and automatic domain-decomposition techniques are adopted to accelerate flow and adjoint solutions in the second-step optimization. As the first step in this method, the initial optimization is first performed by a genetic algorithm coupled with a kriging model. Next, a reduced-order model is set up via proper orthogonal decomposition with the reuse of ready-made flow snapshots in the first-step optimization. A proper-orthogonal-decomposition Petrov-Galerkin method is investigated to provide fast flow predictions in the second-step optimization. With the assistance of an error estimation method and an automatic domain-decomposition method that are presented in this paper, the sensitive domain in the computational grid is split out. To improve the accuracy, predictions provided by proper orthogonal decomposition in the sensitive domain are further corrected by computational fluid dynamics modifications. Meanwhile, in order to accelerate gradient solutions, the adjoint equation of this proper orthogonal decomposition and domain-decomposition-based flow analysis method is derived and discussed. The optimization results of a two-dimensional airfoil design test and a three-dimensional wing design test highlight the efficiency of the proposed method when compared with results of either a gradient-free optimization method or a traditional two-step method.

Approximation of nonlinear discrete-time system using FA-based neural network

In this paper, we focus on an important nature inspired optimization method known as Firefly Algorithm (FA). Weight initialization of neural network is a random matrix between certain interval; FA is used for weight initialization instead of random weights to Back-Propagation Neural Network (BPN). This work emphasis on approximation of nonlinear discrete-time system using FA-based BPN, i.e., FABPN. Neural network behaves as granular computing methodologies, which depends on some important parameters including initial weight matrix. FA is gradient-free optimization method used for balanced premature convergence and stagnation. We consider Mean Square Error (MSE) and Mean Absolute Percentage Error (MAPE) as objective functions. The proposed method is based on characteristics of fireflies, i.e., updating population, moving fireflies, and best solution. The performance of FA-based Back-Propagation Network (FABPN) has been demonstrated using two nonlinear discrete-time systems and one real example. Results of the FABPN envisage that the proposed method exhibits sky-scraping identification accuracy and tracking error in comparison with Particle Swarm Optimization-based BPN (PSOBPN). Tracking error using proposed method is lies on compact set in the closed neighborhood of zero.

A predictive framework to elucidate venous stenosis: CFD & shape optimization

The surgical creation of vascular accesses for renal failure patients provides an abnormally high flow rate conduit in the patient’s upper arm vasculature that facilitates the hemodialysis treatment. These vascular accesses, however, are very often associated with complications that lead to access failure and thrombotic incidents, mainly due to excessive neointimal hyperplasia (NH) and subsequently stenosis. Development of a framework to monitor and predict the evolution of the venous system post access creation can greatly contribute to maintaining access patency. Computational fluid dynamics (CFD) has been exploited to inspect the non-homeostatic wall shear stress (WSS) distribution that is speculated to trigger NH in the patient cohort under investigation. Thereafter, CFD in liaison with a gradient-free shape optimization method has been employed to analyze the deformation modes of the venous system enduring non-physiological hemodynamics. It is observed that the optimally evolved shapes and their corresponding hemodynamics strive to restore the homeostatic state of the venous system to a normal, pre-surgery condition. It is concluded that a CFD-shape optimization coupling that seeks to regulate the WSS back to a well-defined physiological WSS target range can accurately predict the mode of patient-specific access failure.

Dimensionality reduction for production optimization using polynomial approximations

The objective of this paper is to introduce a novel paradigm to reduce the computational effort in waterflooding global optimization problems while realizing smooth well control trajectories amenable for practical deployments in the field. In order to overcome the problems of slow convergence and non-smooth impractical control strategies, often associated with gradient-free optimization (GFO) methods, we introduce a generalized approach which represent the controls by smooth polynomial approximations either by a polynomial function or by a piecewise polynomial interpolation, which we denote as function control method (FCM) and interpolation control method (ICM), respectively. Using these approaches, we aim to optimize the coefficients of the selected functions or the interpolation points in order to represent the well-control trajectories along a time horizon. Our results demonstrate significant computational savings, due to a substantial reduction in the number of control parameters, as we seek the optimal polynomial coefficients or the interpolation points to describe the control trajectories as opposed to directly searching for the optimal control values (bottom hole pressure) at each time interval. We demonstrate the efficiency of the method on two and three-dimensional models, where we found the optimal variables using a parallel dynamic-neighborhood particle swarm optimization (PSO). We compared our FCM-PSO and ICM-PSO to the traditional formulation solved by both gradient-free and gradient-based methods. In all comparisons, both FCM and ICM show very good to superior performances.

A new design approach based on differential evolution algorithm for geometric optimization of magnetorheological brakes

In recent years, various types of magnetorheological brakes (MRBs) have been proposed and optimized by different optimization algorithms that are integrated in commercial software such as ANSYS and Comsol Multiphysics. However, many of these optimization algorithms often possess some noteworthy shortcomings such as the trap of solutions at local extremes, or the limited number of design variables or the difficulty of dealing with discrete design variables. Thus, to overcome these limitations and develop an efficient computation tool for optimal design of the MRBs, an optimization procedure that combines differential evolution (DE), a gradient-free global optimization method with finite element analysis (FEA) is proposed in this paper. The proposed approach is then applied to the optimal design of MRBs with different configurations including conventional MRBs and MRBs with coils placed on the side housings. Moreover, to approach a real-life design, some necessary design variables of MRBs are considered as discrete variables in the optimization process. The obtained optimal design results are compared with those of available optimal designs in the literature. The results reveal that the proposed method outperforms some traditional approaches.

Modified genetic algorithm using Box Complex method: Application to optimal control problems

Genetic algorithm (GA) is a popular stochastic optimization technique for past couple of decades and has been successfully applied to numerous applications of single and multi-objective optimization problems. Various modifications in GA are proposed in open literature to increase convergence rate and probability of obtaining global minimum by increasing population diversity. Box Complex is a gradient free optimization method having good convergence property. To enhance convergence property of GA, we in this work propose an extension of GA by combining the global search property of GA with a convergence property of Box Complex method. We add one or more population members created by Box Complex method using the current population and replace the equal number of worst population members every generation. A comparison study of the proposed GA with conventional GA and widely accepted jumping gene GA (JG GA) is presented in this work. We have considered two benchmark optimization functions, namely Rosenbrock's and Ackley's Path function. We also carry out the comparison of GAs for three optimal control problems. One of them is the maximization of product concentration with multiple reactions in a batch reactor. Minimization of the off-spec product during product grade transition in a polymerization reactor is considered as the second optimal control problem. The third test application is optimal control of a non-isothermal plug flow reactor.There are two user defined parameters in the proposed algorithm, namely number of Box Complex Members (BCM), and expansion/contraction factor α. Effect of both these parameters on the convergence profile have been presented in this work for the proposed GA. A statistical summary of ten simulation runs for the proposed GA, JG GA, and conventional GA has been discussed for each of the five applications.

Improving the Ensemble-Optimization Method Through Covariance-Matrix Adaptation

Summary Ensemble optimization (referred to throughout the remainder of the paper as EnOpt) is a rapidly emerging method for reservoir-model-based production optimization. EnOpt uses an ensemble of controls to approximate the gradient of the objective function with respect to the controls. Current implementations of EnOpt use a Gaussian ensemble of control perturbations with a constant covariance matrix, and thus a constant perturbation size, during the entire optimization process. The covariance-matrix-adaptation evolutionary strategy is a gradient-free optimization method developed in the “machine learning” community, which also uses an ensemble of controls, but with a covariance matrix that is continually updated during the optimization process. It was shown to be an efficient method for several difficult but small-dimensional optimization problems and was recently applied in the petroleum industry for well location and production optimization. In this study, we investigate the scope to improve the computational efficiency of EnOpt through the use of covariance-matrix adaptation (referred to throughout the remainder of the paper as CMA-EnOpt). The resulting method is applied to the waterflooding optimization of a small multilayer test model and a modified version of the Brugge benchmark model. The controls used are inflow-control-valve settings at predefined time intervals for injectors and producers with undiscounted net present value as the objective function. We compare EnOpt and CMA-EnOpt starting from identical covariance matrices. For the small model, we achieve only slightly higher (0.7 to 1.8%) objective-function values and modest speedups with CMA-EnOpt compared with EnOpt. Significantly higher objective-function values (10%) are obtained for the modified Brugge model. The possibility to adapt the covariance matrix, and thus the perturbation size, during the optimization allows for the use of relatively large perturbations initially, for fast exploration of the control space, and small perturbations later, for more-precise gradients near the optimum. Moreover, the results demonstrate that a major benefit of CMA-EnOpt is its robustness with respect to the initial choice of the covariance matrix. A poor choice of the initial matrix can be detrimental to EnOpt, whereas the CMA-EnOpt performance is near-independent of the initial choice and produces higher objective-function values at no additional computational cost.

An improved SPSA algorithm for system identification using fuzzy rules for training neural networks

Simultaneous perturbation stochastic approximation (SPSA) belongs to the class of gradient-free optimization methods that extract gradient information from successive objective function evaluation. This paper describes an improved SPSA algorithm, which entails fuzzy adaptive gain sequences, gradient smoothing, and a step rejection procedure to enhance convergence and stability. The proposed fuzzy adaptive simultaneous perturbation approximation (FASPA) algorithm is particularly well suited to problems involving a large number of parameters such as those encountered in nonlinear system identification using neural networks (NNs). Accordingly, a multilayer perceptron (MLP) network with popular training algorithms was used to predicate the system response. We found that an MLP trained by FASPSA had the desired accuracy that was comparable to results obtained by traditional system identification algorithms. Simulation results for typical nonlinear systems demonstrate that the proposed NN architecture trained with FASPSA yields improved system identification as measured by reduced time of convergence and a smaller identification error.