Derivative-free Optimization Algorithm Research Articles

Computation of the output of a function with fuzzy inputs based on a low-rank tensor approximation

We apply a derivative-free optimization method based on novel low-rank tensor methods to the problem of propagating fuzzy uncertainty through a continuous real-valued function. Adhering to Zadeh's extension principle, such a problem can be reformulated as a sequence of optimization problems over nested search spaces. The optimization method we use is based on a low-rank tensor approximation of the function sampled on a grid and a search for the minimal and maximal entries in this low-rank tensor. In contrast to classical fuzzy uncertainty propagation methods, such as the vertex method and the transformation method, the method we propose does not exhibit an inherent exponential scaling for increasing dimension of the search space. Obviously, no derivative-free optimization algorithm can exist which shows sub-exponential scaling with the dimension for all possible continuous functions. The algorithm that we present here, however, can exploit a specific type of structure and regularity (beyond continuity) that is often present in real-world optimization problems. We illustrate this with some high-dimensional numerical examples where the presented method clearly outperforms some established derivative-free optimization codes.

Using a derivative-free optimization method for multiple solutions of inverse transport problems

Identifying unknown components of an object that emits radiation is an important problem for national and global security. Radiation signatures measured from an object of interest can be used to infer object parameter values that are not known. This problem is called an inverse transport problem. An inverse transport problem may have multiple solutions and the most widely used approach for its solution is an iterative optimization method. This paper proposes a stochastic derivative-free global optimization algorithm to find multiple solutions of inverse transport problems. The algorithm is an extension of a multilevel single linkage (MLSL) method where a mesh adaptive direct search (MADS) algorithm is incorporated into the local phase. Numerical test cases using uncollided fluxes of discrete gamma-ray lines are presented to show the performance of this new algorithm.

Trade-off Analysis for Infrastructure Management: New Approaches to Cross-asset Challenges

As transportation infrastructure managers pursue performance-based management, increased scrutiny is rightfully imposed by the stakeholders (tax-payers and their legislative representatives).The performance of public transportation agencies is not evaluated on a single asset type (i.e. on Pavement, or Bridges alone) but on the system as a whole (cross-assets). Many commercial software packages are focused on managing a single asset type making cross-asset analysis difficult. However, a main problem for transportation agency managers is how to split available budget among different types of assets to provide the best overall performance to the public stakeholders.This paper focuses on two different approaches to the Cross-Asset Problem (CAP) and demonstrates, using real data examples, how optimal budget distribution for various asset types can be found for large agencies.The paper formulates individual asset management as an integer optimization problem (IP).It then discusses two approaches to the CAP. The first approach assumes that overall agency performance can be expressed as a linear combination of individual asset type performances. In this case the CAP can be formulated as an Integer Optimization Problem (IP) where its objective and constraints are defined as a simple linear combination of the objectives/constraints for corresponding asset types. Model formulation, running times and optimal budget distributions using real transportation agency data are presented for this approach.The second approach requires no assumptions on the CAPs objective formula and considers separate asset type problems as a black box which, for a given budget distribution, returns the best overall performance for that asset type. A Derivative Free Optimization algorithm is presented for this setup, showing running time and final budget distributions for several examples.Finally, the paper outlines the advantages and disadvantages of each approach and provides guidance on when each approach should be used.

Optimal Robot Placement for Tasks Execution

Automotive assembly cells are cluttered environments, including robots, workpieces, and fixtures. Due to high volumes and several product variants assembled in the same cell, robot placement is crucial to increase flexibility and throughput. In this paper, we propose a novel method to optimize the base position of an industrial robot with the objective to reach all predefined tasks and minimize cycle time: robot inverse kinematics and collision avoidance are integrated together with a derivative-free optimization algorithm. This approach is successfully used to find feasible solutions on industrial test cases, showing up to 20% cycle time improvement.

A second-order globally convergent direct-search method and its worst-case complexity

Direct-search algorithms form one of the main classes of algorithms for smooth unconstrained derivative-free optimization, due to their simplicity and their well-established convergence results. They proceed by iteratively looking for improvement along some vectors or directions. In the presence of smoothness, first-order global convergence comes from the ability of the vectors to approximate the steepest descent direction, which can be quantified by a first-order criticality (cosine) measure. The use of a set of vectors with a positive cosine measure together with the imposition of a sufficient decrease condition to accept new iterates leads to a convergence result as well as a worst-case complexity bound. In this paper, we present a second-order study of a general class of direct-search methods. We start by proving a weak second-order convergence result related to a criticality measure defined along the directions used throughout the iterations. Extensions of this result to obtain a true second-order optimality one are discussed, one possibility being a method using approximate Hessian eigenvectors as directions (which is proved to be truly second-order globally convergent). Numerically guaranteeing such a convergence can be rather expensive to ensure, as it is indicated by the worst-case complexity analysis provided in this paper, but turns out to be appropriate for some pathological examples.

Delaunay-based derivative-free optimization via global surrogates, part I: linear constraints

The derivative-free global optimization algorithms developed in Part I of this study, for linearly constrained problems, are extended to nonconvex n-dimensional problems with convex constraints. The initial $$n+1$$n+1 feasible datapoints are chosen in such a way that the volume of the simplex generated by these datapoints is maximized; as the algorithm proceeds, additional feasible datapoints are added in such a way that the convex hull of the available datapoints efficiently increases towards the boundaries of the feasible domain. Similar to the algorithms developed in Part I of this study, at each step of the algorithm, a search function is defined based on an interpolating function which passes through all available datapoints and a synthetic uncertainty function which characterizes the distance to the nearest datapoints. This uncertainty function, in turn, is built on the framework of a Delaunay triangulation, which is itself based on all available datapoints together with the (infeasible) vertices of an exterior simplex which completely contains the feasible domain. The search function is minimized within those simplices of this Delaunay triangulation that do not include the vertices of the exterior simplex. If the outcome of this minimization is contained within the circumsphere of a simplex which includes a vertex of the exterior simplex, this new point is projected out to the boundary of the feasible domain. For problems in which the feasible domain includes edges (due to the intersection of multiple twice-differentiable constraints), a modified search function is considered in the vicinity of these edges to assure convergence.

Global optimization of general constrained grey-box models: new method and its application to constrained PDEs for pressure swing adsorption

This paper introduces a novel methodology for the global optimization of general constrained grey-box problems. A grey-box problem may contain a combination of black-box constraints and constraints with a known functional form. The novel features of this work include (i) the selection of initial samples through a subset selection optimization problem from a large number of faster low-fidelity model samples (when a low-fidelity model is available), (ii) the exploration of a diverse set of interpolating and non-interpolating functional forms for representing the objective function and each of the constraints, (iii) the global optimization of the parameter estimation of surrogate functions and the global optimization of the constrained grey-box formulation, and (iv) the updating of variable bounds based on a clustering technique. The performance of the algorithm is presented for a set of case studies representing an expensive non-linear algebraic partial differential equation simulation of a pressure swing adsorption system for $$\hbox {CO}_{2}$$CO2. We address three significant sources of variability and their effects on the consistency and reliability of the algorithm: (i) the initial sampling variability, (ii) the type of surrogate function, and (iii) global versus local optimization of the surrogate function parameter estimation and overall surrogate constrained grey-box problem. It is shown that globally optimizing the parameters in the parameter estimation model, and globally optimizing the constrained grey-box formulation has a significant impact on the performance. The effect of sampling variability is mitigated by a two-stage sampling approach which exploits information from reduced-order models. Finally, the proposed global optimization approach is compared to existing constrained derivative-free optimization algorithms.

Physics-based Surrogate Optimization of Francis Turbine Runner Blades, Using Mesh Adaptive Direct Search and Evolutionary Algorithms

A robust multi-fidelity optimization methodology has been developed, focusing on efficiently handling industrial runner design of hydraulic Francis turbines. The computational task is split between low- and high-fidelity phases in order to properly balance the CFD cost and required accuracy in different design stages. In the low-fidelity phase, a physics-based surrogate optimization loop manages a large number of iterative optimization evaluations. Two derivative-free optimization methods use an inviscid flow solver as a physics-based surrogate to obtain the main characteristics of a good design in a relatively fast iterative process. The case study of a runner design for a low-head Francis turbine indicates advantages of integrating two derivative-free optimization algorithms with different local- and global search capabilities.

On the properties of positive spanning sets and positive bases

The concepts of positive span and positive basis are important in derivative-free optimization. In fact, a well-known result is that if the gradient of a continuously differentiable objective function on \(\mathbb{R}^n\) is nonzero at a point, then one of the vectors in any positive basis (or any positive spanning set) of \(\mathbb{R}^n\) is a descent direction for the objective function from that point. This article summarizes the basic results and explores additional properties of positive spanning sets, positively independent sets and positive bases that are potentially useful in the design of derivative-free optimization algorithms. In particular, it provides construction procedures for these special sets of vectors that were not previously mentioned in the literature. It also proves that invertible linear transformations preserve positive independence and the positive spanning property. Moreover, this article introduces the notion of linear equivalence between positive spanning sets and between positively independent sets to simplify the analysis of their structures. Linear equivalence turns out to be a generalization of the concept of structural equivalence between positive bases that was introduced by Coope and Price (SIAM J Optim 11:859–869, 2001). Furthermore, this article clarifies which properties of linearly independent sets, spanning sets and ordinary bases carry over to positively independent sets, positive spanning sets, and positive bases. For example, a linearly independent set can always be extended to a basis of a linear space but a positively independent set cannot always be extended to a positive basis. Also, the maximum size of a linearly independent set in \(R^n\) is n but there is no limit to the size of a positively independent set in \(\mathbb{R}^n\) when \(n \ge 3\). Whenever possible, the results are proved for the more general case of frames of convex cones instead of focusing only on positive bases of linear spaces. In addition, this article discusses some algorithms for determining whether a given set of vectors is positively independent or whether it positively spans a linear subspace of \(\mathbb{R}^n\). Finally, it provides an algorithm for extending any finite set of vectors to a positive spanning set of \(\mathbb{R}^n\) using only a relatively small number of additional vectors.

Coupling Multi-objective Constrained Optimization, Life Cycle Assessment, and Detailed Process Simulation for Potable Water Treatment Chains

This paper deals with the optimization of key decision variables of a potable water production plant. To this end we propose a multi-objective constrained optimization approach which trades-off operational costs and environmental impact while satisfying outlet water drinkability criteria. The proposed solution approach consists in coupling a meta-heuristic global optimization algorithmand EVALEAU, which is a state-of-the-art process modelling - life cycle assessment (LCA) tool for simulating potable water treatment chains. The proposed eco-design optimization approach is successfully illustrated on a real-world model of a water production plant, and the relative performances of two highly praised stateof- the-art derivative-free global optimization algorithms,namely the Strength Pareto Evolutionary Algorithm (SPEA2) and Non-dominated Sorting Genetic Algorithm (NSGA-II),are compared.

Optimal control of high-order harmonics for the generation of an isolated ultrashort attosecond pulse with two-color midinfrared laser fields

We present an efficient high-order-harmonic optimal control scheme for the generation of the ultrabroad supercontinuum spectrum and an isolated ultrashort attosecond pulse in gases with a two-color mid-IR laser field. The optimal control scheme is implemented using a derivative-free unconstrained optimization algorithm called NEWUOA (NEW Unconstrained Optimization Algorithm). For illustration, the high-order-harmonic generation (HHG) of a hydrogen atom is considered for optimization. It is shown that optimally shaped laser waveforms can greatly enhance and extend the HHG plateau and efficiently generate an isolated ultrashort attosecond pulse. Moreover, by performing accurate semiclassical simulations and a detailed wavelet time-frequency analysis, we found that the optimized supercontinuum harmonics corresponding to long-trajectory electrons are responsible for an isolated ultrashort 21-as pulse.

Optimizing horizontal alignment of roads in a specified corridor

Finding an optimal alignment connecting two end-points in a specified corridor is a complex problem that requires solving three interrelated sub-problems, namely the horizontal alignment, vertical alignment and earthwork optimization problems. In this research, we developed a novel bi-level optimization model combining those three problems. In the outer level of the model, we optimize the horizontal alignment and in the inner level of the model a vertical alignment optimization problem considering earthwork allocation is solved for a fixed horizontal alignment. Derivative-free optimization algorithms are used to solve the outer problem. The result of our model gives an optimal horizontal alignment in the form of a linear-circular curve and an optimal vertical alignment in the form of a quadratic spline. Our model is tested on real-life data. The numerical results show that our approach improves the road alignment designed by civil engineers by 27% on average, resulting in potentially millions of dollars of savings.

Data fitting by G1 rational cubic Bézier curves using harmony search

A metaheuristic algorithm, called Harmony Search (HS) is implemented for data fitting by rational cubic Bézier curves. HS is a derivative-free real parameter optimization algorithm, and draws an inspiration from the musical improvisation process of searching for a perfect state of harmony. HS is suitable for multivariate non-linear optimization problem. It is mainly achieved by data fitting using rational cubic Bézier curves with G1 continuity for every joint of segments of the whole data sets. This approach has significant contributions in making the technique automated. HS is used to optimize positions of middle points and values of the shape parameters. Test outline images and comparative experimental analysis are presented to show effectiveness and robustness of the proposed method. Statistical testing between HS and two other different metaheuristic algorithms is used in the analysis on several outline images. All of the algorithms improvised a near optimal solution but the result that is obtained by the HS is better than the results of the other two algorithms.

A derivative-free approach for a simulation-based optimization problem in healthcare

Hospitals have been challenged in recent years to deliver high quality care with limited resources. Given the pressure to contain costs, developing procedures for optimal resource allocation becomes more and more critical in this context. Indeed, under/overutilization of emergency room and ward resources can either compromise a hospital’s ability to provide the best possible care, or result in precious funding going toward underutilized resources. Simulation-based optimization tools then help facilitating the planning and management of hospital services, by maximizing/minimizing some specific indices (e.g. net profit) subject to given clinical and economical constraints. In this work, we develop a simulation-based optimization approach for the resource planning of a specific hospital ward. At each step, we first consider a suitably chosen resource setting and evaluate both efficiency and satisfaction of the restrictions by means of a discrete-event simulation model. Then, taking into account the information obtained by the simulation process, we use a derivative-free optimization algorithm to modify the given setting. We report results for a real-world problem coming from the obstetrics ward of an Italian hospital showing both the effectiveness and the efficiency of the proposed approach.

Co-Production of Olefins, Fuels, and Electricity from Conventional Pipeline Gas and Shale Gas with Near-Zero CO2 Emissions. Part II: Economic Performance

In this paper, techno-economic analyses of a polygeneration system for the production of olefins, transportation fuels and electricity are performed, considering various process options. Derivative-free optimization algorithms were coupled with Aspen Plus simulation models to determine the optimum product portfolio as a function of a wide variety of market prices. The optimization results show that the proposed plant is capable of producing olefins with the same production costs as traditional petrochemical routes while having effectively zero process CO2 emissions (including the utilities). This provides an economic and more sustainable alternative to traditional naphtha cracking.

Tumor growth prediction with reaction-diffusion and hyperelastic biomechanical model by physiological data fusion.

The goal of tumor growth prediction is to model the tumor growth process, which can be achieved by physiological modeling and model personalization from clinical measurements. Although image-driven frameworks have been proposed with promising results, several issues such as infinitesimal strain assumptions, complicated personalization procedures, and the lack of functional information, may limit their prediction accuracy. In view of these issues, we propose a framework for pancreatic neuroendocrine tumor growth prediction, which comprises a FEM-based tumor growth model with coupled reaction-diffusion equation and nonlinear biomechanics. Physiological data fusion of structural and functional images is used to improve the subject-specificity of model personalization, and a derivative-free global optimization algorithm is adopted to facilitate the complicated model and accommodate flexible choices of objective functions. With this flexibility, we propose an objective function accounting for both the tumor volume difference and the root-mean-squared error of intracellular volume fractions. Experiments were performed on synthetic and clinical data to verify the parameter estimation capability and the prediction performance. Comparisons of using different biomechanical models and objective functions were also performed. From the experimental results of eight patient data sets, the average recall, precision, Dice coefficient, and relative volume difference between predicted and measured tumor volumes were 84.5 ± 6.9%, 85.8 ± 8.2%, 84.6 ± 1.7%, and 14.2 ± 8.4%, respectively.

Integrating PCA and Streamline Information for History Matching Channelized Reservoirs

This article, written by Special Publications Editor Adam Wilson, contains highlights of paper SPE 170636, “Integration of Principal-Component Analysis and Streamline Information for the History Matching of Channelized Reservoirs,” by C. Chen, SPE, Shell International Exploration and Production; G. Gao, SPE, Shell Global Solutions US; J. Honorio, Massachusetts Institute of Technology; P. Gelderblom, SPE, Shell Global Solutions International; E. Jimenez, Qatar Shell GTL; and T. Jaakkola, Massachusetts Institute of Technology, prepared for the 2014 SPE Annual Technical Conference and Exhibition, Amsterdam, 27–29 October. The paper has not been peer reviewed. Although principal-component analysis (PCA) has been applied widely to reduce the number of parameters characterizing a reservoir, its disadvantages are well-recognized. A work flow was proposed to integrate cumulative-distribution-function-based PCA (CDF-PCA) and streamline information for assisted history matching on a two-facies channelized reservoir. The CDF-PCA was developed to reconstruct reservoir models by use of only a few hundred principal components. It inherits the advantage of PCA to capture the main features or trends of spatial correlations among properties, and, more importantly, it can properly correct the smoothing effect of PCA. Introduction Both object-based and multipointstatistics-based models generate relatively more geologically realistic channel bodies compared with conventional two-point geostatistics-based techniques. However, conditioning such models to production data and correctly sampling the posterior probability distribution are challenging problems. One of the major challenges is that the number of para meters to be tuned during history matching is too large to be handled effectively by available history- matching work flows, especially when the adjoint gradient is unavailable. Another challenge is that the models obtained after history matching generally violate or distort the geological and geostatistical characteristics of the original or prior models. One of the authors’ major objectives was to reconstruct channelized geological and reservoir models with much fewer uncertain parameters so that the models can be conditioned to production data through automatic/assisted history matching with use of model-based derivative-free optimization algorithms. The geological and reservoir models obtained by history matching have to capture major flow dynamics and are also constrained by the histogram of lithology distribution and the correlations between reservoir properties of the original model. The following four important questions must be addressed when regenerating geological realizations using new parameters: Are new parameters able to regenerate both static models and dynamic models? Are the models regenerated by new parameters still geologically realistic and relevant? In other words, do the new geological and reservoir models satisfy geologists’ requirements? Are the realizations generated from sampling the probability distribution of these new uncertain parameters equivalent to (but not biased from) the realizations generated from sampling the prior probability distribution? Is it feasible to history match production data by tuning these new parameters?

Statistical Surrogate Formulations for Simulation-Based Design Optimization

Typical challenges of simulation-based design optimization include unavailable gradients and unreliable approximations thereof, expensive function evaluations, numerical noise, multiple local optima, and the failure of the analysis to return a value to the optimizer. One possible remedy to alleviate these issues is to use surrogate models in lieu of the computational models or simulations and derivative-free optimization algorithms. In this work, we use the R dynaTree package to build statistical surrogates of the blackboxes and the direct search method for derivative-free optimization. We present different formulations for the surrogate problem (SP) considered at each search step of the mesh adaptive direct search (MADS) algorithm using a surrogate management framework. The proposed formulations are tested on 20 analytical benchmark problems and two simulation-based multidisciplinary design optimization (MDO) problems. Numerical results confirm that the use of statistical surrogates in MADS improves the efficiency of the optimization algorithm.

An exact penalty function-based differential search algorithm for constrained global optimization

Differential search (DS) is a recently developed derivative-free global heuristic optimization algorithm for solving unconstrained optimization problems. In this paper, by applying the idea of exact penalty function approach, a DS algorithm, where an S-type dynamical penalty factor is introduced so as to achieve a better balance between exploration and exploitation, is developed for constrained global optimization problems. To illustrate the applicability and effectiveness of the proposed approach, a comparison study is carried out by applying the proposed algorithm and other widely used evolutionary methods on 24 benchmark problems. The results obtained clearly indicate that the proposed method is more effective and efficient over the other widely used evolutionary methods for most these benchmark problems.

A Derivative-Free Mesh Optimization Algorithm for Mesh Quality Improvement and Untangling

We propose a derivative-free mesh optimization algorithm, which focuses on improving the worst element quality on the mesh. The mesh optimization problem is formulated as a min-max problem and solved by using a downhill simplex (amoeba) method, which computes only a function value without needing a derivative of Hessian of the objective function. Numerical results show that the proposed mesh optimization algorithm outperforms the existing mesh optimization algorithm in terms of improving the worst element quality and eliminating inverted elements on the mesh.