Deterministic Global Optimization Research Articles

Bounding convex relaxations of process models from below by tractable black-box sampling

Several chemical engineering applications demand global optimization of nonconvex process models, including safety verification and determination of thermodynamic equilibria. Methods for deterministic global optimization typically generate crucial bounding information by minimizing convex relaxations of the process model. However, gradients or subgradients of these convex relaxations may be unavailable in practice for several reasons, which may hinder computation of this bounding information. This article shows that useful, correct affine underestimators and lower bounds of convex relaxations may be generated tractably just by black-box sampling. No additional assumptions are required, and no subgradients or gradients must be computed at any point. Variants of these methods are presented to account for numerical error or noise in the sampling procedure. Several numerical examples are presented for illustration, including application of the new sampling-based underestimators in global optimization problems.

COMANDO: A Next-Generation Open-Source Framework for Energy Systems Optimization

Existing open-source modeling frameworks dedicated to energy systems optimization typically utilize (mixed-integer) linear programming ((MI)LP) formulations, which lack granularity for technical system design and operation. We present COMANDO, an open-source Python package for component-oriented modeling and optimization for nonlinear design and operation of integrated energy systems. COMANDO allows to assemble system models from component models including nonlinear, dynamic and discrete characteristics. Based on a single system model, different deterministic and stochastic problem formulations can be obtained by varying objective function and underlying data, and by applying automatic or manual reformulations. The flexible open-source implementation allows for the integration of customized routines required to solve challenging problems, e.g., initialization, problem decomposition, or sequential solution strategies. We demonstrate features of COMANDO via case studies, including automated linearization, dynamic optimization, stochastic programming, and the use of nonlinear artificial neural networks (ANNs) as surrogate models in a reduced-space formulation for deterministic global optimization.

Obey validity limits of data-driven models through topological data analysis and one-class classification

Data-driven models are becoming increasingly popular in engineering, on their own or in combination with mechanistic models. Commonly, the trained models are subsequently used in model-based optimization of design and/or operation of processes. Thus, it is critical to ensure that data-driven models are not evaluated outside their validity domain during process optimization. We propose a method to learn this validity domain and encode it as constraints in process optimization. We first perform a topological data analysis using persistent homology identifying potential holes or separated clusters in the training data. In case clusters or holes are identified, we train a one-class classifier, i.e., a one-class support vector machine, on the training data domain and encode it as constraints in the subsequent process optimization. Otherwise, we construct the convex hull of the data and encode it as constraints. We finally perform deterministic global process optimization with the data-driven models subject to their respective validity constraints. To ensure computational tractability, we develop a reduced-space formulation for trained one-class support vector machines and show that our formulation outperforms common full-space formulations by a factor of over 3000, making it a viable tool for engineering applications. The method is ready-to-use and available open-source as part of our MeLOn toolbox (https://git.rwth-aachen.de/avt.svt/public/MeLOn).

Optimization-based convex relaxations for nonconvex parametric systems of ordinary differential equations

Novel convex and concave relaxations are proposed for the solutions of parametric ordinary differential equations (ODEs), to aid in furnishing bounding information for deterministic global dynamic optimization methods. These relaxations are constructed as the solutions of auxiliary ODE systems with embedded convex optimization problems, whose objective functions employ convex and concave relaxations of the original ODE right-hand side. Unlike established relaxation methods, any valid convex and concave relaxations for the original right-hand side are permitted in the approach, including the McCormick relaxations and the $$\alpha $$ BB relaxations. In general, tighter such relaxations will necessarily translate into tighter relaxations for the ODE solutions, thus providing tighter bounding information for an overarching global dynamic optimization method. Notably, if McCormick relaxations are employed in the new approach, then the obtained relaxations are guaranteed to be at least as tight as state-of-the-art ODE relaxations based on generalized McCormick relaxations. The new relaxations converge rapidly to the original system as the considered parametric subdomain shrinks. Several examples are presented for comparison with established ODE relaxations, based on a proof-of-concept implementation in MATLAB. In a global optimization case study, the new ODE relaxations are shown to lead to fewer branch-and-bound global optimization iterations than state-of-the-art relaxations.

Review on the monograph Deterministic Global Optimization: An Introduction to the Diagonal Approach, Springer, 2017, written by Yaroslav D. Sergeyev and Dmitri E. Kvasov

This is a review of the book ‘Deterministic Global Optimization: An Introduction to the Diagonal Approach’.

A deterministic semi-infinite global optimization method to design slotless permanent magnet machines

PurposeIn this work, a method to design a slotless permanent magnet machine (SPMM) based on the joint use of an analytical model and deterministic global optimization algorithms is addressed. The purpose of this study is to propose to include torque ripples as an extra constraint in the optimization phase involving de facto the study of a semi-infinite optimization problem.Design/methodology/approachBased on the use of a well-known analytical model describing the electromagnetic behavior of an SPMM, this analytical model has been supplemented by the calculus of the dynamic torque and its ripples to carry out a more accurate optimized sizing method of such an electromechanical converter. As a consequence, the calculated torque depends on a continuous variable, namely, the rotor angular position, resulting in the definition of a semi-infinite optimization problem. The way to solve this kind of semi-infinite problem by discretizing the rotor angular position by using a deterministic global optimization solver, that is to say COUENNE, via the AMPL modeling language is addressed.FindingsIn this study, the proposed approach is validated on some numerical tests based on the minimization of the magnet volume. Efficient global optimal solutions with torque ripples about 5% (instead of 30%) can be so obtained.Research limitations/implicationsThe analytical model does not use results from the solution of two-dimensional field equations. A strong assumption is put forward to approximate the distribution of the magnetic flux density in the air gap of the SPMM.Originality/valueThe problem to design an SPMM can be efficiently formulated as a semi-infinite global optimization problem. This kind of optimization problems are hard to solve because they involve an infinity of constraints (coming from a constraint on the torque ripple). The authors show in this paper that by using analytical models, a discretization method and a deterministic global optimization code COUENNE, this problem is efficiently tackled. Some numerical results show that the deterministic global solution of the design can be reached even if the step of discretization is small.

Linearization of McCormick relaxations and hybridization with the auxiliary variable method

The computation of lower bounds via the solution of convex lower bounding problems depicts current state-of-the-art in deterministic global optimization. Typically, the nonlinear convex relaxations are further underestimated through linearizations of the convex underestimators at one or several points resulting in a lower bounding linear optimization problem. The selection of linearization points substantially affects the tightness of the lower bounding linear problem. Established methods for the computation of such linearization points, e.g., the sandwich algorithm, are already available for the auxiliary variable method used in state-of-the-art deterministic global optimization solvers. In contrast, no such methods have been proposed for the (multivariate) McCormick relaxations. The difficulty of determining a good set of linearization points for the McCormick technique lies in the fact that no auxiliary variables are introduced and thus, the linearization points have to be determined in the space of original optimization variables. We propose algorithms for the computation of linearization points for convex relaxations constructed via the (multivariate) McCormick theorems. We discuss alternative approaches based on an adaptation of Kelley’s algorithm; computation of all vertices of an n-simplex; a combination of the two; and random selection. All algorithms provide substantial speed ups when compared to the single point strategy used in our previous works. Moreover, we provide first results on the hybridization of the auxiliary variable method with the McCormick technique benefiting from the presented linearization strategies resulting in additional computational advantages.

Machine Learning and Simulation-Optimization Coupling for Water Distribution Network Contamination Source Detection.

This paper presents and explores a novel methodology for solving the problem of a water distribution network contamination event, which includes determining the exact source of contamination, the contamination start and end times and the injected contaminant concentration. The methodology is based on coupling a machine learning algorithm for predicting the most probable contamination sources in a water distribution network with an optimization algorithm for determining the values of contamination start time, end time and injected contaminant concentration for each predicted node separately. Two slightly different algorithmic frameworks were constructed which are based on the mentioned methodology. Both algorithmic frameworks utilize the Random Forest algorithm for classification of top source contamination node candidates, with one of the frameworks directly using the stochastic fireworks optimization algorithm to determine the contamination start time, end time and injected contaminant concentration for each predicted node separately. The second framework uses the Random Forest algorithm for an additional regression prediction of each top node’s start time, end time and contaminant concentration and is then coupled with the deterministic global search optimization algorithm MADS. Both a small sized (92 potential sources) network with perfect sensor measurements and a medium sized (865 potential sources) benchmark network with fuzzy sensor measurements were used to explore the proposed frameworks. Both algorithmic frameworks perform well and show robustness in determining the true source node, start and end times and contaminant concentration, with the second framework being extremely efficient on the fuzzy sensor measurement benchmark network.

Automatic Convexity Deduction for Efficient Function’s Range Bounding

Reliable bounding of a function’s range is essential for deterministic global optimization, approximation, locating roots of nonlinear equations, and several other computational mathematics areas. Despite years of extensive research in this direction, there is still room for improvement. The traditional and compelling approach to this problem is interval analysis. We show that accounting convexity/concavity can significantly tighten the bounds computed by interval analysis. To make our approach applicable to a broad range of functions, we also develop the techniques for handling nondifferentiable composite functions. Traditional ways to ensure the convexity fail in such cases. Experimental evaluation showed the remarkable potential of the proposed methods.

Review on the SpriengerBriefs monograph “deterministic global optimization: an introduction to the diagonal approach”

Review on the SpriengerBriefs monograph “deterministic global optimization: an introduction to the diagonal approach”

On the Extension of the DIRECT Algorithm to Multiple Objectives

Deterministic global optimization algorithms like Piyavskii–Shubert, direct, ego and many more, have a recognized standing, for problems with many local optima. Although many single objective optimization algorithms have been extended to multiple objectives, completely deterministic algorithms for nonlinear problems with guarantees of convergence to global Pareto optimality are still missing. For instance, deterministic algorithms usually make use of some form of scalarization, which may lead to incomplete representations of the Pareto optimal set. Thus, all global Pareto optima may not be obtained, especially in nonconvex cases. On the other hand, algorithms attempting to produce representations of the globally Pareto optimal set are usually based on heuristics. We analyze the concept of global convergence for multiobjective optimization algorithms and propose a convergence criterion based on the Hausdorff distance in the decision space. Under this light, we consider the well-known global optimization algorithm direct, analyze the available algorithms in the literature that extend direct to multiple objectives and discuss possible alternatives. In particular, we propose a novel definition for the notion of potential Pareto optimality extending the notion of potential optimality defined in direct. We also discuss its advantages and disadvantages when compared with algorithms existing in the literature.

Globally optimal working fluid mixture composition for geothermal power cycles

Numerical optimization is very useful for design and operation of energy processes. As the design has a major impact on the economics of the system, it is desirable to find a global optimum in the presence of local optima. So far, deterministic global optimization with detailed thermodynamic models incorporated has been limited to single fluid energy systems. We extend our previously presented approach [Schweidtmann et int Mitsos, COMPUT CHEM ENG (2019)] from single-species working fluids to binary mixtures with variable composition. First, we create accurate thermodynamic data for two selected binary mixtures using a thermodynamic library. Using this data, we train artificial neural networks, select them based on desired accuracy, and include them in the process model. The resulting hybrid model is then optimized with the open-source solver MAiNGO. We perform thermodynamic optimizations of geothermal power plants, considering both organic Rankine and Kalina cycle. For each cycle, we identify the globally optimal design, operation, and working fluid composition for the selected binary fluid mixtures within tractable CPU times. We show how a second mixture component enables improved ORC performance.

Deterministic global superstructure-based optimization of an organic Rankine cycle

Organic Rankine cycles (ORCs) offer a high structural design flexibility. The best process structure can be identified via the optimization of a superstructure, which considers design alternatives simultaneously. In this contribution, we apply deterministic global optimization to a geothermal ORC superstructure, thus guaranteeing to find the best solution. We implement a hybrid mechanistic data-driven model, employing artificial neural networks as thermodynamic surrogate models. This approach is beneficial as we optimize the problem in a reduced space using the optimization solver MAiNGO. We further introduce redundant constraints that are only considered for the lower-bounding problem of the branch-and-bound algorithm. We perform two separate optimizations, one maximizing power output and one minimizing levelized cost of electricity. The optimal solutions of both objectives differ from each other, but both have three pressure levels. Global optimization is necessary as there exist suboptimal local solutions for both flowsheet configuration and design with fixed configurations.

Global optimization of large-scale MIQCQPs via cluster decomposition: Application to short-term planning of an integrated refinery-petrochemical complex

Integrated refinery-petrochemical facilities are complex systems that require advanced decision-support tools for optimal short-term planning of their operations. The problem can be formulated as a mixed-integer quadratically constrained quadratic program (MIQCQP), in which discrete decisions select operating modes for the process units, while the entire process network is represented by input-output relationships based on bilinear expressions describing yields and stream properties, pooling equations, fuels blending indices and cost indicators. We develop a novel decomposition-based algorithm for deterministic global optimization that divides the network into small clusters according to their functionality. Inside each cluster, we derive a mixed-integer linear programming (MILP) relaxation based on piecewise McCormick envelopes, dynamically partitioning the variables that belong to the cluster and reducing their domains through optimality-based bound tightening. Results for an industrial case study in Colombia show profit improvements above 10% and significantly reduced optimality gaps compared with the state-of-the-art global optimization solvers BARON and ANTIGONE.

Multidimensional Global Optimization and Robustness Analysis in the Context of Protein-Ligand Binding.

Accuracy of protein-ligand binding free energy calculations utilizing implicit solvent models is critically affected by parameters of the underlying dielectric boundary, specifically, the atomic and water probe radii. Here, a global multidimensional optimization pipeline is developed to find optimal atomic radii specifically for protein-ligand binding calculations in implicit solvent. The computational pipeline has these three key components: (1) a massively parallel implementation of a deterministic global optimization algorithm (VTDIRECT95), (2) an accurate yet reasonably fast generalized Born implicit solvent model (GBNSR6), and (3) a novel robustness metric that helps distinguish between nearly degenerate local minima via a postprocessing step of the optimization. A graph-based "kT-connectivity" approach to explore and visualize the multidimensional energy landscape is proposed: local minima that can be reached from the global minimum without exceeding a given energy threshold (kT) are considered to be connected. As an illustration of the capabilities of the optimization pipeline, we apply it to find a global optimum in the space of just five radii: four atomic (O, H, N, and C) radii and water probe radius. The optimized radii, ρW = 1.37 Å, ρC = 1.40 Å, ρH = 1.55 Å, ρN = 2.35 Å, and ρO = 1.28 Å, lead to a closer agreement of electrostatic binding free energies with the explicit solvent reference than two commonly used sets of radii previously optimized for small molecules. At the same time, the ability of the optimizer to find the global optimum reveals fundamental limits of the common two-dielectric implicit solvation model: the computed electrostatic binding free energies are still almost 4 kcal/mol away from the explicit solvent reference. The proposed computational approach opens the possibility to further improve the accuracy of practical computational protocols for binding free energy calculations.

Multi-scale membrane process optimization with high-fidelity ion transport models through machine learning

Innovative membrane technologies optimally integrated into large separation process plants are essential for economical water treatment and disposal. However, the mass transport through membranes is commonly described by nonlinear differential-algebraic mechanistic models at the nano-scale, while the process and its economics range up to large-scale. Thus, the optimal design of membranes in process plants requires decision making across multiple scales, which is not possible using standard tools. In this work, we embed artificial neural networks (ANNs) as surrogate models in the deterministic global optimization to bridge the gap of scales. This methodology allows for deterministic global optimization of membrane processes with accurate transport models – avoiding the utilization of inaccurate approximations through heuristics or short-cut models. The ANNs are trained based on data generated by a one-dimensional extended Nernst-Planck ion transport model and extended to a more accurate two-dimensional distribution of the membrane module, that captures the filtration-related decreasing retention of salt. We simultaneously design the membrane and plant layout yielding optimal membrane module synthesis properties along with the optimal plant design for multiple objectives, feed concentrations, filtration stages, and salt mixtures. The developed process models and the optimization solver are available open-source, enabling computational resource-efficient multi-scale optimization in membrane science.

Deterministic global optimization of steam cycles using the IAPWS-IF97 model

The IAPWS-IF97 (Wagner et al. (2000) J Eng Gas Turbines Power 122:150) is the state-of-the-art model for the thermodynamic properties of water and steam for industrial applications and is routinely used for simulations of steam power cycles and utility systems. Its use in optimization-based design, however, has been limited because of its complexity. In particular, deterministic global optimization of problems with the IAPWS-IF97 is challenging because general-purpose methods lead to rather weak convex and concave relaxations, thus resulting in slow convergence. Furthermore, the original domains of many functions from the IAPWS-IF97 are nonconvex, while common global solvers construct relaxations over rectangular domains. Outside the original domains, however, many of the functions take very large values that lead to even weaker relaxations. Therefore, we develop tighter relaxations of relevant functions from the IAPWS-IF97 on the basis of an analysis of their monotonicity and convexity properties. We modify the functions outside their original domains to enable tighter relaxations, while we keep them unchanged on their original domains where they have physical meaning. We discuss the benefit of the relaxations for three case studies on the design of bottoming cycles of combined cycle power plants using our open-source deterministic global solver MAiNGO. The derived relaxations result in drastic reductions in computational time compared with McCormick relaxations and can make design problems tractable for global optimization.

A Solution of Implicit Model of Series-Parallel Photovoltaic Arrays by Using Deterministic and Metaheuristic Global Optimization Algorithms

The implicit model of photovoltaic (PV) arrays in series-parallel (SP) configuration does not require the LambertW function, since it uses the single-diode model, to represent each submodule, and the implicit current-voltage relationship to construct systems of nonlinear equations that describe the electrical behavior of a PV generator. However, the implicit model does not analyze different solution methods to reduce computation time. This paper formulates the solution of the implicit model of SP arrays as an optimization problem with restrictions for all the variables, i.e., submodules voltages, blocking diode voltage, and strings currents. Such an optimization problem is solved by using two deterministic (Trust-Region Dogleg and Levenberg Marquard) and two metaheuristics (Weighted Differential Evolution and Symbiotic Organism Search) optimization algorithms to reproduce the current–voltage (I–V) curves of small, medium, and large generators operating under homogeneous and non-homogeneous conditions. The performance of all optimization algorithms is evaluated with simulations and experiments. Simulation results indicate that both deterministic optimization algorithms correctly reproduce I–V curves in all the cases; nevertheless, the two metaheuristic optimization methods only reproduce the I–V curves for small generators, but not for medium and large generators. Finally, experimental results confirm the simulation results for small arrays and validate the reference model used in the simulations.

Global dynamic optimization using edge-concave underestimator

Optimization of problems with embedded system of ordinary differential equations (ODEs) is challenging and the difficulty is amplified due to the presence of nonconvexity. In this article, a deterministic global optimization method is presented for systems consisting of an objective function and constraints with integral terms and an embedded set of nonlinear parametric ODEs. The method is based on a branch-and-bound algorithm that uses a new class of underestimators recently proposed by Hasan (J Glob Optim 71:735–752, 2018). At each node of the branch-and-bound tree, instead of using a convex relaxation, an edge-concave underestimator or the linear facets of its convex envelope is used to compute a lower bound. The underestimator is constructed by finding valid upper bounds on the diagonal elements of the Hessian matrix of the nonconvex terms. Time dependent bounds on the state variables and diagonal elements of the Hessian are obtained by solving an auxiliary set of ODEs that is derived using the notion of differential inequalities. The performance of the edge-concave relaxation is compared to other approaches on several test problems.

New bounding schemes and algorithmic options for the Branch-and-Sandwich algorithm

We consider the global solution of bilevel programs involving nonconvex functions. Deterministic global optimization algorithms for the solution of this challenging class of optimization problems have started to emerge over the last few years. We present new schemes to generate valid bounds on the solution of nonconvex inner and outer problems and examine new strategies for branching and node selection. We integrate these within the Branch-and-Sandwich algorithm (Kleniati and Adjiman in J Glob Opt 60:425–458, 2014), which is based on a branch-and-bound framework and enables the solution of a wide range of problems, including those with nonconvex inequalities and equalities in the inner problem. The impact of the proposed modifications is demonstrated on an illustrative example and 10 nonconvex bilevel test problems from the literature. It is found that the performance of the algorithm is improved for all but one problem (where the CPU time is increased by 2%), with an average reduction in CPU time of 39%. For the two most challenging problems, the CPU time required is decreased by factors of over 3 and 10.