Deterministic Global Optimization Method Research Articles

Multiobjective optimization by using cutting angle methods and hypervolume criterion

We translate a multiobjective optimization problem into a single objective Lipschitz problem by using the hypervolume criterion of the Pareto set. Deterministic global optimization methods allow one to track the whole Pareto front rather than converging to a single non-dominated solution. We augmented an efficient hypervolume computation technique with hypervolume increment strategy, and established Lipschitzianity of the resulting objective function. We used two deterministic Lipschitz optimization methods together with the hypervolume objective and benchmarked them against some state-of-the-art alternative multiobjective optimization methods, establishing the competitiveness of the proposed approach.

A branch-and-bound algorithm with growing datasets for large-scale parameter estimation

The solution of nonconvex parameter estimation problems with deterministic global optimization methods is desirable but challenging, especially if large measurement datasets are considered. We propose to exploit the structure of this class of optimization problems to enable their solution with the spatial branch-and-bound algorithm. In detail, we start with a reduced dataset in the root node and progressively augment it, converging to the full dataset. We show for nonlinear programs (NLPs) that our algorithm converges to the global solution of the original problem considering the full dataset. The implementation of the algorithm extends our open-source solver MAiNGO. A numerical case study with a mixed-integer nonlinear program (MINLP) from chemical engineering and a dynamic optimization problem from biochemistry both using noise-free measurement data emphasizes the potential for savings of computational effort with our proposed approach.

Exploring synchronization and lift suppression in fluid flow around vibrating cylinder: a parallel CFD and global optimization investigation

In this study, we employ a parallel Computational Fluid Dynamics (CFD) code integrated with the VTDIRECT95 algorithm, a parallel deterministic global optimization method, to conduct global optimization for an oscillating circular cylinder. We conduct numerical simulations for the flow at a Reynolds number of 500 within the parameter range of 0.1≤Ax≤0.3 and 0.5fst≤fex≤2.5fst where Ax represents the inline oscillating amplitude, fex denotes the forcing oscillation frequency, and fst corresponds to the frequency of a stationary cylinder. To enhance computational efficiency, a combination of VTdirect and a CFD solver is utilized to efficiently identify the synchronization region, thereby reducing computational resources. The results reveal a significant reduction in the lift coefficient within the synchronized region compared to unsynchronized regimes. Furthermore, the study delves into the underlying flow physics behind synchronization and lift suppression. By synchronizing the shedding of vortices, their detrimental effects are nullified, resulting in a reduction in lift. Moreover, the research examines the influence of three-dimensional (3-D) flow by comparing 2-D and 3-D simulations at two different Reynolds numbers. It demonstrates that accounting for 3-D effects yields more accurate predictions of fluid behavior. Synchronization maps and root mean square (rms) lift coefficient plots illustrate the impact of Reynolds number and movement frequency on lift suppression. The findings indicate that achieving synchronization in 3-D flow necessitates stronger amplitudes and higher frequencies. At higher Reynolds numbers, the wake structures become unstable, leading to intricate vortical patterns. Consequently, the synchronization curve shifts towards higher amplitudes and frequencies in 3-D simulations. Understanding these phenomena is vital for reducing lift force in practical applications. This research significantly contributes to knowledge regarding synchronization and lift suppression in fluid flow around vibrating cylinders.

Collaborative granular sieving: A deterministic multievolutionary algorithm for multimodal optimization problems

Evolutionary algorithms (EAs) that integrate niching techniques are among the most effective methods for multimodal optimization problems. However, most algorithmic contributions are based on empirical performance observations rather than rigorous mathematical convergence support; this makes most existing methods parameter sensitive. Inspired by a recently proposed deterministic global optimization method, granular sieving (GrS), an extended global optimization method named collaborative GrS (Co-GrS) and a novel deterministic multi-EA design framework are proposed in this paper. The innovations are threefold. (1) Existing EAs are stochastic methods, and this paper introduces the principle of deterministic global optimization into EA for the first time in the literature. (2) A deterministic multi-EA framework is designed and implemented in the paper; from the perspective of population evolution, an easy-to-operate survival-of-the-fittest strategy based on mathematical principles is established in Co-GrS. (3) Unlike existing stochastic EAs, where the reproducibility of optimal solutions is achieved in a statistical sense, Co-GrS does not involve random parameters, and it automatically runs the algorithm only once with pre-set fixed parameters to find all optimal solutions. The experimental results demonstrate the effectiveness and competitiveness of our method compared to 16 state-of-the-art multimodal algorithms on the CEC’2013 benchmark suite.

Optimization-based convex relaxations for nonconvex parametric systems of ordinary differential equations

Novel convex and concave relaxations are proposed for the solutions of parametric ordinary differential equations (ODEs), to aid in furnishing bounding information for deterministic global dynamic optimization methods. These relaxations are constructed as the solutions of auxiliary ODE systems with embedded convex optimization problems, whose objective functions employ convex and concave relaxations of the original ODE right-hand side. Unlike established relaxation methods, any valid convex and concave relaxations for the original right-hand side are permitted in the approach, including the McCormick relaxations and the $$\alpha $$ BB relaxations. In general, tighter such relaxations will necessarily translate into tighter relaxations for the ODE solutions, thus providing tighter bounding information for an overarching global dynamic optimization method. Notably, if McCormick relaxations are employed in the new approach, then the obtained relaxations are guaranteed to be at least as tight as state-of-the-art ODE relaxations based on generalized McCormick relaxations. The new relaxations converge rapidly to the original system as the considered parametric subdomain shrinks. Several examples are presented for comparison with established ODE relaxations, based on a proof-of-concept implementation in MATLAB. In a global optimization case study, the new ODE relaxations are shown to lead to fewer branch-and-bound global optimization iterations than state-of-the-art relaxations.

Global dynamic optimization using edge-concave underestimator

Optimization of problems with embedded system of ordinary differential equations (ODEs) is challenging and the difficulty is amplified due to the presence of nonconvexity. In this article, a deterministic global optimization method is presented for systems consisting of an objective function and constraints with integral terms and an embedded set of nonlinear parametric ODEs. The method is based on a branch-and-bound algorithm that uses a new class of underestimators recently proposed by Hasan (J Glob Optim 71:735–752, 2018). At each node of the branch-and-bound tree, instead of using a convex relaxation, an edge-concave underestimator or the linear facets of its convex envelope is used to compute a lower bound. The underestimator is constructed by finding valid upper bounds on the diagonal elements of the Hessian matrix of the nonconvex terms. Time dependent bounds on the state variables and diagonal elements of the Hessian are obtained by solving an auxiliary set of ODEs that is derived using the notion of differential inequalities. The performance of the edge-concave relaxation is compared to other approaches on several test problems.

Simultaneous rational design of ion separation membranes and processes

Economically viable water treatment process plants for drinking water purification are a prerequisite for sustainable supply of safe drinking water in the future. However, modern membrane process development experiences a disconnect in this domain: the synthesis of the membrane and the design of the process are decoupled. We propose an optimization strategy to simultaneously design the performance of layer-by-layer nanofiltration membrane modules along with the separation process. This approach achieves overall optimal performance by extending the search space and thus exploiting synergies. Better separation performances at a lower cost as compared to conventional optimization strategies can be achieved. The key feature of this optimization framework is the integration of artificial neural networks. This machine-learning technique describes the membrane performance as a function of its synthesis protocol. We optimize the design problem rigorously by a deterministic global nonlinear optimization method. Thus, this framework yields membrane synthesis protocols and membrane processes that are optimally tailored to the desired separation task. In a showcase, the simultaneous membrane synthesis and process optimization design achieve immediately favorable results with lower impurities at comparable costs. The process investment and operation costs are compared to a state of the art commercially available membrane for nanofiltration.

Global optimization of stiff dynamical systems

AbstractWe present a deterministic global optimization method for nonlinear programming formulations constrained by stiff systems of ordinary differential equation (ODE) initial value problems (IVPs). The examples arise from dynamic optimization problems exhibiting both fast and slow transient phenomena commonly encountered in model‐based systems engineering applications. The proposed approach utilizes unconditionally stable implicit integration methods to reformulate the ODE‐constrained problem into a nonconvex nonlinear program (NLP) with implicit functions embedded. This problem is then solved to global optimality in finite time using a spatial branch‐and‐bound framework utilizing convex/concave relaxations of implicit functions constructed by a method which fully exploits problem sparsity. The algorithms were implemented in the Julia programming language within the EAGO.jl package and demonstrated on five illustrative examples with varying complexity relevant in process systems engineering. The developed methods enable the guaranteed global solution of dynamic optimization problems with stiff ODE–IVPs embedded.

Guaranteed global optimization of thin-film optical systems

A parallel deterministic global optimization algorithm for thin-film multilayer optical coatings is developed. This algorithm enables locating a global solution to an optimization problem in this class to within a user-specified tolerance. More specifically, the algorithm is a parallel branch-and-bound method with applicable bounds on the merit function computed using Taylor models. This study is the first one, to the best of our knowledge, to attempt guaranteed global optimization of this important class of problems, thereby providing an overview and an assessment of the current state of such techniques in this domain. As a proof of concept on a small scale, the method is illustrated numerically and experimentally in the context of antireflection coatings for silicon solar cells—we design and fabricate a three-layer dielectric stack on silicon that exhibits an average reflectance of (2.53 ± 0.10)%, weighted over a broad range of incident angles and the solar spectrum. The practicality of our approach is assessed by comparing its computational cost relative to traditional stochastic global optimization techniques which provide no guarantees on their solutions. While our method is observed to be significantly more computationally expensive, we demonstrate via our proof of concept that it is already feasible to optimize sufficiently simple practical problems at a reasonable cost, given the current accessibility of cloud computing resources. Ongoing advances in distributed computing are likely to bring more design problems within the reach of deterministic global optimization methods, yielding rigorous guaranteed solutions in the presence of practical manufacturing constraints.

Convergence of Subtangent-Based Relaxations of Nonlinear Programs

Convex relaxations of functions are used to provide bounding information to deterministic global optimization methods for nonconvex systems. To be useful, these relaxations must converge rapidly to the original system as the considered domain shrinks. This article examines the convergence rates of convex outer approximations for functions and nonlinear programs (NLPs), constructed using affine subtangents of an existing convex relaxation scheme. It is shown that these outer approximations inherit rapid second-order pointwise convergence from the original scheme under certain assumptions. To support this analysis, the notion of second-order pointwise convergence is extended to constrained optimization problems, and general sufficient conditions for guaranteeing this convergence are developed. The implications are discussed. An implementation of subtangent-based relaxations of NLPs in Julia is discussed and is applied to example problems for illustration.

Deterministic global derivative-free optimization of black-box problems with bounded Hessian

Obtaining guaranteed lower bounds for problems with unknown algebraic form has been a major challenge in derivative-free optimization. In this work, we present a deterministic global optimization method for black-box problems where the derivatives are not available or it is computationally expensive to obtain. However, a global upper bound on the diagonal Hessian elements is known. An edge-concave underestimator (Hasan in J Glob Optim 71:735–752, 2018) can be then constructed with vertex polyhedral solution. Evaluating this underestimator only at the vertices leads to a valid lower bound on the original black-box problem. We have implemented this lower bounding technique within a branch-and-bound framework and assessed its computational performance in locating $$\epsilon $$-global optimal solution for several box-constrained, nonconvex black-box functions.

GO-APSR: A Globally Optimal Affine Point Set Registration Method

Point set registration under affine transformation is an important problem in computer vision because not only it has many direct applications, but it is also often used as an initial step for non-rigid registration. This problem is challenging when no correspondences between the two point sets are known, and most existing methods start from an initial pose and find a local optimal transformation. This paper presents a deterministic global optimization method for affine point set registration, which is called GO-APSR. We model the two sets to be registered with Gaussian Mixture Models (GMMs) and minimize the L2 distance between the two GMMs under affine transformation. Branch-and-Bound (BnB) is employed to search the transformation parameter space, and we propose a convex quadratic function as the under-estimator of the objective function in each branch. Therefore, calculation of the lower bound in each branch is casted into a bound-constrained convex quadratic programming problem, which can be solved globally and efficiently. Experiment results verify the global optimality of the proposed method and its robustness to noise and outliers. Furthermore, it works very well in the challenging partially-overlap scenarios.

A Deterministic Global Optimization Method for Solving Generalized Linear Multiplicative Programming Problem with Multiplicative Constraints

This paper presents a deterministic global optimization algorithm for solving generalized linear multiplicative programming problem with multiplicative constraints (GLMP). By utilizing equivalent transformation and linear relaxation method, a linear relaxation programming (LRP) of equivalent problem (GLMPH) is established. In the algorithm, lower and upper bounds are simultaneously obtained by solving some linear relaxation programming problems (LRP). Global convergence has been proved and results of some sample examples and a small random experiment show that the proposed algorithm is feasible and efficient.

Decomposition-based Inner- and Outer-Refinement Algorithms for Global Optimization

Traditional deterministic global optimization methods are often based on a Branch-and-Bound (BB) search tree, which may grow rapidly, preventing the method to find a good solution. Motivated by decomposition-based inner approximation (column generation) methods for solving transport scheduling problems with over 100 million variables, we present a new deterministic decomposition-based successive approximation method for general modular and/or sparse MINLPs. The new method, called Decomposition-based Inner- and Outer-Refinement, is based on a block-separable reformulation of the model into sub-models. It generates inner- and outer-approximations using column generation, which are successively refined by solving many easier MINLP and MIP subproblems in parallel (using BB), instead of searching over one (global) BB search tree. We present preliminary numerical results with Decogo (Decomposition-based Global Optimizer), a new parallel decomposition MINLP solver implemented in Python and Pyomo.

On the efficiency of nature-inspired metaheuristics in expensive global optimization with limited budget

Global optimization problems where evaluation of the objective function is an expensive operation arise frequently in engineering, decision making, optimal control, etc. There exist two huge but almost completely disjoint communities (they have different journals, different conferences, different test functions, etc.) solving these problems: a broad community of practitioners using stochastic nature-inspired metaheuristics and people from academia studying deterministic mathematical programming methods. In order to bridge the gap between these communities we propose a visual technique for a systematic comparison of global optimization algorithms having different nature. Results of more than 800,000 runs on 800 randomly generated tests show that both stochastic nature-inspired metaheuristics and deterministic global optimization methods are competitive and surpass one another in dependence on the available budget of function evaluations.

Enclosure of all index-1 saddle points of general nonlinear functions

Transition states (index-1 saddle points) play a crucial role in determining the rates of chemical transformations but their reliable identification remains challenging in many applications. Deterministic global optimization methods have previously been employed for the location of transition states (TSs) by initially finding all stationary points and then identifying the TSs among the set of solutions. We propose several regional tests, applicable to general nonlinear, twice continuously differentiable functions, to accelerate the convergence of such approaches by identifying areas that do not contain any TS or that may contain a unique TS. The tests are based on the application of the interval extension of theorems from linear algebra to an interval Hessian matrix. They can be used within the framework of global optimization methods with the potential of reducing the computational time for TS location. We present the theory behind the tests, discuss their algorithmic complexity and show via a few examples that significant gains in computational time can be achieved by using these tests.

Multi-Cuts Outer Approximation Method for Unit Commitment

This letter introduces a deterministic global optimization methods for unit commitment (UC) problem based on outer approximation method (OAM). The proposed Multi-cuts OAM (MCs-OAM) decomposes the UC problem into a mixed integer linear programming (MILP) master problem and several nonlinear programming (NLP) subproblems, whereas only one NLP in classic OAM. After elaborately designing the terminating criterion for solving the bigger but tighter MILP master problems, MCs-OAM can obtained higher quality solutions with fewer main iterations and less total CPU times, although solving more NLPs consumes more CPU times than OAM. The numerical results on 42 test systems of up to 200 units show that the MCs-OAM is very promising for large scale UC problems because that it can obtain high-quality solutions in reasonable time.

Deterministic global optimization using space-filling curves and multiple estimates of Lipschitz and Hölder constants

In this paper, the global optimization problem miny∈SF(y) with S being a hyperinterval in RN and F(y) satisfying the Lipschitz condition with an unknown Lipschitz constant is considered. It is supposed that the function F(y) can be multiextremal, non-differentiable, and given as a ‘black-box’. To attack the problem, a new global optimization algorithm based on the following two ideas is proposed and studied both theoretically and numerically. First, the new algorithm uses numerical approximations to space-filling curves to reduce the original Lipschitz multi-dimensional problem to a univariate one satisfying the Hölder condition. Second, the algorithm at each iteration applies a new geometric technique working with a number of possible Hölder constants chosen from a set of values varying from zero to infinity showing so that ideas introduced in a popular DIRECT method can be used in the Hölder global optimization. Convergence conditions of the resulting deterministic global optimization method are established. Numerical experiments carried out on several hundreds of test functions show quite a promising performance of the new algorithm in comparison with its direct competitors.

Outer Approximation and Outer-Inner Approximation Approaches for Unit Commitment Problem

This paper proposes a new separable model for the unit commitment (UC) problem and three deterministic global optimization methods for it ensuring convergence to the global optimum within a desired tolerance. By decomposing a multivariate function into several univariate functions, a tighter outer approximation methodology that can be used to improve the outer approximations of several classical convex programming techniques is presented. Based on the idea of the outer approximation (OA) method and the proposed separable model, an outer-inner approximation (OIA) approach is also presented to solve this new formulation of UC problem. In this OIA approach, the UC problem is decomposed into a tighter outer approximation subproblem and an inner approximation subproblem, where the former leads to a better lower bound than the OA method, and the later provides a better upper bound. The simulation results for systems of up to 100 units with 24 h are compared with those of previously published methods, which show that the OIA approach is very promising due to the excellent performance. The proposed approaches are also applied to the large-scale systems of up to 1000 units with 24 h, and systems of up to 100 units with 96 h and 168 h.

Global optimization of bounded factorable functions with discontinuities

A deterministic global optimization method is developed for a class of discontinuous functions. McCormick’s method to obtain relaxations of nonconvex functions is extended to discontinuous factorable functions by representing a discontinuity with a step function. The properties of the relaxations are analyzed in detail; in particular, convergence of the relaxations to the function is established given some assumptions on the bounds derived from interval arithmetic. The obtained convex relaxations are used in a branch-and-bound scheme to formulate lower bounding problems. Furthermore, convergence of the branch-and-bound algorithm for discontinuous functions is analyzed and assumptions are derived to guarantee convergence. A key advantage of the proposed method over reformulating the discontinuous problem as a MINLP or MPEC is avoiding the increase in problem size that slows global optimization. Several numerical examples for the global optimization of functions with discontinuities are presented, including ones taken from process design and equipment sizing as well as discrete-time hybrid systems.