Deterministic Global Optimization Research Articles

A branch-and-bound algorithm with growing datasets for large-scale parameter estimation

The solution of nonconvex parameter estimation problems with deterministic global optimization methods is desirable but challenging, especially if large measurement datasets are considered. We propose to exploit the structure of this class of optimization problems to enable their solution with the spatial branch-and-bound algorithm. In detail, we start with a reduced dataset in the root node and progressively augment it, converging to the full dataset. We show for nonlinear programs (NLPs) that our algorithm converges to the global solution of the original problem considering the full dataset. The implementation of the algorithm extends our open-source solver MAiNGO. A numerical case study with a mixed-integer nonlinear program (MINLP) from chemical engineering and a dynamic optimization problem from biochemistry both using noise-free measurement data emphasizes the potential for savings of computational effort with our proposed approach.

An Algorithm Based on Compute Unified Device Architecture for Estimating Covering Functionals of Convex Bodies

In Chuanming Zong’s program to attack Hadwiger’s covering conjecture, which is a longstanding open problem from Convex and Discrete Geometry, it is essential to estimate covering functionals of convex bodies effectively. Recently, He et al. and Yu et al. provided two deterministic global optimization algorithms having high computational complexity for this purpose. Since satisfactory estimations of covering functionals will be sufficient in Zong’s program, we propose a stochastic global optimization algorithm based on CUDA and provide an error estimation for the algorithm. The accuracy of our algorithm is tested by comparing numerical and exact values of covering functionals of convex bodies including the Euclidean unit disc, the three-dimensional Euclidean unit ball, the regular tetrahedron, and the regular octahedron. We also present estimations of covering functionals for the regular dodecahedron and the regular icosahedron.

Technoeconomic assessment of solar technologies for the hybridization of industrial process heat systems using deterministic global dynamic optimization

The industrial process heat (IPH) sector provides an opportunity to replace terawatts of fossil fuel energy with solar alternatives, such as parabolic trough collectors (PTC) or photovoltaics (PV) with resistive heating and energy storage (ES) for off-peak deployment. PTCs offer significantly higher thermal efficiency, but have stagnated in cost and efficiency improvements over the past decade. In contrast, PVs have seen significant improvements in cost and efficiency. Additionally, the energy generated by PV can be stored electrochemically in batteries. Current optimization-based approaches for design and investment decision making lack guarantees of global optimality that are necessary to ensure the best-possible solutions are obtained for these general nonconvex dynamical models. We present dynamical models for the formal deterministic global optimization-based design and technoeconomic assessment of solar systems to hybridize IPH and reduce natural gas combustion that serve as a general and flexible framework. We compare hybridization strategies of PTC with thermal ES, PV with thermal ES, and PV with battery ES and explore the impacts of natural gas pricing and process scale. For the low- to medium-temperature IPH, PTC with thermal ES is currently at least four times more economically favorable than alternative technologies regardless of location or process scale.

Exploring synchronization and lift suppression in fluid flow around vibrating cylinder: a parallel CFD and global optimization investigation

In this study, we employ a parallel Computational Fluid Dynamics (CFD) code integrated with the VTDIRECT95 algorithm, a parallel deterministic global optimization method, to conduct global optimization for an oscillating circular cylinder. We conduct numerical simulations for the flow at a Reynolds number of 500 within the parameter range of 0.1≤Ax≤0.3 and 0.5fst≤fex≤2.5fst where Ax represents the inline oscillating amplitude, fex denotes the forcing oscillation frequency, and fst corresponds to the frequency of a stationary cylinder. To enhance computational efficiency, a combination of VTdirect and a CFD solver is utilized to efficiently identify the synchronization region, thereby reducing computational resources. The results reveal a significant reduction in the lift coefficient within the synchronized region compared to unsynchronized regimes. Furthermore, the study delves into the underlying flow physics behind synchronization and lift suppression. By synchronizing the shedding of vortices, their detrimental effects are nullified, resulting in a reduction in lift. Moreover, the research examines the influence of three-dimensional (3-D) flow by comparing 2-D and 3-D simulations at two different Reynolds numbers. It demonstrates that accounting for 3-D effects yields more accurate predictions of fluid behavior. Synchronization maps and root mean square (rms) lift coefficient plots illustrate the impact of Reynolds number and movement frequency on lift suppression. The findings indicate that achieving synchronization in 3-D flow necessitates stronger amplitudes and higher frequencies. At higher Reynolds numbers, the wake structures become unstable, leading to intricate vortical patterns. Consequently, the synchronization curve shifts towards higher amplitudes and frequencies in 3-D simulations. Understanding these phenomena is vital for reducing lift force in practical applications. This research significantly contributes to knowledge regarding synchronization and lift suppression in fluid flow around vibrating cylinders.

Towards global parameter estimation exploiting reduced data sets

We focus on deterministic global optimization (DGO) for nonconvex parameter estimation problems. Realistic and accurate solutions often require a fitting against very large measurement data sets, resulting in intractable size for DGO. Thus, we aim at accelerating the branch-and-bound algorithm by using reduced data sets for constructing valid bounds. We focus on fitting the equation of state for propane, which is of high interest for the chemical industry. The resulting estimation problem is a challenging nonconvex mixed-integer nonlinear optimization problem. We investigate the validity of using reduced data sets by comparing how the lower and upper bounds change when replacing the full data set with different reduced data sets. We account for regions with high and low quality fits by considering the results for the whole feasible region and 100 different subregions. Our results indicate that both regions containing solution candidates and regions containing only low quality fits can be identified based on reduced data sets. Moreover, we observe that the average CPU time per branch-and-bound iteration typically decreases if reduced data sets are used.

A rigorous deterministic global optimization approach for the derivation of secondary information in digital maps

We derive a generic system that constructs an optimization model for an emergency stop scenario on the highway, based on map data from high definition maps that are used in Advanced Driver Assistance Systems (ADAS) and in Highly Automated Driving (HAD). New additional situative and scenario-based information is computed by applying a global maximization approach to the model. For this purpose, we develop two new rigorous and deterministic branch-and-bound algorithms that both determine the certified global optimal value up to a predefined tolerance. The underlying interval optimization algorithm, which uses first-order techniques, is enhanced by one of two second-order methods that are applied for specifically selected intervals. We investigate two approaches that either compute a concave overestimator for the objective function or approximate the function with a quadratic polynomial using Taylor expansion. We show the limits of interval arithmetic in our problem, especially for the interval versions of the derivatives, and present a local linearization of the curve data that improves the results significantly. The presented novel method for deriving secondary information is compared to state of the art methods on two exemplary and for the automotive context representative scenarios to show the advantages of our approach.

Collaborative granular sieving: A deterministic multievolutionary algorithm for multimodal optimization problems

Evolutionary algorithms (EAs) that integrate niching techniques are among the most effective methods for multimodal optimization problems. However, most algorithmic contributions are based on empirical performance observations rather than rigorous mathematical convergence support; this makes most existing methods parameter sensitive. Inspired by a recently proposed deterministic global optimization method, granular sieving (GrS), an extended global optimization method named collaborative GrS (Co-GrS) and a novel deterministic multi-EA design framework are proposed in this paper. The innovations are threefold. (1) Existing EAs are stochastic methods, and this paper introduces the principle of deterministic global optimization into EA for the first time in the literature. (2) A deterministic multi-EA framework is designed and implemented in the paper; from the perspective of population evolution, an easy-to-operate survival-of-the-fittest strategy based on mathematical principles is established in Co-GrS. (3) Unlike existing stochastic EAs, where the reproducibility of optimal solutions is achieved in a statistical sense, Co-GrS does not involve random parameters, and it automatically runs the algorithm only once with pre-set fixed parameters to find all optimal solutions. The experimental results demonstrate the effectiveness and competitiveness of our method compared to 16 state-of-the-art multimodal algorithms on the CEC’2013 benchmark suite.

Convex and concave envelopes of artificial neural network activation functions for deterministic global optimization

Convex and concave envelopes of artificial neural network activation functions for deterministic global optimization

Optimal therapy design with tumor microenvironment normalization

AbstractTumor microenvironment (TME) normalization improves efficacy by increasing anticancer nanocarrier delivery by restoring transvascular pressure gradients that induce convection. However, transport depends on TME biophysics, normalization dose, and nanocarrier size. With increased understanding, we could use computation to personalize normalization amount and nanocarrier size. Here, we use deterministic global dynamic optimization with novel bounding routines to validate mechanistic models against in vivo data. We find that normalization with dexamethasone increases the maximum transvascular convection rate of nanocarriers by 48‐fold, the tumor volume fraction with convection by 61%, and the total amount of convection by 360%. Nonetheless, 22% of the tumor still lacks convection. These findings underscore both the effectiveness and limits of normalization. Using artificial neural network surrogate modeling, we demonstrate the feasibility of rapidly determining the dexamethasone dose and nanocarrier size to maximize accumulation. Thus, this digital testbed quantifies transport and performs therapy design.

Simultaneous deterministic global flowsheet optimization and heat integration: Comparison of formulations

Heat integration can be considered in flowsheet optimization by including the minimum utility demand from pinch analysis in the objective and constraints. This often results in better process performance than conducting these steps separately. However, it comes with increased computational cost, especially for global optimization. This cost depends both on the problem formulation and on the solver. In this work, we compare several existing and new smooth, nonsmooth, and mixed-integer formulations. Furthermore, we test different choices of optimization variables and constraints reaching from full- to reduced-space formulations. In the reduced-space formulations, heat integration can be included with few, one, or even zero additional variables beyond the pure flowsheet optimization problem. For the considered case studies, this can significantly reduce the solution time of various global solvers, in particular for our open-source solver MAiNGO. Depending on the case study and solver, either nonsmooth or mixed-integer formulations are the fastest to solve.

Global dynamic optimization with Hammerstein–Wiener models embedded

Hammerstein–Wiener models constitute a significant class of block-structured dynamic models, as they approximate process nonlinearities on the basis of input–output data without requiring identification of a full nonlinear process model. Optimization problems with Hammerstein–Wiener models embedded are nonconvex, and thus local optimization methods may obtain suboptimal solutions. In this work, we develop a deterministic global optimization strategy that exploits the specific structure of Hammerstein–Wiener models to extend existing theory on global optimization of systems with linear dynamics. At first, we discuss alternative formulations of the dynamic optimization problem with Hammerstein–Wiener models embedded, demonstrating that careful selection of the optimization variables of the problem can offer significant numerical advantages to the solution approach. Then, we develop convex relaxations for the proposed optimization problem and discuss implementation aspects to obtain the global solution focusing on a control parametrization technique. Finally, we apply our optimization strategy to case studies comprising both offline and online dynamic optimization problems. The results confirm an improved computational performance of the proposed solution approach over alternative options not exploiting the linear dynamics for all considered examples. They also underline the tractability of deterministic global dynamic optimization when using few control intervals in online applications like nonlinear model predictive control.

Globally optimal shape and spin pole determination with light-curve inversion

ABSTRACT Light-curve inversion is an established technique in determining the shape and spin states of an asteroid. However, the front part of the processing pipeline, which recovers the spin pole and area of each facet, is a non-convex optimization problem. Hence, any local iterative optimization scheme can only promise a locally optimal solution. Apart from the obvious downsides of getting a non-optimal solution and the need for an initialization scheme, another major implication is that it creates an ambiguous scenario – which is to be blamed for the remaining residual? The inaccuracy of the modelling, the integrity of the data, or the non-global algorithm? We address the last uncertainty in this paper by embedding the spin pole and area vector determination module in a deterministic global optimization framework. To the best of our knowledge, this is the first attempt to solve these parameters globally. Specifically, given calibrated light-curve data, a scattering model for the object, and spin period, our method outputs the globally optimal spin pole and area vector solutions. One theoretical contribution of this paper is the introduction of a lower bound error function that is derived based on (1) the geometric relationship between the incident and scattered light on a surface and (2) the uncertainty of the gap between the observed and estimated brightness at a particular epoch in a light curve. We validated our method’s ability in achieving global minimum with both simulated and real light-curve data. We also tested our method on the real light curves of four asteroids.

The application of nonlinear programming on ration formulation for dairy cattle

The objective of this study was to compare the application of iterative linear programming (iteLP), sequential quadratic programming (SQP), and mixed-integer nonlinear programming-based deterministic global optimization (MINLP_DGO) on ration formulation for dairy cattle based on Nutrient Requirements of Dairy Cattle (NRC, 2001). Least-cost diets were formulated for lactating cows, dry cows, and heifers. Nutrient requirements including energy, protein, and minerals, along with other limitations on dry matter intake, neutral detergent fiber, and fat were considered as constraints. Five hundred simulations were conducted, with each simulation randomly selecting 3 roughages and 5 concentrates from the feed table in NRC (2001) as the feed resource for each of 3 animal groups. Among the 500 simulations for lactating cows, 57, 45, and 21 simulations did not yield a feasible solution when using iteLP, SQP, and MINLP_DGO, respectively. All the simulations for dry cows and heifers were feasible when using SQP and MINLP_DGO, but 49 and 11 infeasible simulations occurred when using iteLP for dry cows and heifers, respectively. The average ration costs per animal per day of the feasible solutions obtained by iteLP, SQP, and MINLP_DGO were $4.78 (±0.71), $4.45 (±0.65), and $4.44 (±0.65) for lactating cows; $2.39 (±0.52), $1.48 (±0.26), and $1.48 (±0.26) for dry cows; and $0.98 (±0.72), $0.97 (±0.15), and $0.91 (±0.14) for heifers, respectively. The average computation time of iteLP, SQP, and MINLP_DGO were 0.59 (±1.87) s, 1.15 (±0.62) s, and 58.69 (±68.45) s for lactating cows; 0.041 (±0.070) s, 0.76 (±0.37) s, and 14.84 (±39.09) s for dry cows; and 1.60 (±2.90) s, 0.51 (±0.19) s, and 16.45 (±45.56) s for heifers, respectively. In conclusion, iteLP had limited capability of formulating least-cost diets when nonlinearity existed in the constraints. Both SQP and MINLP_DGO handled the nonlinear constraints well, with SQP being faster, whereas MINLP_DGO was able to return a feasible solution under some situations where SQP could not.

Flexible operation of modular electrochemical CO2 reduction processes

Electrochemical CO2 reduction (eCO2R) is an emerging technology that is capable of producing various organic chemicals from CO2, but its high electricity cost is a big economic obstacle. One solution to reduce the cumulative electricity cost is demand side management, i.e., to adjust the power load based on time-variant electricity prices. However, varying the power load of CO2-electrolyzers often leads to changes in Faraday efficiency towards target components and thereby influences the product composition. Such deviations from the target product composition may be undesired for downstream processes. We tackle this challenge by proposing a flexible operating scheme for a modular eCO2R process. We formulate the economically optimal operation of an eCO2R process with multiple electrolyzer stacks as a parallel-machine scheduling problem. Adjusting the power load of each sub-process properly, we can save electricity costs while the desired product composition is met at any time. We apply an algorithm based on wavelet transform to solve the resulting large-scale nonlinear scheduling problem in tractable time. We solve each optimization problem with a deterministic global optimization software MAiNGO. We examine flexible operation of a modular eCO2R process for syngas production. The case studies show that the modular structure enables savings in the cumulative electricity cost of the eCO2R process via flexible operation while deviations in the syngas composition could be reduced. Also, the maximum ramping speed of the entire process is found to be a key parameter that strongly influences the cost saving.

Precise control of water and wastewater treatment systems with non-ideal heterogeneous mixing models and high-fidelity sensing

Non-ideal heterogeneous mixing models are developed and incorporated within advanced closed-loop control strategies utilizing high-resolution sensing to maximize the resiliency and minimize the energy consumption of water treatment processes with intelligent model-based decision-making approaches. The proposed non-ideal heterogeneous mixing models capture continuity (heat and mass conservation), yet are extremely simple with few parameters, so they lend themselves to fast online prediction (with extrapolation capabilities) and regular recalibration. Further, they are more accurate than computational fluid dynamics (CFD) (60% less error) and symbolic regression data-driven models (73% less error). Real-time high-resolution sensor data are collected for observing spatiotemporal responses of state variables (conductivity, pH, and temperature) to transient influent shocks. Deterministic global dynamic optimization is used for training and recalibration of the non-ideal heterogeneous mixing models to guarantee the best-possible fits to the sensor data. The models are then deployed within standard model-predictive control and two economic model-predictive control strategies to demonstrate model-based decision-making for disturbance rejection and optimal operation of aeration in a continuous flow nitrification system utilizing high-resolution sensor data from several spatial positions. The new technology platform, consisting of high-resolution sensors, non-ideal heterogeneous mixing modeling, deterministic global dynamic optimization, and model-predictive control, offers superior performance over current approaches in water and wastewater treatment processes.

Deterministic global optimization with Gaussian processes embedded

Gaussian processes (Kriging) are interpolating data-driven models that are frequently applied in various disciplines. Often, Gaussian processes are trained on datasets and are subsequently embedded as surrogate models in optimization problems. These optimization problems are nonconvex and global optimization is desired. However, previous literature observed computational burdens limiting deterministic global optimization to Gaussian processes trained on few data points. We propose a reduced-space formulation for deterministic global optimization with trained Gaussian processes embedded. For optimization, the branch-and-bound solver branches only on the free variables and McCormick relaxations are propagated through explicit Gaussian process models. The approach also leads to significantly smaller and computationally cheaper subproblems for lower and upper bounding. To further accelerate convergence, we derive envelopes of common covariance functions for GPs and tight relaxations of acquisition functions used in Bayesian optimization including expected improvement, probability of improvement, and lower confidence bound. In total, we reduce computational time by orders of magnitude compared to state-of-the-art methods, thus overcoming previous computational burdens. We demonstrate the performance and scaling of the proposed method and apply it to Bayesian optimization with global optimization of the acquisition function and chance-constrained programming. The Gaussian process models, acquisition functions, and training scripts are available open-source within the “MeLOn—MachineLearning Models for Optimization” toolbox (https://git.rwth-aachen.de/avt.svt/public/MeLOn).

Bounding convex relaxations of process models from below by tractable black-box sampling

Several chemical engineering applications demand global optimization of nonconvex process models, including safety verification and determination of thermodynamic equilibria. Methods for deterministic global optimization typically generate crucial bounding information by minimizing convex relaxations of the process model. However, gradients or subgradients of these convex relaxations may be unavailable in practice for several reasons, which may hinder computation of this bounding information. This article shows that useful, correct affine underestimators and lower bounds of convex relaxations may be generated tractably just by black-box sampling. No additional assumptions are required, and no subgradients or gradients must be computed at any point. Variants of these methods are presented to account for numerical error or noise in the sampling procedure. Several numerical examples are presented for illustration, including application of the new sampling-based underestimators in global optimization problems.

COMANDO: A Next-Generation Open-Source Framework for Energy Systems Optimization

Existing open-source modeling frameworks dedicated to energy systems optimization typically utilize (mixed-integer) linear programming ((MI)LP) formulations, which lack granularity for technical system design and operation. We present COMANDO, an open-source Python package for component-oriented modeling and optimization for nonlinear design and operation of integrated energy systems. COMANDO allows to assemble system models from component models including nonlinear, dynamic and discrete characteristics. Based on a single system model, different deterministic and stochastic problem formulations can be obtained by varying objective function and underlying data, and by applying automatic or manual reformulations. The flexible open-source implementation allows for the integration of customized routines required to solve challenging problems, e.g., initialization, problem decomposition, or sequential solution strategies. We demonstrate features of COMANDO via case studies, including automated linearization, dynamic optimization, stochastic programming, and the use of nonlinear artificial neural networks (ANNs) as surrogate models in a reduced-space formulation for deterministic global optimization.

Obey validity limits of data-driven models through topological data analysis and one-class classification

Data-driven models are becoming increasingly popular in engineering, on their own or in combination with mechanistic models. Commonly, the trained models are subsequently used in model-based optimization of design and/or operation of processes. Thus, it is critical to ensure that data-driven models are not evaluated outside their validity domain during process optimization. We propose a method to learn this validity domain and encode it as constraints in process optimization. We first perform a topological data analysis using persistent homology identifying potential holes or separated clusters in the training data. In case clusters or holes are identified, we train a one-class classifier, i.e., a one-class support vector machine, on the training data domain and encode it as constraints in the subsequent process optimization. Otherwise, we construct the convex hull of the data and encode it as constraints. We finally perform deterministic global process optimization with the data-driven models subject to their respective validity constraints. To ensure computational tractability, we develop a reduced-space formulation for trained one-class support vector machines and show that our formulation outperforms common full-space formulations by a factor of over 3000, making it a viable tool for engineering applications. The method is ready-to-use and available open-source as part of our MeLOn toolbox (https://git.rwth-aachen.de/avt.svt/public/MeLOn).

Optimization-based convex relaxations for nonconvex parametric systems of ordinary differential equations

Novel convex and concave relaxations are proposed for the solutions of parametric ordinary differential equations (ODEs), to aid in furnishing bounding information for deterministic global dynamic optimization methods. These relaxations are constructed as the solutions of auxiliary ODE systems with embedded convex optimization problems, whose objective functions employ convex and concave relaxations of the original ODE right-hand side. Unlike established relaxation methods, any valid convex and concave relaxations for the original right-hand side are permitted in the approach, including the McCormick relaxations and the $$\alpha $$ BB relaxations. In general, tighter such relaxations will necessarily translate into tighter relaxations for the ODE solutions, thus providing tighter bounding information for an overarching global dynamic optimization method. Notably, if McCormick relaxations are employed in the new approach, then the obtained relaxations are guaranteed to be at least as tight as state-of-the-art ODE relaxations based on generalized McCormick relaxations. The new relaxations converge rapidly to the original system as the considered parametric subdomain shrinks. Several examples are presented for comparison with established ODE relaxations, based on a proof-of-concept implementation in MATLAB. In a global optimization case study, the new ODE relaxations are shown to lead to fewer branch-and-bound global optimization iterations than state-of-the-art relaxations.