Surrogate Modeling for Superstructure Optimization with Generalized Disjunctive Programming

Process simulator-based optimization of biorefinery downstream processes under the Generalized Disjunctive Programming framework

Nested Heuristic and Gradient-based Method for Generalized Disjunctive Programming

Optimal synthesis of distillation columns: Integration of process simulators in a disjunctive programming environment

RF cavity design exploiting a new derivative-free trust region optimization approach

Modern civilian turbofans are complex and nonlinear systems, and they must get even more complex and nonlinear to meet the requirements for noise, emissions, and fuel burn in the future. The increase in the complexity and nonlinearity are due to more advanced component designs and more complex interactions between these components. Consequently, models must get more complex and nonlinear to be useful for design and development in the future. In particular, real-time models must capture all relevant nonlinearity and complexity with acceptable accuracy and execution time to be useful in important applications, such as control system development. For this purpose, several real-time modeling techniques, namely transfer function, piecewise linear, aero-thermodynamic, and surrogate models can be used. Among the real-time modeling techniques, surrogate models promise high computational speed in addition to capturing nonlinearity. However, this potential can only be realized in practice if a turbofan’s state space is sampled sufficiently and efficiently. Because the state space has functional dependencies, typical space filling sampling techniques face problems, such as sampling a significant number of infeasible points and sampling the feasible state space sparsely or incompletely. To overcome the problem due to the functional dependencies in the state space, this paper proposes sampling a civilian turbofan’s state space with Support Vector Machines (SVM) a nonlinear pattern classifier. Thus, the functional dependencies can be captured as a pattern and less infeasible points are sampled while capturing the feasible state space sufficiently and efficiently. SVM were initially trained with a relatively small set of feasible and infeasible points to estimate the boundary between feasible and infeasible regions in the state space. Thus, the remaining sampling points were chosen from the feasible state space to generate more accurate surrogate models for real-time applications without increasing the number of cases. For sampling and accuracy tests, a non-real-time civilian turbofan model with shaft dynamics was developed to be the truth model. The proposed sampling method was compared with space filling samplings which used 5% and 20% perturbations from the steady state at sea level static (SLS) condition. As a result, the reduction in the number of sampled infeasible points and better coverage of the feasible state space were shown. In this paper multilayer perceptron (MLP) with one hidden layer was chosen to demonstrate the accuracy of surrogate models created using the proposed sampling method because of its advantages over other surrogate modeling techniques and fair comparison with work in the literature. The MLPs trained on the data generated by the proposed sampling method were tested in capturing truth model’s small and large signal behavior with step inputs at SLS.

A new, R-graph based, superstructure and corresponding MINLP model for designing conventional distillation columns are presented. A GDP representation (GR) of the superstructure is first constructed, then it is transformed to MINLP representation to which, in turn, additional trivial improvements are added. The new model has been tested on binary mixture examples, and the obtained results are compared to the results of an MINLP model which developed according to the GDP model of Yeomans and Grossmann. 9 The new model yields shorter computation time and provides better local optima. Additionally, the new model has been used for optimizing a complex multicomponent separation system consisting of several distillation columns. In order to handle such a complex system with a huge number of nonlinear equations, the outer approximation algorithm is modified to provide good initial values for the NLP subproblems. Distillation is one of the most widespread processes applied for separating multicomponent liquid mixtures. It is used for working up large volume or stream, and it requires high investment and operation outlays. The significance of the design of economically optimal distillation processes is of no question. Enormous interest has also been addressed to the area of designing optimal heat integrated distillation columns and distillation sequences. Minimizing the cost of a distillation process implies finding the optimal configuration of each individual column, as well. In the present article we consider staged column models only. The number of stages, and the stage numbers of the feed and side-stream points, are discrete decision variables. The total cost of a column may be modeled as the sum of the fix (capital) cost, depending on the number of stages and on the column diameter, and the variable (operation) cost related to the utility consumption. The objective of the design procedure is to find the optimal configuration of the column, which has the minimum total (annualized) cost. In order to model these processes, discrete decisions are required for calculating the number of stages. Optimizing single columns is a well-known procedure; all the basic textbooks outline how to do it in an easy manner. 1 After approximately determining the minimum and the estimated optimal number of theoretical stages, optimizing over the continuous variables is performed at varied values of the discrete variables. This is a 2-dimension discrete array of continuous optimization tasks in the case of a single-feed, two-product column, because there are merely two column sections in this case. This task becomes much more difficult if several feeds and side-products are to be taken into account. The real problem, however, is synthesizing a distillation sequence, or a system of advanced distillation columns, or a complex flowsheet containing distillation units when the number of distillation columns and their connections are not known in advance. In order to solve the complex synthesis problem outlined above, a proper superstructure for a single column, together with a proper generalized disjunctive programming (GDP) model or a proper mixed-integer nonlinear programming (MINLP) model, is a minimum requirement. Once such a model works well for a single column, the problem of more complex flowsheets may also be addressed. Mixed-integer nonlinear programming (MINLP) and generalized disjunctive programming (GDP) are the two exact methodologies commonly applied for solving process engineering problems with discrete decisions. The former includes algebraic equations describing the process, and binary variables related to discrete decisions. The latter method uses logic variables and expressions to represent the problem. 2 Both formulations have been successfully applied to rigorous models of distillation columns. Both methods apply a fixed maximum number of stages, and the actually used stages are selected from this set. Mathematical formulations that represent rigorous models of distillation column configurations fall into two categories: (i) one task-one equipment (OTOE) representations and (ii) variable task-equipment (VTE) representations. 3

Optimization of a Simulated Well Cluster using Surrogate Models

Multi-solution well placement optimization using ensemble learning of surrogate models

Rice drying synthesis is an essential operation that has to be done carefully and cost-effectively. Fast drying can cause fissuring, which lowers the market value of the rice grains. Multipass drying systems are therefore used to bring the moisture content to desired levels gradually. To determine the best configuration of units and their corresponding operating conditions that maximize rice quality and minimize energy consumption, empirical models are used. However, empirical models have limited ranges of validity. Moreover, different mathematical models are possible for the same synthesis problem. This paper proposes a generalized disjunctive programming (GDP) framework for the synthesis problem of rice drying in order to increase the overall range of applicability of the empirical models and establish a consistent solution strategy. The proposed framework is investigated and tested on several case studies. Different drying strategies resulted from solving the synthesis problem with different empirical ...

In the last two decades, a steady stream of research has been devoted to studying various computational aspects of the ordered k-median problem, which subsumes traditional facility location problems (such as median, center, p-centrum, etc.) through a unified modeling approach. Given a finite metric space, the objective is to locate k facilities in order to minimize the ordered median cost function. In its general form, this function penalizes the coverage distance of each vertex by a multiplicative weight, depending on its ranking (or percentile) in the ordered list of all coverage distances. While antecedent literature has focused on mathematical properties of ordered median functions, integer programming methods, various heuristics, and special cases, this problem was not studied thus far through the lens of approximation algorithms. In particular, even on simple network topologies, such as trees or line graphs, obtaining non-trivial approximation guarantees is an open question. The main contribution of this paper is to devise the first provably-good approximation algorithms for the ordered k-median problem. We develop a novel approach that relies primarily on a surrogate model, where the ordered median function is replaced by a simplified ranking-invariant functional form, via efficient enumeration. Surprisingly, while this surrogate model is $$\varOmega ( n^{ \varOmega (1) } )$$-hard to approximate on general metrics, we obtain an $$O(\log n)$$-approximation for our original problem by employing local search methods on a smooth variant of the surrogate function. In addition, an improved guarantee of $$2+\epsilon $$ is obtained on tree metrics by optimally solving the surrogate model through dynamic programming. Finally, we show that the latter optimality gap is tight up to an $$O(\epsilon )$$ term.

ive function are in general the output of a complex simulator for which we don’t have any explicit expression. And the absence of any information on the function gradient narrows the resolution field to algorithms using no first or second order derivatives. There exists many different approaches in derivative free optimization, among which the most popular are space partitioning methods like DIRECT, direct search methods like Nelder Mead or MADS but also evolutionary algorithms like genetic algorithms, evolution strategies or other similar methods. The main drawback for all these methods is that they can suffer from a poor convergence rate and a high computational cost, especially for high dimensional cases. However, they can succeed in finding a global optimum where all other classical methods fail. In this work we consider surrogate based optimization which has already been widely used for many years. A surrogate model is a framework used to minimize a function by sequentially building and minimizing a simpler model (surrogate) of the original function. In this work we build a new surrogate model by using the sparse grid interpolation method. Basically, the sparse grid approach is a grid error-controlled hierarchical approximation method which neglects the basis functions with the smallest supports. This approach was introduced in 1963 by Smolyak in order to evaluate integrals in high dimensions. It was then applied for PDE approximations but also for optimization and more recently for sensitivity analysis. Compared to the first optimization algorithm based on sparse grids proposed by Klimke et al, a local refinement step is constructed here in order to explore the more promising regions. Moreover, no optimization steps are performed over the objective function, which reduces significantly the number of function evaluations employed. In this talk we present our GOSGrid optimization algorithm based on sparse grids. We also compare it with other derivative free global algorithms. The comparison is based on a parameter estimation problem in reservoir engineering.

Optimization of dynamic systems often requires system simulation. Several important classes of dynamic system models have computationally expensive time derivative functions, resulting in simulations that are significantly slower than real time. This makes design optimization based on these models impractical. An efficient two-loop method, based on surrogate modeling, is presented here for solving dynamic system design problems with computationally expensive derivative functions. A surrogate model is constructed for only the derivative function instead of the simulation response. Simulation is performed based on the computationally inexpensive surrogate derivative function; this strategy preserves the nature of the dynamic system, and improves computational efficiency and accuracy compared to conventional surrogate modeling. The inner-loop optimization problem is solved for a given derivative function surrogate model (DFSM), and the outer loop updates the surrogate model based on optimization results. One unique challenge of this strategy is to ensure surrogate model accuracy in two regions: near the optimal point in the design space, and near the state trajectory in the state space corresponding to the optimal design. The initial evidence of method effectiveness is demonstrated first using two simple design examples, followed by a more detailed wind turbine codesign problem that accounts for aeroelastic effects and simultaneously optimizes physical and control system design. In the last example, a linear state-dependent model is used that requires computationally expensive matrix updates when either state or design variables change. Results indicate an order-of-magnitude reduction in function evaluations when compared to conventional surrogate modeling. The DFSM method is expected to be beneficial only for problems where derivative function evaluation expense, and not large problem dimension, is the primary contributor to solution expense (a restricted but important problem class). The initial studies presented here revealed opportunities for potential further method improvement and deeper investigation.

Logic-based MINLP algorithms for the optimal synthesis of process networks

Online tools such as the Truterra Insights Engine (IE) provide key performance indicator (KPI) assessments supporting farmer decisions to sustainably produce and deliver crops to market (<underline>https://www.truterraag.com/</underline>). They enable farmers to analyze and improve the long-term health of their ag-operations and participate in resilient supply chains. For example, IE leverages four physically-based models and algorithms for creating data used to assess cropping system KPIs for water and wind soil erosion, soil organic matter trend, and greenhouse gas emissions. Its scope spans 19 states from Minnesota to Louisiana, and from Kansas to Pennsylvania. Currently, IE supports assessments of six crop rotations (corn, soybeans, wheat), four tillage regimes, six cover crop options, six nitrogen fertilization methods, and six conservation practice combinations across the soils and climates of the 19 states. The existing models used to create the KPI dataset require relatively extensive data input and take significant time to finish a simulation, thus they are not suitable for direct real-time use. By contrast, surrogate models (SM) require many fewer inputs and finish quickly, but are constrained by a defined spatial coverage and providing less precise results. The challenge is to achieve a solution balancing rapid, scientifically valid, sufficiently precise, and applicable results for the regions of interest. Recent advances in machine learning combined with a streamlined SM delineation (Serafin, 2019; Serafin et al., 2021) offer potential for significantly improving surrogate model availability, precision, and applicability. For estimating water induced sheet and rill erosion for KPI assessments we employed the WEPP (Water Erosion Prediction Project) model as the process-based technology for prediction of soil erosion by water at hillslope profile, field, and small watershed scales. The overall objective of this ongoing modeling effort is the creation of a surrogate model for WEPP to produce erosion estimates at the field scale within a given region while: 1) creating surrogate erosion estimates within the range of 10-20% relative RMSE of WEPP results, 2) minimizing the number of required surrogate model inputs, and 3) consistently creating SM erosion results within seconds. We focused our study on rotations of Corn-Soybean, Continuous Corn, Corn-Winter Wheat, Corn-Sorghum in the Central Great Plains (LRR H), creating 5M WEPP simulation scenarios, permuting the inputs for yield, slope steepness and length as well as soil properties. All simulations were executed on a 1200 core Kubernetes cluster, hosting the WEPP CSIP service and supporting Conservation Resources CSIP data services for SSURGO soils, climate and DEM data. Using the CSIP Publish/Subscribe approach (David et al., 2013) for high performance model executions, 2.7M result sets were generated and analyzed. Those data sets were used for ML training with LightGBM, a Light Gradient Boosting Machine open-source framework based on decision tree algorithms for ranking and classification. Exploring various approaches for selecting sensitive and representative input data we generated a surrogate model for 16 sensitive SM inputs for precipitation, yield, crop, slope and soil properties to estimate sheet and rill erosion. Results from SM testing are shown in Figure 1. <fig><graphic xlink:href=23102_files/23102-00.jpg id=E412E382-9B1C-4A55-A8A7-9CDF01C8C2D0></graphic></fig> Using ~1.9M training and ~820K test samples (70/30%), selected using a Kolmogorov–Smirnov test, we achieved a RRMSE of 6.4%, KGE of 0.999, and NSE of 0.998 for training and a RRMSE of 8.1%, KGE of 0.998, and NSE of 0.997 for testing. The generated SM was further integrated into an SM KPI service in CSIP, producing erosion estimates for fields within LRR H in 2.8 seconds of SM runtime, adding time to obtain needed inputs. This work is ongoing and the presented results represent a research snapshot. Currently, the authors are exploring supplementing the current method with an ensemble approach to further align the SM output accuracy with the original WEPP output.

Online tools such as the Truterra Insights Engine (IE) provide key performance indicator (KPI) assessments supporting farmer decisions to sustainably produce and deliver crops to market (<underline>https://www.truterraag.com/</underline>). They enable farmers to analyze and improve the long-term health of their ag-operations and participate in resilient supply chains. For example, IE leverages four physically-based models and algorithms for creating data used to assess cropping system KPIs for water and wind soil erosion, soil organic matter trend, and greenhouse gas emissions. Its scope spans 19 states from Minnesota to Louisiana, and from Kansas to Pennsylvania. Currently, IE supports assessments of six crop rotations (corn, soybeans, wheat), four tillage regimes, six cover crop options, six nitrogen fertilization methods, and six conservation practice combinations across the soils and climates of the 19 states. The existing models used to create the KPI dataset require relatively extensive data input and take significant time to finish a simulation, thus they are not suitable for direct real-time use. By contrast, surrogate models (SM) require many fewer inputs and finish quickly, but are constrained by a defined spatial coverage and providing less precise results. The challenge is to achieve a solution balancing rapid, scientifically valid, sufficiently precise, and applicable results for the regions of interest. Recent advances in machine learning combined with a streamlined SM delineation (Serafin, 2019; Serafin et al., 2021) offer potential for significantly improving surrogate model availability, precision, and applicability. For estimating water induced sheet and rill erosion for KPI assessments we employed the WEPP (Water Erosion Prediction Project) model as the process-based technology for prediction of soil erosion by water at hillslope profile, field, and small watershed scales. The overall objective of this ongoing modeling effort is the creation of a surrogate model for WEPP to produce erosion estimates at the field scale within a given region while: 1) creating surrogate erosion estimates within the range of 10-20% relative RMSE of WEPP results, 2) minimizing the number of required surrogate model inputs, and 3) consistently creating SM erosion results within seconds. We focused our study on rotations of Corn-Soybean, Continuous Corn, Corn-Winter Wheat, Corn-Sorghum in the Central Great Plains (LRR H), creating 5M WEPP simulation scenarios, permuting the inputs for yield, slope steepness and length as well as soil properties. All simulations were executed on a 1200 core Kubernetes cluster, hosting the WEPP CSIP service and supporting Conservation Resources CSIP data services for SSURGO soils, climate and DEM data. Using the CSIP Publish/Subscribe approach (David et al., 2013) for high performance model executions, 2.7M result sets were generated and analyzed. Those data sets were used for ML training with LightGBM, a Light Gradient Boosting Machine open-source framework based on decision tree algorithms for ranking and classification. Exploring various approaches for selecting sensitive and representative input data we generated a surrogate model for 16 sensitive SM inputs for precipitation, yield, crop, slope and soil properties to estimate sheet and rill erosion. Results from SM testing are shown in Figure 1. <fig><graphic xlink:href=23102_files/23102-00.jpg id=E412E382-9B1C-4A55-A8A7-9CDF01C8C2D0></graphic></fig> Using ~1.9M training and ~820K test samples (70/30%), selected using a Kolmogorov–Smirnov test, we achieved a RRMSE of 6.4%, KGE of 0.999, and NSE of 0.998 for training and a RRMSE of 8.1%, KGE of 0.998, and NSE of 0.997 for testing. The generated SM was further integrated into an SM KPI service in CSIP, producing erosion estimates for fields within LRR H in 2.8 seconds of SM runtime, adding time to obtain needed inputs. This work is ongoing and the presented results represent a research snapshot. Currently, the authors are exploring supplementing the current method with an ensemble approach to further align the SM output accuracy with the original WEPP output.