Linear Surrogate Model Research Articles

Batch sequential designs in Bayesian preference elicitation with application to tradespace exploration for vehicle concept design

Engineering design is a complex decision making process that depends on multiple design objectives and often requires frequent information exchange between designers and decision makers (DM), such as product managers or users. A common method to facilitate this information exchange and explore the tradespace of different design objectives is through preference elicitation, where DMs are asked to answer a sequence of queries to inquire preference comparisons between multiple design alternatives. In this work, we generalize the ellipsoid method proposed by Sauré and Vielma (2019) to the batch sequential setting, where multiple queries are selected at a time to be provided to the DM when some operational constraints make a fully sequential questionnaire impractical. In our setting, we use D-error as our selection criterion, the minimization of which leads to more precise estimates of the DM’s utility model parameters. We propose approximations of this criterion using linear surrogate models where the predictors are functions of certain summary statistics of the batch design whose effect can be estimated via offline experiments. Based on these approximations, we propose mixed integer programming formulations for constructing batch designs which can be solved in commercial optimization solvers. Our simulation studies show that the proposed methods provide highly D-efficient batch designs when compared to randomly generated designs. In addition, we show that our methods are comparable to a fully adaptive method when the batch size is small. Lastly, we present an example of using our methodology to aid in tradespace exploration for vehicle concept design.

Optimization design method based on parameter reduction and active subspaces: Redistribution of chordwise loading at blade tips in a transonic axial-flow fan

In practical optimization design, an excessive number of design variables have a highly detrimental influence on the efficiency and accuracy of the final design scheme and expose the optimization problem to the curse of dimensionality. Therefore, incorporating only the most essential variables into an optimization design problem facilitates obtaining accurate and cost-efficient solutions. Reported here is an optimization design method based on parameter reduction and active subspaces, and it is used to redistribute the tip load in a transonic fan. Specifically, a coupled design strategy is developed to reduce the number of parameters needed to describe the three-dimensional blade shape, which leads to far fewer design variables being involved in the optimization design. Moreover, active subspaces are used to perform sensitivity analysis and establish low-dimensional surrogate models. After the coupled design, a blade is represented effectively by only three parameters, each of which has a significant influence on the fan performance. Three one-dimensional active subspaces are established for maximum mass flow rate, maximum total pressure ratio, and maximum efficiency, based on which the linear surrogate models are obtained. Next, the chordwise tip blade loading is optimized, after which the rotor efficiency at the design point is increased by 1.1%, while the total pressure ratio remains nearly unchanged. Finally, the flow field is analyzed to understand the mechanism for this performance improvement, and the results show that the optimized blade loading reduces the aerodynamic losses caused by shock-induced flow separation and the interaction between shocks and tip leakage flows.

Surgery scheduling in flexible operating rooms by using a convex surrogate model of second-stage costs

We study the elective surgery planning problem in a hospital with operating rooms shared by elective and emergency patients. This problem is split in two distinct phases. First, a subset of patients to be operated in the next planning period is selected and the selected patients are assigned to a block and a tentative starting time. Then, in the online phase of the problem, a policy decides how to insert the emergency patients in the schedule and may cancel planned surgeries. The overall goal is to minimize the expectation of a cost function representing the assignment of patient to blocks, case cancellations, overtime, waiting time and idle time. We model the offline problem by a two-stage stochastic program, and show that the optimal second-stage costs can be approximated by a convex piecewise linear surrogate model that can be computed in a preprocessing step. This results in a mixed integer program which can be solved very fast, even for large instances of the problem. We also describe a greedy policy for the online phase of the problem, and analyze the performance of our approach by comparing it to both heuristic methods or approaches relying on sampling average approximation (SAA) on a large set of benchmarking instances. Our simulations indicate that our approach can reduce the expected costs by as much as 30% compared to heuristic methods and it can solve problems with 1000 patients in about one minute, while SAA-approaches fail to obtain good solutions within 30 min on small instances.

Data-driven models of nonautonomous systems

Nonautonomous dynamical systems are characterized by time-dependent inputs, which complicates the discovery of predictive models describing the spatiotemporal evolution of the state variables of quantities of interest from their temporal snapshots. When dynamic mode decomposition (DMD) is used to infer a linear model, this difficulty manifests itself in the need to approximate the time-dependent Koopman operators. Our approach is to approximate the original nonautonomous system with a modified system derived via a local parameterization of the time-dependent inputs. The modified system comprises a sequence of local parametric systems, which are subsequently approximated by a parametric surrogate model using the DRIPS (dimension reduction and interpolation in parameter space) framework. The offline step of DRIPS relies on DMD to build a linear surrogate model, endowed with reduced-order bases for the observables mapped from training data. The online step interpolates on suitable manifolds to construct a sequence of iterative parametric surrogate models; the target/test parameter points on these manifolds are specified by a local parameterization of the test time-dependent inputs. We use numerical experimentation to demonstrate the robustness of our method and compare its performance with that of deep neural networks.

Improving the Accuracy of the Neural Network Models Interpretation of Nonlinear Dynamic Objects

The paper is devoted to the problem of neural network interpretation in the tasks of modeling nonlinear dynamic objects. The purpose of the work is to improve the accuracy of the neural network models interpretation of nonlinear dynamic objects and to determine the scope of their effective application. This goal is achieved by applying analytical models in the form of integral-power series based on multidimensional weight functions. The scientific novelty of the work lies in the use of nonlinear dynamic models in the form of integral-power series based on multidimensional weight functions instead of linear surrogate models. It allows to improve modeling accuracy. The practical usefulness of the work is determination of the effective application area of analytical interpretive models. The practical significance of the obtained results lies in the application of the proposed models for the interpretation of neural network models of nonlinear dynamic objects, which allows to increase the accuracy of interpretation models compared to linear surrogate models.

DRIPS: A framework for dimension reduction and interpolation in parameter space

Reduced-order models are often used to describe the behavior of complex systems, whose simulation with a full model is too expensive, or to extract salient features from the full model's output. We introduce a new model-reduction framework DRIPS (dimension reduction and interpolation in parameter space) that combines the offline local model reduction with the online parameter interpolation of reduced-order bases (ROBs). The offline step of this framework relies on dynamic mode decomposition (DMD) to build a low-rank linear surrogate model, equipped with a local ROB, for quantities of interest derived from the training data generated by repeatedly solving the (nonlinear) high-fidelity model for multiple parameter points. The online step consists of the construction of a parametric reduced-order model for each target/test point in the parameter space, with the interpolation of ROBs done on a Grassman manifold and the interpolation of reduced-order operators done on a matrix manifold. The DMD component enables DRIPS to model (typically low-dimensional) quantities of interest directly, without having to access the (typically high-dimensional and possibly nonlinear) operators in a high-fidelity model that governs the dynamics of the underlying high-dimensional state variables, as required in projection-based reduced-order modeling. A series of numerical experiments suggests that DRIPS yields a model reduction, which is computationally more efficient than the commonly used projection-based proper orthogonal decomposition; it does so without requiring a prior knowledge of the governing equation for quantities of interest. Moreover, for the nonlinear systems considered, DRIPS is more accurate than Gaussian-process interpolation (Kriging).

More Efficient Energy Management for Networked Hybrid AC/DC Microgrids With Multivariable Nonlinear Conversion Losses

Accurate conversion loss models are the keys to guarantee the more efficient operation of networked hybrid ac/dc microgrids (N-ac/dc-MGs). A two-stage stochastic unit commitment (UC) problem is proposed to improve the operational efficiency of N-ac/dc-MGs under uncertain renewable energy generation output and loads. The nonlinear power losses of ac/dc converters and dc/dc converters under different operating modes are formulated as novel multivariate nonlinear functions of both power and voltage. These functions are approximated by linear surrogate models and adopted by the dynamic optimal power flow (OPF), where the voltage and power of the converters can be optimized considering the physical laws among voltages of converters. Embedding the dynamic OPF problem as mixed-integer recourse, a two-stage stochastic programming problem is formulated for the day-ahead operation of N-ac/dc-MGs. To reduce the computational cost, a finite iteration convergent Benders decomposition algorithm is proposed to solve the UC problem. Case studies are performed on hybrid ac/dc MGs and N-ac/dc-MGs. Simulation results reveal that using more accurate conversion loss models, the output commands, especially the dc bus voltage, from the energy management systems will lead to more efficient system operation, and thus, save energy.

On global normal linear approximations for nonlinear Bayesian inverse problems

The replacement of a nonlinear parameter-to-observable mapping with a linear (affine) approximation is often carried out to reduce the computational costs associated with solving large-scale inverse problems governed by partial differential equations (PDEs). In the case of a linear parameter-to-observable mapping with normally distributed additive noise and a Gaussian prior measure on the parameters, the posterior is Gaussian. However, substituting an accurate model for a (possibly well justified) linear surrogate model can give misleading results if the induced model approximation error is not accounted for. To account for the errors, the Bayesian approximation error (BAE) approach can be utilised, in which the first and second order statistics of the errors are computed via sampling. The most common linear approximation is carried out via linear Taylor expansion, which requires the computation of (Fréchet) derivatives of the parameter-to-observable mapping with respect to the parameters of interest. In this paper, we prove that the (approximate) posterior measure obtained by replacing the nonlinear parameter-to-observable mapping with a linear approximation is in fact independent of the choice of the linear approximation when the BAE approach is employed. Thus, somewhat non-intuitively, employing the zero-model as the linear approximation gives the same approximate posterior as any other choice of linear approximations of the parameter-to-observable model. The independence of the linear approximation is demonstrated mathematically and illustrated with two numerical PDE-based problems: an inverse scattering type problem and an inverse conductivity type problem.

Machine learning surrogate models for strain-dependent vibrational properties and migration rates of point defects

Machine learning surrogate models employing atomic environment descriptors have found wide applicability in materials science. In our previous work, this approach yielded accurate and transferable predictions of the vibrational formation entropy of point defects for $\mathcal{O}(N)$ computational cost. The present study investigates the limits of data driven surrogate models in accuracy and applicability for vibrational properties. We propose an improvement of the accuracy by extending the fitting capacity of the model by increasing the dimension of the descriptor space. This is achieved by using a nonlinear relation between descriptors---target observables and when it is possible by including physical relevant information of the underlying energy landscape. The nonlinear extension is used to learn the formation entropy of defects with or without applied strain while including physical information, such as the minimum-saddle point sequences employed for the migration of point defects, is a key ingredient of transition state theory rate approximations. We find excellent predictive power after augmenting the dimensionality of the descriptor space, as demonstrated on large defect databases in $\ensuremath{\alpha}$-iron and amorphous silicon based on semiempirical force fields. The current linear surrogate models are used to investigate the correlation between migration entropy and energy. Our approaches reproduce the Meyer-Neldel compensation law observed from direct calculations in amorphous Si systems. Moreover, the same abstract descriptor space representation for entropy and energy is then used for the statistical correlation analysis. For linear surrogate models, we show that the energy-entropy statistical correlations can be reinterpreted in descriptor space. This provides a simple statistical criterion for the marginal interpretation of the compensation law. More generally, the present work shows how linear surrogate models can accelerate high-throughput workflows, aid the construction of mesoscale material models, and provide new avenues for correlation analysis.

Stick or Spill? Scaling Relationships for the Binding Energies of Adsorbates on Single-Atom Alloy Catalysts

Single-atom alloy catalysts combine catalytically active metal atoms, present as dopants, with the selectivity of coinage metal hosts. Determining whether adsorbates stick at the dopant or spill over onto the host is key to understanding catalytic mechanisms on these materials. Despite a growing body of work, simple descriptors for the prediction of spillover energies (SOEs), i.e., the relative stability of an adsorbate on the dopant versus the host site, are not yet available. Using Density Functional Theory (DFT) calculations on a large set of adsorbates, we identify the dopant charge and the SOE of carbon as suitable descriptors. Combining them into a linear surrogate model, we can reproduce DFT-computed SOEs within 0.06 eV mean absolute error. More importantly, our work provides an intuitive theoretical framework, based on the concepts of electrostatic interactions and covalency, that explains SOE trends and can guide the rational design of future single-atom alloy catalysts.

A methodology for uncertainty quantification and sensitivity analysis for responses subject to Monte Carlo uncertainty with application to fuel plate characteristics in the ATRC

Large-scale reactor simulation often requires the use of Monte Carlo calculation techniques to estimate important reactor parameters. One drawback of these Monte Carlo calculation techniques is they inevitably result in some uncertainty in calculated quantities. The present study includes parametric uncertainty quantification (UQ) and sensitivity analysis (SA) on the Advanced Test Reactor Critical (ATRC) facility housed at Idaho National Laboratory (INL) and addresses some complications due to Monte Carlo uncertainty when performing these analyses. This approach for UQ/SA includes consideration of Monte Carlo code uncertainty in computed sensitivities, consideration of uncertainty from directly measured parameters and a comparison of results obtained from brute-force Monte Carlo UQ versus UQ obtained from a surrogate model. These methodologies are applied to the uncertainty and sensitivity of keff for two sets of uncertain parameters involving fuel plate geometry and fuel plate composition.Results indicate that the less computationally-expensive method for uncertainty quantification involving a linear surrogate model provides accurate estimations for keff uncertainty and the Monte Carlo uncertainty in calculated keff values can have a large effect on computed linear model parameters for parameters with low influence on keff.

Generation of linear-based surrogate models from non-linear functional relationships for use in scheduling formulation

Often functional relationships are well known, but they are too complex to be used efficiently in optimization problems like scheduling formulations. Hence the functions are often replaced by data-based surrogate models. Especially, linear models are often used, since they are easier to solve than non-linear ones. The use of piecewise linear surrogate models allows for an improved consideration of nonlinearities. Although, the number of linear elements must be kept small in order not to lose the advantages of a linear-based formulation. In this work, two approaches for generating piecewise linear surrogate models are proposed, whereby the basic idea of both approaches is the determination of a reduced set of data points that provides an appropriate approximation of the original data via multi-dimensional linear interpolation. The approaches differ in their concepts: One is a numerical algorithm, the other an optimization-based technique. In this contribution, these approaches are described and subsequently compared.

CSO Reduction by Integrated Model Predictive Control of Stormwater Inflows: A Simulated Proof of Concept Using Linear Surrogate Models

AbstractCombined sewer overflows (CSO) of mixed stormwater and wastewater pollute nearby receiving surface waters and pose a risk to the environment and human health. We use “integrated stormwater inflow control” to mitigate CSO by dynamically controlling the inflow of stormwater to the combined sewer system in real time, expanding the physical space of traditional real‐time control. This control is carried out with model predictive control (MPC), which we base on convex optimization including a linear internal surrogate model of the controllable aboveground and belowground infrastructure. A detailed hydrodynamic model is used to evaluate the results and recursively initialize the surrogate model. MPC dynamically decides when to let stormwater enter the sewer system and when to store and convey excess stormwater in the aboveground infrastructure otherwise intended for passive cloudburst management. The performance was quantified in a simulation study in Copenhagen, Denmark, using a 1‐D distributed hydrodynamic model and 32 rain events from 2016, of which 18 caused CSO in the situation without control. Four of the 18 CSO events were avoided with MPC, and the total CSO volume was reduced by 98.4% of the potential reducible volume. For one event, stormwater was unnecessarily kept on the surface because the surrogate model wrongly predicted a CSO. The computational cost was in all cases compatible with an operational implementation. With the invention of proper actuators for control of stormwater inflows, we show that MPC of stormwater inflows may be a viable supplement to more traditional passive ways of managing stormwater in urban areas.

Surrogate-model based MILP for the optimal design of ethylene production from shale gas

We propose a novel algorithm for the optimal design of entire plants by refining piecewise linear surrogate models within an iterative framework. We apply this strategy to a superstructure for ethane-based ethylene production, including steam cracking and alternative technologies, and the separation, utility, carbon dioxide and hydrogen recovery systems. Multivariable piecewise linear surrogate models (SM) based on rigorous Aspen Plus models and capital cost correlations are obtained by solving Generalized Disjunctive Programming problems. Using these surrogates, a Master MILP problem is formulated to determine the optimal design. If convergence criteria are not met, SM are progressively refined in subsequent iterations. The optimal solution is the chemical looping oxidative dehydrogenation technology, whose net present value (NPV) is 12% higher than that of conventional steam cracking, while reducing the ethylene production cost by 15%. Finally, we validate the optimal design with Aspen Plus, obtaining an NPV error of less than 1%.

Spatial Localization for Nonlinear Dynamical Stochastic Models for Excitable Media

Nonlinear dynamical stochastic models are ubiquitous in different areas. Their statistical properties are often of great interest, but are also very challenging to compute. Many excitable media models belong to such types of complex systems with large state dimensions and the associated covariance matrices have localized structures. In this article, a mathematical framework to understand the spatial localization for a large class of stochastically coupled nonlinear systems in high dimensions is developed. Rigorous mathematical analysis shows that the local effect from the diffusion results in an exponential decay of the components in the covariance matrix as a function of the distance while the global effect due to the mean field interaction synchronizes different components and contributes to a global covariance. The analysis is based on a comparison with an appropriate linear surrogate model, of which the covariance propagation can be computed explicitly. Two important applications of these theoretical results are discussed. They are the spatial averaging strategy for efficiently sampling the covariance matrix and the localization technique in data assimilation. Test examples of a linear model and a stochastically coupled FitzHugh-Nagumo model for excitable media are adopted to validate the theoretical results. The latter is also used for a systematical study of the spatial averaging strategy in efficiently sampling the covariance matrix in different dynamical regimes.

Layered reduced-order models for nonlinear aerodynamics and aeroelasticity

A layered reduced-order modeling approach for nonlinear unsteady aerodynamics comprising both linear and nonlinear characteristics is developed. The constructed reduced-order model (ROM) with a parallel connection structure is composed of a linear surrogate model and a nonlinear one. The linear autoregressive with exogenous input (ARX) model is constructed by a training case at small amplitude, while the nonlinear radial basis function neural network (RBFNN) model is identified to model the residual nonlinear responses from the second training case under large amplitude motions. Outputs from both models are superimposed to obtain the total aerodynamic coefficients. Either quasi-steady or unsteady layered model can be constructed by introducing only input delay or considering both input and output delayed effects. The present approach is tested by predicting the unsteady aerodynamic forces, limit cycle oscillations and flutter behaviors of a transonic NACA 64A010 airfoil. Results show that both the quasi-steady and the unsteady models agree well with those from high fidelity simulations within a wide range of amplitude and frequency, and the LCO trends are reasonable after the models are coupled with the structural equations of motion. A better agreement is achieved by the unsteady layered model, even though this model may sometimes become unstable.

Model Predictive Control of a Large-scale River Network

This study investigates the use of Model Predictive Control (MPC) for regulation of river flows. During the past decade, MPC has emerged for controlling open water systems, such as irrigation and drainage channels. Compared to a full river network, irrigation and drainage systems are of relatively small scale. The aim of the present work is to investigate MPC as a tool for control of releases from gates and dams in a large-scale river network using the Murrumbidgee River in New South Wales, Australia, as case study. The Murrumbidgee River has around 1300 kilometers of river reaches, and the travel time through the valley is of the order of one month. The research has focused on four points: 1) Configuring linear surrogate models to describe the characteristics of reaches and weir pools; 2) Formulating the control problem and its objectives; 3) Using MPC with a receding horizon to solve the control problem; and 4) Testing the accuracy of the calculated control action, by using it as forcing in a detailed hydraulic model. The tests show that a reliable computation of optimal releases from regulators throughout the river is obtained, despite the linear approximation of the dynamics. The tests also show that the computation time for setting up and solving the optimization problem is no more than a few minutes on today's laptops.

Reconstructing building stock to replicate energy consumption data

The paper introduces an approach to replicate building stock energy data using energy survey data. For demonstration of the approach, the research uses energy consumption data for office buildings in Chicago from Commercial Building Energy Consumption Survey (CBECS) 2003. The replication starts from derivation of the energy use distribution for a building stock in a specific location from the survey data. Then probabilistic methods are used to map building stock model space to real-world data space reflecting a weather adjustment of the energy survey data. The approach leverages a linear surrogate model of the physics-based reduced order normative energy model. The normative building energy model can rapidly estimate the building energy performance with respect to its design and operational characteristics. The research investigates a statistical procedure to inversely estimate building parameters using regression and Bayesian inference model based on the Markov Chain Monte Carlo (MCMC) sampling techniques. The research serves a new paradigm of the building stock aggregation that can lead to an efficient energy model, which contributes the body of knowledge of energy modeling beyond the single building scale.

Model based optimisation of friction stir welding processes

This paper proposes a model based approach for the optimisation of friction stir welding processes. The proposed approach starts with building a model of the process. For this study, the thermal model developed by Chao and his associate for friction stir welding of AA 2195-T8 is replicated using Fluent. Once developed, the thermal model is then used to simulate the process. Two surrogate models, one linear and one non-linear, are constructed to relate three process parameters with maximum temperature at a selected location, using the simulation data generated by the thermal model. A constrained optimisation model is next formulated, which is eventually solved by five population based metaheuristrics to find the optimal solutions for the studied friction stir welding process. The optimal solutions are primarily constrained by the lower bound of the temperature. The lower this lower bound temperature is the higher travel speed can go. The linear surrogate model results in a slightly better optimal solution than the non-linear model when the temperature constraint is loose and the converse is true when the temperature constraint is tight. Comparing all five metaheuristics, differential evolution has the best performance, followed by particle swarm optimisation, ant colony optimisation, genetic algorithm, and lastly harmony search.