Data-driven Surrogate Model Research Articles

Surrogate Modeling for Solving OPF: A Review

The optimal power flow (OPF) problem, characterized by its inherent complexity and strict constraints, has traditionally been approached using analytical techniques. OPF enhances power system sustainability by minimizing operational costs, reducing emissions, and facilitating the integration of renewable energy sources through optimized resource allocation and environmentally aligned constraints. However, the evolving nature of power grids, including the integration of distributed generation (DG), increasing uncertainties, changes in topology, and load variability, demands more frequent OPF solutions from grid operators. While conventional methods remain effective, their efficiency and accuracy degrade as computational demands increase. To address these limitations, there is growing interest in the use of data-driven surrogate models. This paper presents a critical review of such models, discussing their limitations and the solutions proposed in the literature. It introduces both Analytical Surrogate Models (ASMs) and learned surrogate models (LSMs) for OPF, providing a thorough analysis of how they can be applied to solve both DC and AC OPF problems. The review also evaluates the development of LSMs for OPF, from initial implementations addressing specific aspects of the problem to more advanced approaches capable of handling topology changes and contingencies. End-to-end and hybrid LSMs are compared based on their computational efficiency, generalization capabilities, and accuracy, and detailed insights are provided. This study includes an empirical comparison of two ASMs and LSMs applied to the IEEE standard six-bus system, demonstrating the key distinctions between these models for small-scale grids and discussing the scalability of LSMs for more complex systems. This comprehensive review aims to serve as a critical resource for OPF researchers and academics, facilitating progress in energy efficiency and providing guidance on the future direction of OPF solution methodologies.

Data-Driven Surrogate Modeling with Microstructure-Sensitivity of Viscoplastic Creep in Grade 91 Steel

AbstractTo support the development of advanced steel alloys tailored to withstand extreme conditions, it is imperative to account for the mechanical performance of components, while considering the influence of local microstructure on the macroscopic response. To this end, this study focuses on the development of microstructure-sensitive constitutive models for the mechanical response of Grade 91 steel exposed to extreme thermo-mechanical environments. Polynomial chaos expansion (PCE) surrogates are used to emulate high-fidelity polycrystal simulations of the viscoplastic response of Grade 91 steel as a function of the microstructure fingerprint (e.g., dislocations and precipitates). To cover a wide temperature–stress domain, two separate PCE surrogates—one that captures softening and the other that captures hardening behavior—are combined using another (sparse) Gaussian process regression model. The resulting constitutive creep surrogate model is integrated within the MOOSE finite element framework to simulate the intricate effects of microstructure, in particular MX-phase precipitates, on a component with a graded microstructure. Surrogate sensitivity analysis is applied to quantify the relevant impact of spatially varying microstructure on the creep response in a test-case involving a Grade 91 alloy with a prototypical weld.

Data-driven surrogate model for wind turbine damage equivalent load

Abstract. Aeroelastic simulations are employed to assess wind turbines in accordance with IEC standards in the time domain. These analyses enable the evaluation of fatigue and extreme loads experienced by wind turbine components. Such simulations are essential for several reasons, including but not limited to reducing safety margins in wind turbine component design by accounting for a wide range of uncertainties in wind and wave conditions and fulfilling the requirements of the digital twin, which necessitates a comprehensive set of simulations for calibration. Thus, it is essential to develop computationally efficient yet accurate models that can replace costly aeroelastic simulations and data processing. To address this challenge, we propose a data-driven approach to construct surrogate models for the damage equivalent load (DEL) based on aeroelastic simulation outputs. Our method provides a quick and efficient way to calculate DEL using wind input signals without the need for time-consuming aeroelastic simulations. Our study focuses on utilizing a sequential machine learning (ML) method to map wind speed time series to DEL. Additionally, we demonstrate the versatility of the developed and trained surrogate models by testing them on a wind turbine in the wake and applying transfer learning to enhance their predictive accuracy.

Data-driven homogenisation of viscoelastic porous elastomers: feedforward versus knowledge-based neural networks

A computational framework is established to implement time-dependent data-driven surrogate constitutive models for the homogenised mechanical response of porous elastomers at large strains. The aim is to enhance the computational efficiency of multiscale analyses through the use of these surrogate models. To achieve this, explicit finite element (FE) simulations are conducted to predict the homogenised response of a cubic unit cell of a porous elastomer, using two different viscoelastic descriptions of the parent material, subject to pseudo-random, multiaxial, non-proportional histories of macroscopic strains. The histories of homogenised variables extracted from each set of FE predictions form a training dataset, which is used to train two different surrogate models, both relying on artificial neural networks (NNs). The first model predicts the increment in macroscopic stress over a simulation step, as a function of the macroscopic stress and strain at the beginning of the step, of the prescribed macroscopic strain increment, and of the corresponding time increment. The second model uses the same inputs and outputs but tests a knowledge-based modelling approach: it relies on the aid of an additional nonlinear elastic constitutive model, which is time- and path-independent and known a priori. This model represents an existing base of knowledge which is augmented and corrected by an NN after training on viscoelastic data. The data-driven surrogate model, therefore, learns the viscoelastic behaviour of the unit cell starting from knowledge of its elastic response. The two surrogate models are found to have comparable and very high accuracies at capturing the homogenised unit cell response to highly complex loading cases across a range of viscoelastic properties. Hyperparameter optimisation shows that the second, knowledge-based model requires a simpler NN and therefore incurs a smaller computational cost.

Nonlinear parametric models of viscoelastic fluid flows.

Reduced-order models (ROMs) have been widely adopted in fluid mechanics, particularly in the context of Newtonian fluid flows. These models offer the ability to predict complex dynamics, such as instabilities and oscillations, at a considerably reduced computational cost. In contrast, the reduced-order modelling of non-Newtonian viscoelastic fluid flows remains relatively unexplored. This work leverages the sparse identification of nonlinear dynamics (SINDy) algorithm to develop interpretable ROMs for viscoelastic flows. In particular, we explore a benchmark oscillatory viscoelastic flow on the four-roll mill geometry using the classical Oldroyd-B fluid. This flow exemplifies many canonical challenges associated with non-Newtonian flows, including transitions, asymmetries, instabilities, and bifurcations arising from the interplay of viscous and elastic forces, all of which require expensive computations in order to resolve the fast timescales and long transients characteristic of such flows. First, we demonstrate the effectiveness of our data-driven surrogate model to predict the transient evolution and accurately reconstruct the spatial flow field for fixed flow parameters. We then develop a fully parametric, nonlinear model capable of capturing the dynamic variations as a function of the Weissenberg number. While the training data are predominantly concentrated on a limit cycle regime for moderate , we show that the parametrized model can be used to extrapolate, accurately predicting the dominant dynamics in the case of high Weissenberg numbers. The proposed methodology represents an initial step in applying machine learning and reduced-order modelling techniques to viscoelastic flows.

Prediction of typical gas components in cigarette smoke based on transformer

Abstract Harm reduction of cigarette products has become one of the primary objectives of the entire tobacco industry. The prediction of typical gas components (TGC) and dynamic adjustment of process parameters are crucial for cigarette coke reduction and harm reduction. In this study, a deep learning (DL)-assisted TGC prediction framework is proposed to predict CO and NI simultaneously. Firstly, a large number of experimental and simulation parameters of cigarette process are collected as the data base for subsequent model construction. Then, the feature importance analysis of the process parameters was carried out by combining the mechanism of the combustion process through the mutual information method. Finally, DL models based on Multilayer Perceptron, One-Dimension Convolutional Neural Network and Transformer (TF) was developed as data-driven surrogate models to establish the mapping relationship between process parameters and TGC. The results show that TF model generalizes best and can predict TGC quickly and accurately with R2 over 0.99. This work will provide a valuable predictive and decision-making tool for cigarette harm reduction.

Determination of unsteady wing loading using tuft visualization

Unsteady separated flow affects the aerodynamic performance of many large-scale objects, posing challenges for accurate assessment through low-fidelity simulations. Full-scale wind tunnel testing is often impractical due to the object’s physical scale. Small-scale wind tunnel tests can approximate the aerodynamic loading, with tufts providing qualitative validation of surface flow patterns. This investigation demonstrates that tufts can quantitatively estimate unsteady integral aerodynamic lift and pitching moment loading on a wing. We present computational and experimental data for a NACA0012 wing, capturing unsteady surface flow and force coefficients beyond stall. Computational data for varying angles of attack and Reynolds numbers contain the lift coefficient and surface flow. Experimental data, including lift and moment coefficients for a tuft-equipped NACA0012 wing, were obtained at multiple angles of attack and constant Reynolds number. Our results show that a data-driven surrogate model can predict lift and pitching moment fluctuations from visual tuft observations.

Enhancing subsurface multiphase flow simulation with Fourier neural operator

Accurate numerical modeling of multiphase flow in subsurface oil and gas reservoirs is critical for optimizing hydrocarbon recovery. However, traditional physics-based algorithms face substantial computational hurdles due to the need for fine grid resolution and the inherent geological heterogeneity. To overcome these challenges, data-driven surrogate models solving the flow governing partial differential equations (PDEs) offer a promising alternative to enhance the efficiency of hydrocarbon production operations. In this study, we employ the Fourier Neural Operator (FNO) to extract spectral information from the reservoir properties, thereby facilitating the solution of coupled porous flow PDEs. Our focus is on two-phase flow dynamics, specifically exploring how water injection enhances reservoir pressure and displaces oil. This scenario involves solving a set of nonlinearly coupled PDEs with highly heterogeneous coefficients. Numerical results demonstrate that the developed FNO accurately predicts the reservoir pressure distributions. We further observe that the FNO's zero-shot super-resolution capability is sensitive to abrupt local changes in the reservoir pressure near injection and production wells. To enhance its accuracy, we propose a multi-fidelity FNO model that exhibits better adaptability across various grid configurations. After moderate training on graphics processing units (GPUs), the FNO achieves a speedup of three orders of magnitude compared to traditional numerical PDE solvers. Our experiments confirm the FNO's potential to replace repetitive physics-based simulations, significantly advancing computational efficiency in the uncertainty quantification of reservoir performance.

Output power prediction of stratospheric airship solar array based on surrogate model under global wind field

Stratospheric airships are lighter-than-air vehicles capable of continuous flying for months. The energy balance of the airship is the key to long-duration flights. The stratospheric airship is entirely powered by the solar array. It is necessary to accurately predict the output power of the array for any flight state. Because of the uneven solar radiation received by the solar array, the traditional model based on components has a slow simulation speed. In this study, a data-driven surrogate modeling approach for prediction the output power of the solar array is proposed. The surrogate model is trained using the samples obtained from the high-accuracy simulation model. By using the input parameter preprocessor, the accuracy of the surrogate model in predicting the output power of the solar array is improved to 98.65%. In addition, the predictive speed of the surrogate model is ten million times faster than the traditional simulation model. Finally, the surrogate model is used to predict the energy balance of stratospheric airships flying throughout the year under actual global wind fields.

Physics-constrained polynomial chaos expansion for scientific machine learning and uncertainty quantification

We present a novel physics-constrained polynomial chaos expansion as a surrogate modeling method capable of performing both scientific machine learning (SciML) and uncertainty quantification (UQ) tasks. The proposed method possesses a unique capability: it seamlessly integrates SciML into UQ and vice versa, which allows it to quantify the uncertainties in SciML tasks effectively and leverage SciML for improved uncertainty assessment during UQ-related tasks. The proposed surrogate model can effectively incorporate a variety of physical constraints, such as governing partial differential equations (PDEs) with associated initial and boundary conditions constraints, inequality-type constraints (e.g., monotonicity, convexity, non-negativity, among others), and additional a priori information in the training process to supplement limited data. This ensures physically realistic predictions and significantly reduces the need for expensive computational model evaluations to train the surrogate model. Furthermore, the proposed method has a built-in uncertainty quantification (UQ) feature to efficiently estimate output uncertainties. To demonstrate the effectiveness of the proposed method, we apply it to a diverse set of problems, including linear/non-linear PDEs with deterministic and stochastic parameters, data-driven surrogate modeling of a complex physical system, and UQ of a stochastic system with parameters modeled as random fields.

Optimization of dual-layer flow field in a water electrolyzer using a data-driven surrogate model

Serious bubble clogging in flow-field channels will hinder the water supply to the electrode of proton exchange membrane water electrolyzer (PEMWE), deteriorating the cell performance. In order to address this issue, the dual-layer flow field design has been proposed in our previous study. In this study, the VOF (volume of fluid) method is utilized to investigate the effects of different degassing layer and base heights on the bubble behavior in channel and determine the time for the bubbles to detach from the electrode surface. However, it is very time-consuming to get the optimal combination of base layer and degassing layer heights due to the large number of potential cases, which needs to be calculated through computation-intensive physical model. Therefore, machine learning methods are adopted to accelerate the optimization. A data-driven surrogate model based on deep neural network (DNN) is developed and successfully trained using data obtained by the physical VOF method. Based on the highly efficient surrogate, genetic algorithm (GA) is further utilized to determine the optimal heights of base layer and degassing layer. Finally, the reliability of the optimization was validated by bubble visualization in channel and electrochemical characterization in PEMWE through experiments.

End-to-end reinforcement learning of Koopman models for economic nonlinear model predictive control

(Economic) nonlinear model predictive control ((e)NMPC) requires dynamic models that are sufficiently accurate and computationally tractable. Data-driven surrogate models for mechanistic models can reduce the computational burden of (e)NMPC; however, such models are typically trained by system identification for maximum prediction accuracy on simulation samples and perform suboptimally in (e)NMPC. We present a method for end-to-end reinforcement learning of Koopman surrogate models for optimal performance as part of (e)NMPC. We apply our method to two applications derived from an established nonlinear continuous stirred-tank reactor model. The controller performance is compared to that of (e)NMPCs utilizing models trained using system identification, and model-free neural network controllers trained using reinforcement learning. We show that the end-to-end trained models outperform those trained using system identification in (e)NMPC, and that, in contrast to the neural network controllers, the (e)NMPC controllers can react to changes in the control setting without retraining.

Artificial Neural Networks as Efficient Models of Proton Exchange Membrane Fuel Cells

Abstract Utilization of machine learning methodologies, particularly artificial neural networks (ANNs), presents a good approach to accurately model physical systems. Such predictive simulations offer the ability to predict system performance across diverse operational conditions without the need of use mathematical descriptions. This approach contrast with traditional, time-consuming and unstable approaches reliant on partial differential equations. In this study, ANN methodology is used to obtain the characteristics of proton exchange membrane fuel cells (PEMFCs). PEMFC are low temperature devices which convert chemical energy into electricity. This devices are promising applications in the automotive sector. Utilizing data gained from computational fluid dynamics simulations of PEMFCs, was explored various data collection techniques and network architectures to check their impact on predictive fidelity. Conclusions show that the ANN-based framework enables rapid prediction of current-voltage characteristics, achieving accuracy levels surpassing 90%. Practical of machine learning model implications were discussed, accenting its utility in optimizing PEMFC operational parameters and its potential integration within digital twin frameworks as a data-driven surrogate model. This study underscores the efficiency of machine learning techniques in advancing the comprehension and optimization of complex physical systems such as PEM-FCs, thereby paving the way for their use in engineering and energy sectors.

Data-driven surrogate modeling and optimization of supercritical jet into supersonic crossflow

For the design and optimization of advanced aero-engines, the prohibitively computational resources required for numerical simulations pose a significant challenge, due to the extensive exploration of design parameters across a vast design space. Surrogate modeling techniques offer a viable alternative for efficiently emulating numerical results within a notably compressed timeframe. This study introduces parametric Reduced-Order Models (ROMs) based on Convolutional AutoEncoders (CAE), Fully Connected AutoEncoders (FCAE), and Proper Orthogonal Decomposition (POD) to fast emulate spatial distributions of physical variables for a supercritical jet into a supersonic crossflow under different operating conditions. To further accelerate the decision-making process, an optimization model is developed to enhance fuel-oxidizer mixing efficiency while minimizing total pressure loss. Results indicate that CAE-based ROMs exhibit superior prediction accuracy while FCAE-based ROMs show inferior predictive accuracy but minimal uncertainty. The latter may be ascribed to the markedly greater number of hyperparameters. POD-based ROMs underperform in regions of strong nonlinear flow dynamics, coupled with higher overall prediction uncertainties. Both AE- and POD-based ROMs achieve online predictions approximately 9 orders of magnitude faster than conventional simulations. The established optimization model enables the attainment of Pareto-optimal frontiers for spatial mixing deficiencies and total pressure recovery coefficient.

An offline data-driven dual-surrogate framework considering prediction error for history matching

High computer power has long been a critical ingredient that affects the effectiveness and efficiency of history matching. Data-driven surrogate modeling as an efficient strategy can accelerate the history-matching process by constructing machine learning-based models with high computing speed but reduced accuracy. However, the applicability of surrogate models for different history-matching problems is uncertain due to the influence of data quality and quantity, model architectures, and hyperparameters. To overcome this issue, an offline data-driven dual-surrogate framework (ODDF) that considers the prediction error of surrogate models for history matching is proposed, where one surrogate model predicts the production data of reservoirs and the other one learns the prediction error of the former surrogate. The first surrogate model considers the time-series characteristics of production data using a recurrent neural network, while the second surrogate model regards the two-dimensional spatial correlation characteristics of multivariate prediction error using a fully convolutional neural network. Furthermore, an enhanced error model is applied to incorporate the prediction error into the objective function to reduce the influence of the prediction error on inversion results. Based on this hybrid framework, one can improve the prediction accuracy of surrogate models in history matching when the architectures or hyperparameters of surrogate models are not optimal. Additionally, one can obtain satisfactory results for history matching and uncertainty quantification based on surrogate modeling. The proposed framework is validated on the history matching of two- and three-dimensional reservoir models. The results show that the proposed method is robust in constructing the surrogate models and predicting the production data of reservoirs, which improves the efficiency and reliability of history matching.

Graph neural network-based surrogate modelling for real-time hydraulic prediction of urban drainage networks

Physics-based models are computationally time-consuming and infeasible for real-time scenarios of urban drainage networks, and a surrogate model is needed to accelerate the online predictive modelling. Fully-connected neural networks (NNs) are potential surrogate models, but may suffer from low interpretability and efficiency in fitting complex targets. Owing to the state-of-the-art modelling power of graph neural networks (GNNs) and their match with urban drainage networks in the graph structure, this work proposes a GNN-based surrogate of the flow routing model for the hydraulic prediction problem of drainage networks, which regards recent hydraulic states as initial conditions, and future runoff and control policy as boundary conditions. To incorporate hydraulic constraints and physical relationships into drainage modelling, physics-guided mechanisms are designed on top of the surrogate model to restrict the prediction variables with flow balance and flooding occurrence constraints. According to case results in a stormwater network, the GNN-based model is more cost-effective with better hydraulic prediction accuracy than the NN-based model after equal training epochs, and the designed mechanisms further limit prediction errors with interpretable domain knowledge. As the model structure adheres to the flow routing mechanisms and hydraulic constraints in urban drainage networks, it provides an interpretable and effective solution for data-driven surrogate modelling. Simultaneously, the surrogate model accelerates the predictive modelling of urban drainage networks for real-time use compared with the physics-based model.

Prediction of Frequency-Dependent Optical Spectrum for Solid Materials: A Multioutput and Multifidelity Machine Learning Approach.

The frequency-dependent optical spectrum is pivotal for a broad range of applications from material characterization to optoelectronics and energy harvesting. Data-driven surrogate models, trained on density functional theory (DFT) data, have effectively alleviated the scalability limitations of DFT while preserving its chemical accuracy, expediting material discovery. However, prevailing machine learning (ML) efforts often focus on scalar properties such as the band gap, overlooking the complexities of optical spectra. In this work, we employ deep graph neural networks (GNNs) to predict the frequency-dependent complex-valued dielectric function across the infrared, visible, and ultraviolet spectra directly from the crystal structures. We explore multiple architectures for the spectral multioutput representation of the dielectric function and utilize various multifidelity learning strategies, such as transfer learning and fidelity embedding, to address the challenges associated with the scarcity of high-fidelity DFT data. Additionally, we model key solar cell absorption efficiency metrics, demonstrating that learning these parameters is enhanced when integrated through a learning bias within the learning of the frequency-dependent absorption coefficient. This study demonstrates that leveraging multioutput and multifidelity ML techniques enables accurate predictions of optical spectra from crystal structures, providing a versatile tool for rapidly screening materials for optoelectronics, optical sensing, and solar energy applications across an extensive frequency spectrum.

Surrogate-model-based dwell time optimization for atmospheric pressure plasma jet finishing

With the rapidly increasing requirements for high-precision optical elements in modern optical systems, atmospheric pressure plasma jet (APPJ) finishing has attracted wide attention due to its high material removal efficiency and superior surface quality. However, the temperature fluctuation in the localized processing footprint, induced by the heated plasma and varying dwell time, emerges as a critical factor that oscillates the etching rate and thus degrades the deterministic removal control. Existing dwell time solution algorithms only apply under the assumption that the material removal rate exhibits temporal and spatial constancy, which is not the case in APPJ finishing. To address this challenge, a surrogate-model-based dwell time optimization approach for the temperature-dependent tool influence function (TIF) is proposed in this study. Firstly, a temperature-dependent TIF is proposed to quantitatively describe the thermal effect, and the corresponding calibration method is established. Subsequently, a thermodynamic simulation model is established to simulate the temperature of the localized processing footprint. By integrating the temperature-dependent TIF with the temperature prediction model, a novel dwell time optimization algorithm is proposed to alleviate the error caused by the temperature fluctuation. Given the prohibitive long computation time of the thermodynamic simulation model, a data-driven surrogate model based on Long Short Term Memory (LSTM), capable of rapid temperature prediction, is trained through the customized simulation dataset. The effectiveness of the proposed surrogate-model-based dwell time optimization algorithm is verified through simulations and experiments. Compared to traditional dwell time solution algorithms, the proposed approach yields a 47.12 % improvement in machining precision.

A dual-porosity flow-net model for simulating water-flooding in low-permeability fractured reservoirs

The physics-based data-driven flow-network models with high computational efficiency have received great attention as the promising surrogate models for reservoir numerical simulation. However, the existing flow-network models developed for water-flooding reservoirs fail to consider different seepage characteristics of matrix and fractures and cannot be straightly applied to simulate the water-flooding process in low-permeability fractured reservoirs. In this study, we combine the flow-network model with the dual-porosity model to propose a new physics-based data-driven surrogate model, namely the Dual-Porosity Flow-Network Model (Flow-Net-DP), which can consider the non-Darcy flow in tight matrix and the stress-sensitive effects and preferential flow characteristic in fractures. Specifically, we refer to the dual-porosity model and use two channels to represent the connections between wells: one indicates the fracture system, the other represents the matrix system, and the fluid exchange between these two systems is considered by using a transfer function. Besides, each channel is discretized into one-dimensional grids, and a fully implicit scheme with Newton iteration is used to calculate pressure and saturation. Moreover, we establish an automated history matching method by using the Ensemble Smoother with Multiple Data Assimilation (ESMDA) algorithm to calibrate model parameters of Flow-Net-DP, and develop a production optimization method by using the Differential Evolution (DE) algorithm. Finally, the numerical simulation, history matching, and production optimization are conducted on different numerical examples to validate the capability of Flow-Net-DP. The results indicate that the Flow-Net-DP can provide a better description of water flooding process in low-permeability fractured reservoirs compared with the existing flow-network model, especially the rapid water breakthrough caused by the preferential flow characteristic in fractures. Furthermore, both history matching and production optimization based on the Flow-Net-DP yield satisfactory outcomes. For instance, when the optimization results are similar to the full-order simulation model, the optimization speed of Flow-Net-DP is increased by more than five times.

Enhancing CFD solver with Machine Learning techniques

This study addresses the computational challenges in fluid flow simulations arising from demanding computational grids, required to capture the temporal and length scales involved. Our approach focuses on the pressure solver, as this is a resource-intensive component in Computational Fluid Dynamics (CFD) solvers. We achieve this by integrating a Machine Learning (ML) surrogate model with an incompressible fluid flow solver. We created two variants of an ML-enhanced CFD solver which were able to reduce the number of iterations required by the CFD pressure solver during unsteady flow simulations. Consequently, the simulations yielded comparable drag coefficients and Strouhal numbers, accompanied by an eightfold decrease in execution time. The performance enhancements are attributed to reduced computational effort per temporal iteration and early-stage forcing on the simulation dynamical behavior when using the ML-based surrogate models. This research introduces an approach to enhance the computational efficiency of fluid flow analyses by incorporating surrogate models to aid the pressure solver in CFD simulations. We propose a Hybrid CFD solver, ie. a physics-informed solver enhanced with data-driven surrogate models.