Surrogate Model Research Articles

Probabilistic surrogate modeling of damage equivalent loads on onshore and offshore wind turbines using mixture density networks

Abstract. The use of load surrogates in offshore wind turbine site assessment has gained attention as a way to speed up the lengthy and costly siting process. We propose a novel probabilistic approach using mixture density networks to map 10 min average site conditions to the corresponding load statistics. The probabilistic framework allows for the modeling of the uncertainty in the loads as a response to the stochastic inflow conditions. We train the data-driven model on the OpenFAST simulations of the IEA 10 MW reference wind turbine (IEA-10MW-RWT) and compare the predictions to the widely used Gaussian process regression. We show that mixture density networks can recover the accurate mean response in all load channels with values for the coefficient of determination (R2) greater than 0.95 on the test dataset. Mixture density networks completely outperform Gaussian process regression in predicting the quantiles, showing an excellent agreement with the reference. We compare onshore and offshore sites for training to conclude the need for a more extensive training dataset in offshore cases due to the larger feature space and more noise in the data.

Heat transfer performance optimization of turbine blade internal cooling channel with pin-fins and recessed-extruded endwalls

This paper involves implementing and optimizing a novel recessed-extruded endwall design for a rectangular cooling channel equipped with pin-fins. The study encompasses an in-depth exploration of the flow dynamics, pondering the generation and sustenance of vortices within the channel, using Reynolds-averaged Navier-Stokes analysis. The study delves into the evaluation of heat transfer characteristics, considering crucial parameters such as Nusselt number, friction factor, and overall heat transfer performance. These attributes are contrasted with the corresponding values observed in cases featuring flat endwalls, concerning Reynolds numbers spanning from 7400 to 36,000. A parametric study of the channel with recessed-extruded endwalls is conducted, aiming to comprehend the influence of geometric factors on channel heat transfer. A single-objective optimization process focusing on enhancing heat transfer effectiveness, quantified through the heat transfer efficiency index (HTEI), is conducted with three design variables. The results indicate that the new endwall configuration significantly enlarges the high heat transfer areas near the pin-fins compared to the flat endwall across all Reynolds numbers. This novel endwall design substantially elevates the heat transfer levels adjacent to the pin-fins, showcasing an impressive 62.4% increase in HTEI at Re = 21,500 compared to the flat endwall, with a protrusion height of 2.5% of the channel height. Furthermore, by increasing the height, the recessed-extruded endwall configuration can achieve a remarkable 91.1% increase in HTEI compared to the flat endwall. This achievement is reached with a protrusion height equivalent to 5% of the channel height at Re = 7400. Through design optimization using the radial basis neural network (RBNN) surrogate model and particle swarm optimization (PSO) technique, the HTEI of the recessed-extruded endwall design shows enhancements of 91.7% and 18.0% compared with the flat endwall and reference designs, respectively, at a Reynolds number of 21,500. The present work demonstrates the potential for enhancing the heat transfer of pin-fins channels by improving endwall design.

Machine learning-augmented modeling on the formation of Si-dominated Non-β″ early-stage precipitates in Al-Si-Mg alloys with Si supersaturation induced by non-equilibrium solidification

Recent experiments have shown that Al-Si-Mg alloys solidified under high cooling rates may lead to the nucleation of Si-enriched clusters that are remarkably different from the conventional Mg-Si co-clusters (e.g. β″ particles), and yet the responsible mechanism remains to be elucidated. Here we tackle the problem using a multiscale modeling framework that integrates atomistic modeling, energy landscape sampling, and lattice-based kinetic Monte Carlo (kMC) simulation. The migration energy barriers for vacancy-mediated diffusion amid complex local chemical environments are predicted on-the-fly using a surrogate machine learning model. We discover that the actual vacancy-Si migration barriers are much lower than those assumed in the classic linear interpolation approximation. Such a strong deviation from conventional wisdom, in conjunction with differing Si solute composition, can lead to a great variety in the nucleated early-stage precipitates. More specifically, a high-level supersaturation of Si solute (i.e.xSi/(xSi+xMg)>0.75) would lead to an unexpectedly high enrichment of Si in the nucleated clusters with the Si:Mg ratio up to 5∼6; while at a lower-level supply of Si solute the Mg-Si co-clusters (i.e. Si:Mg ratio around 1∼2) are nucleated instead. These findings provide a viable explanation for the diverse types of early-stage precipitates observed in various experiments, from Si-enriched precipitates in high-pressure die cast Al alloys to β″ particles in conventional casting and/or heat-treated alloys. The implications of our findings are also discussed.

A new active learning surrogate model for time- and space-dependent system reliability analysis

This study introduces a novel method for time- and space-dependent system reliability analysis, integrating an active learning surrogate model with an innovative parallel updating strategy. A global Kriging model is developed to represent the signs of random samples using efficient global optimization. From a Bayesian perspective, the prediction probabilities of random sample signs within the time-space domain are calculated, and the sample with the lowest prediction probability is chosen to update the global Kriging model. The system extremum for each sample in the time-space domain is determined, and the corresponding random variables, time-space coordinates, and failure modes are selected. To further decrease iteration times, a parallel updating strategy that considers both the predicted probability and the correlation among candidate samples is proposed. Additionally, a new stopping criterion is introduced to balance accuracy and efficiency, terminating the updating process appropriately. The method's accuracy and efficiency are validated through three examples.

A novel interpretable semi-supervised graph learning model for intelligent fault diagnosis of hydraulic pumps

Although deep learning has gained popularity in the field of fault diagnosis, its limitations are also equally apparent, including: (1) heavy reliance on a substantial volume of labeled samples; (2) a lack of interpretability. To confront these issues, this article proposes a novel interpretable semi-supervised graph learning model for intelligent fault diagnosis of hydraulic pumps. A comparison between the raw data and the model's hidden layer representations is conducted to minimize feature loss. The model commences by preliminarily learning fault information that is intermixed with noise, leveraging a substantial corpus of unlabeled data. In response to the intricacy of downstream tasks, an interpretable feature reconstruction module is introduced. This module employs a nonlinear surrogate model to fit and elucidate the learned features, embedding the explanation scores into the features to reconstruct the samples, a process utilized for model fine-tuning. The feature reconstruction module capitalizes on the explanatory power of the surrogate model, guiding the model to concentrate more on features with significant impact. This method not only provides interpretability during model training but also expedites the convergence speed of the model. Finally, two hydraulic pump experiment cases are used to verify the effectiveness of the model, and the results show that our method has obvious advantages in reducing label dependence and increasing model reliability for decision making.

Probability density estimation of polynomial chaos and its application in structural reliability analysis

Polynomial chaos expansion (PCE) is a widely used approach for establishing the surrogate model of a time-consuming performance function for the convenience of uncertainty quantification of a stochastic structure. However, it remains difficult to calculate the probability density function (PDF) of the PCE accurately for general cases, though the PDF, as a complete representation of a random variable, is often required in some uncertainty problems. To address this problem, this paper proposes a semi-analytical method to compute the PDF of a PCE. This method derives the closed-form solutions of characteristic functions (CFs) of the first- and second-order PCEs, while an equivalent parabolization technique is proposed to provide the approximate solutions of CFs of higher-order PCEs. Then, the PDF of the PCE can be obtained by the Fourier transform of the resulting CF. Three numerical examples are investigated to demonstrate the accuracy, applicability, and efficiency of the proposed method for probability density estimation of PCE in structural reliability analysis.

Industrial-scale 12-layered-bed vacuum pressure swing adsorption for fuel cell-grade H2 production from carbon-captured steam methane reforming syngas

Vacuum pressure swing adsorption (VPSA) is primarily used to produce high-purity H2 from CO2-rich syngas in steam methane reforming (SMR) plants. Since carbon capture is essential, the performance of current H2 VPSA processes is affected by feeding carbon-captured SMR syngas (CO2-lean SMR syngas). This study conducted the performance evaluation and optimization for a fuel cell-grade H2 production by an industrial-scale 12-layered-bed VPSA process with a 24-step cycle from CO2-lean SMR syngas. The multi-scale dynamic model for the VPSA process was validated with petrochemical industry data from a current SMR-VPSA plant. A sensitivity analysis using CO2-lean SMR syngas indicated the high loss of H2 during desorption steps, suggesting the need for optimization of the VPSA process. Owing to the complex interconnectivity of 12 beds via 24 steps, the framework of a dynamic model-based deep neural network (DNN) and DNN-based optimization was developed to design and optimize the VPSA process. Even without changing equipment or adsorbent configuration, the VPSA process achieved fuel cell-grade H2 production (>99.9999 % H2 and > 88 % recovery with < 0.2 ppm CO). The results indicate that CO2 capture rates, ranging from 90 to 99 %, can be independently determined under techno-economic and environmental conditions, as this capture range of CO2-lean SMR syngas gives a minor impact on the performance of VPSA process. The proposed three-step framework, which encompasses dynamic modeling, surrogate modeling, and multi-objective optimization, provides a systematic and feasible approach for designing and optimizing the VPSA process in an SMR plant integrated with a carbon capture process, to achieve a low-carbon economy.

Hydrodynamic optimization and performance verification of fairings for round-ended cofferdams using dam-break wave experiments and numerical simulations

Round-ended cofferdams are crucial for large-scale marine projects but are vulnerable to the complex marine environment, including waves, tides, and storms. Enhancing the hydrodynamic performance of these cofferdams is essential for improving safety and efficiency in marine construction. This study introduces a novel fairing design aimed at reducing water flow resistance during the quasi-stable stage of a tsunami. The flow field is analyzed using computational fluid dynamics (CFD) simulations, and an adaptive surrogate model is employed to optimize the parameterized fairing shape. The results demonstrate that the optimized shape significantly suppresses vortex shedding, leading to a 16.59 % reduction in the drag coefficient and a 44.3 % reduction in the lift coefficient under flow conditions. Experimental and numerical simulations of dam-break waves are conducted for two designs, R1 and R0. By comparing their flow field and force characteristics, it is found that R1 effectively separates waves during the impulse stage, reduces wave climb height, and decreases impact loads. In the quasi-stable stage, R1 mitigates the blockage effect, reducing the liquid level difference between the front and back, and thus lowering flow forces. Experimental data further reveals that when the downstream is a dry riverbed, R1′s load reduction is particularly notable, with maximum reductions of 45.62 % in the impulse stage and 28.75 % in the quasi-stable stage. When the riverbed is wet, the maximum load reduction rates are 18.04 % and 8.72 %, respectively. Therefore, R1 not only reduces the resistance of round-end cofferdams under water currents but also under extreme wave forces, providing valuable insights for advancing ocean engineering design.

Synthesizing source directivity using DeepONet room acoustic predictions

Incorporating source directivity when simulating spatial room impulse responses for virtual environments is crucial for achieving natural results. Directly injecting the directional source as the initial condition for a wave equation to solve would require re-running a numerical solver for each new directivity pattern, source position or orientation. Here we show that learned surrogate models can lead to large efficiency benefits if such variation shall be simulated. Although training such models is a computationally expensive process, performing inference over them is typically fast. Here we show how to exploit that property by training a deep neural operator network (DeepONet) only using omni-directional sources, and predicting the pressure on a variable grid of points, to which directivity filters are applied to generate arbitrary patterns. We show the fit of directictivity patterns to their re-synthesized versions from the output of a DeepONet.

Using surrogate models to accelerate load step methods for nonlinear finite element problems in hyperelasticity

Using surrogate models to accelerate load step methods for nonlinear finite element problems in hyperelasticity

Optimization of pressure management strategies for geological CO2 storage using surrogate model-based reinforcement learning

Injecting greenhouse gas (e.g. CO2) into deep underground reservoirs for permanent storage can inadvertently lead to fault reactivation, caprock fracturing and greenhouse gas leakage when the injection-induced stress exceeds the critical threshold. It is essential to monitor the evolution of pressure and the movement of the CO2 plume closely during the injection to allow for timely remediation actions or rapid adjustments to the storage design. Extraction of pre-existing fluids at various stages of the injection process, referred as pressure management, can mitigate associated risks and lessen environmental impact. However, identifying optimal pressure management strategies typically requires thousands of simulations, making the process computationally prohibitive. This paper introduces a novel surrogate model-based reinforcement learning method for devising optimal pressure management strategies for geological CO2 sequestration efficiently. Our approach comprises of two steps. The first step involves developing a surrogate model using the embed to control method, which employs an encoder-transition-decoder structure to learn dynamics in a latent or reduced space. The second step, leveraging this proxy model, utilizes reinforcement learning to find an optimal strategy that maximizes economic benefits while satisfying various control constraints. The reinforcement learning agent receives the latent state representation and immediate reward tailored for CO2 sequestration and choose real-time controls which are subject to predefined engineering constraints in order to maximize the long-term cumulative rewards. To demonstrate its effectiveness, this framework is applied to a compositional simulation model where CO2 is injected into saline aquifer. The results reveal that our surrogate model-based reinforcement learning approach significantly optimizes CO2 sequestration strategies, leading to notable economic gains compared to baseline scenarios.

Investigating the oxidation characteristic of a hydro-processed bio-jet fuel: Experimental and modeling study

A rapid transition from conventional jet fuels to sustainable aviation fuels (SAFs) is imperative in order to reduce carbon emissions. Hydro-processed esters and fatty acids synthetic paraffinic kerosene (HEFA-SPK), as a type of SAF, exhibits broad applications. In this study, a new HEFA-SPK named ZH-HEFA was investigated. The fuel comprises 14% n-alkanes, 85% iso-alkanes and only 1% cycloalkanes by weight, with the majority of alkanes ranging from C9 to C17. Oxidation experiments of the fuel were conducted using an atmospheric pressure flow reactor at temperatures ranging from 550 K to 1075 K under three equivalence ratios (0.5, 1.0 and 1.5). Species mole fraction profiles were measured by an on-line gas chromatographic (GC). For comparison purposes, an experiment was also performed on RP-3, a conventional jet fuel commonly used in China, under the equivalence of 0.5. Compared to RP-3, ZH-HEFA exhibited significantly stronger low temperature reactivity and higher combustion conversion rates while demonstrating considerably lower yields of aromatics at high temperatures. The kinetic simulation of ZH-HEFA was achieved by proposing two surrogates and their corresponding kinetic models. Surrogate S-1 consisted solely of n-dodecane, while S-2 comprised 35% n-dodecane and 65% 2,6,10-trimethyl dodecane by weight. Both surrogate models were validated by the experimental data. S-1 exhibited a closer resemblance to the global oxidation characteristics of ZH-HEFA, whereas S-2 demonstrated improved accuracy in predicting the formation of small hydrocarbon intermediates during the fuel oxidation. Rate of production analysis revealed that the branched alkane component in S-2 possessed more pathways and greater capability than S-1 in generating C3 intermediates, which are important for the generation of aromatics. Furthermore, both models displayed good predictive performance for the auto-ignition properties of HEFA-SPK fuels.-

The ozone radiative forcing of nitrogen oxide emissions from aviation can be estimated using a probabilistic approach

Reliable prediction of aviation’s environmental impact, including the effect of nitrogen oxides on ozone, is vital for effective mitigation against its contribution to global warming. Estimating this climate impact however, in terms of the short-term ozone instantaneous radiative forcing, requires computationally-expensive chemistry-climate model simulations that limit practical applications such as climate-optimised planning. Existing surrogates neglect the large uncertainties in their predictions due to unknown environmental conditions and missing features. Relative to these surrogates, we propose a high-accuracy probabilistic surrogate that not only provides mean predictions but also quantifies heteroscedastic uncertainties in climate impact estimates. Our model is trained on one of the most comprehensive chemistry-climate model datasets for aviation-induced nitrogen oxide impacts on ozone. Leveraging feature selection techniques, we identify essential predictors that are readily available from weather forecasts to facilitate the implementation therein. We show that our surrogate model is more accurate than homoscedastic models and easily outperforms existing linear surrogates. We then predict the climate impact of a frequently-flown flight in the European Union, and discuss limitations of our approach.

Spectral expansion methods for prediction uncertainty quantification in systems biology

Uncertainty is ubiquitous in biological systems. For example, since gene expression is intrinsically governed by noise, nature shows a fascinating degree of variability. If we want to use a model to predict the behaviour of such an intrinsically stochastic system, we have to cope with the fact that the model parameters are never exactly known, but vary according to some distribution. A key question is then to determine how the uncertainties in the parameters affect the model outcome. Knowing the latter uncertainties is crucial when a model is used for, e.g., experimental design, optimisation, or decision-making. To establish how parameter and model prediction uncertainties are related, Monte Carlo approaches could be used. Then, the model is evaluated for a huge number of parameters sets, drawn from the multivariate parameter distribution. However, when model solutions are computationally expensive this approach is intractable. To overcome this problem, so-called spectral expansion (SE) methods have been developed to quantify prediction uncertainty within a probabilistic framework. Such SE methods have a basis in, e.g., computational mathematics, engineering, physics, and fluid dynamics, and, to a lesser extent, systems biology. The computational costs of SE schemes mainly stem from the calculation of the expansion coefficients. Furthermore, SE effectively leads to a surrogate model which captures the dependence of the model on the uncertainty parameters, but is much simpler to execute compared to the original model. In this paper, we present an innovative scheme for the calculation of the expansion coefficients. It guarantees that the model has to be evaluated only a restricted number of times. Especially for models of high complexity this may be a huge computational advantage. By applying the scheme to a variety of examples we show its power, especially in challenging situations where solutions slowly converge due to high computational costs, bifurcations, and discontinuities.

Fatigue reliability analysis of bogie frames considering parameter uncertainty

With the increasing service life of bogie frames, the risk of fatigue failure becomes significant, making fatigue reliability analysis crucial for ensuring operational safety. However, accurately analyzing fatigue reliability presents a significant challenge with uncertain factors such as load fluctuations, unstable material shaping, and dimensional manufacturing deviations. To address this, this paper establishes a comprehensive active learning reliability framework based on surrogate models, enabling high-fidelity modeling and precise fatigue reliability analysis of welded frames under parameter uncertainty. The material utilization method was developed using APDL for secondary development to efficiently evaluate frame fatigue failure indicators. The effectiveness of this method was validated by combining the improved Goodman-Smith fatigue limit diagram and test bench fatigue tests, which helped identify the locations on the frame most prone to fatigue fractures. An Atom Search Optimization-BP Neural Network surrogate model was established with the objective of maximum material utilization, and the fatigue reliability of the bogie frame was obtained by combining the active learning function and the Monte Carlo method. The results show that the uncertainty design parameters greatly impact the fatigue reliability of critical welded structures. The proposed method improves the accuracy and efficiency of the fatigue reliability analysis of the bogie frame.

The physical information LSTM surrogate model for establishing a digital twin model of reciprocating air compressors

Reciprocating air compressors play a crucial role in industrial production processes. However, due to the complex structure and long operating time of reciprocating air compressors, real-time monitoring to grasp the operating status of reciprocating air compressors has become particularly important. Digital twin is a technology that can reflect the behavior of physical entities in real time, accurately predicting and evaluating the operation status of reciprocating air compressors. However, the establishment of a digital twin model for reciprocating air compressors requires a significant amount of computational resources, which can result in the inability to meet the requirements of real-time performance evaluation. To overcome this limitation, this paper proposes a method for constructing a digital twin model of reciprocating air compressors based on a surrogate model. The surrogate model is constructed based on a long short-term memory neural network with physical information(PILSTM). This model can accurately describe the changes in cylinder pressure by combining physical information. According to the characteristics of cylinder pressure changes, regularization formulas are added to ensure the smoothness of the predicted pressure. The experimental results show that the digital twin model based on the surrogate model has high prediction accuracy and real-time performance. Therefore, this model provides a new method for monitoring the operating status of reciprocating air compressors.

Surrogate model for reservoir performance prediction with time-varying well control based on depth generative network

This paper proposes a novel intelligent method for defining and solving the reservoir performance prediction problem within a manifold space, fully considering geological uncertainty and the characteristics of reservoirs performance under time-varying well control conditions, creating a surrogate model for reservoir performance prediction based on Conditional Evolutionary Generative Adversarial Networks (CE-GAN). The CE-GAN leverages conditional evolution in the feature space to direct the evolution of the generative network in previously uncontrollable directions, and transforms the problem of reservoir performance prediction into an image evolution problem based on permeability distribution, initial reservoir performance and time-varying well control, thereby enabling fast and accurate reservoir performance prediction under time-varying well control conditions. The experimental results in basic (egg model) and actual water-flooding reservoirs show that the model predictions align well with numerical simulations. In the basic reservoir model validation, the median relative residuals for pressure and oil saturation are 0.5% and 9.0%, respectively. In the actual reservoir model validation, the median relative residuals for both pressure and oil saturation are 4.0%. Regarding time efficiency, the surrogate model after training achieves approximately 160-fold and 280-fold increases in computational speed for the basic and actual reservoir models, respectively, compared with traditional numerical simulations. The reservoir performance prediction surrogate model based on the CE-GAN can effectively enhance the efficiency of production optimization.

Advanced surrogate model for electron-scale turbulence in tokamak pedestals

We derive an advanced surrogate model for predicting turbulent transport at the edge of tokamaks driven by electron temperature gradient (ETG) modes. Our derivation is based on a recently developed sensitivity-driven sparse grid interpolation approach for uncertainty quantification and sensitivity analysis at scale, which informs the set of parameters that define the surrogate model as a scaling law. Our model reveals that ETG-driven electron heat flux is influenced by the safety factor $q$ , electron beta $\beta _e$ and normalized electron Debye length $\lambda _D$ , in addition to well-established parameters such as the electron temperature and density gradients. To assess the trustworthiness of our model's predictions beyond training, we compute prediction intervals using bootstrapping. The surrogate model's predictive power is tested across a wide range of parameter values, including within-distribution testing parameters (to verify our model) as well as out-of-bounds and out-of-distribution testing (to validate the proposed model). Overall, validation efforts show that our model competes well with, or can even outperform, existing scaling laws in predicting ETG-driven transport.

Advanced Sobol sensitivity analysis of a 1:4-scale prestressed concrete containment vessel using an ANN-based surrogate model

The safety of nuclear power plants can be enhanced by performing sensitivity analyses to identify and evaluate factors that significantly impact their structural integrity. In this study, an artificial neural network-based surrogate model was developed using an experimentally validated finite element model. A Sobol sensitivity analysis was performed using this model to quantitatively assess the impact of material uncertainty on the internal pressure–displacement behavior of containment buildings at various internal pressure levels. The results indicate that the compressive strength of concrete and the prestressing force in the hoop direction critically influence the behavior of containment buildings, with their importance varying according to the internal pressure level. Furthermore, the proposed surrogate model can efficiently substitute for expensive and complex finite element analyses, enabling effective exploration of the multidimensional uncertainty space and quantitative assessment of the major factors affecting structural behavior. The process presented herein facilitates the quantification of factors under various uncertainty scenarios.

A DSMC-CFD coupling method using surrogate modelling for low-speed rarefied gas flows

A new Micro-Macro-Surrogate (MMS) hybrid method is presented that couples the Direct Simulation Monte Carlo (DSMC) method with Computational Fluid Dynamics (CFD) to simulate low-speed rarefied gas flows. The proposed MMS method incorporates surrogate modelling instead of direct coupling of DSMC data with the CFD, addressing the limitations CFD has in accurately modelling rarefied gas flows, the computational cost of DSMC for low-speed and multiscale flows, as well as the pitfalls of noise in conventional direct coupling approaches. The surrogate models, trained on the DSMC data using Bayesian inference, provide noise-free and accurate corrections to the CFD simulation enabling it to capture the non-continuum physics. The MMS hybrid approach is validated by simulating low-speed, steady-state, force-driven rarefied gas flows in a canonical 1D parallel-plate system, where corrections to the boundary conditions and stress tensor are considered and shows excellent agreement with DSMC benchmark results. A comparison with the typical domain decomposition DSMC-CFD hybrid method is also presented, to demonstrate the advantages of noise-avoidance in the proposed approach. The method also inherently captures the uncertainty arising from micro-model fluctuations, allowing for the quantification of noise-related uncertainty in the predictions. The proposed MMS method demonstrates the potential to enable multiscale simulations where CFD is inaccurate and DSMC is prohibitively expensive.