Regression Surrogate Model Research Articles

System fragility analysis of highway bridge using multi-output Gaussian process regression surrogate model

This study presents a multi-output Gaussian process regression (GPR) surrogate model for seismic-fragility analysis of bridge structural systems. Multi-output GPR can model the correlations among multiple outputs, accuracy and stability are achieved with fewer training data, which reduces the computational cost of fragility analysis. Furthermore, the explainability of the constructed surrogate model is implemented by adopting an automatic relevance-determination (ARD) kernel in the GPR. The estimated hyperparameters can provide the contribution of the uncertainty of each input parameter to the outputs. The fragility analysis using the multi-output GPR surrogate model was verified by applying it to a seismic isolation highway bridge with multiple spans and a curved geometry. The effectiveness of the multi-output GPR was demonstrated by the construction of an accurate and stable surrogate model with 46 inputs and 28 outputs. The relative contributions of the uncertainties to the structural properties and input earthquake loads could also be understood. The fragility curves, at both the component and system levels, were appropriately obtained using a sufficient number of samples in a Monte Carlo calculation. Furthermore, the failure modes were evaluated, identifying which structural components contributed to the system failure. This enabled discussions on structural system failures from the viewpoint of the structural dynamic characteristics of the bridge and earthquake-load properties.

On Using the Distributor as a Multi Degree-of-Freedom System to Mitigate the Pressure Pulsation in an Axial Turbine at Speed-No-Load

Abstract Hydraulic axial turbines are more frequently utilized for grid regulation purposes. Sometimes, they must be operated at speed-no-load (SNL) conditions, which is characterized for some machines by a varying number of large vortical flow structures extending from the vaneless space to the draft tube, introducing detrimental pressure pulsations throughout the turbine. A recent study shows that the vortices can be mitigated by individually controlling the guide vanes. Since optimization of the distributor layout is linked with a large degree-of-freedom, machine learning is deployed to assist in finding an optimal setup cost-effectively. A reduced numerical computational-fluid-dynamics (CFD) model is built and used to generate input for Gaussian process regression surrogate models by performing 2000 steady-state simulations with varying distributor layouts. The surrogate models suggest that the optimal layout is to open seven out of 20 guide vanes in succession while keeping the remaining ones closed. However, this configuration induces large radial forces on the runner, and after implementing some modifications by trial and error, detailed time-dependent CFD simulations show that placing 4 + 3 opened guide vanes on opposite sides of the runner axis is better; it reduces the pressure peaks corresponding to a two- and three-vortex configuration, and the maximal pressure pulsations by as much as 88% in the vaneless space compared to regular SNL operation. Meanwhile, the radial force on the runner is reduced by more than 83%, and pressure pulsations on the runner blades by more than 55%, compared to the surrogate models' optimal layout prediction.

Explainable Graph Neural Networks: An Application to Open Statistics Knowledge Graphs for Estimating House Prices

In the rapidly evolving field of real estate economics, the prediction of house prices continues to be a complex challenge, intricately tied to a multitude of socio-economic factors. Traditional predictive models often overlook spatial interdependencies that significantly influence housing prices. The objective of this study is to leverage Graph Neural Networks (GNNs) on open statistics knowledge graphs to model these spatial dependencies and predict house prices across Scotland’s 2011 data zones. The methodology involves retrieving integrated statistical indicators from the official Scottish Open Government Data portal and applying three representative GNN algorithms: ChebNet, GCN, and GraphSAGE. These GNNs are compared against traditional models, including the tabular-based XGBoost and a simple Multi-Layer Perceptron (MLP), demonstrating superior prediction accuracy. Innovative contributions of this study include the use of GNNs to model spatial dependencies in real estate economics and the application of local and global explainability techniques to enhance transparency and trust in the predictions. The global feature importance is determined by a logistic regression surrogate model while the local, region-level understanding of the GNN predictions is achieved through the use of GNNExplainer. Explainability results are compared with those from a previous work that applied the XGBoost machine learning algorithm and the SHapley Additive exPlanations (SHAP) explainability framework on the same dataset. Interestingly, both the global surrogate model and the SHAP approach underscored the comparative illness factor, a health indicator, and the ratio of detached dwellings as the most crucial features in the global explainability. In the case of local explanations, while both methods showed similar results, the GNN approach provided a richer, more comprehensive understanding of the predictions for two specific data zones.

Enhancing PEM fuel cell dynamic performance: Co-optimization of cathode catalyst layer composition and operating conditions using a novel surrogate model

Optimizing the cathode catalyst layer (CCL) composition and operating conditions to enhance the dynamic performance of proton exchange membrane fuel cells garners significant attention. Although machine learning surrogate models are efficient for fuel cell analysis and optimization, the varied voltage dynamic response patterns (e.g., loading failure, voltage undershoot, and voltage hysteresis) challenge regression surrogate models designed for steady-state performance predictions. In response, this study introduces a joint framework combining classification and regression models for dynamic performance prediction. For training, a transient, two-phase, non-isothermal fuel cell model with integrated catalyst agglomerate is developed. The dynamic voltage deviation (σV) is proposed as an index to characterize the dynamic performance of the fuel cell. This joint surrogate model achieves correlation coefficients of 0.9976 and 0.9961 for predicting σV in training and test sets, respectively. Through this model, sensitivity analyses of the CCL composition and operating conditions are conducted to quantify their impact and interactions on the fuel cell's dynamic performance. Besides, the analysis reveals a trade-off between dynamic performance and steady-state output. To balance these, a multi-objective optimization is conducted. The results indicate that, compared to the base case, dynamic and steady-state performance improved by 44 % and 8 %, respectively.

Mitigating Measurement Inaccuracies in Digital Twins of Construction Machinery through Multi-Objective Optimization.

The advent of digital twins facilitates the generation of high-fidelity replicas of actual systems or assets, thereby enhancing the design's performance and feasibility. When developing digital twins, precise measurement data is essential to ensure alignment between the actual and digital models. However, inherent uncertainties in sensors and models lead to disparities between observed and predicted (simulated) behaviors. To mitigate these uncertainties, this study originally proposes a multi-objective optimization strategy utilizing a Gaussian process regression surrogate model, which integrates various uncertain parameters, such as load angle, bucket cylinder stroke, arm cylinder stroke, and boom cylinder stroke. This optimization employs a genetic algorithm to indicate the Pareto frontiers regarding the pressure exerted on the boom, arm, and bucket cylinders. Subsequently, TOPSIS is applied to ascertain the optimal candidate among the identified Pareto optima. The findings reveal a substantial congruence between the experimental and numerical outcomes of the devised virtual model, in conjunction with the TOPSIS-derived optimal parameter configuration.

Retraining surrogate models in increasingly restricted design spaces: a novel building energy model calibration method

Surrogate (i.e. meta) models can approximate building energy models (BEMs) accurately and quickly, hence they have been widely used in BEM calibration studies. Typically, the surrogate models are trained a single time over the entire unknown building parameter space with a design such as Latin hypercube sampling. In this article, a multiple polynomial regression surrogate model is, instead, retrained with increasingly restricted designs. In each training repetition, the bounds of the design narrow around the unknown building parameter values that minimize the error between the surrogate model’s predictions and the measured energy. This ‘cascading surrogate’ calibration method finds CVRMSE values that are much lower than those of a powerful black box optimizer in a case study with simulated ‘measured’ data. However, the method has similar performance to the black box optimizer in a case study with real hourly measured energy, probably since the BEM was not configured accurately enough.

Optimization of the film cooling hole for minimizing stress concentration factor based on surrogate model

Film cooling is a typical cooling technology in gas turbines that can lower heat load by spreading coolant on the blade's surface. However, it faces the problem of stress concentration. The present work focuses on optimizing the film cooling hole for minimizing the stress concentration factor based on the surrogate model. The stress concentration factor is the ratio of the maximum stress in a plate with a film hole subjected to frictionless support constraints in the sidewalls at a homogeneous temperature to the average sidewall stress. An optimization platform is built to implement the optimization, which contains a finite element parallel computation module, a support vector regression surrogate model, and a two-layer optimization module for adding sample points around the candidate optimal point. After 50 rounds of optimization process, the optimization results converge. The lowest stress concentration factor obtained is about 1.79, 71.5 % lower than that of the traditional cylinder film hole. Considering that the reduction of the stress concentration factor should not be accompanied by the reduction of the film cooling effectiveness, this work tests the film effectiveness of four candidate holes by means of pressure-sensitive paint experiments. Geo4 has the lowest stress concentration factor and the maximum film effectiveness when both factors are considered.

A Novel Inversion Method for Permeability Coefficients of Concrete Face Rockfill Dam Based on Sobol-IDBO-SVR Fusion Surrogate Model

The accurate and efficient inversion of permeability coefficients is significant for the scientific assessment of seepage safety in concrete face rockfill dams. In addressing the optimization challenge of permeability coefficients with few samples, multiple parameters, and strong nonlinearity, this paper proposes a novel intelligent inversion method based on the Sobol-IDBO-SVR fusion surrogate model. Firstly, the Sobol sequence sampling method is introduced to extract high-quality combined samples of permeability coefficients, and the equivalent continuum seepage model is utilized for the forward simulation to obtain the theoretical hydraulic heads at the seepage monitoring points. Subsequently, the support vector regression surrogate model is used to establish the complex mapping relationship between the permeability coefficients and hydraulic heads, and the convergence performance of the dung beetle optimization algorithm is effectively enhanced by fusing multiple strategies. On this basis, we successfully achieve the precise inversion of permeability coefficients driven by multi-intelligence technologies. The engineering application results show that the permeability coefficients determined based on the inversion of the Sobol-IDBO-SVR model can reasonably reflect the seepage characteristics of the concrete face rockfill dam. The maximum relative error between the measured and the inversion values of the hydraulic heads at each monitoring point is only 0.63%, indicating that the inversion accuracy meets the engineering requirements. The method proposed in this study may also provide a beneficial reference for similar parameter inversion problems in engineering projects such as bridges, embankments, and pumping stations.

Performance prediction and optimization of hydrogenation feed pump based on particle swarm optimization – least squares support vector regression surrogate model

Due to high power consumption and low energy efficiency of the hydrogenation feed multistage pump, conducting structural optimization design and reducing energy losses for this pump is necessary. In this study, an optimization method based on the particle swarm optimization (PSO) algorithm and least squares support vector regression (LSSVR) surrogate model is proposed. The head and efficiency are taken as the optimization objectives of the multistage pump, and the optimization design of the two impellers is carried out. Through the optimization of the surrogate model, the head and efficiency of the multistage pump are increased by 3.99% and 2.91%, respectively. It is found that the optimized impeller has more stable flow characteristics than original impeller. Further, the entropy production method is introduced to analyze the energy loss of pump. The result shows that the pump optimized by PSO-LSSVR surrogate model can reduce 9% energy consumption under nominal flow rate. It is attributed to the inhibitory effect of the optimization scheme on the horseshoe vortex at leading edge and the wake vortex at trailing edge of impeller. This study provides a new idea for the optimization design of high efficiency and low energy consumption of multistage centrifugal pumps.

Generic fully coupled framework for reliability assessment of offshore wind turbines under typical limit states

Offshore wind turbines (OWTs) generally experience harsh marine environments with uncertainties in marine loads, soil properties, and material properties; therefore, the structural integrity assessment of OWTs is a challenging task. Most of the previously proposed methods for reliability assessment are time-consuming and neglect the coupling effects of aerodynamics, hydrodynamics, and OWT structural dynamics under complex environmental loads. In this study, a fully coupled framework for the reliability assessment of OWTs is proposed, in which a dynamic simulation tool in the time domain is combined with surrogate models. The established Kriging, multivariate regression, and support vector regression surrogate models are compared to determine the most feasible model. Furthermore, the influence of different solving methods, such as first-order reliability method (FORM), importance sampling (IS), and subset simulation (SS), on the estimation of structural reliability indexes are discussed. The superiority of the estimated reliability indexes in the safety assessment of OWTs is proved using the SS method based on the Kriging surrogate model. Subsequently, the potential failure modes of an OWT under different operational states are revealed, indicating that excessive tower vibrations must be eliminated for stable power production from OWTs. Furthermore, the SS method is found to be feasible as it ensures the estimation accuracy of the reliability indexes of the parked OWT based on Kriging and the other surrogate models. Finally, the significance of soil–structure interactions in the reliability index estimation is investigated, and the equivalent coupled spring boundary is proposed from the perspectives of structural safety and simulation accuracy.

Deep Gaussian process for enhanced Bayesian optimization and its application in additive manufacturing

ABSTARCT Engineering design problems typically require optimizing a quality measure by finding the right combination of controllable input parameters. In Additive Manufacturing (AM), the output characteristics of the process can often be non-stationary functions of the process parameters. Bayesian Optimization (BO) is a methodology to optimize such “black-box” functions, i.e., the input–output relationship is unknown and expensive to compute. Optimization tasks involving “black-box” functions widely use BO with Gaussian Process (GP) regression surrogate model. Using GPs with standard kernels is insufficient for modeling non-stationary functions, while GPs with non-stationary kernels are typically over-parameterized. On the other hand, a Deep Gaussian Process (DGP) can overcome GPs’ shortcomings by considering a composition of multiple GPs. Inference in a DGP is challenging due to its structure resulting in a non-Gaussian posterior, and using DGP as a surrogate model for BO is not straightforward. Stochastic Imputation (SI)-based inference is promising in speed and accuracy for BO. This work proposes a bootstrap aggregation-based procedure to effectively utilize the SI-based inference for BO with a DGP surrogate model. The proposed BO algorithm DGP-SI-BO is faster and empirically better than the state-of-the-art BO method in optimizing non-stationary functions. Several analytical test functions and a case study in metal AM simulation demonstrate the applicability of the proposed method.

Systematic Calibration and Validation Approach for Discrete Element Method (DEM) Modeling of Corn Under Varying Moisture Contents (MC)

Abstract. Discrete Element Method (DEM) based simulation of grain threshing and conveyance processing at relatively dry to wet grain moisture content range has been limited. A comprehensive five-step framework of a DEM grain model calibration and validation of flow and grain impacts on equipment was investigated. The framework was examined by developing DEM models for corn grain moisture conditions sampled during crop harvesting, and simulated hopper discharge corn flow, and corn impact forces on a grain paddle elevator of a crop harvesting machine. Five bulk material response parameters were obtained from two calibration experiments of the angle of repose and direct shear test at the MC levels of 11%, 16%, and 26%. Latin hypercube sampling statistical design of experiment points, Gaussian Process Regression surrogate model, and optimization algorithms were successfully implemented to generate Hertz-Mindlin (HM) with Johnson-Kendall-Roberts (JKR) cohesive model of corn at three moisture content levels. The DEM simulation results predict the corn mass flow rate of hopper discharge with relative errors of -5%, -4%, and 1% for corn at 11%, 16%, and 26% MC levels, respectively. The DEM also predicts corn impact forces on a paddle elevator with a relative error of less than 11%. The established DEM calibration methodology, used for simulation of crop threshing, and harvested grain handling processes at extreme grain moisture content, can accelerate the simulation-based design of crop harvesting equipment. Keywords: Calibration, Corn, Discrete Element Method, Moisture Content, Sensitivity analysis, Validation.

Multi-objective optimization study of hydrothermal aging on NOx-assisted soot catalytic combustion and secondary pollution emission in regeneration of CeO2 catalytic diesel particulate filter

The present study aimed to explore the effect of hydrothermal aging on the heterogeneous catalytic combustion process of soot in CeO2 catalytic diesel particular filter and the formation of secondary pollutants (N2O and CO).The present study was based on a fixed-bed experimental bench for NOx-assisted soot catalytic oxidation in CeO2-CDPF with different soot loading and catalyst coating amounts. A non-linear regression surrogate model and a multi-objective optimization decision approach were used to analyze the trade-off relationship between the design variables and the optimization objectives. The results showed that lower levels of soot loading (2.5 g/L), CeO2 catalyst coating amount (2.5 g/L) and higher NOx concentration (1000 ppm) in fresh catalysts were favorable for improving the performance of soot-assisted catalytic combustion and minimizing the formation of N2O and CO. However, it was observed that higher levels of catalyst coating amount (10 g/L) resulted in better soot catalytic combustion performance with lower N2O and CO emissions. Global sensitivity analysis showed a positive correlation between N2O and CO peak concentrations and peak temperatures with soot loading, but a negative correlation with catalyst loading. Over time, hydrothermal aging of CDPF caused the inhibitory effect of CeO2 catalyst on N2O to shift from negative to positive correlation, while also decreasing the synergistic performance of NOx in soot catalytic combustion.

A Pointwise-Optimal Ensemble of Surrogate Models

AbstractThe ensemble of surrogate models is commonly used to replace computationally expensive simulations due to their superior prediction accuracy and robustness compared to individual models. This paper proposes a new pointwise ensemble of surrogate models, namely, a pointwise-optimal ensemble of surrogate models (POEMs). To address the limitations of the cross-validation (CV) error in evaluating the performance of regression surrogate models, this paper introduces the compensated cross-validation error, which is more reliable in selecting better individual surrogate models and improving the accuracy of surrogate model ensembles. To overcome the limitations of CV error in calculating pointwise weight factors, this paper designs and solves an optimization problem at training points to obtain corresponding pointwise weight factors. Additionally, this paper proposes two weight calculation methods to be applied in the interpolation and extrapolation regions, respectively, to reduce the instability of ensembles caused by extrapolation. Thirty test functions are employed to investigate the appropriate hyperparameters of POEM and the Friedman test is used to verify the rationality of the α value. The thirty test functions are also used to examine the performance of POEM and compare it with state-of-the-art ensemble surrogate models. Furthermore, POEM is applied to a large-aperture mirror holder optimization case to verify its superiority. The results demonstrate that POEM presents better accuracy and robustness than individual surrogates and other compared ensembles of surrogate models.

Optimization of V-shaped regulating ball valve profile based on proxy optimization algorithm

PurposeThe purpose of this study is to address the issue that the traditional V-shaped ball valve profile shape is limiting the flow control characteristics in a series structure and to optimize the design profile by proposing an open-hole profile.Design/methodology/approachThis paper proposes a Gaussian process regression surrogate model based on the genetic algorithm optimization of swarm intelligence, combined with the Expected Improvement point addition criterion, to optimize and correct the design profile. The flow regulation performance of the optimized V-shaped regulating ball valve is verified through a combination of numerical simulation and experiment.FindingsThe results demonstrate that the optimized V-shaped regulating ball valve has higher flow regulation accuracy and a more stable flow regulation process. After optimization, the flow characteristic curve of the spool is closer to the ideal equal percentage characteristic. The simulation results of the flow field are consistent with the experimental results.Originality/valueThe proposed method significantly reduces the optimization time, has higher efficiency and solves the problem that traditional optimization methods struggle with, which is ensuring optimal flow regulation performance. Compared to the traditional trial-and-error optimization method, the proposed method is more effective. The feasibility of the method is supported by experimental results.

Machine Learning Surrogates of a Fuel Matrix Degradation Process Model for Performance Assessment of a Nuclear Waste Repository

Spent nuclear fuel repository simulations are currently not able to incorporate detailed fuel matrix degradation (FMD) process models due to their computational cost, especially when large numbers of waste packages breach. The current paper uses machine learning to develop artificial neural network and k-nearest neighbor regression surrogate models that approximate the detailed FMD process model while being computationally much faster to evaluate. Using fuel cask temperature, dose rate, and the environmental concentrations of CO3 2−, O2, Fe2+, and H2 as inputs, these surrogates show good agreement with the FMD process model predictions of the UO2 degradation rate for conditions within the range of the training data. A demonstration in a full-scale shale repository reference case simulation shows that the incorporation of the surrogate models captures local and temporal environmental effects on fuel degradation rates while retaining good computational efficiency.

TL-GPRSM: A python software for constructing transfer learning Gaussian process regression surrogate model with explainability

This paper presents a software for the Transfer Learning Gaussian Process Regression Surrogate Model (TL-GPRSM). This software implements sampling and regression, which are essential for constructing surrogate models. Transfer learning is also supported. The implementation supports estimating the degree of effect of transfer learning to detect any loss of accuracy due to transfer learning. Estimation of the contribution of each input factor to the prediction is also supported so that the validity of the surrogate model’s predictions can be known during training. The source code is available on GitHub, including implementation and how to use it.

Optimization of Artificial Viscosity in Production Codes Based on Gaussian Regression Surrogate Models

Optimization of Artificial Viscosity in Production Codes Based on Gaussian Regression Surrogate Models

Transfer learning Gaussian process regression surrogate model with explainability for structural reliability analysis under variation in uncertainties

In this paper, a Gaussian process regression surrogate model with transfer learning (TL-GPRSM) is introduced to reduce the computational cost of structural reliability analysis by using the input–output relationship of the source analysis having similarities with that of the target analysis. In addition, automatic relevance determination (ARD) is introduced for providing the explainability of the constructed model and confidence assurance of transfer learning. Two verifications were conducted: (i) the surrogate modeling of the live-load performance analysis of a steel bridge with corrosion by applying the source analysis in the undamaged condition and (ii) seismic performance analysis of a bridge pier, which required nonlinear time-history analyses for various earthquake loads. The results showed that TL-GPRSM was especially effective in surrogate modeling of the performance analysis with linear numerical calculations. Moreover, it was shown that the effectiveness of transfer learning in each modeling and explainability of the constructed model could be discussed based on the contributions of the model parameters estimated through ARD.

A fast data-driven optimization method of multi-area combined economic emission dispatch

To provide more feasible schemes for power system optimization management and operation, the interconnection of power grids in different areas is inevitable. Naturally, the multi-area combined economic/emission dispatch (MACEED) problem becomes a more important decision problem. However, as the dimensions of MACEED problems increase, existing studies may not obtain feasible scheduling decisions in a suitable time. On this background, MACEED problems are transferred into computational expensive problems. In order to solve high-dimensional, large-scale MACEED problems, a data-driven surrogate-assisted approach is proposed. First, a feature engineering-based support vector regression surrogate model is proposed to replace the traditional objective functions in high-dimensional MACEED problems. Second, knowledge distillation is applied as a freezing and fine-tuning mechanism for the improved support vector regression surrogate models, which significantly reduces the time required to build surrogate models. Third, an improved non-dominated sorting genetic algorithm-III is proposed to obtain feasible solutions to high-dimensional MACEED problem. The proposed algorithm enhances the convergence and diversity of optimal solutions. The effectiveness of the proposed data-driven approach is demonstrated through simulations of a four-area 40-unit test system under different constraints.