Machine Learning Surrogate Model Research Articles

Real-time hybrid simulation incorporating machine learning for deep-water bridges subjected to seismic ground motion with fluid-structure dynamic interaction

The investigation of fluid-structure dynamic interaction (FSI) of deep-water structures subjected to seismic events with underwater shaking table testing method has been restricted by similarity problems and limited experimental capabilities. To address this issue, the authors previously proposed a real-time hybrid simulation (RTHS) method for FSI under seismic loading. In this study, the proposed method is applied to a deep-water bridge to demonstrate the low-cost and high-efficiency features of the proposed method. A total of 100 sets of experimental samples were analyzed, providing details and statistical information of the structural responses with FSI effects during seismic events. In the proposed RTHS application, a machine learning surrogate model was established for real-time calculation of the complex numerical substructure, and data exchange for complex physical-numerical boundaries. Additionally, a solution for RTHS when complex physical substructure boundary conditions can only be partially satisfied has been proposed, further contributing to the versatility of the study. Through the utilization of such advanced computational methods, the authors were able to explore the intricate coupled effect between fluid and structure in a deep-water environment. Such efforts hold the promise of enhancing our understanding of this complex phenomenon and may ultimately inform the design and construction of more resilient structures in the future.

Machine-learning surrogate model for accelerating the search of stable ternary alloys

Machine-learning surrogate model for accelerating the search of stable ternary alloys

Techno-economic analysis of an indirect solar dryer with thermal energy storage: An approach with machine learning algorithms for decision making

Machine learning models effectively forecast and improve engineering systems as solar dryers, making them valuable replacements for traditional physics-based models. Also, according to the literature, there is a research gap in solar drying regarding the lack of a comprehensive comparison of machine learning algorithms, neglecting the importance of cross-validation and tuning parameters. Therefore, this research focuses on employing machine learning models as effective surrogate models for solar dryers that attend to this gap, reducing the computational burden. A methodology for developing and comparing machine learning surrogate models of three indirect solar drying technologies (conventional, energy storage with beach sand, and storage with limestone) is proposed. The algorithms were trained using the database generated from a tomato solar drying process, where the solar dryer type, solar irradiance, ambient temperature, wind velocity, and relative humidity were considered model inputs to estimate the absorber plate, solar collector outlet air, and drying chamber outlet air temperatures. Subsequently, from the best predictive algorithm, a techno-economic analysis was conducted annually. The results revealed a high accuracy (R2 of 0.9917) and similar values regarding metrics that measure precision and variance. The solar dryer with thermal energy storage using beach sand exhibited the highest thermal and drying efficiency values (52.52 % and 12.57 %, respectively), while the limestone variant reported the shortest drying time (22 h). The economic evaluation indicated minimal monetary loss (3.09 %) compared to the current system’s annual projection. Finally, the framework presented in this research emphasizes using machine learning models to forecast the system’s energetic and economic performance indicators and contributes as a substitute for experimental studies that involve high cost and long duration. This work lays the bases and guides future research where machine learning algorithms are needed to compare energetic systems’ best computational and techno-economic performance.

Machine-Learned Surrogate Models for Efficient Oil Well Placement Under Operational Reservoir Constraints

Summary Recent predictive analytics and soft computing methods enhanced the exploration of new hydrocarbon reserves. Machine learning (ML) has showed a promising role in oil and gas explorations in recent years. Among the applications, determining a proper location for injection and production wells along with their optimum operating conditions is a complex problem. This research aims to develop a unified process using surrogate proxy models to address this issue. Five robust ML models, (i) extreme gradient boosting (XGBoost), (ii) light gradient boosting machine (LightGBM), (iii) gradient boosting with categorical features support (CatBoost), (iv) support vector regression (SVR), and (v) multilayer perceptron (MLP), are implemented to create surrogate proxy models for estimating the net present value (NPV) of an oil reservoir. A systematic approach is used to find the best-fit hyperparameter inputs for these models. The objective of this method was to refine a broad set of hyperparameters through a random cross-validation search technique. This grid cross-validation method investigates the space narrowed in more accurate intervals. Four reservoir scenarios are considered: (i) production from a single well in a homogeneous reservoir, (ii) production from a single well in a heterogeneous channelized reservoir, (iii) production from multiple wells in a heterogeneous reservoir, and (iv) waterflooding into a heterogeneous reservoir. A reservoir simulator is implemented to create a data set of reservoir realizations with various input parameters (i.e., well location, number of wells‚ production-injection well distance, and interwell angles) in a broad range of operating conditions. The prediction of gradient boosting and MLP models showed a better fit to the simulated data with an R-squared (R2) above 95% in the first three scenarios and 75% in the fourth scenario. The results indicate that the implemented proxies are promising approaches to efficiently estimate the NPV of the reservoir models both during primary and secondary recovery scenarios.

Beyond price taker: Conceptual design and optimization of integrated energy systems using machine learning market surrogates

Future electricity generation systems must be optimized to provide flexibility that counteracts the variability of non-dispatchable renewable energy sources and ensures the reliability and safety of critical infrastructure, including the electric grid. The current state-of-the-art is to co-optimize the design and operation of integrated energy systems (IES) treating historical or predicted time-series electricity prices as fixed parameters. Recent literature has shown the limitations of this price taker assumption, which neglects how IES optimization decisions influence market outcomes. As such, this paper proposes a new optimization formulation that uses machine learning surrogate models, trained from a library of annual market operation simulations, to embed IES market interactions into the co-optimization problem directly. Using a thermal generator example built in the open-source IDAES computational environment, we show that the price taker approach routinely over-predicts annual revenues by 8% or more compared to a validation simulation, where the proposed approach has a typical relative error of 1% or less.

Synergistic integration of deep neural networks and finite element method with applications of nonlinear large deformation biomechanics

Patient-specific finite element analysis (FEA) holds great promise in advancing the prognosis of cardiovascular diseases by providing detailed biomechanical insights such as high-fidelity stress and deformation on a patient-specific basis. Albeit feasible, FEA that incorporates three-dimensional, complex patient-specific geometry can be time-consuming and unsuitable for time-sensitive clinical applications. To mitigate this challenge, machine learning (ML) models, e.g., deep neural networks (DNNs), have been increasingly utilized as potential alternatives to finite element method (FEM) for biomechanical analysis. So far, efforts have been made in two main directions: (1) learning the input-to-output mapping of traditional FEM solvers and replacing FEM with data-driven ML surrogate models; (2) solving equilibrium equations using physics-informed loss functions of neural networks. While these two existing strategies have shown improved performance in terms of speed or scalability, ML models have not yet provided practical advantages over traditional FEM due to generalization issues. This has led us to the question: instead of abandoning or replacing the traditional FEM framework that can reliably solve biomechanical problems, can we integrate FEM and DNNs to enhance performance?In this study, we propose a synergistic integration of DNNs and FEM to overcome their individual limitations. Using biomechanical analysis of the human aorta as the test bed, we demonstrated two novel integrative strategies in forward and inverse problems. For the forward problem, we developed DNNs with state-of-the-art architectures to predict a nodal displacement field, and this initial DNN solution was then updated by a FEM-based refinement process, yielding a fast and accurate computing framework. For the inverse problem of heterogeneous material parameter identification, our method employs DNN as a regularizer of the spatial distribution of material parameters, aiding the optimizer in locating the optimal solution. In our demonstrative examples, despite that the DNN-only forward models yielded small displacement errors in most test cases; stress errors were considerably large, and for some test cases, the peak stress errors were greater than 50%. Our DNN-FEM integration eliminated these non-negligible errors in DNN-only models and was magnitudes faster than the FEM-only approach. Additionally, compared to FEM-only inverse method with errors greater than 50%, our DNN-FEM inverse approach significantly improved the parameter identification accuracy and reduced the errors to less than 1%.

A surrogate model for the economic evaluation of renewable hydrogen production from biomass feedstocks via supercritical water gasification

Supercritical water gasification is a promising technology for renewable hydrogen production from high moisture content biomass. This work produces a machine learning surrogate model to predict the Levelised Cost of Hydrogen over a range of biomass compositions, processing capacities, and geographic locations. The model is published to facilitate early-stage economic analysis (doi.org/10.6084/m9.figshare.22811066). A process simulation using the Gibbs reactor provided the training data using 40 biomass compositions, five processing capacities (10–200 m3/h), and three geographic locations (China, Brazil, UK). The levelised costs ranged between 3.81 and 18.72 $/kgH2 across the considered parameter combinations. Heat and electricity integration resulted in low process emissions averaging 0.46 kgCO2eq/GJH2 (China and Brazil), and 0.37 kgCO2eq/GJH2 (UK). Artificial neural networks were most accurate when compared to random forests and support vector regression for the surrogate model during cross-validation, achieving an accuracy of MAPE: <4.6%, RMSE: <0.39, and R2: >0.99 on the test set.

Impact study of numerical discretization accuracy on parameter reconstructions and model parameter distributions

In optical nano metrology numerical models are used widely for parameter reconstructions. Using the Bayesian target vector optimization method we fit a finite element numerical model to a Grazing Incidence x-ray fluorescence data set in order to obtain the geometrical parameters of a nano structured line grating. Gaussian process, stochastic machine learning surrogate models, were trained during the reconstruction and afterwards sampled with a Markov chain Monte Carlo sampler to determine the distribution of the reconstructed model parameters. The numerical discretization parameters of the used finite element model impact the numerical discretization error of the forward model. We investigated the impact of the polynomial order of the finite element ansatz functions on the reconstructed parameters as well as on the model parameter distributions. We showed that such a convergence study allows to determine numerical parameters which allows for efficient and accurate reconstruction results.

Combining synchrotron X-ray diffraction, mechanistic modeling and machine learning for in situ subsurface temperature quantification during laser melting.

Laser melting, such as that encountered during additive manufacturing, produces extreme gradients of temperature in both space and time, which in turn influence microstructural development in the material. Qualification and model validation of the process itself and the resulting material necessitate the ability to characterize these temperature fields. However, well established means to directly probe the material temperature below the surface of an alloy while it is being processed are limited. To address this gap in characterization capabilities, a novel means is presented to extract subsurface temperature-distribution metrics, with uncertainty, from in situ synchrotron X-ray diffraction measurements to provide quantitative temperature evolution data during laser melting. Temperature-distribution metrics are determined using Gaussian process regression supervised machine-learning surrogate models trained with a combination of mechanistic modeling (heat transfer and fluid flow) and X-ray diffraction simulation. The trained surrogate model uncertainties are found to range from 5 to 15% depending on the metric and current temperature. The surrogate models are then applied to experimental data to extract temperature metrics from an Inconel 625 nickel superalloy wall specimen during laser melting. The maximum temperatures of the solid phase in the diffraction volume through melting and cooling are found to reach the solidus temperature as expected, with the mean and minimum temperatures found to be several hundred degrees less. The extracted temperature metrics near melting are determined to be more accurate because of the lower relative levels of mechanical elastic strains. However, uncertainties for temperature metrics during cooling are increased due to the effects of thermomechanical stress.

Machine Learning Surrogate Model for Acceleration of Ferroelectric Phase-Field Modeling

Machine Learning Surrogate Model for Acceleration of Ferroelectric Phase-Field Modeling

Digital twins of multiple energy networks based on real-time simulation using holomorphic embedding method, Part II: Data-driven simulation

Digital twins can act as a transformative role in improving the operational performance of multiple energy networks (MEN) by examining the impact of implementing newer technologies, extra equipment, control strategies, etc. The objective of this series of papers is to present digital twins of MEN that can be simulated in real-time using the holomorphic embedding method. While Part I concentrated on mechanism-driven modeling of the holomorphic embedding-based model (HEM), this paper (Part II) focuses on data-driven simulation to ensure the twin is synchronized with actual physical objects. A parametric synchronization method (PSM) is proposed, which assists HEM in closely matching the actual dynamic behavior with time-varying characteristics. A machine learning surrogate model (MLSM) is proposed to accelerate the search of HEM’s convergence radius, which is critical to maintaining the twin’s real-time computational performance. Finally, the finalized digital twins are tested on the OPAL-RT simulation platform equipped with a real-time simulator. In a medium-sized MEN test case with a minor time step of 0.01s, the digital twins can be validated with a faster than real-time performance even without the assistance of parallel computing.

Interpretable discovery of semiconductors with machine learning

Machine learning models of material properties accelerate materials discovery, reproducing density functional theory calculated results at a fraction of the cost1–6. To bridge the gap between theory and experiments, machine learning predictions need to be distilled in the form of interpretable chemical rules that can be used by experimentalists. Here we develop a framework to address this gap by combining evolutionary algorithm-powered search with machine-learning surrogate models. We then couple the search results with supervised learning and statistical testing. This strategy enables the efficient search of a materials space while providing interpretable design rules. We demonstrate its effectiveness by developing rules for the design of direct bandgap materials, stable UV emitters, and IR perovskite emitters. Finally, we conclusively show how DARWIN-generated rules are statistically more robust and applicable to a wide range of applications including the design of UV halide perovskites.

Using machine learning surrogate modeling for faster QSP VP cohort generation.

Virtual patients (VPs) are widely used within quantitative systems pharmacology (QSP) modeling to explore the impact of variability and uncertainty on clinical responses. In one method of generating VPs, parameters are sampled randomly from a distribution, and possible VPs are accepted or rejected based on constraints on model output behavior. This approach works but can be inefficient (i.e., the vast majority of model runs typically do not result in valid VPs). Machine learning surrogate models offer an opportunity to improve the efficiency of VP creation significantly. In this approach, surrogate models are trained using the full QSP model and subsequently used to rapidly pre-screen for parameter combinations that result in feasible VPs. The overwhelming majority of parameter combinations pre-vetted using the surrogate models result in valid VPs when tested in the original QSP model. This tutorial presents this novel workflow and demonstrates how a surrogate model software application can be used to select and optimize the surrogate models in a case study. We then discuss the relative efficiency of the methods and scalability of the proposed method.

Toward a population synthesis of disks and planets

Aims. We want to find the distribution of initial conditions that best reproduces disc observations at the population level. Methods. We first ran a parameter study using a 1D model that includes the viscous evolution of a gas disc, dust, and pebbles, coupled with an emission model to compute the millimetre flux observable with ALMA. This was used to train a machine learning surrogate model that can compute the relevant quantity for comparison with observations in seconds. This surrogate model was used to perform parameter studies and synthetic disc populations. Results. Performing a parameter study, we find that internal photoevaporation leads to a lower dependency of disc lifetime on stellar mass than external photoevaporation. This dependence should be investigated in the future. Performing population synthesis, we find that under the combined losses of internal and external photoevaporation, discs are too short lived. Conclusions. To match observational constraints, future models of disc evolution need to include one or a combination of the following processes: infall of material to replenish the discs, shielding of the disc from internal photoevaporation due to magnetically driven disc winds, and extinction of external high-energy radiation. Nevertheless, disc properties in low-external-photoevaporation regions can be reproduced by having more massive and compact discs. Here, the optimum values of the α viscosity parameter lie between 3 × 10−4 and 10−3 and with internal photoevaporation being the main mode of disc dispersal.

An interpretable boosting-based predictive model for transformation temperatures of shape memory alloys

In this study, we demonstrate how the incorporation of appropriate feature engineering together with the selection of a Machine Learning (ML) algorithm that best suits the available dataset, leads to the development of a predictive model for transformation temperatures that can be applied to a wide range of shape memory alloys. We develop a gradient boosting ML surrogate model capable of predicting Martensite Start, Martensite Finish, Austenite Start, and Austenite Finish transformation temperatures with an average accuracy of more than 95% by explicitly taking care of potential distribution changes when modeling different alloy systems. We included heat treatment, rolling, extrusion processing parameters, and alloy system categorical features in the model input features to achieve more accurate and realistic results. In addition, using Shapley values, which are calculated based on the average marginal contribution of features to all possible coalitions, this study was able to gain insights into the governing features and their effect on predicted transformation temperatures, providing a unique opportunity to examine the critical parameters and features in martensite transformation temperatures.

Machine learning-aided reliability analysis of rainfall-induced landslide of root-reinforced slopes

Estimating the failure probability of rainfall-induced landslides is often challenging as the triggering mechanism is influenced by a number of parameters whose uncertainty is difficult to quantify and, in practice, is neglected. The reinforcing effect of vegetation on natural slopes adds to the complexity of the stability analysis. In this study, we present the application of a coupled hydro-mechanical model for the effect of plant roots on soil shear strength. First, a deterministic approach is adopted. Then, a reliability analysis of a root-reinforced slope subjected to rainfall is performed by considering the inherent variability of the soil and root properties. The probability of failure is estimated with machine learning surrogate models, which approximate the nonlinear relationship between constitutive parameters and slope displacements at different time steps. The machine learning algorithms are trained on a small dataset. The extreme gradient boosting is the best-performing algorithm with R2 ≥ 0.975 and is then employed to estimate the probability of failure on a larger dataset of one million datapoints with higher accuracy.

Forecasting atmospheric turbulence conditions from prior environmental parameters using artificial neural networks.

Atmospheric turbulence (C n2) modeling has been proposed by physics-based models, but they are unable to capture the many cases. Recently, machine learning surrogate models have been used to learn the relationship between local meteorological conditions and turbulence strength. These models predict C n2 at time t from weather at time t. This work expands modeling capabilities by proposing a technique to forecast 3h of future turbulence conditions at 30min intervals from prior environmental parameters using artificial neural networks. First, local weather and turbulence measurements are formatted to pairs of the input sequence and output forecast. Next, a grid search is used to find the best combination of model architecture, input variables, and training parameters. The architectures investigated are the multilayer perceptron and three variants of the recurrent neural network (RNN): the simple RNN, the long short-term memory RNN (LSTM-RNN), and the gated recurrent unit RNN (GRU-RNN). A GRU-RNN architecture that uses 12h of prior inputs is found to have the best performance. Finally, this model is applied to the test dataset and analyzed. It is shown that the model has generally learned the relationship between prior environmental and future turbulence conditions.

A Biaxial Discrete Diode Position Sensor for Rapid Postevent Structural Damage Assessment

The interstory drift ratio (IDR) is a key engineering parameter utilized by several codes and standards to quantify structural performance and correlate with potential structural damage following a seismic event. A number of methods have been explored for measuring building interstory drift, including approaches that rely on accelerometers, linear variable differential transformers (LVDTs), global position systems, combinations of cameras and optical sensors, and, most recently, on machine learning surrogate models. Nevertheless, limitations related to accuracy, cost, and challenges associated with data processing and interpretation still hinder the broad utilization of such methods for rapid post-event damage assessment. This paper presents a recently developed optical sensor, termed a biaxial discrete diode position sensor (2DDPS), which is able to directly measure earthquake transient and residual interstory drift in the two horizontal directions and transmit structure observables to offsite locations in near real-time. The sensor performance is herein evaluated through a series of shake-table tests carried out on the single 2DDPS to provide a statistical basis for sensor performance evaluations and on the 2DDPS installed on a 3D two-story frame subject to realistic earthquake biaxial excitation to evaluate system performance under realistic deployment conditions. The experimental campaign design and execution has been supported by advanced computational models. Results of the investigation demonstrate that the 2DDPS can reliably measure transient and residual bidirectional interstory drifts with an accuracy of the order of a millimeter. A critical review of the observed error and strategies to reduce it are also provided. Evidence from the presented study demonstrates that the 2DDPS has the potential to be utilized as a key tool for rapid, postevent decision-making and field inspection prioritization.

Physics-separating artificial neural networks for predicting initial stages of Al sputtering and thin film deposition in Ar plasma discharges

Simulations of Al thin film sputter depositions rely on accurate plasma and surface interaction models. Establishing the latter commonly requires a higher level of abstraction and means to dismiss the fundamental atomic fidelity. Previous works on sputtering processes addressed this issue by establishing machine learning surrogate models, which include a basic surface state (i.e. stoichiometry) as static input. In this work, an evolving surface state and defect structure are introduced to jointly describe sputtering and growth with physics-separating artificial neural networks. The data describing the plasma–surface interactions (PSIs) stem from hybrid reactive molecular dynamics/time-stamped force bias Monte Carlo simulations of Al neutrals and Ar+ ions impinging onto Al(001) surfaces. It is demonstrated that the fundamental processes are comprehensively described by taking the surface state as well as defect structure into account. Hence, a machine learning PSI surrogate model is established that resolves the inherent kinetics with high physical fidelity. The resulting model is not restricted to input from modeling and simulation, but may similarly be applied to experimental input data.

Highly efficient reliability analysis of anisotropic heterogeneous slopes: machine learning-aided Monte Carlo method

Machine learning (ML) algorithms are increasingly used as surrogate models to increase the efficiency of stochastic reliability analyses in geotechnical engineering. This paper presents a highly efficient ML-aided reliability technique that is able to accurately predict the results of a Monte Carlo (MC) reliability study and yet performs 500 times faster. A complete MC reliability analysis on anisotropic heterogeneous slopes consisting of 120,000 simulated samples is conducted in parallel to the proposed ML-aided stochastic technique. Comparing the results of the complete MC study and the proposed ML-aided technique, the expected errors of the proposed method are realistically examined. Circumventing the time-consuming computation of factors of safety for the training datasets, the proposed technique is more efficient than previous methods. Different ML models, including random forest, support vector machine and artificial neural networks, are presented, optimised and compared. The effects of the size and type of training and testing datasets are discussed. The expected errors of the ML predicted probability of failure are characterised by different levels of soil heterogeneity and anisotropy. Using only 1% of MC samples to train ML surrogate models, the proposed technique can accurately predict the probability of failure with mean errors limited to 0.7%. The proposed technique reduces the computational time required for our study from 306 days to only 14 h, providing 500 times higher efficiency.