Machine Learning Surrogate Model Research Articles

Machine-Learning-Driven High-Throughput Screening for High-Energy Density and Stable NASICON Cathodes.

The Na super ionic conductor (NASICON), which has outstanding structural stability and a high operating voltage, is an appealing material for overcoming the limits of low specific energy and larger volume distortion of sodium-ion batteries. In this study, to discover ideal NASICON cathode materials, a screening platform based on density functional theory (DFT) calculations and machine learning (ML) is developed. A training database was generated utilizing the previous 124 545 electrode databases, and a test set of 3126 potential NASICON structures [NaxMyM'1-y(PO4)3] with 27 dopants at the metal site and 6 dopants at the polyanion central site was constructed. The developed ML surrogate model identifies 796 materials that satisfy the following criteria: formation energy of <0.0 eV/atom, energy above hull of ≤0.025 eV/atom, volume change of ≤4%, and theoretical capacity of ≥50 mAh/g. The thermodynamically stable configurations of doped NASICON structures were then selected using machine learning interatomic potential (MLIP), enabling rapid consideration of various dopant site configurations. DFT calculations are followed on 796 screened materials to obtain energy density, average voltage, and volume change. Finally, 50 candidates with an average voltage of ≥3.5 V are identified. The suggested platform accelerates the exploration for optimal NASICON materials by narrowing the focus on materials with desired properties, saving considerable resources.

Design and validation of refractory alloys using machine learning, CALPHAD, and experiments

Refractory multicomponent alloys (RMCAs) have garnered attention as potential materials for high-temperature structural applications, due to their excellent mechanical properties. However, conventional alloy design has limitations in terms of constrained compositional space and a lack of computational databases with adequate coverage. To address this, we present a design framework that leverages machine learning (ML), the CALculation of PHAse Diagram (CALPHAD) method, and experimental validation to efficiently develop refractory alloys. The present study focuses on the Mo-Nb-W ternary system. Six ternary alloys were inversely designed by means of the conditional generative adversarial network (cGAN) and fabricated via arc melting. The ternary alloys exhibit a single BCC phase which is consistent with CALPHAD calculations as well as Scheil simulations. The present interactive design loop between the ML surrogate model and experiments is demonstrated through the accurate hardness prediction, resulting in cGAN models capable of rapid exploration of the higher-order design space. The hardness of the Mo-Nb-W alloys is in the range of 5–6 GPa due to their solid solution strengthening.

Large-scale graph-machine-learning surrogate models for 3D-flowfield prediction in external aerodynamics

The article presents the application of inductive graph machine learning surrogate models for accurate and efficient prediction of 3D flow for industrial geometries, explicitly focusing here on external aerodynamics for a motorsport case. The final aim is to build a surrogate model that can provide quick predictions, bypassing in this way the unfeasible computational burden of traditional computational fluid dynamics (CFD) simulations. We investigate in this contribution the usage of graph neural networks, given their ability to smoothly deal with unstructured data, which is the typical context for industrial simulations. We integrate an efficient subgraph-sampling approach with our model, specifically tailored for large dataset training. REV-GNN is the chosen graph machine learning model, that stands out for its capacity to extract deeper insights from neighboring graph regions. Additionally, its unique feature lies in its reversible architecture, which allows keeping the memory usage constant while increasing the number of network layers. We tested the methodology by applying it to a parametric Navier–Stokes problem, where the parameters control the surface shape of the industrial artifact at hand, here a motorbike.

Thermal Conductivity Analysis of Polymer‐Derived Nanocomposite via Image‐Based Structure Reconstruction, Computational Homogenization, and Machine Learning

Macroscopic thermal properties of engineered or inherent composites depend substantially on the composite structure and the interface characteristics. While it is acknowledged that unveiling such dependency relation is essential for materials design, the complexity involved in, e.g., microstructure representation and limited data impedes the research progress. Herein, this issue is tackled by machine learning techniques on image‐based microstructure and property data predicted from physics simulations, along with experimental validation. The methodology is demonstrated for the model system ultrahigh‐temperature ceramic nanocomposite. The structure is reconstructed from scanning electron microscope images, and is resolved by a diffuse‐interface representation, which is advantageous in handling complicated structure and interface properties. Subsequently, hierarchical finite element homogenization is carried out to evaluate the effective thermal conductivity. A thorough comparison between the computed results and experimentally measured data, conducted across diverse temperatures and varying interface thermal resistances, reveals a high level of agreement. The observed agreement allows for the inverse estimation of the interface thermal resistance, a parameter typically challenging to ascertain directly through experimental means. Utilizing comprehensive data, a machine learning surrogate model has been meticulously trained to accurately predict the effective thermal conductivity of composite structures with exceptional performance.

A machine learning surrogate model for time of flight diffraction measurements of rough defects

Understanding the uncertainty in nondestructive evaluation measurements of rough defects requires a stochastic analysis due to the random variation between the morphology of different defects. In previous studies, large numbers of finite element models of randomly generated rough defects have been run in order to gain an insight into the statistics of the uncertain results. This approach is limited due to the time taken to run each individual model (typically of the order of minutes per model) and so the total number of models run is limited. In this paper, a surrogate model approach is proposed which produces close approximations to an original finite element model of a time-of-flight diffraction measurement, but in a small fraction of the time (<1 ms). The surrogate model is trained on pairs of input (rough defect) and output (A-scan) data from the finite element model; the surrogate then learns the relationship between inputs and outputs and is then able to generate new results. The architecture employed is a machine learning sequence-to-sequence approach similar to those used in natural language algorithms, based on the idea of ‘translating’ between defect to A-scan. In this study, a surrogate model was trained on 2160 finite element results which spanned across a range of ultrasonic incidence angles, rough defect correlation length and root-mean-square roughness. The surrogate model is shown to produce A-scans with close agreement to those produced by the original finite element model; taking approximately 0.425 ms instead of approximately 8 min. Millions of results are then possible in seconds, rather than the decades that would be required for finite element studies; more complete stochastic analysis are therefore now viable.

Bayesian Proper Orthogonal Decomposition for Learnable Reduced-Order Models With Uncertainty Quantification

Designing and/or controlling complex systems in science and engineering relies on appropriate mathematical modeling of systems dynamics. Classical differential equation based solutions in applied and computational mathematics are often computationally demanding. Recently, the connection between reduced-order models of high-dimensional differential equation systems and surrogate machine learning models has been explored. However, the focus of both existing reduced-order and machine learning models for complex systems has been how to best approximate the high fidelity model of choice. Due to high complexity and often limited training data to derive reduced-order or machine learning surrogate models, it is critical for derived reduced-order models to have reliable uncertainty quantification at the same time. In this paper, we propose such a novel framework of Bayesian reduced-order models naturally equipped with uncertainty quantification as it learns the distributions of the parameters of the reduced-order models instead of their point estimates. In particular, we develop <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">learnable Bayesian proper orthogonal decomposition</i> (BayPOD) that learns the distributions of both the POD projection bases and the mapping from the system input parameters to the projected scores/coefficients so that the learned BayPOD can help predict high-dimensional systems dynamics/fields as quantities of interest in different setups with reliable uncertainty estimates. The developed learnable BayPOD inherits the capability of embedding physics constraints when learning the POD-based surrogate reduced-order models, a desirable feature when studying complex systems in science and engineering applications where the available training data are limited. Furthermore, the proposed BayPOD method is an end-to-end solution, which unlike other surrogate-based methods, does not require separate POD and machine learning steps. The results from a real-world case study of the pressure field around an airfoil have shown the potential of learnable BayPOD as a new family of reduced-order models with reliable uncertainty estimates.

Machine-Learning-Assisted Construction of Ternary Convex Hull Diagrams.

In the search for novel intermetallic ternary alloys, much of the effort goes into performing a large number of ab initio calculations covering a wide range of compositions and structures. These are essential to building a reliable convex hull diagram. While density functional theory (DFT) provides accurate predictions for many systems, its computational overheads set a throughput limit on the number of hypothetical phases that can be probed. Here, we demonstrate how an ensemble of machine-learning (ML) spectral neighbor-analysis potentials (SNAPs) can be integrated into a workflow for the construction of accurate ternary convex hull diagrams, highlighting regions that are fertile for materials discovery. Our workflow relies on using available binary-alloy data both to train the SNAP models and to create prototypes for ternary phases. From the prototype structures, all unique ternary decorations are created and used to form a pool of candidate compounds. The SNAPs ensemble is then used to prerelax the structures and screen the most favorable prototypes before using DFT to build the final phase diagram. As constructed, the proposed workflow relies on no extra first-principles data to train the ML surrogate model and yields a DFT-level accurate convex hull. We demonstrate its efficacy by investigating the Cu-Ag-Au and Mo-Ta-W ternary systems.

Calibration of the 2-Phase Bubble Tracking Model for Liquid Mercury Target Simulation with Machine Learning Surrogate Models

The Spallation Neutron Source (SNS) at Oak Ridge National Laboratory is one of the most powerful accelerator-driven neutron sources in the world. The intense protons strike on SNS’s mercury target to provide bright neutron beams, which also leads to severe fluid-structure interactions inside the target. Prediction of resultant loading on the target is difficult particularly when helium gas is injected into mercury to reduce the loading and mitigate the pitting damage on vessel walls. A 2-phase material model that incorporates the Rayleigh-Plesset (R-P) model is expected to address this multi-physics problem. However, several uncertain parameters in the R-P model require intensive simulations to determine their optimal values. With the help of machine learning and the measured target strain, we have studied the major uncertain parameters in this R-P model and developed a framework to identify optimal parameters that significantly reduce the discrepancy between simulations and experimental strains. The preliminary results show the possibility of using this mercury/helium mixture and surrogate models to predict a better match of target strain response when the helium gas is injected.

Machine Learning Surrogate Model Optimized by Improved Sparrow Search Algorithm for Multi-Objective Optimization of Permanent Magnet Synchronous Motor Direct-Drive Pump

Machine Learning Surrogate Model Optimized by Improved Sparrow Search Algorithm for Multi-Objective Optimization of Permanent Magnet Synchronous Motor Direct-Drive Pump

Robust optimization of heat-transfer-enhancing microtextured surfaces based on machine learning surrogate models

Currently, there is a growing industry-wide focus on enhancing the thermohydraulic performances of devices, with the goal of achieving more efficient energy management. However, due to manufacturing constraints, there has been a tendency to focus on simple geometric designs. Recently, the emergence of additive manufacturing techniques and advancements in artificial intelligence have enabled new possibilities in this field. In this work, a pioneering exploration of a novel methodology for optimization of microfins to enhance the heat transfer on flat surfaces is presented. The optimization is performed using data-driven algorithms to accelerate the evaluation of the performance of these surfaces; these algorithms are trained on a database of 15,694 numerical simulations of enhanced surfaces. The performance evaluation criterion (PEC), equal to the ratio between the thermal and hydraulic performance parameters of the geometry, is used as the objective function. To avoid an optimization that focuses solely on the compactness of the fins while ignoring their shape, the optimal geometry is sought, which proves to be a challenge. Hence, an optimization method that classifies the surfaces based on their periodicity is proposed, focusing on improving the performance in terms of the morphology. Results present a PEC augmentation range from +0.08 to +0.28.

Clinamen2: Functional-style evolutionary optimization in Python for atomistic structure searches

Clinamen2 is a versatile functional-style Python implementation of the covariance matrix adaptation evolution strategy (CMA-ES) utilizing Cholesky decomposition. On top of a problem-agnostic core algorithm, the software package offers a suite of utilities and library code enabling applications to important atomistic structure searches. Features include massively distributed computation and the BI-Population restart scheme. This article details the general code structure and introduces examples that illustrate some relevant applications for the materials science and chemistry worlds, including interfacing to density-functional-theory codes and machine-learned surrogate models. The functional design renders the code modular and adaptable, and makes the creation of interfaces to other atomistic software straightforward. Program summaryProgram Title: Clinamen2CPC Library link to program files:https://doi.org/10.17632/x7syr2txsd.1Developer's repository link:https://github.com/Madsen-s-research-group/clinamen2-public-releasesCode Ocean capsule:https://codeocean.com/capsule/4950229Licensing provisions: Apache-2.0Programming language: PythonSupplementary material:Nature of problem: Find optimal atomistic structures retaining full flexibility in the choice of the optimization target, the methodological approach and its implementation (e.g. CPU- vs. GPU-heavy calculations). Enable interfacing with relevant software, including but not limited to density-functional-theory (DFT) codes and machine-learning (ML) solutions.Solution method: The covariance matrix adaptation evolution strategy (CMA-ES) algorithm is implemented using Cholesky decompositions for efficiency. The core algorithm and application examples for specific problems are implemented in functional-style Python free of side effects, with data classes to keep track of the state of the evolution. Dask is used for job control to enable highly distributed workflows. Advanced strategies like BI-Population CMA-ES are easy to implement and illustrated in the examples.

On the advantages of radial variation in assembly design for a nuclear reactor core

The geometric flexibility of additively manufactured metals and ceramics generates a vast, open design space requiring advanced modeling and simulation tools for physics simulations and the rigorous definition of design problems. This effort deploys artificial intelligence (AI) and machine learning (ML) algorithms to inform the design space, to enhance evaluation of potential designs, and to generate optimized results more efficiently.This report documents efforts of the Transformational Challenge Reactor (TCR) program to leverage advanced modeling and simulation techniques driven by AI/ML algorithms on high-performance computing (HPC) systems to yield more optimized TCR core designs. A multiphysics ML surrogate model was developed to run on the HPC architectures. The surrogate model is trained on high-fidelity simulation data of coupled neutronics and thermofluidics and is used to quickly evaluate thousands of candidate core designs in parallel, thus driving the evolution of the cooling channel shapes to minimize temperature peaking and material stress. The flexibility of additive manufacturing allows for the design of unique assemblies for each radial ring of the reactor core, which drives an increase in the selected performance metrics.One of the unique contributions made by this work is the capability to computationally demonstrate a 26.5% improvement in the selected reactor core performance metric by exploiting the freedom of axially variable assembly design for a nuclear reactor core. At the same pressure, the drop in the core decreased by a factor of 2.6, and the maximum temperature dropped by 10.7%. Additionally, the radially optimized design uses less materials, with a 15% decrease in total fuel and a 17.5% decrease in total moderator volume. These are the first quantitative results to allow for variation in assembly design in the core’s different radial rings.

Uncertainty quantification of CO[formula omitted] plume migration in highly channelized aquifers using probabilistic convolutional neural networks

Characterization of the CO2 plume evolution in deep aquifers under geological uncertainties is crucial for designing geological carbon storage (GCS) projects. In this study, we are concerned with uncertainty quantification (UQ) of CO2 plume migration in uncertain permeability fields following non-Gaussian distributions. Specifically, we are concerned with the effect of intra-facies heterogeneity on plume migration in a multi-facies sedimentary setting. To alleviate the huge computational burden required for UQ, machine learning (ML) methods are broadly used to develop surrogate models. However, the quality of ML surrogate models is often not qualified. Here we present a probabilistic convolutional neural network (pCNN) method for fast surrogate modeling of the CO2 plume evolution. The dense and residual connections are adopted in the pCNN’s network architecture to enhance its performance in approximating the highly complex input–output mapping of the GCS model induced by non-Gaussian permeability heterogeneity. In addition, a Bayesian training strategy is utilized to train the pCNN surrogate model, enabling it to give predictions together with an estimate of the surrogate uncertainty. As a result, uncertainties associated with the pCNN surrogate can be quantified in addition to the model parameter uncertainty. The proposed method is demonstrated using a 3-D GCS model with an uncertain, non-Gaussian distributed, multi-facies permeability field. Results show that pCNN is able to provide a reliable and fast approximation of the highly complex GCS model. The pCNN-derived UQ estimates, obtained with negligible additional computational costs, show a good agreement with those derived from the computationally intensive GCS model. The generalization performance of pCNN is further investigated using “out-of-distribution” permeability samples. For such permeability fields, pCNN can still provide relatively accurate approximations, only showing slightly increased predictive errors and uncertainties relative to those having similar distributions to the training samples.

Effect of reservoir heterogeneity on well placement prediction in CO2-EOR projects using machine learning surrogate models: Benchmarking of boosting-based algorithms

Rising Carbon Dioxide (CO2) levels from human activities are driving climate change. Carbon capture and storage (CCS) during enhanced oil recovery (EOR) in underground reservoirs offer both environmental and economic benefits. This method boosts oil production, cuts greenhouse gas emissions, and supports sustainable energy. Precise well placement in CO2-EOR is a crucial task for effective oil displacement, but traditional reservoir simulators are costly. This study explores and compares boosting algorithms, as fast surrogate models, to achieve accurate well placement during CO2-EOR in light oil carbonate reservoirs. The research considers various reservoir scenarios with different geological heterogeneity levels (i.e., homogeneous, moderately heterogeneous, and highly heterogeneous reservoirs). Various parameters, such as injection and production well locations, the distance between production and injection wells in an inverted five-spot pattern, pattern angle, and injection and production rates are explored using a compositional reservoir simulator to assess their impact on the well placement problem. A comprehensive analysis of various boosting algorithms, including AdaBoost, CatBoost, Gradient Boosting, LightGBM, and XGBoost is performed using the simulated dataset to assess their efficacy. The results demonstrate that LightGBM outperformed the other algorithms with the lowest Mean Absolute Error and Root Mean Square Error of 115.3 × 106 $ and 188.2 × 106 $, respectively. Additionally, it demonstrates exceptional speed, averaging 3 to 8 times faster than other boosting algorithms in the three reservoir scenarios. This superior performance coupled with its efficient runtime makes LightGBM the ideal choice for the study objectives. Moreover, the mass balance approach highlights the significant CO2 storage efficiency, emphasizing the effectiveness of CO2-EOR in storing CO2 in underground heterogeneous reservoirs.

Bayesian calibration for Lamb wave propagation on a composite plate using a machine learning surrogate model

This paper presents a new framework for stochastic updating of a finite element model for a composite plate, considering the influence of temperature on Lamb wave propagation. The framework involves deterministic updating to optimize mechanical properties and stochastic updating to derive probability density functions for key parameters. It utilizes sensitivity analysis and Bayesian inference with Markov-Chain Monte Carlo simulations and the Metropolis–Hastings sampling algorithm. This paper proposes a machine learning surrogate model based on artificial neural networks to improve computational efficiency. This surrogate modeling approach allows parallelized Monte Carlo simulations, reducing updating time significantly without compromising the accuracy of the resulting probability density functions for model parameters. These advancements show a promising way to enhance composite plate modeling and Lamb wave propagation studies, providing a more efficient and accurate approach to verify and validate finite element models with potential applications in engineering simulations.

Bayesian experimental design and parameter estimation for ultrafast spin dynamics

Advanced experimental measurements are crucial for driving theoretical developments and unveiling novel phenomena in condensed matter and materials physics, which often suffer from the scarcity of large-scale facility resources, such as x-ray or neutron scattering centers. To address these limitations, we introduce a methodology that leverages the Bayesian optimal experimental design paradigm to efficiently uncover key quantum spin fluctuation parameters from x-ray photon fluctuation spectroscopy (XPFS) data. Our method is compatible with existing theoretical simulation pipelines and can also be used in combination with fast machine learning surrogate models in the event that real-time simulations are unfeasible. Our numerical benchmarks demonstrate the superior performance in predicting model parameters and in delivering more informative measurements within limited experimental time. Our method can be adapted to many different types of experiments beyond XPFS and spin fluctuation studies, facilitating more efficient data collection and accelerating scientific discoveries.

A comparison of machine learning surrogate models of street-scale flooding in Norfolk, Virginia

Low-lying coastal cities, exemplified by Norfolk, Virginia, face the challenge of street flooding caused by rainfall and tides, which strain transportation and sewer systems and can lead to personal and property damage. While high-fidelity, physics-based simulations provide accurate predictions of urban pluvial flooding, their computational complexity renders them unsuitable for real-time applications. Using data from Norfolk rainfall events between 2016 and 2018, this study compares the performance of a previous surrogate model based on a random forest algorithm with two deep learning models: Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU). The comparison of deep learning to the random forest algorithm is motivated by the desire to utilize a machine learning architecture that allows for the future inclusion of common uncertainty quantification techniques and the effective integration of relevant, multi-modal features.

Physics-informed neural networks as surrogate models of hydrodynamic simulators

In response to growing concerns surrounding the relationship between climate change and escalating flood risk, there is an increasing urgency to develop precise and rapid flood prediction models. Although high-resolution flood simulations have made notable advancements, they remain computationally expensive, underscoring the need for efficient machine learning surrogate models. As a result of sparse empirical observation and expensive data collection, there is a growing need for the models to perform effectively in ‘small-data’ contexts, a characteristic typical of many scientific problems. This research combines the latest developments in surrogate modelling and physics-informed machine learning to propose a novel Physics-Informed Neural Network-based surrogate model for hydrodynamic simulators governed by Shallow Water Equations. The proposed method incorporates physics-based prior information into the neural network structure by encoding the conservation of mass into the model without relying on calculating continuous derivatives in the loss function. The method is demonstrated for a high-resolution inland flood simulation model and a large-scale regional tidal model. The proposed method outperforms the existing state-of-the-art data-driven approaches by up to 25 %. This research demonstrates the benefits and robustness of physics-informed approaches in surrogate modelling for flood and hydroclimatic modelling problems.

Piecewise linear trees as surrogate models for system design and planning under high-frequency temporal variability

The design and planning of systems subject to high-frequency time-varying conditions (e.g., prices, resource supplies, and customer demand) requires the solution of multi-period optimization problems, which have to account for operational aspects that are often described by complex nonlinear models. Accordingly, to overcome the computational challenges associated with the solution of the above problems, we present a framework to build computationally efficient and yet accurate optimization models. We also propose a general method to use trained piecewise linear (PWL) trees as surrogate models to approximate nonlinearities in relatively high dimensions and embed these trees onto mathematical optimization models. We show that, for some datasets, embedding PWL trees leads to models that result in a better balance between accuracy and computational performance when compared with approaches based on other machine-learning surrogate models. We showcase the applicability of the proposed framework via a case study on maintenance optimization of building cooling systems.

Learning a surrogate Pekeris waveguide model

Underwater sound propagation models predict transmission loss (TL) given environmental parameters, such as seafloor sediment parameters. Also of interest is the reverse: Can seafloor parameters, such as sediment density, sound speed, and attenuation, be inferred from ocean acoustic data? This inverse problem can be addressed using information geometry tools for parameter identifiability analysis and model reduction, where we identify parameters that can be removed from a model without sacrificing accuracy. Critical to these methods for model reduction is the ability to evaluate derivatives of the model’s TL predictions with respect to model parameters. Automatic differentiation (AD) allows for rapid evaluation of a model’s derivatives but may face some challenges for implementation with sound propagation models, including models being written in “legacy code” and having non-differentiable points in the function space. Our solution is to train a surrogate machine learning (ML) model which can circumvent these challenges, and which we can apply AD to. We demonstrate this method using the Pekeris waveguide model of the ocean. We compare the surrogate and original model outputs and demonstrate the process of model reduction using a ML surrogate model. [Work supported by Office of Naval Research.]