Data-driven Surrogate Research Articles

Proposal of a Data-Driven Surrogate Models with the Aim to Estimate the Mechanical Degradation of Cathodes in Lithium-Ion Batteries

Battery aging, a complex process crucial to the longevity and effectiveness of lithium-ion batteries, relies on the intricate interplay between electrochemical reactions and structural changes within battery electrodes. An essential element of battery aging involves the deterioration of electrode materials, which occurs due to the mechanical stresses induced during the insertion and extraction of ions within the active material particles.Ion intercalation, which governs the charging and discharging behavior of lithium-ion batteries, creates concentration variations within the active material particles. These variations, affected by transport processes and kinetics, lead to mechanical stresses as ions are spatial confined within the particles. These stresses are a cause of electrode degradation, resulting in particle fractures and the deterioration of electrode integrity.Understanding electrode degradation requires considering phase transitions experienced by common cathode materials like Lithium Nickel Manganese Cobalt Oxide (NMC) or Lithium Iron Phosphate (LFP) during ion intercalation. The transition between lithium-poor and lithium-rich phases, especially in the transition zone, creates significant mechanical stresses due to concentration differences. These stresses often exceed the material's resilience, causing particle fractures and impairing electrode functionality.Despite advances in computational techniques, accurately simulating these complex processes remains challenging due to the computational resources required. This has led researchers to explore innovative methods such as data-driven modeling, including physics-informed neural networks (PINNs). Here the PINN integrate the transport equation into the training algorithm to predict concentration profiles within active material particles, which lead to a improved accuracy of the concentration estimation inside the particle without computational bottlenecks. The proposed PINN possess a novel design, to precisely solve the moving boundary Stefan-Problem of the active material in phase transition.Classical neural networks (ANNs) are also valuable for estimating the electrical potential of active material particles, using results from existingmicrostructural simulations to develop models capturing electrochemical-mechanical interactions within batteries.Furthermore, to understand the extent of particle fracture and its impact on electrode performance, a phase-field-like damage model is integrated into the grey-box model. The governing equations are derived from the stationary point of the potential energy functional of the active particle system.The aim is to establish an operational range resilient to battery aging by systematically studying mechanical stresses and degradation under various operating conditions. This approach, combining experimental data, computational simulations, and theoretical frameworks, promises to advance our understanding of battery aging and drive the development of durable battery technologies

Tutorial on Molecular Latent Space Simulators (LSSs): Spatially and Temporally Continuous Data-Driven Surrogate Dynamical Models of Molecular Systems.

The inherently serial nature and requirement for short integration time steps in the numerical integration of molecular dynamics (MD) calculations place strong limitations on the accessible simulation time scales and statistical uncertainties in sampling slowly relaxing dynamical modes and rare events. Molecular latent space simulators (LSSs) are a data-driven approach to learning a surrogate dynamical model of the molecular system from modest MD training trajectories that can generate synthetic trajectories at a fraction of the computational cost. The training data may comprise single long trajectories or multiple short, discontinuous trajectories collected over, for example, distributed computing resources. Provided the training data provide sufficient sampling of the relevant thermodynamic states and dynamical transitions to robustly learn the underlying microscopic propagator, an LSS furnishes a global model of the dynamics capable of producing temporally and spatially continuous molecular trajectories. Trained LSS models have produced simulation trajectories at up to 6 orders of magnitude lower cost than standard MD to enable dense sampling of molecular phase space and large reduction of the statistical errors in structural, thermodynamic, and kinetic observables. The LSS employs three deep learning architectures to solve three independent learning problems over the training data: (i) an encoding of the high-dimensional MD into a low-dimensional slow latent space using state-free reversible VAMPnets (SRVs), (ii) a propagator of the microscopic dynamics within the low-dimensional latent space using mixture density networks (MDNs), and (iii) a generative decoding of the low-dimensional latent coordinates back to the original high-dimensional molecular configuration space using conditional Wasserstein generative adversarial networks (cWGANs) or denoising diffusion probability models (DDPMs). In this software tutorial, we introduce the mathematical and numerical background and theory of LSS and present example applications of a user-friendly Python package software implementation to alanine dipeptide and a 28-residue beta-beta-alpha (BBA) protein within simple Google Colab notebooks.

Data-Driven Surrogate Modeling with Microstructure-Sensitivity of Viscoplastic Creep in Grade 91 Steel

To support the development of advanced steel alloys tailored to withstand extreme conditions, it is imperative to account for the mechanical performance of components, while considering the influence of local microstructure on the macroscopic response. To this end, this study focuses on the development of microstructure-sensitive constitutive models for the mechanical response of Grade 91 steel exposed to extreme thermo-mechanical environments. Polynomial chaos expansion (PCE) surrogates are used to emulate high-fidelity polycrystal simulations of the viscoplastic response of Grade 91 steel as a function of the microstructure fingerprint (e.g., dislocations and precipitates). To cover a wide temperature–stress domain, two separate PCE surrogates—one that captures softening and the other that captures hardening behavior—are combined using another (sparse) Gaussian process regression model. The resulting constitutive creep surrogate model is integrated within the MOOSE finite element framework to simulate the intricate effects of microstructure, in particular MX-phase precipitates, on a component with a graded microstructure. Surrogate sensitivity analysis is applied to quantify the relevant impact of spatially varying microstructure on the creep response in a test-case involving a Grade 91 alloy with a prototypical weld.

A Data-Driven Surrogate Modeling for Train Rescheduling in High-Speed Railway Networks Under Wind-Caused Speed Restrictions

A Data-Driven Surrogate Modeling for Train Rescheduling in High-Speed Railway Networks Under Wind-Caused Speed Restrictions

Atomistic simulation assisted error-inclusive Bayesian machine learning for probabilistically unraveling the mechanical properties of solidified metals

Solidification phenomenon has been an integral part of the manufacturing processes of metals, where the quantification of stochastic variations and manufacturing uncertainties is critically important. Accurate molecular dynamics (MD) simulations of metal solidification and the resulting properties require excessive computational expenses for probabilistic stochastic analyses where thousands of random realizations are necessary. The adoption of inadequate model sizes and time scales in MD simulations leads to inaccuracies in each random realization, causing a large cumulative statistical error in the probabilistic results obtained through Monte Carlo (MC) simulations. In this work, we present a machine learning (ML) approach, as a data-driven surrogate to MD simulations, which only needs a few MD simulations. This efficient yet high-fidelity ML approach enables MC simulations for full-scale probabilistic characterization of solidified metal properties considering stochasticity in influencing factors like temperature and strain rate. Unlike conventional ML models, the proposed hybrid polynomial correlated function expansion here, being a Bayesian ML approach, is data efficient. Further, it can account for the effect of uncertainty in training data by exploiting mean and standard deviation of the MD simulations, which in principle addresses the issue of repeatability in stochastic simulations with low variance. Stochastic numerical results for solidified aluminum are presented here based on complete probabilistic uncertainty quantification of mechanical properties like Young’s modulus, yield strength and ultimate strength, illustrating that the proposed error-inclusive data-driven framework can reasonably predict the properties with a significant level of computational efficiency.

An Approach to Integrate Domain Knowledge into Feature Engineering to Enhance Data-Driven Surrogate Models of Simulations

Surrogate models are approximations of time-consuming computer simulations. These approximations may be derived by applying machine learning algorithms on a training set of simulation data. Due to the long runtime of these simulations, the amount of available simulation data to train surrogate models is however limited. This leads to a subpar performance of the resulting surrogate models. Here, feature engineering is important to improve the prediction performance of these models. However, automated feature engineering often results in surrogate models that no longer adhere to laws of physics. Therefore, we present an approach to integrate domain knowledge into feature engineering for the data-driven creation of surrogate models. This reduces the prediction error of surrogate models and ensures they adhere to laws of physics and are thus more reliable. We validate our approach with a prototypical implementation and a use case for simulations in virtual product development, and we show that our approach significantly improves the prediction quality of the resulting surrogate models.

Battery Modeling: Fusing Equivalent Circuit Models with Data-Driven Surrogate Modeling

Electrochemical impedance spectroscopy (EIS) is widely used to characterize electrochemical energy conversion systems. The traditional analysis with equivalent circuit models (ECM) has recently been augmented by a transform based distribution of relaxation times (DRT) analysis which allows one to reduce the ambiguity in the construction of ECMs and thus overfitting. Experimentally determined ECM parameters vary depending on operating conditions and the lifetime history of battery usage. Here we focus on State of Health (SOH) and State of Charge (SOC) as a basis for operando diagnosis and functionality optimization in the setting of fast-charging. Within pure ECM approaches, aging effects can only be represented to a limited extent, as aging is related to a variety of different factors whose impact on cell impedance are not sufficiently understood, yet. The highly complex interplay of factors motivates the development of data-driven Machine Learning (ML) models as a basis for future battery management systems. We present ML enabled ECMs based on experimental impedance analyses and a data-driven ML approach that computationally samples an abstract target space for classification and recognition of cells at vastly different SOC/SOH combinations.

Data-driven friction force prediction model for hydraulic actuators using deep neural networks

Hydraulic actuators convert fluid pressure into mechanical motion. They are widely used in many industrial and aerospace applications due to their reliability, high speed, high force output, smooth operation, and shock compensation ability. Because of their importance and wide adoption, it is vital to enable precise modeling of such devices. Fortunately, various modeling methods exist for hydraulic actuators and hydraulically driven systems, ranging from lookup tables or simple equations reflecting the system’s main features using lumped fluid theory to sophisticated and realistic fluid dynamics models. However, accurately accounting for friction that can depend nonlinearly on several state variables remains a core challenge in achieving high-fidelity hydraulic modeling. Therefore, several computational friction models are available, and their parameters must be identified or guessed. Another concern refers to simulation efficiency when complex models are considered. This study introduces a data-driven surrogate based on deep neural networks to address the challenge of practical and effective modeling of friction in hydraulic actuators. The surrogate is trained as a predictor using synthetic data generated from LuGre friction, demonstrating excellent accuracy and efficiency.

Thermal parameter determination in underdetermined spacecraft thermal models using surrogate modeling

This study introduces a data-driven surrogating modeling approach for determining thermal parameters in undetermined thermal mathematical models (TMM) used in spacecraft systems. Maintaining appropriate temperature levels for onboard components is crucial for the success and safety of space missions. However, accurately determining thermal parameters in the undetermined TMM presents a challenge. To address this, we propose an approach that combines TMM correlation with test data and data-driven surrogate modeling.Our approach minimizes an objective function that measures the deviation between predicted temperatures from a surrogate model, incorporating thermal parameters, and steady state temperatures obtained from thermal vacuum balance test results. This optimization is performed using a genetic algorithm while imposing constraints to ensure the physical validity of the thermal parameters and correlation criteria related to temperatures.We demonstrate the effectiveness and validity of our approach by applying the obtained thermal parameters to the TMM. The resulting calculated temperatures exhibit excellent agreement with the test results. This methodology offers a user-friendly and reliable way to determine thermal parameters in undetermined TMM within spacecraft systems, ultimately enhancing temperature control and mission success.

A Frequency-Based Ground Motion Clustering Approach for Data-Driven Surrogate Modeling of Bridges

A Frequency-Based Ground Motion Clustering Approach for Data-Driven Surrogate Modeling of Bridges

Inferring electrochemical performance and parameters of Li-ion batteries based on deep operator networks

Li-ion battery is a complex physicochemical system that generally takes observable current and terminal voltage as input and output, while leaving some unobservable quantities, e.g., Li-ion concentration, for serving as internal variables (states) of the system. On-line estimation for the unobservable states plays a key role in battery management system since they reflect battery safety and degradation conditions. Several kinds of models that map from current to voltage have been established for state estimation, such as accurate but inefficient physics-based models, and efficient but sometimes inaccurate equivalent circuit and black-box models. To realize accuracy and efficiency simultaneously in battery modeling, we propose to build a data-driven surrogate for a battery system while incorporating the underlying physics as constraints. In this work, we innovatively treat the functional mapping from current curve to terminal voltage as a composite of operators, which is approximated by the powerful deep operator network (DeepONet). Its learning capability is firstly verified through a predictive test for Li-ion concentration at two electrodes. In this experiment, the physics-informed DeepONet is found to be more robust than the purely data-driven DeepONet, especially in temporal extrapolation scenarios. A composite surrogate is then constructed for mapping current curve and solid diffusivity to terminal voltage with three operator networks, in which two parallel physics-informed DeepONets are firstly used to predict Li-ion concentration at two electrodes, and then based on their surface values, a DeepONet is built to give terminal voltage predictions. Since the surrogate is differentiable anywhere, it is endowed with the ability to learn from data directly, which was validated by using terminal voltage measurements to estimate input parameters. The proposed surrogate built upon operator networks possesses great potential to be applied in on-board scenarios, since it integrates efficiency and accuracy by incorporating underlying physics, and also leaves an interface for model refinement through a totally differentiable model structure.

Buried object characterization by data-driven surrogates and regression-enabled hyperbolic signature extraction

This work addresses artificial-intelligence-based buried object characterization using FDTD-based electromagnetic simulation toolbox of a Ground Penetrating Radar (GPR) to generate B-scan data. In data collection, FDTD-based simulation tool, gprMax is used. The task is to estimate geophysical parameters of a cylindrical shape object of various radii, buried at different positions in the dry soil medium simultaneously and independently of each other. The proposed methodology capitalizes on a fast and accurate data-driven surrogate model developed for object characterization in terms of its vertical and lateral position, and the size. The surrogate is constructed in a computationally efficient manner as compared to methodologies using 2D B-scan image. This is achieved by operating at the level of hyperbolic signatures extracted from the B-scan data through linear regression, which effectively reduces the dimensionality and the size of data. The proposed methodology relies on reducing of 2D B-scan image to 1D data including variation of reflected electric fields’ amplitudes with respect to the scanning aperture. The input of the surrogate model is the extracted hyperbolic signature obtained through linear regression executed on the background subtracted B-scan profiles. The hyperbolic signatures encode information about the geophysical parameters of the buried object, including depth, lateral position, and radius, all of which can be extracted using proposed methodology. Parametric estimation of the object radius and the estimation of the location parameters simultaneously is a challenging problem. Applying the application of processing steps on B-scan profiles incurs high computational costs, which is a limitation of the current methodologies. The metamodel itself is rendered using a novel deep-learning-based modified multilayer perceptron (M2LP) framework. The presented object characterization technique is favourably benchmarked against the state-of-the-art regression techniques, including Multilayer Perceptron (MLP), Support Vector Regression Machine (SVRM), and Convolutional Neural Network (CNN). The verification results demonstrate the average mean absolute error of 10 mm, and the average relative error of 8 percent, both corroborating the relevance of the proposed M2LP framework. In addition, the presented methodology provides a well-structured relation between the geophysical parameters of object and the extracted hyperbolic signatures. For the sake of supplementary verification under realistic scenarios, it is also applied for scenarios involving noisy data. The environmental and internal noise of the GPR system and their effect is analyzed as well. Furthermore, the proposed surrogate modeling approach is validated using measurement data, which is indicative of suitability of the approach to handle physical measurements as data sources.

Physically recurrent neural networks for path-dependent heterogeneous materials: Embedding constitutive models in a data-driven surrogate

Driven by the need to accelerate numerical simulations, the use of machine learning techniques is rapidly growing in the field of computational solid mechanics. Their application is especially advantageous in concurrent multiscale finite element analysis (FE2) due to the exceedingly high computational costs often associated with it and the high number of similar micromechanical analyses involved. To tackle the issue, using surrogate models to approximate the microscopic behavior and accelerate the simulations is a promising and increasingly popular strategy. However, several challenges related to their data-driven nature compromise the reliability of surrogate models in material modeling. The alternative explored in this work is to reintroduce some of the physics-based knowledge of classical constitutive modeling into a neural network by employing the actual material models used in the full-order micromodel to introduce non-linearity. Thus, path-dependency arises naturally since every material model in the layer keeps track of its own internal variables. For the numerical examples, a composite Representative Volume Element with elastic fibers and elasto-plastic matrix material is used as the microscopic model. The network is tested in a series of challenging scenarios and its performance is compared to that of a state-of-the-art Recurrent Neural Network (RNN). A remarkable outcome of the novel framework is the ability to naturally predict unloading/reloading behavior without ever seeing it during training, a stark contrast with popular but data-hungry models such as RNNs. Finally, the proposed network is applied to FE2 examples to assess its robustness for application in nonlinear finite element analysis.

A physics-informed convolutional neural network for the simulation and prediction of two-phase Darcy flows in heterogeneous porous media

The physics-informed neural network (PINN) is a general deep learning framework for simulating flows with limited or no labeled data. In the current study, we develop a physics-informed convolutional neural network (PICNN) for simulating transient two-phase Darcy flows in heterogeneous reservoir models with source/sink terms in the absence of labeled data, where the finite volume method (FVM) is adopted to approximate the PDE residual in the loss function such that flux continuity between neighboring cells of different properties is defined rigorously, and a well model is adopted to approximate the high pressure gradient near sources or sinks. The implicit-pressure explicit-saturation (IMPES) scheme is employed such that only a single CNN needs to be trained per time step. Dirichlet boundary conditions are not a mandatory requirement for PICNN-based implicit solver but act as labeled data that can help enhance accuracy. The proposed approach is validated in homogeneous and heterogeneous reservoirs and aspects including efficiency and accuracy are discussed. In addition, we demonstrate that the CNN structure can be trained as a data-driven surrogate for two-phase Darcy flows given sufficient labeled samples.

Deep learning-enhanced ensemble-based data assimilation for high-dimensional nonlinear dynamical systems

Data assimilation (DA) is a key component of many forecasting models in science and engineering. DA allows one to estimate better initial conditions using an imperfect dynamical model of the system and noisy/sparse observations available from the system. Ensemble Kalman filter (EnKF) is a DA algorithm that is widely used in applications involving high-dimensional nonlinear dynamical systems. However, EnKF requires evolving large ensembles of forecasts using the dynamical model of the system. This often becomes computationally intractable, especially when the number of states of the system is very large, e.g., for weather prediction. With small ensembles, the estimated background error covariance matrix in the EnKF algorithm suffers from sampling error, leading to an erroneous estimate of the analysis state (initial condition for the next forecast cycle). In this work, we propose hybrid ensemble Kalman filter (H-EnKF), which is applied to a two-layer quasi-geostrophic turbulent flow as a test case. This framework utilizes a pre-trained deep learning-based data-driven surrogate that inexpensively generates and evolves a large data-driven ensemble of the states to accurately compute the background error covariance matrix with smaller sampling errors. The H-EnKF framework outperforms EnKF with only dynamical model or only the data-driven surrogate, and estimates a better initial condition without the need for any ad-hoc localization strategies. H-EnKF can be extended to any ensemble-based DA algorithm, e.g., particle filters, which are currently too expensive to use for high-dimensional systems.

Data-driven surrogate optimized and intensified extractive distillation process for clean separation of isopropanol from water: A sustainable alternative

Isopropanol (IPA) is an important chemical because of its wide use in chemical and related process industries. In the production of IPA, an important processing step is its recovery and purification. The objective of this paper is to analyze different processes to find an energy efficient and environmentally friendly alternative for clean separation of isopropanol from water. A six-step design methodology is employed, which starts with the conventional extractive distillation process (EDP). Design parameters of EDP are first optimized by using a proposed artificial neural network (ANN) based data-driven surrogate (ANNDS). Using the optimal EDP as a reference design, improvements (intensification and/or integration) are considered through the so-called distillation-membrane hybrid process (DMHP), by heat pump vapor recompression (HP-EDP-I and HP-EDP-II), and dividing wall column without (E-DWC) and with (HP-E-DWC) heat pump, respectively. The results show that DMHP can be considered as a retrofit option with an energy reduction of 2.45% for the whole EDP. As compared to EDP, energy consumed in HP-EDP-I, HP-EDP-II, E-DWC, and HP-E-DWC can be reduced by 37.7%, 44.1%, 40.0%, and 56.4%, respectively. Moreover, CO2 emission is reduced by 42.6%, 50.4%, 40.0%, and 59.0%, respectively. Therefore, HP-E-DWC process is found to be the most sustainable alternative for the clean separation of isopropanol from water among the proposed processes. With further consideration of economic performance, HP-E-DWC is found to have the best performance with high separation capacity (>37500 kg/h), while E-DWC is found to be a favorable choice for a small plant with low capital cost and intermediate environmental impacts.

Data-driven strategies for optimization of integrated chemical plants

Operation optimization over large-scale integrated chemical plants is an inherently complex problem. We propose a surrogate-based optimization approach to optimize the operation of an industrial site that addresses both short-term market change and long-term maintenance plans. We develop a platform for automating the simulation and construction of surrogate models with a propagation error mitigation strategy. We are the first to investigate the impact of different levels of abstraction for surrogate models in site-level optimization. We also develop a deterministic, discrete-time optimization model that uses data-driven surrogate models. By optimizing a rolling horizon model with the above optimization model as the underlying model for each planning interval, we show that the plant level of abstraction is the superior approach. We demonstrate how data-driven surrogates can help address site-level process optimization by abstracting the process site network to a level that balances relevant details with tractability.

When stakes are high: Balancing accuracy and transparency with Model-Agnostic Interpretable Data-driven suRRogates

Technological advancements allow to develop high-performance black box predictive models. However, strictly regulated industries (like banking and insurance) ask for transparent decision-making algorithms. We therefore present a procedure to develop a Model-Agnostic Interpretable Data-driven suRRogate (maidrr) suited for structured tabular data. Knowledge is extracted from a black box via partial dependence effects. These are used to perform smart feature engineering by grouping variable values. This results in a segmentation of the feature space with automatic variable selection. A transparent generalized linear model (GLM) is fit to the features in categorical format and their relevant interactions. This GLM serves as a global surrogate to the original black box and replaces it in production. We demonstrate our R package maidrr with a case study on general insurance claim frequency modeling for six publicly available datasets. Our maidrr GLM closely approximates a gradient boosting machine (GBM) black box and outperforms both a linear and tree surrogate as benchmarks.

Self-supervised optimization of random material microstructures in the small-data regime

While the forward and backward modeling of the process-structure-property chain has received a lot of attention from the materials’ community, fewer efforts have taken into consideration uncertainties. Those arise from a multitude of sources and their quantification and integration in the inversion process are essential in meeting the materials design objectives. The first contribution of this paper is a flexible, fully probabilistic formulation of materials’ optimization problems that accounts for the uncertainty in the process-structure and structure-property linkages and enables the identification of optimal, high-dimensional, process parameters. We employ a probabilistic, data-driven surrogate for the structure-property link which expedites computations and enables handling of non-differential objectives. We couple this with a problem-tailored active learning strategy, i.e., a self-supervised selection of training data, which significantly improves accuracy while reducing the number of expensive model simulations. We demonstrate its efficacy in optimizing the mechanical and thermal properties of two-phase, random media but envision that its applicability encompasses a wide variety of microstructure-sensitive design problems.

Critical Responses of Flexible Pavements Under Superheavy Loads and Data-Driven Surrogate Model

Critical Responses of Flexible Pavements Under Superheavy Loads and Data-Driven Surrogate Model