Machine Learning Surrogate Model Research Articles

Exploring the spatial effects influencing the EGFR/ERK pathway dynamics with machine learning surrogate models

The fate of cells is regulated by biochemical reactions taking place inside of them, known as intracellular pathways. Cells display a variety of characteristics related to their shape, structure and contained fluid, which influences the diffusion of proteins and their interactions. To gain insights into the spatial effects shaping intracellular regulation, we apply machine learning (ML) to explore a previously developed spatial model of the epidermal growth factor receptor (EGFR) signaling. The model describes the reactions between molecular species inside of cells following the transient activation of EGF receptors. To train our ML models, we conduct 10,000 numerical simulations in parallel where we calculate the cumulative activation of molecules and transcription factors under various conditions such as different diffusion speeds, inactivation rates, and cell structures. We take advantage of the low computational cost of ML algorithms to investigate the effects of cell and nucleus sizes, the diffusion speed of proteins, and the inactivation rate of the Ras molecules on the activation strength of transcription factors. Our results suggest that the predictions by both neural networks and random forests yielded minimal mean square error (MSEs), while linear generalized models displayed a significant larger MSE. The exploration of the surrogate models has shown that smaller cell and nucleus radii as well, larger diffusion coefficients, and reduced inactivation rates increase the activation of transcription factors. These results are confirmed by numerical simulations. Our ML algorithms can be readily incorporated within multiscale models of tumor growth to embed the spatial effects regulating intracellular pathways, enabling the use of complex cell models within multiscale models while reducing the computational cost.

Topology-Agnostic Graph U-Nets for Scalar Field Prediction on Unstructured Meshes

Abstract Machine-learned surrogate models to accelerate lengthy computer simulations are becoming increasingly important as engineers look to streamline the product design cycle. In many cases, these approaches offer the ability to predict relevant quantities throughout a geometry, but place constraints on the form of the input data. In a world of diverse data types, a preferred approach would not restrict the input to a particular structure. In this paper, we propose Topology-Agnostic Graph U-Net (TAG U-Net), a graph convolutional network that can be trained to input any mesh or graph structure and output a prediction of a target scalar field at each node. The model constructs coarsened versions of each input graph and performs a set of convolution and pooling operations to predict the node-wise outputs on the original graph. By training on a diverse set of shapes, the model can make strong predictions, even for shapes unlike those seen during training. A 3-D additive manufacturing dataset is presented, containing Laser Powder Bed Fusion simulation results for thousands of parts. The model is demonstrated on this dataset, and it performs well, predicting both 2-D and 3-D scalar fields with a median R-squared > 0.85 on test geometries.

A GAN-based stepwise full-field mechanical prediction model for architected metamaterials

The exploration of mechanical metamaterials, characterized by unique unit cells with significant macroscopic mechanical properties, is of crucial importance. Traditional methods relying on bioinspired structures lack broad design space and controllability to some extent, especially when practical requirements have no biological counterparts. Hence, there is a pressing need to advance methodologies for the expansive design of unit cells, catering to diverse practical demands. Inspired by architected material, a latent design paradigm gains prominence for its extensive possibilities and unique distribution. However, wide design space faces challenges in accurately assessing the deformation history and stress states of diverse units, requiring validation through experiments or simulations, and imposing efficiency constraints on design. Using patterns generated by a mimetic corrosion algorithm and computational simulations, a GAN-based machine learning surrogate model is constructed to predict the history of deformation and stress fields of architected unit cells with accuracy and efficiency, in which a stepwise mapping strategy is proposed to augment comprehension of mechanical information in image processing. In distinct cases covering both explored domains and several unexplored domains, the model achieves effective predictions, demonstrating the robustness and adaptability of the model in extrapolating predictions of both time domain and design space. The proposed method demonstrates efficacy, stability, and generalizability, pioneering a solution for the mechanical history prediction of architected metamaterials in a sparse data setting, thereby overcoming the limitations of the training set.

Rapid and precise calibration of soil microparameters for high-fidelity discrete element models in vehicle mobility simulation

In the realm of numerical simulations concerning vehicle mobility, the establishment of a high-fidelity soil discrete element model often necessitates substantial parameter adjustments to align with the mechanical responses of actual soil. In pursuit of a rapid and precise calibration of the microparameters of the soil model, this paper describes an approach rooted in machine learning surrogate models. This method calibrates the corresponding discrete element microparameters based on the macroscopic Mohr–Coulomb parameters derived from actual soil direct shear tests. The distinct contribution lies in the creation of a dataset that bridges the soil microparameters and macroparameters through simulated direct shear tests, which serves as training data for machine learning algorithms. Additionally, an adaptive particle swarm optimization neural network algorithm is proposed to represent the nonlinear relationships among parameters within the dataset, thus achieving intelligent calibration. To verify the reliability of the proposed soil calibration model in the context of vehicle mobility simulations, a co-simulation is conducted using a vehicle multibody dynamics simulation model and the calibrated soil model, with validation conducted across multiple criteria.

ReLU surrogates in mixed-integer MPC for irrigation scheduling

Efficient water management in agriculture is important for mitigating the growing freshwater scarcity crisis. Mixed-integer Model Predictive Control (MPC) has emerged as an effective approach for addressing the complex scheduling problems in agricultural irrigation. However, the computational complexity of mixed-integer MPC still poses a significant challenge, particularly in large-scale applications. This study proposes an approach to enhance the computational efficiency of mixed-integer MPC-based irrigation schedulers by employing Rectified Linear Unit (ReLU) surrogate models to describe the soil moisture dynamics of the agricultural field. By leveraging the mixed-integer linear representation of the ReLU operator, the proposed approach transforms the mixed-integer MPC-based scheduler with a quadratic cost function into a mixed-integer quadratic program, which is the simplest class of mixed-integer nonlinear programming problems that can be efficiently solved using global optimization solvers. The effectiveness of this approach is demonstrated through comparative studies conducted on a large-scale agricultural field across two growing seasons, involving other machine learning surrogate models, specifically Long Short-Term Memory (LSTM) networks, and the triggered irrigation scheduling method. The ReLU-based approach significantly reduces solution times—by up to 99.5%—while achieving comparable performance to the LSTM approach in terms of water savings and Irrigation Water Use Efficiency (IWUE). Moreover, the ReLU-based approach achieves enhanced performance in terms of irrigation water savings and IWUE compared to the triggered approach.

Boosting Barlow Twins Reduced Order Modeling for Machine Learning‐Based Surrogate Models in Multiphase Flow Problems

AbstractWe present an innovative approach called boosting Barlow Twins reduced order modeling (BBT‐ROM) to enhance the reliability of machine learning surrogate models for multiphase flow problems. BBT‐ROM builds upon Barlow Twins reduced order modeling that leverages self‐supervised learning to effectively handle linear and nonlinear manifolds by constructing well‐structured latent spaces of input parameters and output quantities. To address the challenge of high contrast data in multiphase flow problems due to injection wells and faults, we employ a boosting algorithm within BBT‐ROM. This algorithm sequentially trains a set of weak models (i.e., inaccurate models), improving prediction accuracy through ensemble learning. To evaluate the performance of BBT‐ROM, we conduct three three‐dimensional multiphase flow problems, including waterflooding and geologic carbon storage (GCS), with varying numbers of input parameter cases and model domain features. The results demonstrate that BBT‐ROM excels at predicting non‐wetting phase saturation (e.g., oil or saturation) and fluid pressure, with average relative errors ranging from 0.5% to 3%. Importantly, BBT‐ROM showcases robustness when faced with limited input parameter space during GCS testing.

Simulation-Based Optimization for Vertiport Location Selection: A Surrogate Model With Machine Learning Method

We present Vertiport-informed Surrogate-Based Optimization with Machine Learning Surrogates (VinS), a novel framework for solving the vertiport location problem for urban air mobility operations. The primary focus of this work is on the optimization of vertiport locations to facilitate efficient urban air transportation. Our framework helps choose not only the optimal vertiport locations but also the optimal number of vertiports. We develop a simulation model to analyze the impacts of various vertiport location configurations on the efficiency of the transportation network. To expedite the simulation process, a surrogate model is trained using machine learning algorithms, effectively reducing the computational time for evaluating a given vertiport location configuration. With the machine learning surrogate models, we apply a genetic algorithm to find the optimal set of vertiport locations. An empirical study was performed in the San Francisco Bay Area, from which we found that given the optimal set of vertiport locations, we further reduced the total travel time in the entire transportation system in the Bay Area compared with the sampled sets by 0.05% (400 h) on average. We release our framework at: https://github.com/Xuan-1998/LPSim .

Machine Learning Surrogate Model Optimized by Improved Sparrow Search Algorithm for Multi-Objective Optimization of Permanent Magnet Synchronous Motor Direct-Drive Pump

Machine Learning Surrogate Model Optimized by Improved Sparrow Search Algorithm for Multi-Objective Optimization of Permanent Magnet Synchronous Motor Direct-Drive Pump

Derivative learning of tensorial quantities-Predicting finite temperature infrared spectra from first principles.

We develop a strategy that integrates machine learning and first-principles calculations to achieve technically accurate predictions of infrared spectra. In particular, the methodology allows one to predict infrared spectra for complex systems at finite temperatures. The method's effectiveness is demonstrated in challenging scenarios, such as the analysis of water and the organic-inorganic halide perovskite MAPbI3, where our results consistently align with experimental data. A distinctive feature of the methodology is the incorporation of derivative learning, which proves indispensable for obtaining accurate polarization data in bulk materials and facilitates the training of a machine learning surrogate model of the polarization adapted to rotational and translational symmetries. We achieve polarization prediction accuracies of about 1% for the water dimer by training only on the predicted Born effective charges.

Auxetic kirigami structure-based self-powered strain sensor with customizable performance using machine learning

Recently, soft material based wearable sensors have discovered numerous applications in healthcare, sports monitoring, and virtual reality/augmented reality (VR/AR) systems. For these sensors, fulfilling user-specified requirements rather than just improving the sensor performance has become an important issue. In this study, a self-powered piezo-transmittance type strain sensor based on auxetic structures was optimized for configurable and user-specified characteristics using a machine-learning surrogate model. The sensor mechanism is based on the optical transmittance change induced by the gap opening of the auxetic kirigami structure. The sensor performance was analyzed according to the geometric design variables, and the optimal design was determined using Bayesian and Gaussian process to maximize the sensor performance for different purposes. The optimally designed geometries were used for self-powered sensors on a structural health monitoring (SHM) system, a human motion monitoring (HMM) system for monitoring sports performance and incorporated into an AR system.

Groundwater LNAPL Contamination Source Identification Based on Stacking Ensemble Surrogate Model

Groundwater LNAPL (Light Non-Aqueous Phase Liquid) contamination source identification (GLCSI) is essential for effective remediation and risk assessment. Addressing the GLCSI problem often involves numerous repetitive forward simulations, which are computationally expensive and time-consuming. Establishing a surrogate model for the simulation model is an effective way to overcome this challenge. However, how to obtain high-quality samples for training the surrogate model and which method should be used to develop the surrogate model with higher accuracy remain important questions to explore. To this end, this paper innovatively adopted the quasi-Monte Carlo (QMC) method to sample from the prior space of unknown variables. Then, this paper established a variety of individual machine learning surrogate models, respectively, and screened three with higher training accuracy among them as the base-learning models (BLMs). The Stacking ensemble framework was utilized to integrate the three BLMs to establish the ensemble surrogate model for the groundwater LNAPL multiphase flow numerical simulation model. Finally, a hypothetical case of groundwater LNAPL contamination was designed. After evaluating the accuracy of the Stacking ensemble surrogate model, the differential evolution Markov chain (DE-MC) algorithm was applied to jointly identify information on groundwater LNAPL contamination source and key hydrogeological parameters. The results of this study demonstrated the following: (1) Employing the QMC method to sample from the prior space resulted in more uniformly distributed and representative samples, which improved the quality of the training data. (2) The developed Stacking ensemble surrogate model had a higher accuracy than any individual surrogate model, with an average R2 of 0.995, and reduced the computational burden by 99.56% compared to the inversion process based on the simulation model. (3) The application of the DE-MC algorithm effectively solved the GLCSI problem, and the mean relative error of the identification results of unknown variables was less than 5%.

Reconstructing blood flow in data-poor regimes: a vasculature network kernel for Gaussian process regression.

Blood flow reconstruction in the vasculature is important for many clinical applications. However, in clinical settings, the available data are often quite limited. For instance, transcranial Doppler ultrasound is a non-invasive clinical tool that is commonly used in clinical settings to measure blood velocity waveforms at several locations. This amount of data is grossly insufficient for training machine learning surrogate models, such as deep neural networks or Gaussian process regression. In this work, we propose a Gaussian process regression approach based on empirical kernels constructed by data generated from physics-based simulations-enabling near-real-time reconstruction of blood flow in data-poor regimes. We introduce a novel methodology to reconstruct the kernel within the vascular network. The proposed kernel encodes both spatiotemporal and vessel-to-vessel correlations, thus enabling blood flow reconstruction in vessels that lack direct measurements. We demonstrate that any prediction made with the proposed kernel satisfies the conservation of mass principle. The kernel is constructed by running stochastic one-dimensional blood flow simulations, where the stochasticity captures the epistemic uncertainties, such as lack of knowledge about boundary conditions and uncertainties in vasculature geometries. We demonstrate the performance of the model on three test cases, namely, a simple Y-shaped bifurcation, abdominal aorta and the circle of Willis in the brain.

Part-scale microstructure prediction for laser powder bed fusion Ti-6Al-4V using a hybrid mechanistic and machine learning model

Laser powder bed fusion (LPBF) Ti-6Al-4V is widely studied for use in structural applications in aerospace and medical industries, but mechanical anisotropy and microstructural inhomogeneity prohibits its wider adoption. Although successful microstructure prediction models have been developed, a remaining challenge is their limited integration across length/time scales and validation by experimental studies. This work proposes a physics-augmented machine learning surrogate model to unite predictions of LPBF temperature, β phase morphology and texture, and α/α’ formation into a single framework that is calibrated and validated with experiments. First, a phase field (PF) model of the martensitic β→α’ transformation is developed and calibrated using data from in-situ synchrotron cyclic heating/cooling studies quantifying the variation of α phase fraction with time. In parallel, an established finite difference-Monte Carlo (FDMC) model predicts the part-scale temperature profile and β grain formation during solidification. A dataset is developed using LPBF cyclic temperature descriptors from the FDMC model as inputs and corresponding α/α’ phase fraction and width from the PF model as outputs. Five machine learning (ML) regression models are tested and optimized, having mean absolute error in testing ≤ 4 %, and the k-nearest neighbors (KNN) model is selected as the best performing. The KNN model is called at the nodal level during post-processing of the FDMC model to replace and downscale the response of the PF model. The combined agility and accuracy of the hybrid FDMC-ML model enables part-scale microstructure predictions that can be further used for property predictions to accelerate AM process optimization.

Enhancing PEM fuel cell dynamic performance: Co-optimization of cathode catalyst layer composition and operating conditions using a novel surrogate model

Optimizing the cathode catalyst layer (CCL) composition and operating conditions to enhance the dynamic performance of proton exchange membrane fuel cells garners significant attention. Although machine learning surrogate models are efficient for fuel cell analysis and optimization, the varied voltage dynamic response patterns (e.g., loading failure, voltage undershoot, and voltage hysteresis) challenge regression surrogate models designed for steady-state performance predictions. In response, this study introduces a joint framework combining classification and regression models for dynamic performance prediction. For training, a transient, two-phase, non-isothermal fuel cell model with integrated catalyst agglomerate is developed. The dynamic voltage deviation (σV) is proposed as an index to characterize the dynamic performance of the fuel cell. This joint surrogate model achieves correlation coefficients of 0.9976 and 0.9961 for predicting σV in training and test sets, respectively. Through this model, sensitivity analyses of the CCL composition and operating conditions are conducted to quantify their impact and interactions on the fuel cell's dynamic performance. Besides, the analysis reveals a trade-off between dynamic performance and steady-state output. To balance these, a multi-objective optimization is conducted. The results indicate that, compared to the base case, dynamic and steady-state performance improved by 44 % and 8 %, respectively.

Machine-learning structural reconstructions for accelerated point defect calculations

Defects dictate the properties of many functional materials. To understand the behaviour of defects and their impact on physical properties, it is necessary to identify the most stable defect geometries. However, global structure searching is computationally challenging for high-throughput defect studies or materials with complex defect landscapes, like alloys or disordered solids. Here, we tackle this limitation by harnessing a machine-learning surrogate model to qualitatively explore the structural landscape of neutral point defects. By learning defect motifs in a family of related metal chalcogenide and mixed anion crystals, the model successfully predicts favourable reconstructions for unseen defects in unseen compositions for 90% of cases, thereby reducing the number of first-principles calculations by 73%. Using CdSexTe1−x alloys as an exemplar, we train a model on the end member compositions and apply it to find the stable geometries of all inequivalent vacancies for a range of mixing concentrations, thus enabling more accurate and faster defect studies for configurationally complex systems.

Accelerating Molecular Vibrational Spectra Simulations with a Physically Informed Deep Learning Model.

In recent years, machine learning (ML) surrogate models have emerged as an indispensable tool to accelerate simulations of physical and chemical processes. However, there is still a lack of ML models that can accurately predict molecular vibrational spectra. Here, we present a highly efficient multitask ML surrogate model termed Vibrational Spectra Neural Network (VSpecNN), to accurately calculate infrared (IR) and Raman spectra based on dipole moments and polarizabilities obtained on-the-fly via ML-enhanced molecular dynamics simulations. The methodology is applied to pyrazine, a prototypical polyatomic chromophore. The VSpecNN-predicted energies are well within the chemical accuracy (1 kcal/mol), and the errors for VSpecNN-predicted forces are only half of those obtained from a popular high-performance ML model. Compared to the ab initio reference, the VSpecNN-predicted frequencies of IR and Raman spectra differ only by less than 5.87 cm-1, and the intensities of IR spectra and the depolarization ratios of Raman spectra are well reproduced. The VSpecNN model developed in this work highlights the importance of constructing highly accurate neural network potentials for predicting molecular vibrational spectra.

Machine learning-aided first-principles calculations of redox potentials

We present a method combining first-principles calculations and machine learning to predict the redox potentials of half-cell reactions on the absolute scale. By applying machine learning force fields for thermodynamic integration from the oxidized to the reduced state, we achieve efficient statistical sampling over a broad phase space. Furthermore, through thermodynamic integration from machine learning force fields to potentials of semi-local functionals, and from semi-local functionals to hybrid functionals using Δ-machine learning, we refine the free energy with high precision step-by-step. Utilizing a hybrid functional that includes 25% exact exchange (PBE0), this method predicts the redox potentials of the three redox couples, Fe3+/Fe2+, Cu2+/Cu+, and Ag2+/Ag+, to be 0.92, 0.26, and 1.99 V, respectively. These predictions are in good agreement with the best experimental estimates (0.77, 0.15, 1.98 V). This work demonstrates that machine-learned surrogate models provide a flexible framework for refining the accuracy of free energy from coarse approximation methods to precise electronic structure calculations, while also facilitating sufficient statistical sampling.

Predicting ground surface deformation induced from CO2 plume movement using machine learning

Carbon capture and storage (CCS), which involves injecting carbon dioxide (CO2) into subsurface, is an increasingly popular process for mitigating human caused greenhouse gas emissions. In order to ensure the safety and efficacy of CCS implementation, it is necessary to possess a comprehensive understanding of the complex behaviour of CO2 plumes within geological formations and their potential impact on ground surface deformation. Therefore, conducting research and analysis on these critical aspects is of vital importance. This research provides a methodology to anticipate ground surface deformations, which result from the motion of CO2 plumes utilising an advanced machine learning (ML) technique. The ML surrogate model has been developed using conditional Generative Adversarial Networks (cGAN). The dataset used for the model training and testing comprises ground surface measurements (tiltmeters), reservoir properties, as well as pressure/volume data. The model has been trained and tested using a set of samples created using a forward finite element model. Results show that the surrogate model is capable of predicting reasonably accurate results while running much faster than the forward model.

A constrained machine learning surrogate model to predict the distribution of water-in-oil emulsions in electrostatic fields

Accurately describing the evolution of water droplet size distribution in crude oil is fundamental for evaluating the water separation efficiency in dehydration systems. Enhancing the separation of an aqueous phase dispersed in a dielectric oil phase, which has a significantly lower dielectric constant than the dispersed phase, can be achieved by increasing the water droplet size through the application of an electrostatic field in the pipeline. Mathematical models, while being accurate, are computationally expensive. Herein, we introduced a constrained machine learning (ML) surrogate model developed based on a population balance model. This model serves as a practical alternative, facilitating fast and accurate predictions. The constrained ML model, utilizing an extreme gradient boosting (XGBoost) algorithm tuned with a genetic algorithm (GA), incorporates the key parameters of the electrostatic dehydration process, including droplet diameter, voltage, crude oil properties, temperature, and residence time as input variables, with the output being the number of water droplets per unit volume. Furthermore, we modified the objective function of the XGBoost algorithm by incorporating two penalty terms to ensure the model’s predictions adhere to physical principles. The constrained model demonstrated accuracy on the test set, with a mean squared error of 0.005 and a coefficient of determination of 0.998. The efficiency of the model was validated through comparison with the experimental data and the results of the population balance mathematical model. The analysis shows that the initial droplet diameter and voltage have the highest influence on the model, which aligns with the observed behaviour in the real-world process.

Machine-Learning-Driven High-Throughput Screening for High-Energy Density and Stable NASICON Cathodes.

The Na super ionic conductor (NASICON), which has outstanding structural stability and a high operating voltage, is an appealing material for overcoming the limits of low specific energy and larger volume distortion of sodium-ion batteries. In this study, to discover ideal NASICON cathode materials, a screening platform based on density functional theory (DFT) calculations and machine learning (ML) is developed. A training database was generated utilizing the previous 124 545 electrode databases, and a test set of 3126 potential NASICON structures [NaxMyM'1-y(PO4)3] with 27 dopants at the metal site and 6 dopants at the polyanion central site was constructed. The developed ML surrogate model identifies 796 materials that satisfy the following criteria: formation energy of <0.0 eV/atom, energy above hull of ≤0.025 eV/atom, volume change of ≤4%, and theoretical capacity of ≥50 mAh/g. The thermodynamically stable configurations of doped NASICON structures were then selected using machine learning interatomic potential (MLIP), enabling rapid consideration of various dopant site configurations. DFT calculations are followed on 796 screened materials to obtain energy density, average voltage, and volume change. Finally, 50 candidates with an average voltage of ≥3.5 V are identified. The suggested platform accelerates the exploration for optimal NASICON materials by narrowing the focus on materials with desired properties, saving considerable resources.