Robust Surrogate Model Research Articles

Composition driven machine learning for unearthing high-strength lightweight multi-principal element alloys

High-strength, lightweight alloys are highly desired in the aerospace, automotive and marine industries. The optimization of such alloys is a multifaceted process, characterized by the consideration of diverse objectives and constraints. Various optimization methodologies exist, spanning from intricate first-principle approaches to the application of sophisticated machine learning algorithms. These algorithms might incorporate input features encompassing elemental composition, microstructural attributes, and thermodynamic properties to enhance prediction accuracies. In this work, we aim to streamline this complexity by employing solely the alloy's elemental composition as the input feature for the machine learning algorithm, improving the hardness while reducing the density of the alloy. We have curated a comprehensive database comprising 544 multi-principal element alloys and developed a robust surrogate model based on these compositions. This composition-driven model is subsequently coupled with principal component analysis (PCA) to facilitate the selection process. Remarkably, through a mere three iterations involving 14 samples, we successfully identified an alloy with an effective specific hardness surpassing the training database maximum by 8.6 %. The proposed composition-driven machine learning delineates a simplified approach for conducting optimization across multiple target material properties.

A Robust Recurrent Neural Networks-Based Surrogate Model for Thermal History and Melt Pool Characteristics in Directed Energy Deposition.

In directed energy deposition (DED), accurately controlling and predicting melt pool characteristics is essential for ensuring desired material qualities and geometric accuracies. This paper introduces a robust surrogate model based on recurrent neural network (RNN) architectures-Long Short-Term Memory (LSTM), Bidirectional LSTM (Bi-LSTM), and Gated Recurrent Unit (GRU). Leveraging a time series dataset from multi-physics simulations and a three-factor, three-level experimental design, the model accurately predicts melt pool peak temperatures, lengths, widths, and depths under varying conditions. RNN algorithms, particularly Bi-LSTM, demonstrate high predictive accuracy, with an R-square of 0.983 for melt pool peak temperatures. For melt pool geometry, the GRU-based model excels, achieving R-square values above 0.88 and reducing computation time by at least 29%, showcasing its accuracy and efficiency. The RNN-based surrogate model built in this research enhances understanding of melt pool dynamics and supports precise DED system setups.

Offline Model-Based Optimization via Policy-Guided Gradient Search

Offline optimization is an emerging problem in many experimental engineering domains including protein, drug or aircraft design, where online experimentation to collect evaluation data is too expensive or dangerous. To avoid that, one has to optimize an unknown function given only its offline evaluation at a fixed set of inputs. A naive solution to this problem is to learn a surrogate model of the unknown function and optimize this surrogate instead. However, such a naive optimizer is prone to erroneous overestimation of the surrogate (possibly due to over-fitting on a biased sample of function evaluation) on inputs outside the offline dataset. Prior approaches addressing this challenge have primarily focused on learning robust surrogate models. However, their search strategies are derived from the surrogate model rather than the actual offline data. To fill this important gap, we introduce a new learning-to-search perspective for offline optimization by reformulating it as an offline reinforcement learning problem. Our proposed policy-guided gradient search approach explicitly learns the best policy for a given surrogate model created from the offline data. Our empirical results on multiple benchmarks demonstrate that the learned optimization policy can be combined with existing offline surrogates to significantly improve the optimization performance.

Efficient sampling techniques for surrogate-based optimization with thermoelastic application

In many real-life models, the objective function may be very expensive to calculate or sometimes a black-box function. Therefore, a surrogate model is required to reduce the cost of the calculations. To build a robust surrogate model, a good sampling technique is needed. This paper suggests two new sampling techniques for surrogate-based optimization, depending on four random walk steps, which in turn give a good chance for building an accurate approximation for the original function. In addition to that, one of these new sampling techniques has a new feature that was not used before. The main advantage of our two suggested sampling techniques is that there is no optimization subproblem will be solved for the next sample point, which in turn saves CPU time. Furthermore, we applied our two sampling techniques to a 1D thermal shock problem for a half-space in the context of the fractional order theory of thermoelasticity to reduce the number of function evaluations and consequently CPU time. Laplace transform techniques are used to obtain the solution of the system in the frequency domain. Applying our algorithms to the few points obtained by using Laplace’s inverse, we can reach the optimal solution for all functions filed. The predictions of our techniques are discussed and compared with the classical method. 2D and 3D graphs are plotted for our numerical results. The temperature, displacement, and stress distributions are computed and represented graphically for different times and several fractional parameters.

Sequential Deep Operator Networks (S-DeepONet) for predicting full-field solutions under time-dependent loads

Deep Operator Network (DeepONet), a recently introduced deep learning operator network, approximates linear and nonlinear solution operators by taking parametric functions (infinite-dimensional objects) as inputs and mapping them to solution functions in contrast to classical neural networks that need re-training for every new set of parametric inputs. In this work, we have extended the classical formulation of DeepONets by introducing sequential learning models like the gated recurrent unit (GRU) and long short-term memory (LSTM) in the branch network to allow for accurate predictions of the solution contour plots under parametric and time-dependent loading histories, known as Sequential DeepONets (S-DeepONets). Two example problems, one on transient heat transfer and the other on path-dependent plastic loading, were shown to demonstrate the capabilities of the new architectures compared to the benchmark DeepONet model with a feed-forward neural network (FNN) in the branch. Despite being more computationally expensive, the GRU- and LSTM-DeepONets lowered the prediction error by half (0.06% vs. 0.12%) compared to FNN-DeepONet in the heat transfer problem, and by 2.5 times (0.85% vs. 3%) in the plasticity problem. In all cases, the proposed DeepONets achieved a prediction R2 value of above 0.995, indicating superior accuracy. Results show that once trained, the proposed S-DeepONets can accurately predict the final full-field solution over the entire domain and are at least two orders of magnitude faster than direct finite element simulations, rendering it an accurate and robust surrogate model for rapid preliminary evaluations.

Adaptive Basis Function Selection Enhanced Multisurrogate-Assisted Evolutionary Algorithm for Production Optimization

Summary Surrogate-assisted evolutionary algorithms (SAEAs) have become a popular approach for solving reservoir production optimization problems. The radial-basis-function network (RBFN) is a robust surrogate model technology suitable for reservoir development with numerous wells and a long production lifetime. There are several types of basis functions for constructing RBFN models. However, existing research shows that selecting the basis function with competitive performance for the current optimization problem is challenging without prior knowledge. In conventional SAEAs, the basis function is often predetermined, but its prediction accuracy for the problem at hand cannot be guaranteed. Furthermore, canonical SAEAs usually employ only one surrogate model for the entire optimization process. However, relying on a single surrogate model for optimization increases the probability of search direction misdirection due to prediction deviations. In this paper, a novel method named adaptive basis function selection enhanced multisurrogate-assisted evolutionary algorithm (ABMSEA) is introduced for production optimization. This method mainly includes two innovations. First, by training and testing different types of basis functions, the one with the best prediction performance is adaptively selected. Second, the ensemble model is constructed using the bootstrap sampling method, comprising multiple global surrogate models based on the selected best basis function. To search for a set of solutions that perform well on multiple surrogates, we employ an efficient multiobjective optimization (MOO) algorithm called nondominated sorting genetic algorithm II (NSGA-II). This algorithm uses the surrogates themselves as objective functions, aiming to find solutions that yield favorable results across multiple surrogates. The proposed method improves the efficiency of production optimization while enhancing global search capabilities. To evaluate the effectiveness of ABMSEA, we conduct tests on four 100D benchmark functions, a three-channel model, and an egg model. The obtained results are compared with those obtained from differential evolution (DE) and three other surrogate-model-based methods. The experimental results demonstrate that ABMSEA exhibits an accurate selection of competitive basis functions for the current optimization period while maintaining high optimization efficiency and avoiding local optima. Consequently, our method enables optimal well control, leading to the attainment of the highest net present value (NPV).

Development and validation of asthma risk prediction models using co-expression gene modules and machine learning methods

Asthma is a heterogeneous respiratory disease characterized by airway inflammation and obstruction. Despite recent advances, the genetic regulation of asthma pathogenesis is still largely unknown. Gene expression profiling techniques are well suited to study complex diseases including asthma. In this study, differentially expressed genes (DEGs) followed by weighted gene co-expression network analysis (WGCNA) and machine learning techniques using dataset generated from airway epithelial cells (AECs) and nasal epithelial cells (NECs) were used to identify candidate genes and pathways and to develop asthma classification and predictive models. The models were validated using bronchial epithelial cells (BECs), airway smooth muscle (ASM) and whole blood (WB) datasets. DEG and WGCNA followed by least absolute shrinkage and selection operator (LASSO) method identified 30 and 34 gene signatures and these gene signatures with support vector machine (SVM) discriminated asthmatic subjects from controls in AECs (Area under the curve: AUC = 1) and NECs (AUC = 1), respectively. We further validated AECs derived gene-signature in BECs (AUC = 0.72), ASM (AUC = 0.74) and WB (AUC = 0.66). Similarly, NECs derived gene-signature were validated in BECs (AUC = 0.75), ASM (AUC = 0.82) and WB (AUC = 0.69). Both AECs and NECs based gene-signatures showed a strong diagnostic performance with high sensitivity and specificity. Functional annotation of gene-signatures from AECs and NECs were enriched in pathways associated with IL-13, PI3K/AKT and apoptosis signaling. Several asthma related genes were prioritized including SERPINB2 and CTSC genes, which showed functional relevance in multiple tissue/cell types and related to asthma pathogenesis. Taken together, epithelium gene signature-based model could serve as robust surrogate model for hard-to-get tissues including BECs to improve the molecular etiology of asthma.

Flow imaging as an alternative to non-intrusive measurements and surrogate models through vision transformers and convolutional neural networks

A numerical framework is proposed whereby flow imaging data are leveraged to extract relevant information from flowfield visualizations. To this end, a vision transformer (ViT) model is developed to predict quantities of interest from images of unsteady flows. Here, the unsteady pressure distribution, the aerodynamic coefficients, and the skin friction coefficient are computed for an airfoil under dynamic stall as an example. The network is capable of identifying relevant flow features present in the images and associate them to the airfoil response. Results demonstrate that the model is effective in interpolating and extrapolating between flow regimes and for different airfoil motions, meaning that ViT-based models may offer a promising alternative for sensors in experimental campaigns and for building robust surrogate models of complex unsteady flows. In addition, we uniquely treat the image semantic segmentation as an image-to-image translation task that infers semantic labels of structures from the input images in a supervised way. Given an input image of the velocity field, a resulting convolutional neural network generates synthetic images of any corresponding fluid property of interest. In particular, we convert the velocity field data into pressure in order to subsequently estimate the pressure distribution over the airfoil in a robust manner. This approach proves to be effective in mapping between flowfield properties.

Fast transonic flow prediction enables efficient aerodynamic design

A deep learning framework is proposed for real-time transonic flow prediction. To capture the complex shock discontinuity of transonic flow, we introduce the residual network ResNet and deconvolutional neural networks to learn the nonlinear discontinuity phenomenon in transonic flow, which is affected by the Mach number, angle of attack, Reynolds number, and aerodynamic shape. In our framework, flow field variables on actual grid points are utilized in the neural network training to avoid the interpolation operation and the input of spatial position with a point cloud that is required with traditional convolutional neural networks. To investigate and validate the proposed framework, transonic flows around two-dimensional airfoils and three-dimensional wings are utilized to verify its effectiveness and prediction accuracy. The results prove that the model is able to efficiently learn the transonic flow field under the influence of the Mach number, angle of attack, Reynolds number, and aerodynamic shape. Significantly, some essential physical features, such as shock strength and location, flow separation, and the boundary layer, are accurately captured by this model. Furthermore, it is shown that our framework is able to make accurate predictions of the pressure distribution and aerodynamic coefficients. Thus, the present work provides an efficient and robust surrogate model for computational fluid dynamics simulation that enhances the efficiency of complex aerodynamic shape design optimization tasks and represents a step toward the realization of the digital twin concept.

The Monocytic Cell Line THP-1 as a Validated and Robust Surrogate Model for Human Dendritic Cells.

We have implemented an improved, cost-effective, and highly reproducible protocol for a simple and rapid differentiation of the human leukemia monocytic cell line THP-1 into surrogates for immature dendritic cells (iDCs) or mature dendritic cells (mDCs). The successful differentiation of THP-1 cells into iDCs was determined by high numbers of cells expressing the DC activation markers CD54 (88%) and CD86 (61%), and the absence of the maturation marker CD83. The THP-1-derived mDCs are characterized by high numbers of cells expressing CD54 (99%), CD86 (73%), and the phagocytosis marker CD11b (49%) and, in contrast to THP-1-derived iDCs, CD83 (35%) and the migration marker CXCR4 (70%). Treatment of iDCs with sensitizers, such as NiSO4 and DNCB, led to high expression of CD54 (97%/98%; GMFI, 3.0/3.2-fold induction) and CD86 (64%/96%; GMFI, 4.3/3.2-fold induction) compared to undifferentiated sensitizer-treated THP-1 (CD54, 98%/98%; CD86, 55%/96%). Thus, our iDCs are highly suitable for toxicological studies identifying potential sensitizing or inflammatory compounds. Furthermore, the expression of CD11b, CD83, and CXCR4 on our iDC and mDC surrogates could allow studies investigating the molecular mechanisms of dendritic cell maturation, phagocytosis, migration, and their use as therapeutic targets in various disorders, such as sensitization, inflammation, and cancer.

Data augmentation for disruption prediction via robust surrogate models

The goal of this work is to generate large statistically representative data sets to train machine learning models for disruption prediction provided by data from few existing discharges. Such a comprehensive training database is important to achieve satisfying and reliable prediction results in artificial neural network classifiers. Here, we aim for a robust augmentation of the training database for multivariate time series data using Student $t$ process regression. We apply Student $t$ process regression in a state space formulation via Bayesian filtering to tackle challenges imposed by outliers and noise in the training data set and to reduce the computational complexity. Thus, the method can also be used if the time resolution is high. We use an uncorrelated model for each dimension and impose correlations afterwards via colouring transformations. We demonstrate the efficacy of our approach on plasma diagnostics data of three different disruption classes from the DIII-D tokamak. To evaluate if the distribution of the generated data is similar to the training data, we additionally perform statistical analyses using methods from time series analysis, descriptive statistics and classic machine learning clustering algorithms.

Thermomechanical analysis of the effect of contact forces in heat conduction of composite materials through multiscale contact finite element modeling

The present study proposes a thermomechanical surface contact finite element formulation for studying interfacial heat conductance in correlation to micromechanical contact forces. The proposed methodology is applied for modeling heat surface conductivity of Carbon Nanotubes (CNT). The atomic lattice of CNTs is initially modeled using the molecular structural mechanics approach and reduced to an equivalent 2D continuum element. The equivalent 2D continuum element is used as a robust and efficient surrogate model for the construction of full length CNTs in contact. Coupled structural and heat pde’s have been used to describe the corresponding thermomechanical problem which are discretized using 2D quadrilateral finite elements, along with surface contact elements based on the Node to Node and Node to Segment formulations. The problem has been treated as a nonlinear steady state problem and solved in the frame of a Newton-Raphson incremental-iterative scheme. Extensive sensitivity analysis was performed with respect to several influencing parameters, such as initial contact angle, boundary conditions and CNT overlapping. Based on this analysis, the effect of overlapping and approaching angle on the total heat exchange was quantified. It has been demonstrated that in most cases, increasing the total contact force can produce an almost linear increase in heat exchange. However, in cases of small approaching angles between CNTs an extensive contact force could heavily deform the contacting geometries producing a non-linear heat exchange mechanism.

Uncertainty quantification of aeroelastic wings flutter using an optimized machine learning approach

This study outlines the flutter characteristics of aeroelastic wings under unsteady aerodynamic loading based on an efficient support vector machine assisted k-method. First, the aeroelastic wing flutter speed and flutter frequency are obtained using k-method. Then, the uncertain input parameters distribution is modeled by probability density functions. These parameters are propagated to the aeroelastic wing equations. The Monte Carlo simulation using 12 parallel logical threads is carried out to obtain the flutter speed and the flutter frequency distribution. An optimal robust surrogate model is trained by limited numbers of input and output using support vector machine. Monte Carlo simulation is also carried out in conjunction with the machine learning based k-method computational framework for obtaining the complete probabilistic description of flutter speed and flutter frequency. The coupled support vector regression based k-method is a novel approach that is first used in the aeroelastic wings flutter. The present method is found to reduce the computational time and cost significantly without compromising the accuracy of results.

Atikokan Digital Twin: Machine learning in a biomass energy system

The Atikokan Generating Station, operated by Ontario Power Generation, has a 200 MW, biomass-fired tower boiler that operates on a dispatch schedule with a five-minute cycle. The boiler is generally operated in the range of 40–100 MW using two of five burner levels. In order to optimize boiler performance, we propose the implementation of a unique digital twin. Our digital twin abstraction couples Bayesian inference from science-based models and from observations (machine learning) with decision theory to predict operating-variable set points that optimize the physical asset (the boiler) in the presence of uncertainty (artificial intelligence). We focus this paper on the continuous Bayesian machine learning part of the Atikokan Digital Twin; we discuss decision theory in a companion paper. We identify and learn about 12 operational, model, and measured-output parameters and their uncertainties from high-fidelity, science-based simulations of the Atikokan boiler and from the observed measurements at the power plant. Since the goal of the Atikokan Digital Twin is to implement it online in real time, we require fast function evaluations for the quantities of interest extracted from the simulations in the Bayesian analysis. We use Gaussian process regression/interpolation to create accurate, robust surrogate models. We define the Bayesian priors and likelihood function and solve for the posterior distributions of the 12 parameters. We then propagate these distributions (i.e., parameters with uncertainty) into the predicted distributions of 790 quantities of interest to learn about the relative importance of various sources of error including experimental, model, and operating-parameter errors.

A robust surrogate model of a solid oxide cell based on an adaptive polynomial approximation method

Surrogate models that predict the behaviors of solid oxide cells (SOCs) accurately at low computational cost are crucial to the control and optimization of SOC plants. Lumped physical models of SOCs, while widely used in such applications, lack accuracy because of neglected physical details. Data-driven models are the other options of surrogate models, which are proved to be more accurate because these models are identified directly from experiments or numerical simulations. However, due to the time cost of experiments and numerical simulations of SOCs, it is hoped that data-driven models can be constructed from a minimum amount of data. Also, the trained data-driven models should be robust, in other words, insensitive to the data set as well as the initial settings. These requirements are hard to be achieved by existing data-driven models of SOCs, such as lookup tables and artificial neural networks (ANNs). Aiming to preserve robustness and reduce the required amount of data, this paper introduces an adaptive polynomial approximation (APA) method, which is derived from the latest findings of numerical computation science, to the surrogate modeling of SOCs. The obtained models by the APA method are validated by both experiments and simulations. By analyzing the models, the coupling relationship among operating parameters of SOCs is revealed. The physical interpretability makes the APA method distinctive from common data-driven modeling methods. Performance comparison shows that the APA method is more accurate and robust than the existing ones with similar sampling costs. Additionally, the APA method can control the accuracy of the model by setting an error criterion in the algorithm iteration, endowing the APA method with an error control ability as per different accuracy requirements for SOC modeling.

EM-Centric Multiphysics Optimization of Microwave Components Using Parallel Computational Approach

For the high-performance microwave component and system design, besides electromagnetic (EM) physics domain, we also need to consider the operation of the real-world multiphysics (MP) environment that contains the effects of other physics domains. EM-centric MP analysis and design optimization become very important. In this article, for the first time, we develop a novel parallel EM-centric multiphysics optimization (MPO) technique. In our proposed technique, the pole/residue-based transfer function is exploited to build an effective and robust surrogate model. A group of modified quadratic mapping functions is formulated to map the relationships between pole/residues of the transfer function and the design variables. Multiple EM-centric MP evaluations are performed in parallel to generate the training samples for establishing the surrogate model. Using our proposed technique, the surrogate model can be valid in a relatively large neighborhood, which makes an effective and large optimization update in each optimization iteration. The trust region algorithm is performed to guarantee the convergence of the proposed MPO algorithm. Our proposed MPO technique takes a small number of iterations to obtain the optimal EM-centric MP response. Two microwave filter examples are used to demonstrate the validity of the proposed technique.

Deep convolutional neural networks for uncertainty propagation in random fields

AbstractThe development of a reliable and robust surrogate model is often constrained by the dimensionality of the problem. For a system with high‐dimensional inputs/outputs (I/O), conventional approaches usually use a low‐dimensional manifold to describe the high‐dimensional system, where the I/O data are first reduced to more manageable dimensions and then the condensed representation is used for surrogate modeling. In this study, a new solution scheme for this type of problem based on a deep learning approach is presented. The proposed surrogate is based on a particular network architecture, that is, convolutional neural networks. The surrogate architecture is designed in a hierarchical style containing three different levels of model structures, advancing the efficiency and effectiveness of the model in the aspect of training. To assess the model performance, uncertainty quantification is carried out in a continuum mechanics benchmark problem. Numerical results suggest the proposed model is capable of directly inferring a wide variety of I/O mapping relationships. Uncertainty analysis results obtained via the proposed surrogate have successfully characterized the statistical properties of the output fields compared to the Monte Carlo estimates.

Machine Learning Enabled Adaptive Optimization of a Transonic Compressor Rotor With Precompression

In aerodynamic design, accurate and robust surrogate models are important to accelerate computationally expensive computational fluid dynamics (CFD)-based optimization. In this paper, a machine learning framework is presented to speed-up the design optimization of a highly loaded transonic compressor rotor. The approach is threefold: (1) dynamic selection and self-tuning among several surrogate models; (2) classification to anticipate failure of the performance evaluation; and (3) adaptive selection of new candidates to perform CFD evaluation for updating the surrogate, which facilitates design space exploration and reduces surrogate uncertainty. The framework is demonstrated with a multipoint optimization of the transonic NASA rotor 37, yielding increased compressor efficiency in less than 48 h on 100 central processing unit cores. The optimized rotor geometry features precompression that relocates and attenuates the shock, without the stability penalty or undesired reacceleration usually observed in the literature.

Methodology of Robust Optimal Design of the Multistage Axial Compressor on the Basis of Discrete Analogue-Related Data

Consideration was given to the formulation and methodology of the direct nonlinear design problem of structural dimensional circuits in conditions of a priori parametric uncertainty. It was shown that the solution of the direct nonlinear design problem of structural dimensional circuits can be reduced to the multicriteria problems of stochastic optimization with mixed conditions. Mathematical models and methods of the solution of multicriteria problems of stochastic optimization with mixed conditions were analyzed. A.N. Tikhonov method of the regulation of solution of incorrectly formulated problems was used as a computation method for the synthesis of quasi-solutions of incorrectly formulated problems. An effective memetic algorithm for the synthesis of the solutions of MV-problems was suggested. It is based on the genetic algorithm combined with the randomized path tracing method. The suggested methodology allows us to search for rational solutions of the multicriteria problems of system modification through the creation of hierarchic two-level scheme for the synthesis of solutions that includes the construction of robust surrogate models of the systems and processes with subsequent robust estimation of sought values for the parametric data uncertainty. As an example, we obtained the solution data for the problem of robust optimal design of two-stage axial compressor in conditions of a stochastic nature of the input data obtained using the interactive decision support computer system «Concept_Pro_St ® ».

Development of robust surrogate model for economic performance prediction of oil reservoir production under waterflooding process

Geological uncertainty imposes significant challenge in modeling oil reservoirs. In this paper, a new robust modeling methodology is presented to identify a set of robust surrogate models with unstructured uncertainty for economic performance prediction of an uncertain oil reservoir under waterflooding process. The oil reservoir, being treated as a multi-input, multi-output (MIMO) system in terms of injection inputs and production outputs, is identified within a robust surrogate modeling framework using a frequency-based system identification approach. For this purpose, the concept of unstructured uncertainty model is incorporated with the surrogate modeling in the context of a function block having a norm-bounded uncertainty profile. The identified MIMO surrogate model is integrated with a desired nonlinear net present value (NPV) objective function to synthesize a new modified robust surrogate model in a multi-input, single-output (MISO) configuration form. This new modeling strategy enables direct calculation of economic performance prediction for the target oil reservoir. Finally, applicability of the proposed method is evaluated on “Egg Model” as a three-dimensional synthetic oil reservoir with 8 water injection wells and 4 oil production wells. The results clearly demonstrate that economic performance prediction of the oil reservoir, having uncertain permeability field, lies in the bounds corresponding to the worst-case scenarios of the uncertainty model set.