Neural Network Surrogate Model Research Articles

Reliability Optimization of the Honeycomb Sandwich Structure Based on A Neural Network Surrogate Model

Composite radomes are usually located in the nose of aircraft and are important structural components that protect radar antenna. The finite element model of a radome structure is developed and analyzed in this article. Single-objective deterministic and reliability optimization models based on the minimum total mass of the radome structure were established, and the layer thickness of each part of the honeycomb sandwich radome structure was considered a design variable. A multi-objective deterministic and reliability optimization model for a radome structure with a minimum total mass and maximum buckling critical load was established, and a particle swarm optimization algorithm was used to solve the problem. Our optimized results satisfied the constraints and utilization rate of materials, and structural safety was improved.

Optimizing Energy-Efficient jet impingement cooling using an artificial neural network (ANN) surrogate model for high heat flux Semiconductors

As the demand for high-performance electronic devices and electric vehicles (EVs) increases, the importance of energy-efficient thermal management has become more important. This study presents the optimization of two high-heat flux (865.8 W/cm2) semiconductor thermal management systems targeting an EV inverter. The cooling system employs a jet impingement integrated with microposts. The thermal resistance (Rth.avg) and pumping power (Ppump) were evaluated through computational fluid dynamics (CFD) simulations by varying the jet nozzle parameters, micropost variables, and volume flow rate. To ensure the accuracy of the CFD model, experimental validation was also conducted, demonstrating less than a 5 % error. Then a surrogate model predicting Rth.avg and Ppump was developed using the artificial neural network (ANN) method. The sensitivity analysis subsequently identified the primary influencing factors of the system, and the Pareto optimal fronts were determined via the elitist non-dominated sorting genetic algorithm (NSGA-II). Through optimization, the approach provided multiple optimal designs for a broad spectrum of Ppump values, which was feasible within a brief period (less than 200 s) due to the rapid estimations by the ANN-based surrogate model. The experimentally validated design achieved a 65 % improvement in heat transfer coefficient (∼102.96 kW/(m2·K)) at similar pumping power levels (∼0.42 W) compared to previously reported jet impingement studies. Moreover, the design chosen through the optimization (Case B) projected a 140 % enhancement in heat transfer coefficient (∼150.2 kW/(m2·K)) with only a 63 % rise in Ppump (∼0.65 W). By exploring a more diverse range of multi-jet and micropost configurations, we were able to achieve higher performance. This study highlights the potential of combining jet impingement cooling with microposts as a highly attractive strategy for thermal management in high heat flux and multi-hot spot applications.

Deep learning workflow for the inverse design of molecules with specific optoelectronic properties

The inverse design of novel molecules with a desirable optoelectronic property requires consideration of the vast chemical spaces associated with varying chemical composition and molecular size. First principles-based property predictions have become increasingly helpful for assisting the selection of promising candidate chemical species for subsequent experimental validation. However, a brute-force computational screening of the entire chemical space is decidedly impossible. To alleviate the computational burden and accelerate rational molecular design, we here present an iterative deep learning workflow that combines (i) the density-functional tight-binding method for dynamic generation of property training data, (ii) a graph convolutional neural network surrogate model for rapid and reliable predictions of chemical and physical properties, and (iii) a masked language model. As proof of principle, we employ our workflow in the iterative generation of novel molecules with a target energy gap between the highest occupied molecular orbital (HOMO) and the lowest unoccupied molecular orbital (LUMO).

Multi-Objective Optimization of Air-Cooled Perforated Micro-Pin Fin Heat Sink Via an Artificial Neural Network Surrogate Model Coupled With NSGA-II

Abstract This research aims to create an artificial neural network (ANN) regression model for predicting the performance parameters of the perforated micro-pin fin (MPF) heat sinks for various geometric parameters and inflow conditions. A three-dimensional computational fluid dynamics (CFD) simulation system is developed to generate dataset samples under different operational conditions, which are specified using Latin hypercube sampling (LHS). An ANN model is first obtained by optimizing the model hyper-parameters, which are then deployed to learn from the input feature space that consists of perforation diameter, perforation location, and inflow velocity. For accurate training of the ANN, the model is trained over a range of uniformly distributed data points in the input feature space. The developed multi-layer model predicted Nusselt number and friction factor with the mean absolute percentage error of 4.45% and 1.80%, respectively. Subsequently, the developed surrogate model is used in the optimization study to demonstrate the application of the surrogate model. A multi-objective non-dominated sorting genetic algorithm (NSGA-II) is used to perform the optimization of the perforation location, diameter, and inflow conditions. Negative of the Nusselt number and friction factor are chosen as objectives to minimize. A Pareto front is obtained from the optimization study that shows a set of optimal solutions. Thermal performance of the perforated MPF is increased between 11.5% and 39.77%. The optimizer selected a significantly smaller hole diameter at a higher location and a faster speed to maximize the Nusselt number and minimize the friction factor.

An integrated modelling framework for multiple pollution source identification in surface water

Pollution source identification is vital in water safety management. An integrated simulation-optimization modelling framework comprising a process-based hydrodynamic water quality model, artificial neural network surrogate model and particle swarm optimization (PSO) was proposed to achieve rapid, accurate and reliable pollution source identification. In this study, the hydrodynamics and water quality processes in a straight lab-based flume were simulated to test pollution source identification under steady flow conditions. Additionally, the pollution source identification in the unsteady flow conditions was examined using a real-life estuary, specifically the Yangtze River estuary. First, we developed two process-based models to simulate hydrodynamics and water quality in the flume and estuary. Then, the data generated from the process-based models were used to develop surrogate models. Three typical artificial neural networks (ANNs) algorithms: backpropagation (BP), radial basis function (RBF) and general regression neural networks (GRNN) were selected to develop surrogates for process-based models (PBMs), and they were coupled with PSO algorithm to achieve the hybrid modelling framework for pollution source identification. Our results showed that hybrid PBM-ANNs-PSO models could be applied to identify the pollution source and quantify release intensity in spatial distribution when the discharge type was assumed as the point source with a continuous release. Multiple-performance criteria metrics, in terms of the coefficient of determination, root-mean-square error, mean absolute error, evaluated the model performance as “Excellent prediction”. The BP-PSO models consistently appear to be the top-performing source identification model within the developed models, with most cases of relative error (RE) values lower than 5%. The new insights from the hybrid modelling framework would provide useful information for the local government agency to make reasonable decisions regarding pollution source identification issues.

JAX-DIPS: Neural bootstrapping of finite discretization methods and application to elliptic problems with discontinuities

We present a scalable strategy for development of mesh-free hybrid neuro-symbolic partial differential equation solvers based on existing mesh-based numerical discretization methods. Particularly, this strategy can be used to efficiently train neural network surrogate models of partial differential equations by (i) leveraging the accuracy and convergence properties of advanced numerical methods, solvers, and preconditioners, as well as (ii) better scalability to higher order PDEs by strictly limiting optimization to first order automatic differentiation. The presented neural bootstrapping method (hereby dubbed NBM) is based on evaluation of the finite discretization residuals of the PDE system obtained on implicit Cartesian cells centered on a set of random collocation points with respect to trainable parameters of the neural network. Importantly, the conservation laws and symmetries present in the bootstrapped finite discretization equations inform the neural network about solution regularities within local neighborhoods of training points. We apply NBM to the important class of elliptic problems with jump conditions across irregular interfaces in three spatial dimensions. We show the method is convergent such that model accuracy improves by increasing number of collocation points in the domain and preconditioning the residuals. We show NBM is competitive in terms of memory and training speed with other PINN-type frameworks. The algorithms presented here are implemented using JAX in a software package named JAX-DIPS (https://github.com/JAX-DIPS/JAX-DIPS), standing for differentiable interfacial PDE solver. We open sourced JAX-DIPS to facilitate research into use of differentiable algorithms for developing hybrid PDE solvers.

Multi-Objective Surrogate-Based Optimization Method for the Scaling of Combustion Chambers

Abstract In this study, we developed an efficient computer-aided design tool for scaling combustor designs. From a limited number of fluid simulations, an original design is scaled while preserving most of the flow properties, using a multi-objective optimization method and deep neural network or kriging surrogate models. The accuracy and robustness of the method were first tested with a simple geometry. Investigations have shown a strong sensitivity of the surrogate models to the sample distribution, which can be reduced using a strategic sampling method. Subsequently, the geometry was scaled down to a factor of two using both surrogate models while preserving most of the flow features.

A distribution network planning method based on the integration of operation and planning and coordinated with the transmission network

With the increasing integration of renewable energy into the power grid, the traditional roles of the transmission and distribution networks have become less distinct at the operational level. The integration between distribution network planning (DNP) and the transmission and distribution networks operation is crucial to ensure grid stability. Existing research has primarily focused on collaborative operation control between transmission and distribution networks, leaving a gap in integrated DNP, since few works can handle the integer variables. This study proposes a distribution network planning method based on the integration of operation and planning and coordinated with the transmission network. It aims to minimize investment and operational costs while considering local generation units, distributed renewables, and network constraints. Using a heterogeneous decomposition algorithm (HGD), the optimization model alternates between the two networks, assisted by injected parameters for global optimality. A convolutional neural network (CNN) surrogate model is then used to rapidly optimize precise distribution network plans that coordinate with the transmission network. Experimental results on IEEE 30 and IEEE 69 cases demonstrate that the proposed approach offers valuable engineering benefits, reducing iteration counts by up to 20% and improving accuracy compared to other distributed algorithms.

Porous single crystals under triaxial creep loadings: A data-driven modelling approach

The evolution of porosity plays an important role during failure processes in metallic materials at room and elevated temperatures. Recent experiments showed that void growth and coalescence may lead to the creep rupture of single crystal materials, mainly through plastic flow. It is highly challenging to accurately model the combined effects of multiple controlling factors, such as crystal orientation, matrix hardening behaviour, void shape and stress state. In this work, a neural network surrogate model was constructed to predict the behaviour of single crystals containing voids under creep loadings. Training of the neural network model was performed using 3D micromechanical finite element simulation results. A detailed assessment was carried out to examine the interpolation ability of the neural network model with respect to the crystal orientation, stress triaxiality and Lode parameter, as well as its extrapolation ability. Within the interpolation range, the neural network predictions of the deformation behaviour and porosity evolution of porous single crystals agree well with the finite element results of test dataset. The characteristic quantities, including the effective steady state strain rate, critical time and effective strain for the onset of tertiary creep, were predicted within 20% error with respect to the micromechanical simulation results for most of cases considered. Finally, the extrapolation ability of the model in predicting deformation behaviour of porous single crystal is good for the stress triaxiality lower than 1/3. However, the prediction of deformation and void evolution is deteriorated for the stress triaxiality larger than the range of value used for the training.

FE2 Computations with Deep Neural Networks: Algorithmic Structure, Data Generation, and Implementation

Multiscale FE2 computations enable the consideration of the micro-mechanical material structure in macroscopical simulations. However, these computations are very time-consuming because of numerous evaluations of a representative volume element, which represents the microstructure. In contrast, neural networks as machine learning methods are very fast to evaluate once they are trained. Even the DNN-FE2 approach is currently a known procedure, where deep neural networks (DNNs) are applied as a surrogate model of the representative volume element. In this contribution, however, a clear description of the algorithmic FE2 structure and the particular integration of deep neural networks are explained in detail. This comprises a suitable training strategy, where particular knowledge of the material behavior is considered to reduce the required amount of training data, a study of the amount of training data required for reliable FE2 simulations with special focus on the errors compared to conventional FE2 simulations, and the implementation aspect to gain considerable speed-up. As it is known, the Sobolev training and automatic differentiation increase data efficiency, prediction accuracy and speed-up in comparison to using two different neural networks for stress and tangent matrix prediction. To gain a significant speed-up of the FE2 computations, an efficient implementation of the trained neural network in a finite element code is provided. This is achieved by drawing on state-of-the-art high-performance computing libraries and just-in-time compilation yielding a maximum speed-up of a factor of more than 5000 compared to a reference FE2 computation. Moreover, the deep neural network surrogate model is able to overcome load-step size limitations of the RVE computations in step-size controlled computations.

Information-driven optimal experimental design with deep neural network surrogate model for composite materials

The optimal experimental design (OED) provides informative experimental resources and reduces the inherent uncertainty of the experiment. The OED is a framework to maximize performed experiments by controlling design variables. This paper proposes the OED framework combined with deep neural network (DNN) surrogate model to find the optimal design over large and complex design space. The OED has the optimization process to maximize the conditional mutual information (CMI) between hidden properties and composite structural performances. Approximate coordinate exchange (ACE) algorithm was applied to find the optimal design over a high and complicated design space. For fast optimization, surrogate DNN model was used instead of finite element analysis. Two examples were executed for the demonstration of the OED frameworks. From the results of each demonstration, OED provided the results that are consistent with heuristic knowledge.

A Fast Surrogate Model-Based Algorithm Using Multilayer Perceptron Neural Networks for Microwave Circuit Design

This paper introduces a novel algorithm for designing a low-pass filter (LPF) and a microstrip Wilkinson power divider (WPD) using a neural network surrogate model. The proposed algorithm is applicable to various microwave devices, enhancing their performance and frequency response. Desirable output parameters can be achieved for the designed LPF and WPD by using the proposed algorithm. The proposed artificial neural network (ANN) surrogate model is employed to calculate the dimensions of the LPF and WPD, resulting in their efficient design. The LPF and WPD designs incorporate open stubs, stepped impedances, triangular-shaped resonators, and meandered lines to achieve optimal performance. The compact LPF occupies a size of only 0.15 λg × 0.081 λg, and exhibits a sharp response within the transmission band, with a sharpness parameter of approximately 185 dB/GHz. The designed WPD, operating at 1.5 GHz, exhibits outstanding harmonics suppression from 2 GHz to 20 GHz, with attenuation levels exceeding 20 dB. The WPD successfully suppresses 12 unwanted harmonics (2nd to 13th). The obtained results demonstrate that the proposed design algorithm effectively accomplishes the LPF and WPD designs, exhibiting desirable parameters such as operating frequency and high-frequency harmonics suppression. The WPD demonstrates a low insertion loss of 0.1 dB (S21 = 0.1 dB), input and output return losses exceeding 30 dB (S11 = −35 dB, S22 = −30 dB), and an output ports isolation of more than 32 dB (S23 = −32 dB), making it suitable for integration into modern communication systems.

Research on Prediction Method of Bolt Tightening for Aviation Components Based on Neural Network

Aviation components play an important role in national defense and aviation development. Bolt connections are widely used in the assembly of aviation components, due to their simple structure and convenient disassembly. In addition to the impact of elastic interaction, the gap between the tightened parts also makes it very difficult to obtain a uniform bolt load, to achieve the required tightness during the tightening process. However, the impact of elastic interaction can be reduced by selecting the best tightening sequence, and the optimal tightening sequence of aviation components under different gaps can be predicted by constructing a neural network surrogate model. Based on the predicted optimal sequence, the elastic interaction matrix corresponding to the sequence can be obtained. In order to obtain a uniform preload, the initial load of each bolt is calculated according to an elastic interaction matrix. This research has improved the tightness of aviation components and the real-life efficiency of tightening process planning.

Using Bayesian updating for monopile offshore wind turbines monitoring

Digital twins (DTs) are thought to be promising tools for offshore wind turbines (OWTs) in real-time updating, optimized design, intelligent operation, and maintenance. When creating DT analysis tools, it is critical to update model parameters to create high-fidelity, high-accuracy virtual models. In this paper, a Bayesian updating framework based on Transitional Markov Chain Monte Carlo (TMCMC) is proposed to construct the desired virtual model of monopile OWTs to address this difficult problem. The likelihood function for Bayesian inference was created using measurements from physical entity monitoring as well as natural frequencies and damping ratios from the monopile OWT finite element (FE) model. Then, through a series of relatively simple intermediate likelihood functions, it gradually converges to the target function. During this time, the artificial neural network (ANN) surrogate model calculates the intermediate likelihood function of each sample, and a parallel computing strategy is used to enhance the computational efficiency. The results show that the posterior distribution of the uncertain parameters can be successfully estimated from the Markov chain using this method to update the finite element model. In addition, the proposed Bayesian updating method is resistant to measurement noise.

Machine Learning Surrogates of a Fuel Matrix Degradation Process Model for Performance Assessment of a Nuclear Waste Repository

Spent nuclear fuel repository simulations are currently not able to incorporate detailed fuel matrix degradation (FMD) process models due to their computational cost, especially when large numbers of waste packages breach. The current paper uses machine learning to develop artificial neural network and k-nearest neighbor regression surrogate models that approximate the detailed FMD process model while being computationally much faster to evaluate. Using fuel cask temperature, dose rate, and the environmental concentrations of CO3 2−, O2, Fe2+, and H2 as inputs, these surrogates show good agreement with the FMD process model predictions of the UO2 degradation rate for conditions within the range of the training data. A demonstration in a full-scale shale repository reference case simulation shows that the incorporation of the surrogate models captures local and temporal environmental effects on fuel degradation rates while retaining good computational efficiency.

Multi-Objective Optimization of Variable Density Multi-Layer Insulation for Liquid Hydrogen Containers Based on Reduced-Order Surrogate Model

For liquid hydrogen transportation, thermal insulation materials that are lightweight, compact and exhibit high-performance have been pursued for several decades, and variable density multi-layer insulation (VD-MLI) has been regarded as a promising choice. The thermal insulation performance of the insulation materials is important, but is not at the top of the list; many constraints, such as the space and weight of the insulation structures, are imposed on the design of a VD-MLI. Consequently, this makes the optimization of VD-MLIs more complicated. The present authors conducted a multi-objective optimization of a VD-MLI stacked with specific insulation units. The number of repetitions of the basic insulation unit was regarded as the dimensionless design parameter of the VD-MLI. Based on the experimentally validated layer-by-layer (LBL) model for MLI design, the multi-objective optimization of VD-MLI for liquid hydrogen storage was conducted by the combination of proper orthogonal decomposition with a general regression neural network (POD-GRNN) surrogate model optimization framework. The results showed that the optimal solutions for VD-MLI configurations could be achieved under different constraints. The present optimization framework provides a new reference for the optimization of VD-MLI for cryogenic liquid storage.

Machine learning surrogate modeling toward the design of lattice-structured heat sinks fabricated by additive manufacturing

Metal additive manufacturing (AM) enables the fabrication of lattice-structured heat sinks with enhanced heat transfer properties. The lattice structural features for enhancing heat transfer and suppressing pressure loss need to be clarified. To describe the structural features dominating heat transfer and pressure loss, eleven structural parameters consisting of three groups, namely the area, hydraulic diameter, and effective flow path, are proposed and used as inputs for the neural network (NN) surrogate model. The surrogate model precisely and quickly predicts the heat transfer property and pressure loss simulated by computational fluid dynamics. The random forest feature importance is used to select necessary and sufficient structural features. Four parameters describing the bottleneck in the flow pathway, surface area, and intricate structures are selected. Even when only the selected four parameters are used, the NN model maintains a high prediction performance, suggesting that the four parameters are sufficient for describing the heat transfer and pressure loss. Four parameters are used to discuss the design concept for lattice-structured heat sinks. An appropriately intricate lattice structure exhibits improved heat transfer and reduced pressure loss compared to a simple lattice structure. This study provides new insights into the design of lattice-structured heat sinks fabricated by AM.

A physics-informed deep neural network for surrogate modeling in classical elasto-plasticity

In this work, we present a deep neural network architecture that can efficiently surrogate classical elasto-plastic constitutive relations. The network is enriched with crucial physics aspects of classical elasto-plasticity, including additive decomposition of strains into elastic and plastic parts, and nonlinear incremental elasticity. This leads to a Physics-Informed Neural Network (PINN) surrogate model named here as Elasto-Plastic Neural Network (EPNN). Detailed analyses show that embedding these physics into the architecture of the neural network facilitates a more efficient training of the network with less training data, while also enhancing the extrapolation capability for loading regimes outside the training data. The architecture of EPNN is model and material-independent; it can be adapted to a wide range of elasto-plastic material types, including geomaterials; and experimental data can potentially be directly used in training the network. To demonstrate the robustness of the proposed architecture, we adapt its general framework to the elasto-plastic behavior of sands. We use synthetic data generated from material point simulations based on a relatively advanced dilatancy-based constitutive model for granular materials to train the neural network. The superiority of EPNN over regular neural network architectures is demonstrated through predicting unseen strain-controlled loading paths for sands with different initial densities.

Pavement Strategy Optimization of Cable-Stayed Bridges against the Negative Reaction Risks of Auxiliary Piers

Because the adjustment of the stay cable tension and girder counterweight is limited at the operation stage it is a difficult problem to control the negative reaction risk of the auxiliary pier (NRRAP) caused by multisource construction uncertainties and traffic growth. This paper proposes a pavement strategy optimization to control the NRRAP by adjusting the pavement thickness. The pavement strategy optimization is formulated as a reliability-constrained, multiobjective optimization problem, which is resolved by the nondominated sorting genetic algorithm (NSGA-II). A sensitivity analysis and a reliability analysis based on the generalized regression neural network (GRNN) surrogate model were performed to illustrate the significance of the uncertainty level in auxiliary pier negative reactions. The Pareto front examines the balance of construction cost, driving comfort and specified reliability threshold. The efficiency and accuracy of the proposed method are validated in a real cable-stayed bridge, and the results exhibit its advantages in controlling the NRRAP.

Hierarchical MPM-ANN Multiscale Terrain Model for High-Fidelity Off-Road Mobility Simulations: A Coupled MBD-FE-MPM-ANN Approach

AbstractA new hierarchical multiscale terrain model is developed using the material point method (MPM) to enable effective modeling of large terrain deformation for high-fidelity off-road mobility simulations. Unlike the Lagrangian finite element (FE) model, MPM allows for modeling large deformation of a continuum without mesh distortion using material points as moving quadrature points for the background grid. This unique feature is extended to account for complex granular soil material behavior with the hierarchical multiscale modeling approach in the context of off-road mobility simulations. The grain-scale discrete-element (DE) representative volume element (RVE) and its neural network surrogate model (artificial neural network (ANN) RVE) are developed and introduced to the upper-scale MPM model through the scale-bridging algorithm. The DE RVE is used to generate training data for the ANN RVE, allowing for predicting the history-dependent grain-scale soil material behavior efficiently at every material point that moves through the upper-scale MPM background grid. A numerical procedure for modeling the interaction of the nonlinear FE tire model with the MPM-ANN multiscale terrain model is developed considering moving soil patches generalized for the upper-scale MPM terrain model. It is fully integrated into the general off-road mobility simulation framework by leveraging scalable high-performance computing techniques. The predictive ability of the proposed MPM-ANN multiscale off-road mobility model is examined and validated against the full-scale vehicle test data, involving large deformation of soft terrain. The computational benefit from the neural network surrogate model is also demonstrated.