Physics-informed Neural Networks Research Articles

Stable weight updating: A key to reliable PDE solutions using deep learning

Deep learning techniques, particularly neural networks, have revolutionized computational physics, offering powerful tools for solving complex partial differential equations (PDEs). However, ensuring stability and efficiency remains a challenge, especially in scenarios involving nonlinear and time-dependent equations. This paper introduces novel residual-based architectures, namely the Simple Highway Network and the Squared Residual Network, designed to enhance stability and accuracy in physics-informed neural networks (PINNs). These architectures augment traditional neural networks by incorporating residual connections, which facilitate smoother weight updates and improve backpropagation efficiency. Through extensive numerical experiments across various examples—including linear and nonlinear, time-dependent and independent PDEs—we demonstrate the efficacy of the proposed architectures. The Squared Residual Network, in particular, exhibits robust performance, achieving enhanced stability and accuracy compared to conventional neural networks. These findings underscore the potential of residual-based architectures in advancing deep learning for PDEs and computational physics applications.

Advanced deep learning approach with physics-informed neural networks for analysing the thermal variation through a radial fin applicable in heat exchangers

Advanced deep learning approach with physics-informed neural networks for analysing the thermal variation through a radial fin applicable in heat exchangers

Novel physics-informed optimization framework for complex multi-physics problems: Implementation for a sweeping gas membrane distillation module

The application of advanced machine learning algorithms such as deep learning is limited for the surrogate-based process optimization of complex multi-physics problems because high-quality experimental and numerical training samples required for deep-learning model training are expensive and scarce. To address this issue, in this study, a novel physics-informed process optimization (PIPO) framework was introduced. In the first step, surrogate models based on conventional neural networks (NN) were trained using a few available high-quality training samples. In the second step, the trained NN models were coupled to the process optimization solver in which the loss of physical laws was added to the optimizer’s objective function to find optimal design points that satisfy the laws of physics. As a result, the generalization performance of the framework was greatly improved for design targets outside the training range of NN models. PIPO is substantially different from the physics-informed neural networks where the loss of physics is added to the loss function used during NN model training. The PIPO framework was used to optimize a sweeping gas membrane distillation (SGMD) module. Eight input design variables, including process and geometrical parameters, were optimized for different challenging targets to achieve the best SGMD performance in terms of ammonia recovery ratio and concentration. It was shown that for noticeably few training samples of 68 experiments, the proposed framework was able to achieve the optimization targets within a reasonable computational cost. The optimum designs were verified and analyzed in detail by high-resolution computational fluid dynamics models.

Physics-informed neural network classification framework for reliability analysis

Reliability analysis is crucial for quantitatively evaluating structural safety amidst uncertainties, laying the foundation for reliability-based design optimization aimed at augmenting structural reliability and reducing economic expenditures, thereby offering substantial engineering benefits. However, performing reliability analysis on intricate structures, especially those requiring time-intensive finite element models, presents a formidable challenge. This study develops an innovative physics-informed neural network classification (PINNC) model to tackle this issue. In the PINNC model, the loss associated with the structural output state (i.e., safety or failed state) is defined as classification loss. Additionally, the structural output value (i.e., actual structural response) is treated as critical physical information, with its associated loss termed as physical loss. To facilitate a separate calculation of physical loss and classification loss, a parametric sigmoid activation function is used, establishing a link between the structural output value and the structural output state. The total loss is calculated as a weighted sum of both physical and classification losses. Unlike conventional neural network classification models that solely focus on classification loss, the PINNC model integrates both physical and classification losses, markedly improving the model’s efficacy in structural reliability analysis. Moreover, an adaptive framework is developed to incrementally include samples proximal to the limit state surface as new training data, thereby reinforcing the PINNC model’s computational precision. The effectiveness of the proposed PINNC framework in overcoming reliability analysis challenges is demonstrated through various applications.

Parameter identification and platoon control for virtually coupled train set with physics-informed neural network dynamics model

Modelling train dynamics is essential for the platoon control of virtually coupled train sets (VCTS). Variable rail-wheel conditions and high data collection costs pose substantial challenges for both physics-based and data-driven models, especially when unit trains in the VCTS are heterogenous. This study introduces a train dynamics model that integrates parameter identification of basic resistance with physics-informed neural networks. The developed physics-informed neural network for train dynamics modelling (PINN-TDM) identifies key resistance parameters using the Davis formula. By combining the strengths of traditional modelling and data-driven approaches, the PINN-TDM provides accurate predictions of train dynamics, accommodating variations due to changes in rail-wheel conditions and heterogeneity among unit trains. Compared to conventional neural network models that lack physical guidance, the PINN-TDM demonstrates enhanced performance in data-limited conditions. Furthermore, this study incorporates the PINN-TDM into a model predictive control framework and derives the sufficient condition for heterogeneous string stability in the frequency domain using the Laplacian transformation. This enables the analysis of the impact of train dynamics heterogeneity on the stable region. Simulation experiments show that the integrated controller improves the precision of trajectory tracking and the stability of the VCTS under diverse rail-wheel conditions.

Physics-constrained deep learning approach for solving inverse problems in composite laminated plates

The applications of physics-informed neural networks (PINNs) in material parameters identification of composite laminates are currently research highlights. We present an efficient physics-constrained deep learning approach based on PINNs for solving material parameters of composite laminates. We explain the details of incorporating first-order shear deformation theory as physical information into designed loss functions. The performance of proposed approach is demonstrated through a numerical example of idealized composite laminated plates under uniform pressure conditions. The computational errors of material parameters are less than 0.1%. Moreover, the applicability of transfer learning (TL) has been illustrated for inversion of material parameters with few datasets. The ambition is to improve the identification efficiency of carbon and glass fiber reinforced plies material parameters across various loading conditions. Intensive studies are conducted to compare convergence time between with and without TL through a numerical example under thermal-force coupling conditions. The comparison shows PINNs with TL obtain average error of 0.144% and 0.077% in carbon and glass fiber reinforced plies, which is better than without TL of 0.472% and 0.284%, respectively. The proposed work emphasizes the effective application of PINNs-based approach that combines first-order deformation and data-driven solution features for inverse solving material parameters of composite laminates.

Physics-informed neural network for creep-fatigue life prediction of Inconel 617 and interpretation of influencing factors

Providing a comprehensive assessment of the creep-fatigue life of critical structures in nuclear power facilities is important for structural integrity and performance requirements during service. The data-driven modeling method can set aside traditional mechanical theory and use experimental data to drive creep-fatigue performance modeling directly, avoiding complex parameter fitting. A sufficiently robust purely data-driven model must be supported by a large amount of training data, which means significant time and financial costs. To address these issues, a physics-informed neural network (PINN) is proposed in this work to predict the creep-fatigue life of Inconel 617 at high temperatures, where the physical constraints are introduced to enhance the physical fundamentals. The prediction results show that introduced physical constraints promote the stability of prediction performance and analysis based on the strength of physical constraints provides valuable guidance for creep-fatigue modeling to select the suitable architecture for building PINN. The SHapley Additive exPlanations (SHAP) method and dependency analysis are utilized to reveal the intrinsic mechanism of the properties and structure of PINN.

A study of mechanism-data hybrid-driven method for multibody system via physics-informed neural network

A study of mechanism-data hybrid-driven method for multibody system via physics-informed neural network

Physics-Informed Graph-Mesh Networks for PDEs: A hybrid approach for complex problems

The recent rise of deep learning has led to numerous applications, including solving partial differential equations using Physics-Informed Neural Networks. This approach has proven highly effective in several academic cases. However, their lack of physical invariances, coupled with other significant weaknesses, such as an inability to handle complex geometries or their lack of generalization capabilities, make them unable to compete with classical numerical solvers in industrial settings. In this work, a limitation regarding the use of automatic differentiation in the context of physics-informed learning is highlighted. A hybrid approach combining physics-informed graph neural networks with numerical kernels from finite elements is introduced. After studying the theoretical properties of our model, we apply it to complex geometries, in two and three dimensions. Our choices are supported by an ablation study, and we evaluate the generalization capacity of the proposed approach.

Physics-Informed Neural Networks for Prediction of a Flow-Induced Vibration Cylinder

Abstract Flow-induced vibration (FIV) is a common phenomenon in ocean engineering for subsea structures with circular-shaped cross sections. A large amount of computational resources or experimental efforts are required to predict or measure the complicated motions and flow field surrounding a circular cylinder undergoing vortex-induced vibration (VIV). Physics-informed neural networks (PINNs) are powerful deep learning techniques for solving governing partial differential equations (PDEs) of dynamic systems as an alternative to complex numerical methods. In the present study, a framework is built employing PINNs for solving the Navier–Stokes equations to predict flows past an FIV cylinder using sparsely distributed spatiotemporal data inside the domain. The training process involves minimizing the supervised loss of flow data at these sparse points and the residuals of the governing PDEs. For the training of the PINN model, a moving frame around an FIV cylinder is used to collect the training flow data from two-dimensional direct numerical simulation results at a low Reynolds number. The structural displacements of the cylinder are also implemented in the residuals of the equations of the developed PINN. The performance of the PINN is evaluated by comparing the predicted contours of the surrounding flow velocities with the training data. The hydrodynamic forces prediction is achieved using the PINN-obtained flow field predictions combined with the force partitioning method.

Traffic assignment optimization to improve urban air quality with the unified finite-volume physics-informed neural network

Emissions by road transportation constitute the major contributor of air pollutants to threat the public health, especially on mega cities with high-rise buildings and growing population. A practicable and economical remedy is assigning the traffic volume judiciously, termed traffic assignment optimization (TAO). Incorporating the computational fluid dynamics (CFD), this method provides precisely optimized traffic volumes under versatile meteorological conditions. However, the high-dimensional degree of freedoms (DoF) involved in CFD hampers its further application toward the large-scale problem where a massive urban area is considered. With the more complicated urban traffic network, the larger decision variable dimension also impedes the time-efficient acquisition of the optimization results. To alleviate this curse-of-dimensionality, the current work proposes an optimization framework with the surrogate model based on the unified finite-volume physics-informed neural networks (UFV-PINN). The UFV-PINN plays the dual role as the partial differential equation (PDE) solver and the surrogate model for pollutant concentration prediction. It reduces the DoF within the PDE solution process and enables the gradient-based optimization. The current work optimizes the average CO concentration and accumulative travel time in Kowloon peninsula of Hong Kong, using the multiple-gradient descent algorithm (MGDA). This optimization framework is tested with various working conditions. Results indicate that the proposed method attain high accuracy solutions, comparable to the heuristic algorithms. This study is the first attempt to incorporate UFV-PINN into the TAO with air quality consideration. Endowed with the differentiability by UFV-PINN, the framework will facilitate the TAO to provide precise suggestions toward large-scale problems.

A Meshless Solver for Blood Flow Simulations in Elastic Vessels Using a Physics-Informed Neural Network

A Meshless Solver for Blood Flow Simulations in Elastic Vessels Using a Physics-Informed Neural Network

Tutorials: Physics-informed machine learning methods of computing 1D phase-field models

Phase-field models are widely used to describe phase transitions and interface evolution in various scientific disciplines. In this Tutorial, we present two neural network methods for solving them. The first method is based on physics-informed neural networks (PINNs), which enforce the governing equations and boundary/initial conditions in the loss function. The second method is based on deep operator neural networks (DeepONets), which treat the neural network as an operator that maps the current state of the field variable to the next state. Both methods are demonstrated with the Allen–Cahn equation in one dimension, and the results are compared with the ground truth. This Tutorial also discusses the advantages and limitations of each method, as well as the potential extensions and improvements.

Higher-order multi-scale physics-informed neural network (HOMS-PINN) method and its convergence analysis for solving elastic problems of authentic composite materials

The limitations of prohibitive computation and Frequency Principle remain difficult for deep learning methods to effectively resolve multi-scale problems. In this work, a novel higher-order multi-scale physics-informed neural network (HOMS-PINN) framework is developed to compute the elastic problems of authentic composite materials with strongly discontinuous and high-contrast material parameters, which inherits the advantages of higher-order multi-scale method and physics-informed neural network, and halves the computational expense remarkably in comparison with direct PINN simulation. In the numerical framework of HOMS-PINN method, two types of network structures including single neural networks and independent neural networks are designed to mesh-free solve lower-order and higher-order microscopic unit cell (UC) functions, and macroscopic homogenization equations of multi-scale composites, which are then assembled into higher-order multi-scale solutions for multi-scale elastic problems by using automatic differentiation technology. Moreover, transfer learning is introduced to extend HOMS-PINN framework accelerating simulation of multi-scale elastic problems for composite materials with high-contrast mechanical parameters. Furthermore, the error estimate of the proposed HOMS-PINN method is demonstrated under some assumptions. Finally, numerous numerical experiments including high-contrast, multi-inclusion and three-dimensional composites are presented to validate the accuracy and effectiveness of HOMS-PINN method, especially for accurately capturing the sharply oscillatory behaviors in micro-scale. This study offers a robust HOMS-PINN computational framework that enables the simulation and analysis of large-scale mechanical problems of authentic composite materials.

A comprehensive and FAIR comparison between MLP and KAN representations for differential equations and operator networks

Kolmogorov–Arnold Networks (KANs) were recently introduced as an alternative representation model to MLP. Herein, we employ KANs to construct physics-informed machine learning models (PIKANs) and deep operator models (DeepOKANs) for solving differential equations for forward and inverse problems. In particular, we compare them with physics-informed neural networks (PINNs) and deep operator networks (DeepONets), which are based on the standard MLP representation. We find that although the original KANs based on the B-splines parameterization lack accuracy and efficiency, modified versions based on low-order orthogonal polynomials have comparable performance to PINNs and DeepONet, although they still lack robustness as they may diverge for different random seeds or higher order orthogonal polynomials. We visualize their corresponding loss landscapes and analyze their learning dynamics using information bottleneck theory. Our study follows the FAIR principles so that other researchers can use our benchmarks to further advance this emerging topic.

Bending analysis of thin plates with variable stiffness resting on elastic foundation via a two-network strategy physics-informed neural network method

A physics-informed neural network method based on a two-network strategy is introduced to address the bending problem of thin plates with variable stiffness resting on an elastic foundation. This problem is governed by a fourth-order partial differential equation (PDE), and the use of a one-network strategy to solve the PDE directly may lead to convergence issues due to singular points. Following the principles of Kirchhoff plate theory, the governing PDE is equivalently transformed into four second-order PDEs. A two-network strategy is employed for solution. We present numerical examples under various load conditions, plate geometries, foundation types, thicknesses, and material properties. The obtained results are validated against finite element method (FEM) solutions and literature. Discussions on alternative network strategies were conducted, and this study reveals that the presented two-network strategy have proven to be effective and robust. It also facilitates easier imposition of boundary conditions.

A modified physics-informed neural network to fatigue life prediction of deck-rib double-side welded joints

Deck-rib welded joint is an important part of orthotropic steel decks, which are susceptible to fatigue cracks under various load cycles. Therefore, it is essential to use efficient predicting methods to assess the fatigue performance of deck-rib welded joints with new geometry and form in order to ensure a reliable operation of the bridge. A modified physics-informed neural network (PINN) is proposed for the fatigue life prediction of the novel deck-rib double-side welded joints. The modified PINN incorporates implicit physical model as an activation function into the hidden neurons, and explicit physical constraints into the loss function. Additionally, the influence of deck thickness is integrated using limited test sample data based on the observational approach. The 62 experimental data were taken from the literature to compare the performance of the artificial neural network (ANN), traditional PINN and the modified PINN. The results demonstrate that the modified PINN enhances the learning process of the neural network through the joint learning of physical knowledge by activation function and loss function. Furthermore, the fatigue life prediction of the deck-rib double-side welded joints under the modified PINN exhibits physical consistency and accuracy when compared with the traditional ANN and PINN.

Evaluating single multiplicative neuron models in physics-informed neural networks for differential equations

Machine learning is a prominent and highly effective field of study, renowned for its ability to yield favorable outcomes in estimation and classification tasks. Within this domain, artificial neural networks (ANNs) have emerged as one of the most powerful methodologies. Physics-informed neural networks (PINNs) have proven particularly adept at solving physics problems formulated as differential equations, incorporating boundary and initial conditions into the ANN’s loss function. However, a critical challenge in ANNs lies in determining the optimal architecture, encompassing the selection of the appropriate number of neurons and layers. Traditionally, the Single Multiplicative Neuron Model (SMNM) has been explored as a solution to this issue, utilizing a single neuron with a multiplication function in the hidden layer to enhance computational efficiency. This study initially aimed to apply the SMNM within the PINNs framework, targeting the differential equation y′-y=0\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$y'-y=0$$\\end{document} with boundary conditions y(0)=1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$y(0) = 1$$\\end{document} and y(1)=e\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$y(1) = e$$\\end{document}. Upon implementation, however, it was discovered that while the conventional SMNM approach was theorized to offer significant advantages, multiplicative aggregate function led to a failure in convergence. Consequently, we introduced a “mimic single multiplicative neuron model (mimic-SMNM)” employing an architecture with a single neuron, designed to simulate the SMNM’s conceptual advantages while ensuring convergence and computational efficiency. Comparative analysis revealed that the real-PINNs accurately solved the equation, the true SMNM failed to converge, and the mimic model was highlighted for its architectural simplicity and computational feasibility, directly implying it is faster and more efficient than real PINNs for the solution of simple differential equations. Furthermore, our findings demonstrated that our proposed mimic-SMNM model achieves a five-times increase in computational speed compared to real PINNs after 30,000 epochs.

The modified physics-informed neural network (PINN) method for the thermoelastic wave propagation analysis based on the Moore-Gibson-Thompson theory in porous materials

This paper presents novel contributions to both theory and solution methodology in AI-based analysis of solid mechanics. The physics-informed neural network (PINN) method is developed for thermoelastic wave propagation and Moore-Gibson-Thompson (MGT) coupled thermoelasticity analysis of porous media, a first in the field. The coupled thermoelasticity governing equations, based on the MGT heat conduction model, are derived for a porous half-space, with the thermal relaxation coefficient and strain relaxation factor being considered. Mechanical and thermal shock loading boundary conditions are imposed. The behavior of a magnesium-made porous body is analyzed using the PINN method, with highly accurate results being achieved for the system of coupled PDEs. An adaptive hyperparameter tuning approach, integrating a generalized subset design (GSD) and Bayesian optimization algorithm, is used to automatically select the optimal structure based on the L2 relative error. This hybrid methodology eliminates manual adjustment concerns. The proposed method is verified through a thorough comparison with the Lord-Shulman theory of coupled thermoelasticity. The strength of the methodology lies in its ability to operate without domain data, with only boundary and initial points being required. Four example sets are examined to demonstrate the capabilities of the modified PINN, and high-quality predictions of dimensionless fields’ variables over an extended time interval are obtained, confirming its extrapolation abilities.

Physical activation functions (PAFs): An approach for more efficient induction of physics into physics-informed neural networks (PINNs)

In recent years, the evolution of Physics-Informed Neural Networks (PINNs) has reduced the gap between Deep Learning (DL) based methods and analytical/numerical approaches in scientific computing. However, there are still complications in training PINNs and the optimal interleaving of physical models. In this work, we introduce the concept of Physical Activation Functions (PAFs), in which one can use generic AFs with their mathematical expression inherited from the physical description of the evaluated system, instead of solely usage of standard activation functions (AFs) such as tanh, and sigmoid for all the neurons. The expression of PAFs could be selected based on individual terms appearing in the analytical solution, the initial or boundary conditions of the PDE system, or a component in the composition-of-functions type solutions. The PAFs could be applied in NNs, either in explicit, or self-adaptive forms. In the explicit approach, the main activation function of the network is replaced by PAF in some of the neurons of the network. In the self-adaptive implementation approach, the relative impact of PAFs (compared to the base AF) for each neuron was determined automatically. We tested the performance of PAFs in both forward and inverse problems for several PDEs, such as 1D and 2D wave equations, the Advection-Convection equation, the 1D heterogeneous, and 2D diffusion equations, and the Laplace equation. The main advantage of PAFs, compared to using standard AFs, was the more efficient constraining and interleaving of PINNs with the physical phenomena and their underlying mathematical models. The added PAFs significantly improved the predictions of PINNs for the testing data that were out-of-training distribution. Furthermore, applying PAFs reduced the size of the PINNs by up to 75 % in different cases while maintaining the same accuracy. Also, the training process was improved by reducing the value of the total loss term by one to two orders of magnitude. Furthermore, it improved the precision of the calculated properties in the examined inverse problems, for both clean and noisy observational data. It can be concluded that using PAFs helps in generating PINNs with less complexity and more validity for longer ranges of prediction.