Physics-informed Neural Networks Research Articles

Adaptive Deep Fourier Residual method via overlapping domain decomposition

The Deep Fourier Residual (DFR) method is a specific type of variational physics-informed neural network (VPINN). It provides a robust neural network-based solution to partial differential equations (PDEs). The DFR strategy is based on minimizing the dual norm of the weak residual of a PDE, which is equivalent to minimizing the energy norm of the error. To compute the dual norm of the weak residual, the DFR method employs an orthonormal spectral basis of the test space, known for rectangles or cuboids for multiple function spaces.In this work, we extend the DFR method with ideas of traditional domain decomposition (DD). This enables two improvements: (a) to solve problems in more general polygonal domains and (b) to develop an adaptive refinement technique in the test space using a Döfler marking algorithm. In the former case, we retain the desirable equivalence between the employed loss function and the H1 error under non-restrictive assumptions, numerically demonstrating adherence to explicit bounds in the case of the L-shaped domain problem. In the latter, we show how refinement strategies lead to potentially significant improvements against a reference, classical DFR implementation with a test function space of significantly lower dimensionality, allowing us to better approximate singular solutions at a more reasonable computational cost.

Dynamically configured physics-informed neural network in topology optimization applications

The integration of physics-informed neural network (PINN) and topology optimization (TO) is an attractive issue because PINN can avoid the prohibitive data acquisition of solving forward problems compared to traditional machine learning. To enhance the efficiency of this integrated optimization framework, a dynamically configured PINN-based topology optimization (DCPINN-TO) method is proposed in conjunction with an active sampling strategy. The DCPINN comprises two sub-networks with distinct training costs, capable of dynamically adjusting the trainable parameters based on the optimization state of TO. Moreover, the active sampling strategy selectively samples collocations based on the pseudo-densities, which can significantly reduce training costs by decreasing the number of input collocations. Additionally, the Gaussian integral is used to calculate the strain energy of elements to decouple the mapping of the material at the collocations. The proposed method is extended to various scenarios, including those with high resolution, multiple loads, and displacement constraints. Its efficiency and generalization are validated by several illustrative examples. Furthermore, the accuracy of DCPINN versus finite element analysis-based TO (FEA-TO) was also investigated.

PINN-CHK: physics-informed neural network for high-fidelity prediction of early-age cement hydration kinetics

AbstractCement hydration kinetics, characterized by heat generation in early-age concrete, poses a modeling challenge. This work proposes a physics-informed neural network (PINN) named PINN-CHK designed for cement hydration kinetics, to predict early-age temperature rises in cement paste. PINN-CHK leverages data-driven solutions to craft a high-fidelity prediction model, encompassing material properties and maturity functions in cement hydration. Trained on heated cement paste data, it simultaneously fits experimental results and underlying physics, yielding a mesh-free simulation. Incorporating governing partial differential equations (PDEs), and initial and boundary conditions into its loss function, PINN-CHK architecture undergoes rigorous benchmark testing, demonstrating unparalleled predictive accuracy compared to conventional deep-learning methods. It excels in predicting complete temperature fields during spatial–temporal cement hydration, achieving a remarkable relative L2 error as low as 0.00341. PINN-CHK achieves exceptional convergence and accuracy with only 5% of the training data, ushering in a new era in this crucial field. This innovative approach bridges the gap between theory and practice, offering an attractive alternative to conventional finite element solvers for enhanced comprehension of cement hydration kinetics and concrete maturity and strength development in cement-based materials.

Physics-informed deep-learning applications to experimental fluid mechanics

High-resolution reconstruction of flow-field data from low-resolution and noisy measurements is of interest due to the prevalence of such problems in experimental fluid mechanics, where the measurement data are in general sparse, incomplete and noisy. Deep-learning approaches have been shown suitable for such super-resolution tasks. However, a high number of high-resolution examples is needed, which may not be available for many cases. Moreover, the obtained predictions may lack in complying with the physical principles, e.g. mass and momentum conservation. Physics-informed deep learning provides frameworks for integrating data and physical laws for learning. In this study, we apply physics-informed neural networks (PINNs) for super-resolution of flow-field data both in time and space from a limited set of noisy measurements without having any high-resolution reference data. Our objective is to obtain a continuous solution of the problem, providing a physically-consistent prediction at any point in the solution domain. We demonstrate the applicability of PINNs for the super-resolution of flow-field data in time and space through three canonical cases: Burgers’ equation, two-dimensional vortex shedding behind a circular cylinder and the minimal turbulent channel flow. The robustness of the models is also investigated by adding synthetic Gaussian noise. Furthermore, we show the capabilities of PINNs to improve the resolution and reduce the noise in a real experimental dataset consisting of hot-wire-anemometry measurements. Our results show the adequate capabilities of PINNs in the context of data augmentation for experiments in fluid mechanics.

A hierarchically normalized physics-informed neural network for solving differential equations: Application for solid mechanics problems

Physics-informed neural networks (PINNs) usually confront significant difficulties to accurately solve partial differential equations (PDEs) due to many pathologies caused by gradient failures during training process. In this paper, a gradient disease regarding the ill-conditioned loss function of the PINN methodology is intensively investigated in both theoretical and experimental aspects. Particularly, the regular PINN approaches can obtain good predictions for trivial problems but may fail to learn solutions when their PDE coefficients and/or physical domain sizes vary. Besides, the influences of neural network structure to the PINN training quality are inferred from the theoretical analyses and also confirmed by relating experiments, in which using large-width multi-layer perceptrons (MLPs) is proved to be beneficial in stabilizing PINN training process. To overcome the abovementioned restrictions, a novel PINN methodology using hierarchically normalized technique (hnPINN) motivated from the developed theory is devised. The key idea of the proposed hnPINN is, on the one hand, to transform the original PDE system into one of two proposed dimensionless forms to alleviate the negative effects of PDE coefficients and domain size; on the other hand, to use secondary output scalers to flexibly calibrate the gradient flow for training effectively and improving the solution preciseness. The determination of the secondary output scaler is formulated by a heuristic framework inspired from theoretical analyses on the hnPINN gradient flow. The obtained results from some typical PDEs and common problems in solid mechanics strongly confirm the high effectiveness of the hnPINN in terms of solution accuracy, superior convergence and performance stability in comparison to those of the vanilla PINN and the non-dimensionalized PINN (ndPINN). As a preprocessing procedure, the hnPINN method is independent on network architectures and also greatly potential to combine with other state-of-the-art PINN models for effectively solving challenged issues in practice.

Hybrid quantum physics-informed neural networks for simulating computational fluid dynamics in complex shapes

Abstract Finding the distribution of the velocities and pressures of a fluid by solving the Navier-Stokes equations is a principal task in the chemical, energy, and pharmaceutical industries, as well as in mechanical engineering and the design of pipeline systems. With existing solvers, such as OpenFOAM and Ansys, simulations of fluid dynamics in intricate geometries are computationally expensive and require re-simulation whenever the geometric parameters or the initial and boundary conditions are altered. Physics-informed neural networks are a promising tool for simulating fluid flows in complex geometries, as they can adapt to changes in the geometry and mesh definitions, allowing for generalization across fluid parameters and transfer learning across different shapes. We present a hybrid quantum physics-informed neural network that simulates laminar fluid flows in 3D $Y$-shaped mixers. Our approach combines the expressive power of a quantum model with the flexibility of a physics-informed neural network, resulting in a 21\% higher accuracy compared to a purely classical neural network. Our findings highlight the potential of machine learning approaches, and in particular hybrid quantum physics-informed neural network, for complex shape optimization tasks in computational fluid dynamics. By improving the accuracy of fluid simulations in complex geometries, our research using hybrid quantum models contributes to the development of more efficient and reliable fluid dynamics solvers.

Combining Empirical and Physics-Based Models for Solar Wind Prediction

Solar wind modeling is classified into two main types: empirical models and physics-based models, each designed to forecast solar wind properties in various regions of the heliosphere. Empirical models, which are cost-effective, have demonstrated significant accuracy in predicting solar wind at the L1 Lagrange point. On the other hand, physics-based models rely on magnetohydrodynamics (MHD) principles and demand more computational resources. In this research paper, we build upon our recent novel approach that merges empirical and physics-based models. Our recent proposal involves the creation of a new physics-informed neural network that leverages time series data from solar wind predictors to enhance solar wind prediction. This innovative method aims to combine the strengths of both modeling approaches to achieve more accurate and efficient solar wind predictions. In this work, we show the variability of the proposed physics-informed loss across multiple deep learning models. We also study the effect of training the models on different solar cycles on the model’s performance. This work represents the first effort to predict solar wind by integrating deep learning approaches with physics constraints and analyzing the results across three solar cycles. Our findings demonstrate the superiority of our physics-constrained model over other unconstrained deep learning predictive models.

Intelligent vehicle trajectory tracking control based on physics-informed neural network dynamics model

In order to solve the accuracy problem of trajectory tracking control method based on data-driven model, an intelligent vehicle trajectory tracking control method based on physics-informed neural network (PINN) vehicle dynamics model is proposed. Aiming at the problem of poor interpretability of data-driven model, a vehicle dynamics model based on the PINN is established, and the physics-driven deep learning method is used instead of the data-driven deep learning method to obtain the dynamic characteristics of the intelligent vehicle, to benefit from both the physical-based method and the data-driven method. A sequential training method is also proposed to solve the coupling problem when training multiple PINNs simultaneously. The model takes the nonlinearity of the neural network model and physical interpretability into consideration compared to the standard neural network model. Then, based on the PINN vehicle dynamics model, a trajectory tracking controller based on the iterative linear quadratic regulator (ILQR) control algorithm is developed. The optimal control law is derived by optimizing the ILQR control algorithm to implement the intelligent vehicle’s precise and stable tracking for the desired trajectory. The Levenberg-Marquardt (LM) algorithm and line search technology are used and damping factor adjustment rules are set up to enhance the convergence performance of the ILQR control algorithm. In order to verify the effectiveness of the proposed method, the simulation is conducted under the condition of double lane change. The simulation results demonstrate that the proposed method can track the reference trajectory accurately under the limited conditions. Its control performance is much better than other algorithms.

Railway safety through predictive vertical displacement analysis using the PINN-EKF synergy

Scientific Machine Learning (SciML) finds extensive application in daily life, industry, and scientific research. Specifically, in railway data analysis, it utilizes tools such as time series analysis, classification, and data visualization. Among these, monitoring vertical displacement, or the movement of railway components relative to a fixed point, is vital. This indicates changes in the elevation of railway infrastructure, highlighting track settlement and structural shifts. However, relying solely on single sensor measurements for these displacement calculations can introduce inaccuracies. To overcome this, sensor fusion techniques are employed. These techniques employ advanced algorithms to combine data from multiple sensors, thereby enhancing accuracy. They consider the individual characteristics of each sensor, effectively mitigating the limitations of any single sensor. This study introduces a hybrid approach that combines the Extended Kalman Filter (EKF) with the Physics-Informed Neural Network (PINN) to refine predictive data analytics in dynamic railway environments. The experimental results underscore the efficacy of this innovative methodology.

Static analysis of two-side supported 2-ply laminated glass panes through physics-informed neural networks

This paper utilizes an emerging machine learning technique, namely physics-informed neural networks (PINNs), to analyze the static mechanical behavior of two-side supported 2-ply laminated glass panes. The PINN model utilizes prior physical information in the form of partial differential equations during model training, eliminating the need for tremendous amounts of data required by traditional data-driven machine learning methods. The governing equations for the structural behavior of two-side supported 2-ply laminated glass panes are derived, based on which a PINN framework encompassing the neural network architecture and the loss functions is constructed. The training stability is evidently improved after the linearly inducing process, with the relative error maintained within a reasonable range. Extensive validation examples and case studies are provided to demonstrate the accuracy and effectiveness of the proposed method. Composite action analysis results reveal that below 70 % in the design is a conservative estimate for thinner laminated glass. Furthermore, the PINN method achieves exceptional computational efficiency, reducing computational times from six hours to just a few minutes for the uncertainty evaluation of laminated glass, indicating its potential in engineering practice.

Neural-network-designed three-qubit gates robust against charge noise and crosstalk in silicon

Spin qubits in semiconductor quantum dots are a promising platform for quantum computing, however, scaling to large systems is hampered by crosstalk and charge noise. Crosstalk here refers to the unwanted off-resonant rotation of idle qubits during the resonant rotation of the target qubit. For a three-qubit system with crosstalk and charge noise, it is difficult to analytically create gate protocols that produce three-qubit gates, such as the Toffoli gate, directly in a single shot instead of through the composition of two-qubit gates. Therefore, we numerically optimize a physics-informed neural network to produce theoretically robust shaped pulses that generate a Toffoli-equivalent gate. Additionally, robust π2 X and Controlled-Z gates are also presented in this work to create a universal set of gates robust against charge noise. The robust pulses maintain an infidelity of 10−3 for average quasistatic fluctuations in the voltage of up to a few mV instead of tenths of mV for non-robust pulses.

A novel normalized reduced-order physics-informed neural network for solving inverse problems

A novel normalized reduced-order physics-informed neural network for solving inverse problems

Unified finite-volume physics informed neural networks to solve the heterogeneous partial differential equations

The emerging physics informed neural network (PINN) has been recently applied to a wide range of mathematical problems. It is promising to precisely solve the partial differential equations (PDEs) in a fast, flexible manner. Whereas, PINN struggles with poor accuracy and costly computation in case of heterogeneous PDE coefficients. To mitigate these issues, a new PINN, which is known as the unified finite volume PINN (UFV-PINN), is proposed to unify the sub-domain decomposition, finite volume discretization, and conventional numerical solvers. The output by neural network (NN) over the boundaries of agglomerated sub-domains functions as boundary conditions (BCs) for UFV-PINN training. In this connection, the customized differentiable conventional numerical solver further solves the PDEs. The discrepancy between NN prediction and the conventional numerical solution within the sub-domains is taken as the novel training loss, enforcing the conservation law of PDE. For illustration, the Poisson and advection–diffusion equations (ADE) are solved, which are classical but still challenging to PINN in the presence of heterogeneity. Numerical experiments are conducted to compare the performance of the proposed UFV-PINN and the standard PINN, both in terms of accuracy and efficiency. Results indicate that UFV-PINN attains remarkable accuracy improvement with less computation time. Hessian spectrum analysis indicates that the loss Hessian matrix of UFV-PINN is more inclined to be positive definite, highlighting its superior performance over the standard PINN. It is the first time that the FV numerical solver is seamlessly embedded into the PINN training for performance improvement.

Physics-informed neural networks for acoustic boundary admittance estimation

Acoustic simulations often face significant uncertainties due to limited knowledge of acoustic boundary conditions. While measuring the boundary admittance in situ is challenging in practical applications, numerical inverse methods can be used to characterize the boundary conditions based on sound pressure data. However, conventional inverse methods require a validated forward model and can become impractical for computationally expensive simulation models. Over the past years, machine learning approaches have emerged as promising methods for scientific computing and data-driven modeling. Physics-informed neural networks incorporate physical prior knowledge into a neural network by adding the residual of the underlying partial differential equation to the loss function. Training the neural network minimizes the loss function, allowing the network to learn a solution that not only fits the training data but also satisfies the corresponding boundary value problem. In this study, physics-informed neural networks are trained to learn the sound pressure solution within two numerical examples governed by the Helmholtz equation without explicitly specifying the boundary conditions at selected boundaries. After training, the neural network’s prediction of the boundary admittance is evaluated and compared to the ground truth, initially assigned in the finite element reference solution. Additionally, the proposed method is validated using experimental data obtained from an acoustic impedance tube measurement. The results show that physics-informed neural networks can accurately learn the sound pressure field and implicitly solve the inverse problem by providing an accurate estimate of the underlying boundary admittance, even in the case of spatially varying boundary conditions.

GRINN: a physics-informed neural network for solving hydrodynamic systems in the presence of self-gravity

Modeling self-gravitating gas flows is essential to answering many fundamental questions in astrophysics. This spans many topics including planet-forming disks, star-forming clouds, galaxy formation, and the development of large-scale structures in the Universe. However, the nonlinear interaction between gravity and fluid dynamics offers a formidable challenge to solving the resulting time-dependent partial differential equations (PDEs) in three dimensions (3D). By leveraging the universal approximation capabilities of a neural network within a mesh-free framework, physics informed neural networks (PINNs) offer a new way of addressing this challenge. We introduce the gravity-informed neural network (GRINN), a PINN-based code, to simulate 3D self-gravitating hydrodynamic systems. Here, we specifically study gravitational instability and wave propagation in an isothermal gas. Our results match a linear analytic solution to within 1% in the linear regime and a conventional grid code solution to within 5% as the disturbance grows into the nonlinear regime. We find that the computation time of the GRINN does not scale with the number of dimensions. This is in contrast to the scaling of the grid-based code for the hydrodynamic and self-gravity calculations as the number of dimensions is increased. Our results show that the GRINN computation time is longer than the grid code in one- and two- dimensional calculations but is an order of magnitude lesser than the grid code in 3D with similar accuracy. Physics-informed neural networks like GRINN thus show promise for advancing our ability to model 3D astrophysical flows.

Research on model predictive control of autonomous underwater vehicle based on physics informed neural network modeling

In the rapidly evolving field of Autonomous Underwater Vehicles (AUVs), achieving precise control remains a critical endeavor. This study presents a pioneering integration of Model Predictive Control (MPC) with a Physics-Informed Neural Network (PINN), aiming to enhance control system precision and operational efficiency in AUVs. The efficacy of MPC lies in its adept handling of the intricate constraints and inherent nonlinear dynamics intrinsic to AUV systems. Concurrently, the PINN architecture incorporates the fundamental physical laws represented by Partial Differential Equations (PDEs), augmenting the predictive fidelity of the system. Firstly, this research implements the novel PINN-enhanced MPC framework for trajectory tracking and conducts a comparative evaluation against adaptive proportional-integral-derivative (PID) and Gaussian-process-based MPC controllers. This comparative analysis elucidates the advancements in control mechanisms attributable to the PINN integration. Furthermore, this study meticulously assesses the PINN-MPC's proficiency in navigating through static and dynamic obstacles within three-dimensional marine environments, a critical capability for AUV operations. Through extensive and meticulous simulations, the proposed approach demonstrates notable progress in overcoming environmental challenges and executing intricate operational tasks, such as obstacle avoidance, with heightened efficiency and dexterity. This research constitutes a substantial contribution to the theoretical advancement and elucidation of control systems in the AUV domain, bearing profound practical implications. It lays the foundation for the development of increasingly sophisticated, advanced, and reliable AUV missions, signifying a crucial advancement in the realms of underwater exploration and operational technology.

Characterization of partial wetting by CMAS droplets using multiphase many-body dissipative particle dynamics and data-driven discovery based on PINNs

The molten sand that is a mixture of calcia, magnesia, alumina and silicate, known as CMAS, is characterized by its high viscosity, density and surface tension. The unique properties of CMAS make it a challenging material to deal with in high-temperature applications, requiring innovative solutions and materials to prevent its buildup and damage to critical equipment. Here, we use multiphase many-body dissipative particle dynamics simulations to study the wetting dynamics of highly viscous molten CMAS droplets. The simulations are performed in three dimensions, with varying initial droplet sizes and equilibrium contact angles. We propose a parametric ordinary differential equation (ODE) that captures the spreading radius behaviour of the CMAS droplets. The ODE parameters are then identified based on the physics-informed neural network (PINN) framework. Subsequently, the closed-form dependency of parameter values found by the PINN on the initial radii and contact angles are given using symbolic regression. Finally, we employ Bayesian PINNs (B-PINNs) to assess and quantify the uncertainty associated with the discovered parameters. In brief, this study provides insight into spreading dynamics of CMAS droplets by fusing simple parametric ODE modelling and state-of-the-art machine-learning techniques.

Using Physics-Informed Neural Networks (PINNs) for Tumor Cell Growth Modeling

Using Physics-Informed Neural Networks (PINNs) for Tumor Cell Growth Modeling

Efficient Bayesian Physics Informed Neural Networks for inverse problems via Ensemble Kalman Inversion

Bayesian Physics Informed Neural Networks (B-PINNs) have gained significant attention for inferring physical parameters and learning the forward solutions for problems based on partial differential equations. However, the overparameterized nature of neural networks poses a computational challenge for high-dimensional posterior inference. Existing inference approaches, such as particle-based or variance inference methods, are either computationally expensive for high-dimensional posterior inference or provide unsatisfactory uncertainty estimates. In this paper, we present a new efficient inference algorithm for B-PINNs that uses Ensemble Kalman Inversion (EKI) for high-dimensional inference tasks. By reframing the setup of B-PINNs as a traditional Bayesian inverse problem, we can take advantage of EKI's key features: (1) gradient-free, (2) computational complexity scales linearly with the dimension of the parameter spaces, and (3) rapid convergence with typically O(100) iterations. We demonstrate the applicability and performance of the proposed method through various types of numerical examples. We find that our proposed method can achieve inference results with informative uncertainty estimates comparable to Hamiltonian Monte Carlo (HMC)-based B-PINNs with a much reduced computational cost. These findings suggest that our proposed approach has great potential for uncertainty quantification in physics-informed machine learning for practical applications.

SPINNED: Simulation-based physics-informed neural network for deconvolution of dynamic susceptibility contrast MRI perfusion data.

To propose the simulation-based physics-informed neural network for deconvolution of dynamic susceptibility contrast (DSC) MRI (SPINNED) as an alternative for more robust and accurate deconvolution compared to existing methods. The SPINNED method was developed by generating synthetic tissue residue functions and arterial input functions through mathematical simulations and by using them to create synthetic DSC MRI time series. The SPINNED model was trained using these simulated data to learn the underlying physical relation (deconvolution) between the DSC-MRI time series and the arterial input functions. The accuracy and robustness of the proposed SPINNED method were assessed by comparing it with two common deconvolution methods in DSC MRI data analysis, circulant singular value decomposition, and Volterra singular value decomposition, using both simulation data and real patient data. The proposed SPINNED method was more accurate than the conventional methods across all SNR levels and showed better robustness against noise in both simulation and real patient data. The SPINNED method also showed much faster processing speed than the conventional methods. These results support that the proposed SPINNED method can be a good alternative to the existing methods for resolving the deconvolution problem in DSC MRI. The proposed method does not require any separate ground-truth measurement for training and offers additional benefits of quick processing time and coverage of diverse clinical scenarios. Consequently, it will contribute to more reliable, accurate, and rapid diagnoses in clinical applications compared with the previous methods including those based on supervised learning.