Physics-informed Neural Networks Research Articles

Modeling unobserved geothermal structures using a physics-informed neural network with transfer learning of prior knowledge

Deep learning has gained attention as a potentially powerful technique for modeling natural-state geothermal systems; however, its physical validity and prediction inaccuracy at extrapolation ranges are limiting. This study proposes the use of transfer learning in physics-informed neural networks to leverage prior expert knowledge at the target site and satisfy conservation laws for predicting natural-state quantities such as temperature, pressure, and permeability. A neural network pre-trained with multiple numerical datasets of natural-state geothermal systems was generated using numerical reservoir simulations based on uncertainties of the permeabilities, sizes, and locations of geological units. Observed well logs were then used for tuning by transfer learning of the network. Two synthetic datasets were examined using the proposed framework. Our results demonstrate that the use of transfer learning significantly improves the prediction accuracy in extrapolation regions with no observed wells.

Physics-guided neural network-based framework for 3D modeling of slope stability

Physics-informed neural networks (PINN) have gradually attracted attention in the field of geotechnical engineering. This paper proposes a novel PINN-based framework for the three-dimensional (3D) stability analysis of soil slopes. Based on the fundamental theorem of plasticity and limit analysis, the partial differential equations (PDE) with regard to slope collapse are derived and integrated into the physics-guided loss function. A kinematically admissible failure mechanism that rigorously satisfies the Mohr-Coulomb associated flow rule is obtained by minimizing the loss function, thereby circumventing complex mathematical calculations. The discrete points generated by PINN are selected and refined through a series of procedures to represent the failure block of slopes using a two-dimensional matrix. The entire training process of the PINN-based framework is conducted without the need for any labeled data. The resulting discretized failure mechanism is meshfree and capable of accommodating spatially discrete data. A validation exercise is performed to verify the proposed framework by comparing it with the classical 3D rotational failure mechanism. To further consider the impact of external excitation on slope stability, a hybrid PINN framework is developed to assess the stability of slopes subjected to complex external environments. In addition to the PINN to generate a failure mechanism, a parallel PINN is employed to acquire the corresponding spatially discrete data of specified external excitations. The hybrid PINN framework for seismic stability assessment of slopes is demonstrated by way of example, indicating favorable feasibility and applicability of the developed approach. The proposed PINN-based framework provides innovative and promising avenues for 3D slope stability analysis.

Three-dimensional spatiotemporal wind field reconstruction based on LiDAR and multi-scale PINN

In this numerical study, we use a multi-scale version of physics-informed neural network (PINN) for wind field reconstruction from LiDAR measurements. The reference velocity field is obtained from high-fidelity large-eddy simulation of neutral atmospheric boundary layer, and the LiDAR measurement strategy is restricted to that of a real LiDAR. The multi-scale PINN reconstructs the velocity field by minimizing a combination of the L2-error with respect to the LiDAR measurements and residuals of the governing equations. Our most significant finding is that adding a multi-scale layer to PINN enables us to: (i) capture wider range of scales, (ii) reconstruct the flow field outside the LiDAR scanning area, and (iii) reach faster convergence. We also find that for volumetric flow reconstruction and to deal with uncertainty in the wind direction, the reconstruction accuracy can be greatly improved by employing multiple LiDAR devices. Overall, our results show that the normalized errors in the reconstructed wind field are lower than 5% for all the experiments conducted in this study and that the errors do not increase even in the presence of measurement noise levels of typical LiDAR. These findings show the feasibility of three-dimensional dynamic wind field reconstruction across large wind farms using LiDAR devices.

Thermodynamics-Informed Neural Networks for the Design of Solar Collectors: An Application on Water Heating in the Highland Areas of the Andes

This study addresses the challenge of optimizing flat-plate solar collector design, traditionally reliant on trial-and-error and simplified engineering design methods. We propose using physics-informed neural networks (PINNs) to predict optimal design conditions in a range of data that not only characterized the highlands of Ecuador but also similar geographical locations. The model integrates three interconnected neural networks to predict global collector efficiency by considering atmospheric, geometric, and physical variables, including overall loss coefficient, efficiency factors, outlet fluid temperature, and useful heat gain. The PINNs model surpasses traditional simplified thermodynamic equations employed in engineering design by effectively integrating thermodynamic principles with data-driven insights, offering more accurate modeling of nonlinear phenomena. This approach enhances the precision of solar collector performance predictions, making it particularly valuable for optimizing designs in Ecuador’s highlands and similar regions with unique climatic conditions. The ANN predicted a collector overall loss coefficient of 5.199 W/(m2·K), closely matching the thermodynamic model’s 5.189 W/(m2·K), with similar accuracy in collector useful energy gain (722.85 W) and global collector efficiency (33.68%). Although the PINNs model showed minor discrepancies in certain parameters, it outperformed traditional methods in capturing the complex, nonlinear behavior of the data set, especially in predicting outlet fluid temperature (55.05 °C vs. 67.22 °C).

Enhanced physics-informed neural networks without labeled data for weakly and fully coupled transient thermomechanical analysis

In this paper, a series of enhanced physics-informed neural networks (PINN) models without labeled data is proposed to solve the weakly and fully coupled thermomechanical problems. In these models, to better predict the thermal and mechanical responses, PINNs consisting of different deep neural networks (DNN) representing temperature, displacement, and stress are specifically constructed. Furthermore, to elevate the accuracy and avoid possible training failure, several advanced algorithms are developed to ensure the effectiveness of imposing boundary conditions, refining sampling distributions, and enhancing training strategy. A notable aspect of the enhanced PINNs is their independence from expensive, labeled data, relying solely on the temporal and spatial information embedded within the sampling points. The effectiveness and accuracy of the enhanced PINNs are validated through extensive numerical examples, including heat conduction and both weakly and fully coupled thermomechanical problems. The comparation between original PINN and enhanced PINN illustrates the necessity of involving these enhanced methods. The results demonstrate the significant potential of PINN methodologies in engineering areas involving complex thermomechanical processes.

Physics-informed neural networks for efficient aeroacoustic computational based on two time sequence prediction models

Physics-informed neural networks (PINNs) have been proven powerful for solving partial differential equations (PDEs). However, conventional PINNs fail handling time/space variables in the training data distinctly and overlook the temporal relationships among data points, which is critical for accurately modeling dynamic systems. To address this issue, this work proposes two novel PINN models based on the transformer and the convolutional long short-term memory (ConvLSTM) architectures, respectively. An encoder-decoder neural network structure is adopted to enhance handling of time-space variables and a high-order finite-difference method is used to compute partial derivatives. These computations are implemented using the convolution within the PyTorch framework that incorporates both physical and mathematical information into the neural networks. The efficacy of the proposed methods has been evaluated for the wave equation and the Reynolds-Averaged Navier-Stokes equations. Results suggest that the transformer-based PINN outperforms both the conventional and ConvLSTM-based PINN in terms of accuracy, efficiency and generalizability.

Reconstruction of directional sources using physics-informed neural networks

In the context of augmented/virtual reality applications, replicating the directivity of sound sources is a critical step to ensure a high-quality immersive experience. However, accurate directivity measurements require complex experimental procedures involving a high number of microphones. In these scenarios, spatial interpolation allows to reconstruct the sound field with fewer sensors, which can be sparsely distributed around the source. The literature offers several interpolation schemes for sound field reconstruction, although they exhibit limited accuracy due to the lack of knowledge on the underlying physics. Recently, Physics-Informed Neural Networks (PINN) have been used to directly solve partial differential equations and thus better predict the evolution of dynamical systems. In this manuscript, we propose a spatial interpolation scheme based on PINN for the reconstruction of sound source directivity. PINN are used to solve the Helmholtz equation starting from acquisitions of the directivity over a limited set of sparsely distributed points. Results are compared with the outcomes of well-established methods based on Spherical Harmonics Decomposition (SH), Thin Plate Pseudo-Splines Interpolation (SPLI) and Piece-wise Linear Spherical Triangular Interpolation (TRI). PINN prove to provide better reconstruction of the directivity with respect to the proposed comparison methods, even when dealing with highly sparse sampling grids.

Toward mechanistic models “augmented” by machine learning. Example of a drying simulation data set exploited by a Physics Informed Neural Network

Mechanistic modeling of drying is well-established since several decades. Based on fundamental balance equations and driven by relevant material parameters, it can predict the entire process, including configurations that were not observed before. Besides, thanks to their ability to tackle non-linear and dynamics problems, approaches based on Machine Learning (ML) based are capable of coping with complex situations even better than mechanistic modeling. The main drawback of mechanistic models is their complexity as operational tools, namely in providing the whole set of product characteristics, while the main drawback of ML tools is its restriction to the domain paved by the data set. This paper summarizes the physics of the mechanistic formulation and then presents the different possibilities of coupling the mechanistic and ML approaches to obtain an “augmented” mechanistic model. The idea is to merge the advantages of both worlds. Different strategies can be imagined: A full coupled method (hybrid model) A fully decoupled method A cascade coupling The second part of the paper gives examples of a fully decoupled approach the mechanistic model is used to populate a data set, which is further exploited by a neural network. The originality of the work is to compare a classical neural network with a Physics Informed Neural Network.

Optical interferometry based acoustic field characterization using physics informed neural networks

The acousto-optic effect offers a non-contact way of measuring and visualizing acoustic fields in transparent fluids. Acoustic density fluctuations induce changes in the media's refractive index that can be measured using interferometry. While sensing methods based on acousto-optics have gathered increasing attention, many of such methods are only sensitive to projections of the acoustic field along the optical path, limiting the fields that can be characterized to one or two dimensions, or requiring complex experimental setups. Recently, reconstruction methods that abide by the physics of acoustic wave propagation showed unparalleled performance compared with classical reconstruction methods, reducing the sampling requirements and enabling to characterize three-dimensional acoustic fields. Nonetheless, such reconstruction methods rely on discrete approximations which might be poor at describing spatially complex fields. In this work we use physics-informed neural networks (PINNs) to estimate the acoustic pressure over space from interferometer measurements. The network is flexible enough to represent complex acoustic fields, while its output is constrained to fulfill the wave equation. Once trained on the data available for a given experiment, the PINN can estimate the acoustic pressure at arbitrary positions. The approach is tested in an experimental study, showing an improved performance compared with state-of-the-art reconstruction methods.

Comparison of standard Bayesian methods and PINNs for the reconstruction of acoustic fields in 'in situ' combustion chambers

Acoustic measurements, utilizing sensors such as microphones strategically placed, are commonly employed to monitor acoustic systems and extract quantities of interest, such as acoustic reflection coefficients. Typically, a specialized test rig must be constructed to allow for external harmonic acoustic forcing via loudspeakers. In this study, we propose two different methods: physics-informed neural networks (PINNs) and the extended Kalman filter, to reconstruct the acoustic field of a system and assess the acoustic reflection coefficient at a defined boundary. We demonstrate numerically that a single external broadband forcing signal suffices for these methods. This implies that these methods can be applied to 'in situ' configurations, such as industrial combustion chambers, where the broadband nature of combustion noise serves as the acoustic forcing element. Furthermore, we investigate the performance of the two aforementioned methods in reconstructing the acoustic systems and corresponding acoustic boundaries.

Neural Impedance Boundary (NeIB): a neural-network based framework for acoustic surface impedance estimation utilizing sparse measurement data

Amidst recent advancements in the 3D digital representation that have significantly enhanced the modeling of geometric attributes of pre-existing environments, accurate estimation of acoustic boundary conditions remains a complex challenge. This paper presents a novel way to determine what we refer to as a neural boundary field, using physics-informed neural networks (PINN). The aim is to estimate the surface impedance measured in-situ by utilizing several points of sound field pressure. This approach couples the Helmholtz equation with automatic differentiation in the PINN framework to estimate accurately the surface impedance using a hybrid modeling approach where measurement data and domain knowledge in the form of equations for the acoustic waves are combined. As a proof-of-concept, we train the neural networks using 2D sound field data obtained from high-fidelity numerical acoustical simulations that incorporate actual surface materials parameters. We discuss the measurement techniques associated with this method and outline our vision for its application to 3D scene reconstructions in the future.

Physics-informed neural networks for localized assessment in Euler-Bernoulli beams

This study employs the Physics-Informed Neural Networks (PINNs) to address an inverse problem, specifically evaluating local changes in material properties within an Euler-Bernoulli beam through numerical simulations. In our numerical model, a viscoelastic material is strategically introduced to a localized zone of the beam, inducing non-constant and complex alterations in the storage modulus. The training process incorporates the partial differential equation into the neural network's loss function, coupled with essential stability conditions for the machine learning model, enhancing the efficiency of the training process. Notably, this methodology excels in characterizing structural behavior influenced by these intricate local changes in material properties, bypassing the need to identify the entire structure and consequently to deal with uncertainties related to boundary conditions. The effectiveness of the approach is validated through numerical simulations, underscoring its potential for applications in efficiently assessing and understanding local material alterations within complex structural systems.

Predicting fatigue life of multi-defect materials using the fracture mechanics-based physics-informed neural network framework

To address the limitations of traditional methods (such as the S-N curves and Paris’s law) in evaluating the fatigue life of multi-defect materials, this study developed a fracture mechanics-based physics-informed neural network (PINN) to predict the lifetime of multi-defect materials using cyclic loading (Δσ) and the equivalent damage area (AD). Influences of multiple defects were unified characterized through the equivalent damage area, which was calculated based on the M−integral fatigue model. This model reflects the energy evolution of multi-defect fatigue damage. By embedding the prior knowledge of fracture mechanics derived from the M−integral fatigue model into the loss function of PINN, crucial physical information was captured during the training progress, enhancing the interpretability of the neural network. By integrating the advantage of the M−integral fatigue model in characterizing the fatigue performance of multi-defect materials and the nonlinear fitting ability of neural networks, the proposed approach effectively improves the generalization ability and predictive accuracy of limited fatigue data. The presented PINN models accurately forecast the fatigue life of multi-defect materials, with a squared correlation coefficient (R2) exceeding 0.9. The presented methodological framework addresses the existing gap in methods for evaluating the fatigue performance of multi-defect materials and reliance on fatigue testing.

Physics-informed deep learning for structural dynamics under moving load

Physics-informed deep learning has emerged as a promising approach that incorporates physical constraints into the model, reduces the amount of data required, and demonstrates robustness and potential in dealing with limited datasets for a variety of studies. However, several key challenges still exist, with one being the spectral bias problem of deep learning in the simulation of functions with multi-frequency features. To overcome the challenge, this study proposes a novel physics-informed deep learning method, which integrates physics-informed neural network with Fourier transform so as to solve partial differential equations in the frequency domain, thus alleviating the problem of spectral bias of neural networks in the simulation of multi-frequency functions. In addition, the proposed method is used to focus on the forward simulation and parameter inverse identification issues in structural dynamics under moving loads. To illustrate the superiority of the method, the issues of dynamic response of simply supported beams under moving loads are presented as case studies, and the performance of the method in multiple cases is analysed and discussed. The research results demonstrate the feasibility and effectiveness of the method for structural dynamics simulation and parameter inverse identifications using limited datasets.

Assessing physics-informed neural network performance with sparse noisy velocity data

The utilization of data in physics-informed neural network (PINN) may be considered as a necessity as it allows the simulation of more complex cases with a significantly lower computational cost. However, doing so would also make it prone to any issue with the data quality, including its noise. This study would primarily focus on developing a special loss function in the PINN to allow an effective utilization of noisy data. However, a study regarding the data location and amount was also conducted in order to allow a better data utilization in PINN. This study was conducted on a lid-driven cavity flow at Re = 200, 1000, and 5000 with a dataset of less than 100 velocity data and a maximum noise of 10% of the maximum velocity. The results show that by ensuring the data are distributed in a certain configuration, it has zero noise, and by using as much data as possible, the computational cost of PINN can be significantly reduced compared to without using any data at all. For Re = 200, it is 7.4 faster by using data, and this speedup is potentially higher for higher Re cases. For the noise in particular, it does not only make the PINN more inaccurate but also necessitate the usage of more data as this is the only way to make it more accurate. This issue though is capable to be solved with our new method, which only uses the data as an approximate solution, and the governing equation would figure out the details. This method was also shown to be capable to improve the PINN accuracy with the potential to almost completely eliminating the noise effect.

A comprehensive review of advances in physics-informed neural networks and their applications in complex fluid dynamics

Physics-informed neural networks (PINNs) represent an emerging computational paradigm that incorporates observed data patterns and the fundamental physical laws of a given problem domain. This approach provides significant advantages in addressing diverse difficulties in the field of complex fluid dynamics. We thoroughly investigated the design of the model architecture, the optimization of the convergence rate, and the development of computational modules for PINNs. However, efficiently and accurately utilizing PINNs to resolve complex fluid dynamics problems remain an enormous barrier. For instance, rapidly deriving surrogate models for turbulence from known data and accurately characterizing flow details in multiphase flow fields present substantial difficulties. Additionally, the prediction of parameters in multi-physics coupled models, achieving balance across all scales in multiscale modeling, and developing standardized test sets encompassing complex fluid dynamic problems are urgent technical breakthroughs needed. This paper discusses the latest advancements in PINNs and their potential applications in complex fluid dynamics, including turbulence, multiphase flows, multi-field coupled flows, and multiscale flows. Furthermore, we analyze the challenges that PINNs face in addressing these fluid dynamics problems and outline future trends in their growth. Our objective is to enhance the integration of deep learning and complex fluid dynamics, facilitating the resolution of more realistic and complex flow problems.

Physics-informed neural networks coupled with flamelet/progress variable model for solving combustion physics considering detailed reaction mechanism

In recent years, physics-informed neural networks (PINNs) have shown potential as a method for solving combustion physics. However, current efforts using PINNs for the direct predictions of multi-dimensional flames only use global reaction mechanisms. Considering detailed chemistry is crucial for understanding detailed combustion physics, and how to accurately and efficiently consider detailed mechanisms under the framework of PINNs has not been explored yet and is still an open question. To this end, this paper proposes a PINN/flamelet/progress variable (FPV) approach to accurately and efficiently solve combustion physics, considering detailed chemistry. Specifically, the combustion thermophysical properties are tabulated using several control variables, with the FPV model considering detailed chemistry. Then, PINNs are used to solve the governing equations of continuity, momentum, and control variables with the thermophysical properties extracted from the FPV library. The performance of the proposed PINN/FPV approach is assessed for diffusion flames in a two-dimensional laminar mixing layer by comparing it with the computational fluid dynamics (CFD) results. It has been found that the PINN/FPV model can accurately reproduce the flow and combustion fields, regardless of the presence or absence of observation points. The quantitative statistics demonstrated that the mean relative error was less than 10%, and R2 values were all higher than 0.94. The applicability and stability of this model were further verified on other unseen cases with variable parameters. This study provides an efficient and accurate method to consider detailed reaction mechanisms in solving combustion physics using PINNs.

Prediction of spatiotemporal dynamics using deep learning: Coupled neural networks of long short-terms memory, auto-encoder and physics-informed neural networks

Several classic reaction-diffusion models using partial differential equations (PDEs) have been established to elucidate the formation mechanism of vegetation patterns. However, predictive modeling of complex spatiotemporal dynamics using traditional numerical methods can be significantly challenging in many practical scenarios. Physics-Informed Neural Networks (PINNs) provide a new approach to predict the solution of PDEs. However, the generalization of PINNs is not satisfactory when pretrained PINNs is directly used in non-trained space (defined as explorations). This may be attributed to the lack of training in the time dimension. Therefore, a framework (LA-PINNs) is proposed to predict the evolutionary solution of the non-dimensional vegetation-sand model. The framework couples neural networks of Long-Short Terms Memory, Auto-Encoder and Physics-Informed Neural Networks. The predictions of LA-PINNs are much better than those of PINNs. Then we studied the effects of hyperparameters on the accuracy of predictions. Based on training in time dimension by LSTM module and pretraining for quick-training strategy, LA-PINNs can improve the accuracy of explorations.

Physics-informed neural networks for dynamic process operations with limited physical knowledge and data

In chemical engineering, process data are expensive to acquire, and complex phenomena are difficult to fully model. We explore the use of physics-informed neural networks (PINNs) for modeling dynamic processes with incomplete mechanistic semi-explicit differential–algebraic equation systems and scarce process data. In particular, we focus on estimating states for which neither direct observational data nor constitutive equations are available. We propose an easy-to-apply heuristic to assess whether estimation of such states may be possible. As numerical examples, we consider a continuously stirred tank reactor and a liquid-liquid separator. We find that PINNs can infer immeasurable states with reasonable accuracy, even if respective constitutive equations are unknown. We thus show that PINNs are capable of modeling processes when relatively few experimental data and only partially known mechanistic descriptions are available, and conclude that they constitute a promising avenue that warrants further investigation.

The data-driven solutions and inverse problems of some nonlinear diffusion convection-reaction equations based on Physics-Informed Neural Network

The Physics-Informed Neural Network (PINN) has achieved remarkable results in solving partial differential equations (PDEs). This paper aims to solve the forward and inverse problems of some specific nonlinear diffusion convection-reaction equations, thereby validating the practical efficacy and accuracy of data-driven approaches in tackling such equations. In the forward problems, four different solutions of the studied equations are reproduced effectively and the approximation errors can be reduced to 10−5. Experiments indicate that the PINNs method based on adaptive activation functions (PINN-AAF), outperforms the standard PINNs in dealing with inverse problems. The unknown parameters are estimated effectively and the approximation errors can lower to 10−4. Additionally, training rules for both PINN and PINN-AAF are summarized. The results of this study validate the exceptional performance of the data-driven approach in solving the complex nonlinear diffusion convection-reaction equation problems, and provide an effective mechanism for dealing with analogous, intricate nonlinear problems.