Physics-informed Deep Learning Research Articles

Physics-informed deep learning to forecast {widehat{{varvec{M}}}}_{{varvec{m}}{varvec{a}}{varvec{x}}} during hydraulic fracturing

Short-term forecasting of estimated maximum magnitude ({widehat{M}}_{max}) is crucial to mitigate risks of induced seismicity during fluid stimulation. Most previous methods require real-time injection data, which are not always available. This study proposes two deep learning (DL) approaches, along with two data-partitioning methods, that rely solely on preceding patterns of seismicity. The first approach forecasts {widehat{M}}_{max} directly using DL; the second incorporates physical constraints by using DL to forecast seismicity rate, which is then used to estimate {widehat{M}}_{max}. These approaches are tested using a hydraulic-fracture monitoring dataset from western Canada. We find that direct DL learns from previous seismicity patterns to provide an accurate forecast, albeit with a time lag that limits its practical utility. The physics-informed approach accurately forecasts changes in seismicity rate, but sometimes under- (or over-) estimates {widehat{M}}_{max}. We propose that significant exceedance of {widehat{M}}_{max} may herald the onset of runaway fault rupture.

A quantile-regression physics-informed deep learning for car-following model

Car-following models have been extensively studied using either theory-driven models or data-driven models. On the other hand, theory-driven models have low accuracy, whereas data-driven models have poor interpretability and require a high-quality training dataset. Furthermore, the majority of studies focused on predicting short-term trajectory based on a single vehicle. This paper aims to combine physics knowledge with deep learning and stochasticity to completely capture the genuine regularities of car-following behaviors, leveraging the benefits of both physics-based (interpretability) and deep learning-based (data-efficient and generalizability) models. We design a quantile-regression physics-informed deep learning car-following model (QRPIDL-CF). The physics-informed computational graph is integrated with a long short-term memory (LSTM) architecture (abbreviated as PIDL). By incorporating the quantile regression to the loss function of the PIDL model, stochastic behaviors can be produced without prior assumptions of drivers. To validate the proposed model, data from a 25-vehicle platoon experiment was used. The prediction results prove that given the first vehicle’s trajectory and other vehicles’ initial states, it can preserve the high precision in learning car-following behaviors and introduce small disturbances into car-following in a vehicle-platoon just the same as the real driving process. The trajectory prediction of the whole platoon shows that the proposed hybrid QRPIDL-CF model can better perform the macro-traffic behavior of car-following than every single model from the hybrid QRPIDL-CF model.

A physics-informed Transformer model for vehicle trajectory prediction on highways

Autonomous Vehicles (AVs) have made remarkable developments and are anticipated to replace human drivers. In transitioning from human-driven vehicles to fully AVs, one crucial task is to predict the trajectories of the subject vehicle and its surrounding vehicles in real time. Most existing methods of vehicle trajectory prediction on highways are based on physical models or purely data-driven models. However, they either yield unsatisfactory prediction performance or lack model interpretability and physical implications. This paper proposes a Physics-Informed Deep Learning framework that fully leverages the advantages of data-driven and physics-based models to go beyond the existing models. We use the Transformer neural network architecture with self-attention as Physics-Uninformed Neural Network (PUNN) and Intelligent Driver Model (IDM) as physical model to construct of Physics-Informed Transformer-Intelligent Driver Model (PIT-IDM). Extensive experiments have been conducted on two datasets with different traffic environments, i.e., Next Generation SIMulation (NGSIM) data in the US and the Ubiquitous Traffic Eyes (UTE) data in China, to verify model accuracy and efficiency. Compared with the three kinds of baselines by relative and absolute measures of effectiveness, the best performing PIT-IDM reduces longitudinal trajectory prediction errors for long horizons by 5%-50%, some even reduced up to 70%. Extensive empirical analyses have been carried out to verify its excellent spatio-temporal transferability and explore the physics-informed mechanism underlying this deep learning method. The training and inference time analysis indicates that although it takes longer to train PIT-IDM, it requires fewer calls and accumulates fewer errors with less computation time in real-world applications. The overall results further validate the efficacy of this Physics-Informed Deep Learning framework in enhancing model accuracy, interpretability, and transferability.

Physics-informed deep learning for rock physical inversion and its uncertainty analysis

Geological CO2 storage is aiming to inject the carbon dioxide into subsurface formations, and geophysical measurements are then commonly used to monitor the fluid long-term and safe storage for risk assessment during and after the injection. In this process, the rock physical inversion is an essential part for determining the reservoir parameters such as porosity or fluid saturation for potential storage calculation or monitoring fluid migration. We propose a deep learning approach to invert reservoir parameters based on rock properties or seismically inverted results. The rock physics equations are incorporated into the learning process, leading the neural networks as physics informed. We choose the generative adversarial networks to obtain ensemble predictions by varying the input latent vectors, from which the uncertainty analysis is performed. The proposed approach is applied to the Sleipner 2019 Benchmark Model for inverting reservoir porosity and CO2 saturation with rock properties in terms of velocities and bulk density as inputs. The supervised learning and physics-informed neural network are also applied for a comparison; however, both of them cannot access the prediction uncertainty that is important for risk reduction by decision makers.

Simultaneous mapping of nearshore bathymetry and waves based on physics-informed deep learning

This paper uses physics-informed neural networks (PINNs) to simultaneously determine nearshore water depths and wave height fields based on remote sensing of the ocean surface with limited or sparse measurements. Two methods that integrate the knowledge of water wave mechanics and fully connected neural networks are introduced. The first method utilizes observed wave celerity fields and scarce measurements of wave height as training data. The model performance was examined with linear waves over an alongshore varying barred beach and nonlinear waves over an alongshore uniform barred beach. The second method uses scarce wave height and water depth measurements as training points, and the model performance was investigated with water waves over a circular shoal and the alongshore varying barred beach. One advantage of applying PINNs to solve bathymetry inversion problems is that wave height and bathymetry can be simultaneously estimated by PINN models. Thus, the impact of wave amplitude dispersion on depth inversion in nonlinear wave systems can be considered without measuring the entire wave height field. Overall, this study demonstrates the potential of the inverse PINN model as a promising tool for estimating nearshore bathymetry and reconstructing wave fields using observations from different remote sensing platforms.

Physics-informed deep learning for melting heat transfer analysis with model-based transfer learning

We present an adaptive deep collocation method (DCM) based on physics-informed deep learning for the melting heat transfer analysis of a non-Newtonian (Sisko) fluid over a moving surface with nonlinear thermal radiation. Fitted neural network search (NAS) and model based transfer learning (TL) are developed to improve model computational efficiency and accuracy. The governing equations for this boundary-layer flow problem are derived using Buongiorno's and a nonlinear thermal radiation model. Next, similarity transformations are introduced to reduce the governing equations into coupled nonlinear ordinary differential equations (ODEs) subjected to asymptotic infinity boundary conditions. By incorporating physics constraints into the neural networks, we employ the proposed deep learning model to solve the coupled ODEs. The imposition of infinity boundary conditions is carried out by adding an inequality constraint to the loss function, with infinity added to the hyper-parameters of the neural network, which is updated dynamically in the optimization process. The effects of various dimensionless parameters on three profiles (velocity, temperature, concentration) are investigated. Finally, we demonstrate the performance and accuracy of the adaptive DCM with transfer learning through several numerical examples, which can be the promising surrogate model to solve boundary layer problems.

Forecasting small-scale dynamics of fluid turbulence using deep neural networks

Turbulence in fluid flows is characterized by a wide range of interacting scales. Since the scale range increases as some power of the flow Reynolds number, a faithful simulation of the entire scale range is prohibitively expensive at high Reynolds numbers. The most expensive aspect concerns the small-scale motions; thus, major emphasis is placed on understanding and modeling them, taking advantage of their putative universality. In this work, using physics-informed deep learning methods, we present a modeling framework to capture and predict the small-scale dynamics of turbulence, via the velocity gradient tensor. The model is based on obtaining functional closures for the pressure Hessian and viscous Laplacian contributions as functions of velocity gradient tensor. This task is accomplished using deep neural networks that are consistent with physical constraints and explicitly incorporate Reynolds number dependence to account for small-scale intermittency. We then utilize a massive direct numerical simulation database, spanning two orders of magnitude in the large-scale Reynolds number, for training and validation. The model learns from low to moderate Reynolds numbers and successfully predicts velocity gradient statistics at both seen and higher (unseen) Reynolds numbers. The success of our present approach demonstrates the viability of deep learning over traditional modeling approaches in capturing and predicting small-scale features of turbulence.

Physics-informed machine learning for understanding rock moisture dynamics in a sandstone cave

Abstract. Rock moisture, which is a hidden component of the terrestrial hydrological cycle, has received little attention. In this study, frequency domain reflectometry is used to monitor fluctuating rock water content (RWC) in a sandstone cave of the Yungang Grottoes, China. We identified two major cycles of rock moisture addition and depletion, one in summer affected by air vapour concentration and the other in winter caused by freezing–thawing. For the summer-time RWC, by using the long short-term memory (LSTM) network and the SHapley Additive exPlanations (SHAP) method, we find relative humidity, air temperature and wall temperature have contributions to rock moisture, and there is a good match between predicted and measured RWC using the three variables as model inputs. Moreover, by using summer-time vapour concentration and the difference between dew point temperature and wall temperature as input variables of the LSTM network, which belongs to physics-informed machine learning, the predicted RWC has a better agreement with the measured RWC, with increased Nash–Sutcliffe efficiency (NSE) and decreased mean absolute error (MAE) and root mean square error (RMSE). After identifying the causal factors of RWC fluctuations, we also identified the mechanism controlling the inter-day fluctuations of vapour condensation. The increased vapour concentration accompanying a precipitation event leads to transport of water vapour into rock pores, which is subsequently adsorbed onto the surface of rock pores and then condensed into liquid water. With the aid of the physics-informed deep learning model, this study increases understanding of sources of water in caves, which would contribute to future strategies of alleviating weathering in caves.

Physics-informed deep learning method for the refrigerant filling mass flow metering

Refrigerant filling is a process of transitioning from compressed liquid to gas-liquid discrete state, which poses difficulties in monitoring the quality of refrigerant charging. This paper proposed a novel approach based on physics-informed deep learning for refrigerant filling mass flow metering. First, a shock discontinuity flow physical model is established related to the refrigerant filling mass flow rate, velocity and pressure. Additionally, a filling mass metering method based on PINN is proposed, as well as a residual points-adding optimization method for PDEs with discontinuous solution. The results demonstrate that PINN accurately predict mass flow rate, pressure, and velocity for refrigerant filling, and the error relative to numerical solution is less than 3%. Finally, a refrigerant filling experiment was conducted to verify the feasibility of the method of refrigerant mass flow metering.

Variational inference in neural functional prior using normalizing flows: application to differential equation and operator learning problems

Physics-informed deep learning has recently emerged as an effective tool for leveraging both observational data and available physical laws. Physics-informed neural networks (PINNs) and deep operator networks (DeepONets) are two such models. The former encodes the physical laws via the automatic differentiation, while the latter learns the hidden physics from data. Generally, the noisy and limited observational data as well as the over-parameterization in neural networks (NNs) result in uncertainty in predictions from deep learning models. In paper “MENG, X., YANG, L., MAO, Z., FERRANDIS, J. D., and KARNIADAKIS, G. E. Learning functional priors and posteriors from data and physics. Journal of Computational Physics, 457, 111073 (2022)”, a Bayesian framework based on the generative adversarial networks (GANs) has been proposed as a unified model to quantify uncertainties in predictions of PINNs as well as DeepONets. Specifically, the proposed approach in “MENG, X., YANG, L., MAO, Z., FERRANDIS, J. D., and KARNIADAKIS, G. E. Learning functional priors and posteriors from data and physics. Journal of Computational Physics, 457, 111073 (2022)” has two stages: (i) prior learning, and (ii) posterior estimation. At the first stage, the GANs are utilized to learn a functional prior either from a prescribed function distribution, e.g., the Gaussian process, or from historical data and available physics. At the second stage, the Hamiltonian Monte Carlo (HMC) method is utilized to estimate the posterior in the latent space of GANs. However, the vanilla HMC does not support the mini-batch training, which limits its applications in problems with big data. In the present work, we propose to use the normalizing flow (NF) models in the context of variational inference (VI), which naturally enables the mini-batch training, as the alternative to HMC for posterior estimation in the latent space of GANs. A series of numerical experiments, including a nonlinear differential equation problem and a 100-dimensional (100D) Darcy problem, are conducted to demonstrate that the NFs with full-/mini-batch training are able to achieve similar accuracy as the “gold rule” HMC. Moreover, the mini-batch training of NF makes it a promising tool for quantifying uncertainty in solving the high-dimensional partial differential equation (PDE) problems with big data.

Physics-Informed Super-Resolution of Turbulent Channel Flows via Three-Dimensional Generative Adversarial Networks

For a few decades, machine learning has been extensively utilized for turbulence research. The goal of this work is to investigate the reconstruction of turbulence from minimal or lower-resolution datasets as inputs using reduced-order models. This work seeks to effectively reconstruct high-resolution 3D turbulent flow fields using unsupervised physics-informed deep learning. The first objective of this study is to reconstruct turbulent channel flow fields and verify these with respect to the statistics. The second objective is to compare the turbulent flow structures generated from a GAN with a DNS. The proposed deep learning algorithm effectively replicated the first- and second-order statistics of turbulent channel flows of Reτ= 180 within a 2% and 5% error, respectively. Additionally, by incorporating physics-based corrections to the loss functions, the proposed algorithm was also able to reconstruct λ2 structures. The results suggest that the proposed algorithm can be useful for reconstructing a range of 3D turbulent flows given computational and experimental efforts.

PI-LSTM: Physics-informed long short-term memory network for structural response modeling

Deep learning models have achieved remarkable accuracy for structural response modeling. However, these models heavily depend on having a sufficient amount of training data, which can be challenging and time-consuming to collect. Moreover, data-driven models sometimes struggle to adhere to physics constraints. Therefore, in this study, a physics-informed long short-term memory (PI-LSTM) network was applied to structural response modeling by incorporating physics constraints into deep learning. The physics constraints were modified to accommodate the characteristics of both linear and nonlinear cases. The PI-LSTM network, inspired by and compared with existing physics-informed deep learning models (PhyCNN and PhyLSTM), was validated using the numerical simulation results of the single-degree-of-freedom (SDOF) system and the experimental results of the six-story building. Additionally, the PI-LSTM network underwent thorough investigation and validation across the four cases of the SDOF system and numerical simulation results of the six-story building with the comparison of the regular LSTM. The results indicate that the PI-LSTM network outperformed the regular LSTM models in terms of accuracy. Furthermore, the PI-LSTM network exhibited a more concentrated and higher accuracy range when analyzing the results of both the SDOF system and the six-story building. These findings demonstrate that the PI-LSTM network presents a reliable and efficient approach for structural response modeling.

Physics-Informed Deep Learning for Traffic State Estimation: A Survey and the Outlook

For its robust predictive power (compared to pure physics-based models) and sample-efficient training (compared to pure deep learning models), physics-informed deep learning (PIDL), a paradigm hybridizing physics-based models and deep neural networks (DNNs), has been booming in science and engineering fields. One key challenge of applying PIDL to various domains and problems lies in the design of a computational graph that integrates physics and DNNs. In other words, how the physics is encoded into DNNs and how the physics and data components are represented. In this paper, we offer an overview of a variety of architecture designs of PIDL computational graphs and how these structures are customized to traffic state estimation (TSE), a central problem in transportation engineering. When observation data, problem type, and goal vary, we demonstrate potential architectures of PIDL computational graphs and compare these variants using the same real-world dataset.

Physics-Informed Deep Learning for Solving Coupled Electron and Phonon Boltzmann Transport Equations

Electron-phonon (e-ph) coupling and transport are ubiquitous in modern electronic devices. The coupled electron and phonon Boltzmann transport equations (BTEs) hold great potential for the simulation of thermal transport in metal and semiconductor systems. However, solving the BTEs is often computationally challenging owing to their high dimensional complexity and a wide span of heat carrier properties, which hinder large-scale thermal modeling at the device level. In this work, we present a physics-informed neural network framework for solving the coupled electron and phonon BTEs. Instead of relying on labeled data, the proposed framework directly learns the spatiotemporal solutions (i.e., the electron and phonon distribution functions) within a parameterized space by enforcing physical laws. The efficacy of this framework is demonstrated through its ability to accurately resolve temperature profiles in low-dimensional thermal transport problems and visualize the ultrafast electron and phonon dynamics in laser heating experiments on thin metal films. The results indicate that our approach can accurately describe nonequilibrium e-ph energy transfer with improved efficiency, opening alternative avenues for the predictive design and optimization of micro- and nanostructures.

Physics-informed data-driven model for fluid flow in porous media

A physics-informed machine learning model is developed that can replace the numerical simulations of porous media. By learning the communications among grid cells in the numerical domain, this model is capable of accurately predicting flow fields for new sets of simulation runs. Because of the many possible random arrangements of particles and their orientation with respect to each other, generalization of permeability with high accuracy is not trivial — nor is it practical using conventional means. Furthermore, building a comprehensive database for different grain/pore arrangements is not possible because of the cost of running numerical simulations to generate the database that represent all possible arrangements. The objective is to predict grid-level flow fields in porous media as a priori to determining permeability of porous media. The rationale is that once the detailed grid-level dynamics can be accurately predicted using data-driven approach, for any configuration/topology of the porous media, the detailed dynamics could be predicted without any need for new expensive numerical simulation runs. In this work, a physics-informed deep learning model is developed by using the results from Lattice-Boltzmann simulations of randomly distributed circular grains to represent the porous media. A variety of porous structures are developed by changing the number, size, and location of circular grains. The deep U-Net and ResNet neural network architectures are combined to train a deep learning model which avoids the vanishing gradient issues. The continuity and momentum conservation equations are embedded into the loss function of deep learning architecture. Robustness of the developed model is then tested for numerous variations of porous media which have not been used for developing the model.

Milling Surface Roughness Prediction Based on Physics-Informed Machine Learning

Surface roughness is a key indicator of the quality of mechanical products, which can precisely portray the fatigue strength, wear resistance, surface hardness and other properties of the products. The convergence of current machine-learning-based surface roughness prediction methods to local minima may lead to poor model generalization or results that violate existing physical laws. Therefore, this paper combined physical knowledge with deep learning to propose a physics-informed deep learning method (PIDL) for milling surface roughness predictions under the constraints of physical laws. This method introduced physical knowledge in the input phase and training phase of deep learning. Data augmentation was performed on the limited experimental data by constructing surface roughness mechanism models with tolerable accuracy prior to training. In the training, a physically guided loss function was constructed to guide the training process of the model with physical knowledge. Considering the excellent feature extraction capability of convolutional neural networks (CNNs) and gated recurrent units (GRUs) in the spatial and temporal scales, a CNN–GRU model was adopted as the main model for milling surface roughness predictions. Meanwhile, a bi-directional gated recurrent unit and a multi-headed self-attentive mechanism were introduced to enhance data correlation. In this paper, surface roughness prediction experiments were conducted on the open-source datasets S45C and GAMHE 5.0. In comparison with the results of state-of-the-art methods, the proposed model has the highest prediction accuracy on both datasets, and the mean absolute percentage error on the test set was reduced by 3.029% on average compared to the best comparison method. Physical-model-guided machine learning prediction methods may be a future pathway for machine learning evolution.

Reconstructing the Unsaturated Flow Equation From Sparse and Noisy Data: Leveraging the Synergy of Group Sparsity and Physics‐Informed Deep Learning

AbstractData‐driven scientific discovery methods have been developed and applied to discover governing equations from data, involving the attempt to discover the unsaturated flow equation in soils from data. However, an important but unresolved problem is how to reconstruct the unsaturated flow equation from highly noisy and scarce discrete data. In this study, we present a new deep‐learning framework: DeepGS (deep‐learning‐based group sparsity framework), that leverages the synergy of group sparsity and physics‐informed deep learning (PIDL) to reconstruct the latent governing equation for unsaturated flow. In particular, we design a strategy that decomposes the identification of the unsaturated flow equation into two tasks: the determination of the partial differential equation structure and the reconstruction of the nonlinear coefficients. The tasks can be seamlessly handled by group sparse regression and the PIDL approach. Through the training, it realizes the simultaneous reconstruction of soil moisture dynamics and unsaturated flow governing equation. A series of comprehensive numerical experiments are conducted to determine the optimal architecture and test its performance. The results show the efficacy and robustness of DeepGS, which significantly outperform previous methods. We also conclude that accurately reconstructing soil moisture dynamics and spatiotemporal derivatives from noisy and scarce data play a critical role in governing equation discovery. This study further demonstrates the potential of discovering the governing equation for unsaturated flow from data in more complex scenarios, where rich and accurate soil moisture observations are generally intractable to access.

Enhancing Computational Accuracy in Surrogate Modeling for Elastic–Plastic Problems by Coupling S-FEM and Physics-Informed Deep Learning

Physics-informed neural networks (PINNs) provide a new approach to solving partial differential equations (PDEs), while the properties of coupled physical laws present potential in surrogate modeling. However, the accuracy of PINNs in solving forward problems needs to be enhanced, and solving inverse problems relies on data samples. The smoothed finite element method (S-FEM) can obtain high-fidelity numerical solutions, which are easy to solve for the forward problems of PDEs, but difficult to solve for the inverse problems. To the best of the authors’ knowledge, there has been no prior research on coupling S-FEM and PINN. In this paper, a novel approach that couples S-FEM and PINN is proposed. The proposed approach utilizes S-FEM to synthesize high-fidelity datasets required for PINN inversion, while also improving the accuracy of data-independent PINN in solving forward problems. The proposed approach is applied to solve linear elastic and elastoplastic forward and inverse problems. The computational results demonstrate that the coupling of the S-FEM and PINN exhibits high precision and convergence when solving inverse problems, achieving a maximum relative error of 0.2% in linear elasticity and 5.69% in elastoplastic inversion by using approximately 10,000 data points. The coupling approach also enhances the accuracy of solving forward problems, reducing relative errors by approximately 2–10 times. The proposed coupling of the S-FEM and PINN offers a novel surrogate modeling approach that incorporates knowledge and data-driven techniques, enabling it to solve both forward and inverse problems associated with PDEs with high levels of accuracy and convergence.

Physics-Informed Deep-Learning For Elasticity: Forward, Inverse, and Mixed Problems.

Elastography is a medical imaging technique used to measure the elasticity of tissues by comparing ultrasound signals before and after a light compression. The lateral resolution of ultrasound is much inferior to the axial resolution. Current elastography methods generally require both axial and lateral displacement components, making them less effective for clinical applications. Additionally, these methods often rely on the assumption of material incompressibility, which can lead to inaccurate elasticity reconstruction as no materials are truly incompressible. To address these challenges, a new physics-informed deep-learning method for elastography is proposed. This new method integrates a displacement network and an elasticity network to reconstruct the Young's modulus field of a heterogeneous object based on only a measured axial displacement field. It also allows for the removal of the assumption of material incompressibility, enabling the reconstruction of both Young's modulus and Poisson's ratio fields simultaneously. The authors demonstrate that using multiple measurements can mitigate the potential error introduced by the "eggshell" effect, in which the presence of stiff material prevents the generation of strain in soft material. These improvements make this new method a valuable tool for a wide range of applications in medical imaging, materials characterization, and beyond.

Particle Swamp Optimized -Physics-Informed Deep Learning for Heterogeneous Oil and Gas Underground Reservoir Pressure Management.

In recent years, Particle Swarm Optimization (PSO) has been integrated with machine learning algorithms, such as deep learning, to create powerful hybrid methods that can tackle complex optimization problems more effectively. In the domain of oil and gas reservoir management, underground pressure management is crucial to maximize the yield and efficiency of the reservoir. However, the heterogeneity of the reservoir, along with uncertainties in its properties, makes pressure management a complex and challenging task. To address this issue, researchers have proposed Physics-Informed Deep Learning (PIDL) techniques that incorporate domain-specific knowledge, such as the governing physical equations, into the deep learning framework. Particle Swarm Optimized-Physics-Informed Deep Learning (PSO-PIDL) is a novel hybrid approach that combines PSO with PIDL to optimize the pressure management of heterogeneous oil and gas underground reservoirs. In this approach, the PSO algorithm is used to find the optimal solution for the PIDL-based model that incorporates the governing physical equations of the reservoir. PSO-PIDL can effectively handle the uncertainties and heterogeneity of the reservoir, while also incorporating the physical constraints of the problem. Overall, PSO-PIDL is a promising approach for optimizing the pressure management of oil and gas reservoirs. It can help reduce the operational costs and improve the efficiency of the reservoir, while also ensuring the sustainable use of natural resources. Keywords: Physics-Informed Deep Learning, Particle Swarm Optimization, Bidirectional Long-Short-Term-Memory, Heterogeneous Reservoir, DuPont Finite Element Heat and Mass Transfer Code