Physics-informed Deep Learning Research Articles

An adaptive physics-informed deep learning approach for structural nonlinear response prediction

An adaptive physics-informed deep learning approach for structural nonlinear response prediction

Attention-based hybrid network for structural nonlinear response prediction under long-period earthquake

To develop structures that can respond to severe seismic events, models and tools are essential for quickly and precisely assessing and controlling the structure's performance. Long-period ground motions may pose potentially significant hazards to the safety of high-rise buildings located in basins or alluvial plains. However, traditional regression-based methods and pure data-driven machine learning methods are insufficient to address this problem of response prediction, which is characterized by non-stationarity, nonlinearity, and scarcity of samples. To tackle this issue, we propose a novel semi-explicit physics-informed deep learning method. The method incorporates the physical information into the model, thereby achieving better prediction accuracy. Through numerical and experimental validations, including hysteresis experimental data, the actual seismic response record of a six-story hotel, as well as data from single-degree-of-freedom structures across various periods and multi-story buildings in Los Angeles, Seattle, and Boston, we verified the proposed method significantly outperforms traditional regression-based methods and data-driven machine learning methods in terms of accuracy, efficiency, and robustness.

Physics-informed deep learning for structural dynamics under moving load

Physics-informed deep learning has emerged as a promising approach that incorporates physical constraints into the model, reduces the amount of data required, and demonstrates robustness and potential in dealing with limited datasets for a variety of studies. However, several key challenges still exist, with one being the spectral bias problem of deep learning in the simulation of functions with multi-frequency features. To overcome the challenge, this study proposes a novel physics-informed deep learning method, which integrates physics-informed neural network with Fourier transform so as to solve partial differential equations in the frequency domain, thus alleviating the problem of spectral bias of neural networks in the simulation of multi-frequency functions. In addition, the proposed method is used to focus on the forward simulation and parameter inverse identification issues in structural dynamics under moving loads. To illustrate the superiority of the method, the issues of dynamic response of simply supported beams under moving loads are presented as case studies, and the performance of the method in multiple cases is analysed and discussed. The research results demonstrate the feasibility and effectiveness of the method for structural dynamics simulation and parameter inverse identifications using limited datasets.

Joint [Formula: see text] and Image Reconstruction in Low-Field MRI by Physics-Informed Deep-Learning.

We present a model-based image reconstruction approach based on unrolled neural networks which corrects for image distortion and noise in low-field ( B0 ∼ 50 mT) MRI. Utilising knowledge about the underlying physics, a novel network architecture (SH-Net) is introduced which involves the estimation of spherical harmonic coefficients to guarantee a spatially smooth field map estimate. The SH-Net is integrated in an end-to-end trainable model which jointly estimates the B0-field map as well as the image. Experiments were conducted on retrospectively simulated low-field data of human knees. We compare our model to different model-based approaches at distinct noise levels and various B0-field distributions. Our results show that our physics-informed neural network approach outperforms the purely model-based methods by improving the PSNR up to 11.7% and the RMSE up to 86.3%. Our end-to-end trained model-based approach outperforms existing methods in reconstructing image and B0-field maps in the low-field regime. low-field MRI is becoming increasingly more popular as it enables access to MR in challenging situations such as intensive care units or resource poor areas. Our method allows for fast and accurate image reconstruction in such low-field imaging with B0-inhomogeneity compensation under a wide range of various environmental conditions.

Physics-Informed Deep-Learning Models Improve Forecast Scalability, Reliability

_ This article, written by JPT Technology Editor Chris Carpenter, contains highlights of paper URTeC 4042557, “Physics-Informed Deep-Learning Models for Improving Shale and Tight Forecast Scalability and Reliability,” by Kainan Wang, SPE, Lichi Deng, and Yuzhe Cai, SPE, Chevron, et al. The paper has not been peer reviewed. _ In the complete paper, the authors present a workflow that combines probabilistic modeling and deep-learning models trained on an ensemble of physics models to improve scalability and reliability for shale and tight reservoir forecasting. Their approach is applied to synthetic cases and many wells in the Permian Basin. Through hindsight studies, these models have been demonstrated to generate realistic and diverse production curves, capture the physics of unconventional flow, quantify well-production-outlook uncertainty, and help interpretation of subsurface uncertainty. Introduction In the realm of unconventional asset development, scalable forecasting is a key component in forecast reliability. In recent years, data-driven machine-learning models and workflows have emerged as potent tools for predicting well performance, particularly in scenarios where wells share similar reservoir properties, completion designs, and operational conditions. A novel approach known as physics-informed machine learning (PIML) has gained prominence. These models leverage the strengths of machine learning while honoring the constraints imposed by physical laws. By incorporating observational, inductive, or learning bias, they bridge the gap between empirical data and fundamental physics. In this paper, the authors showcase the application of a PIML system tailored specifically for unconventional reservoirs. Their methodology involves the selection of an appropriate physics modeling framework, coupled with the development of multiple machine-learning models that augment the limited field data set. Data Acquisition and Generation Bottomhole pressure (BHP) data are key to reliable forecasting because they reflect constraints to well production. Downhole pressure gauges can provide measurements of BHP accurately, but these measurements are not always available. Numerical models can be used to correlate surface measurements to BHP with some accuracy, taking into consideration the well configuration and fluid properties, yet it is too time-consuming to develop such models for many wells in relation to the speed needed for unconventional reservoir development. An in-house machine-learning model was developed for BHP calculation to further augment the data set of gauge measurements and numerical modeling. Through a workflow that chooses the wells with higher-fidelity data as a training set, and careful calibration and iterations between incrementally adding wells of high-fidelity data to reduce uncertainty, the trained machine- learning model is able to predict BHP accurately for many more wells. Another type of engineering data important for reliable well-performance analysis and forecasting is reservoir-fluid description. In this study, the authors leveraged an internally developed pressure/volume/temperature (PVT) machine-learning model trained on proprietary PVT data, which was also shown to provide reliable predictions to key PVT properties such as saturation pressure, formation volume factor, and viscosity from limited inputs at the reservoir and separator level. These PVT models not only enhance scalability of any downstream forecasting models but also play a crucial role in physics-informed uncertainty analysis for these models.

SELFNet: Denoising Shear Wave Elastography Using Spatial-temporal Fourier Feature Networks

ObjectiveUltrasound-based shear wave elastography offers estimation of tissue stiffness through analysis of the propagation of a shear wave induced by a stimulus. Displacement or velocity fields during the process can contain noise as a result of the limited number of acquisitions. With advances in physics-informed deep learning, neural networks can approximate a physics field by minimizing the residuals of governing physics equations. MethodsIn this research, we introduce a shear wave elastography Fourier feature network (SELFNet) using spatial-temporal random Fourier features within a physics-informed neural network framework to estimate and denoise particle displacement signals. The network uses a sparse mapping to increase robustness and incorporates the governing equations for regularization while simultaneously learning the mapping of the shear modulus. The method was evaluated in datasets from tissue-mimicking phantom of lesions and ex vivo tissue. ResultsThe findings indicate that SELFNet is capable of smoothing out the noise in phantom lesions with different stiffness and sizes, outperforming a reference Gaussian filtering method by 17% in relative ℓ2 error, 45% in reconstruction root-mean-square error. Furthermore, the ablation study suggested that SELFNet can prevent over-fitting through the Fourier feature mapping module. An ex vivo study confirmed its applicability to different types of tissue. ConclusionThe implementation of SELFNet shows promise for shear wave elastography with limited acquisitions. In this context, subject to successful translation, it has the potential to be extended to clinical applications, such as the diagnosis of cancer or liver disease.

Road adhesion coefficient Estimation: Physics-informed deep learning method with vehicle dynamics model

Road adhesion coefficient (RAC) is not only a significant basis for trajectory planning and decision-making of autonomous vehicles but also a critical parameter to determine the control potential of autonomous vehicles under complex operating conditions. Accurate, timely, and reliable estimation of RAC facilitates many function units of autonomous vehicles and improves driving safety. To address the issues of low accuracy and transparency of RAC estimation method based on road surface image recognition, a vehicle dynamics model-informed deep learning (VDMIDL) method for RAC estimation is proposed in this work. The physics information related to vehicle dynamics is integrated into a CNN-based deep learning model via the fusion of vehicle physical features and high-dimensional road surface image features to realize accurate estimation of RAC. The results of the model training show that the proposed VDMIDL model for RAC estimation possesses higher prediction accuracy compared with the conventional deep learning (CDL) model. The vehicle test results indicate that the proposed VDMIDL model can provide more prospective and precise RAC for trajectory planning, decision-making, and control of the vehicle in response to sudden variations in road conditions. The VDMIDL model is applicable to a wide range of road conditions. RAC estimated by the VDMIDL model is accurate and stable under poor road condition in particular. The proposed fusion estimation model incorporates more physics information into deep learning model, endowing more interpretability and transparency to the black-box model.

Innovative load identification with Res-UNet: Integrating phase space reconstruction and physics-informed deep learning

This study developed a specialized load identification model for a specific ocean platform that integrates advanced deep learning techniques with the unique physical characteristics of marine engineering. Key achievements include high-dimensional mapping of data features using phase space reconstruction (PSR) techniques to enhance the predictive capacity of the model through chaos system theory; optimization of ResNet and UNet deep learning networks with attention mechanisms to boost performance; calculation of stiffness, mass, and damping matrices using finite element software; establishment of corresponding state-space equations; and integration of structural dynamics equations with the deep learning network loss function to significantly improve load identification accuracy. In addition, proprietary structural equations for ocean platform structures were developed, underscoring the uniqueness and applicability of the algorithm. The proposed physics-informed Res-UNet model exhibited excellent performance in identifying both periodic and random loads. These improvements were validated through training with finite element simulated data and subsequent experimental applications, demonstrating the effectiveness of the proposed innovative methods.

A physics-informed deep learning approach for combined cycle fatigue life prediction

The prediction of the combined high and low cycle fatigue (CCF) life of welded components was valuable for the reliability of practical engineering structures. In this study, a physics-informed Grey-deep convolutional neural networks (DCNN) model was constructed and the model parameters was determined based on the fatigue behavior characteristics of CCF, which was proved perform well in the CCF life prediction of welded components. The predicted results indicated that the proposed model can realize reliable CCF life prediction with good accuracy (MAE = 0.49) and stability (RMSE = 0.44). Further, the proposed model also exhibited good prediction performance under different loading and geometry conditions, indicating the good universality of the proposed model. And the good universality represented that the proposed model had the potential to be applied in more engineering fields.

Physics-informed deep generative learning for quantitative assessment of the retina

Disruption of retinal vasculature is linked to various diseases, including diabetic retinopathy and macular degeneration, leading to vision loss. We present here a novel algorithmic approach that generates highly realistic digital models of human retinal blood vessels, based on established biophysical principles, including fully-connected arterial and venous trees with a single inlet and outlet. This approach, using physics-informed generative adversarial networks (PI-GAN), enables the segmentation and reconstruction of blood vessel networks with no human input and which out-performs human labelling. Segmentation of DRIVE and STARE retina photograph datasets provided near state-of-the-art vessel segmentation, with training on only a small (n = 100) simulated dataset. Our findings highlight the potential of PI-GAN for accurate retinal vasculature characterization, with implications for improving early disease detection, monitoring disease progression, and improving patient care.

Unsupervised physics-informed deep learning for assessing pulmonary artery hemodynamics

Deep learning advancements have significantly benefited medical applications. One such helpful application is noninvasive fractional flow reserve (FFR) evaluation along the pulmonary artery tree, which aids in planning optimal balloon pulmonary angioplasty. This study proposes a surrogate model that employs an unsupervised physics-informed neural network (UPNN) to predict FFR based on pulmonary CT angiography. To ensure the UPNN strictly follows the unique solution of the governing equations, we implemented a hard boundary conditions enforcement approach. Subsequently, a finite difference convolutional filter was developed to enhance connectivity among neighboring points. This allows the neural networks to propagate boundary conditions into the unseen vessel interior and perceive the geometric structure of the computational domain. We also introduced regularization constraints on the three components contributing to pressure drop within the artery tree. A total of 4500 synthetic pulmonary artery trees were used to train the UPNN with impedance outlet boundary conditions. We found that the limits of agreement of FFR trained by UPNN ranged from −0.05 to 0.04 with computational fluid dynamics (CFD) simulation results. The testing results showed that the correlation coefficient between FFR predicted by UPNN and CFD were 96.1% and 94.7% for synthetic and patient-specific data, respectively. Using invasive FFR as reference, the accuracy of FFR predicted by UPNN and CFD was 81.4% and 84.8%, respectively. These results demonstrate that our UPNN-based surrogate model’s performance in evaluating FFR aligns closely with CFD simulations.

SRS-Net: a universal framework for solving stimulated Raman scattering in nonlinear fiber-optic systems by physics-informed deep learning

As a crucial nonlinear phenomenon, stimulated Raman scattering (SRS) plays multifaceted roles involved in forward and inverse problems. In fibre-optic systems, these roles range from detrimental interference that impairs optical performance to beneficial effects that enables various devices such as Raman amplifier. To obtain solutions of SRS, various numerical methods customized for different scenarios have been proposed. However, these methods are time-consuming, low-efficiency, and experience-orientated, particularly in combined scenarios consisting of both forward and inverse problems. Inspired by physics-informed neural networks, we propose SRS-Net, which combines the efficient automatic differentiation and powerful representation ability of neural networks with the regularization of SRS physical laws, to obtain universal solutions for SRS of forward, inverse, and combined problems. We showcase the intuitive solving procedure and high-speed performance of SRS-Net through extensive simulations covering different scenarios. Additionally, we validate its capabilities in experiments involving the high-fidelity modelling of a wavelength division multiplexing system spanning the C + L-band with approximately 10 THz. The versatility of the SRS-Net framework extends beyond SRS, indicating its potential as a promising universal solution in other engineering problems with nonlinear dynamics governed by partial differential equations.

Learning Feynman integrals from differential equations with neural networks

We perform an exploratory study of a new approach for evaluating Feynman integrals numerically. We apply the recently-proposed framework of physics-informed deep learning to train neural networks to approximate the solution to the differential equations satisfied by the Feynman integrals. This approach relies neither on a canonical form of the differential equations, which is often a bottleneck for the analytical techniques, nor on the availability of a large dataset, and after training yields essentially instantaneous evaluation times. We provide a proof-of-concept implementation within the PyTorch framework, and apply it to a number of one- and two-loop examples, achieving a mean magnitude of relative difference of around 1% at two loops in the physical phase space with network training times on the order of an hour on a laptop GPU.

Theory-data dual driven car following model in traffic flow mixed of AVs and HDVs

To model the car following (CF) behavior of mixed traffic flow composed of autonomous vehicles (AVs) and human-driving vehicles (HDVs) in the future, this paper calibrated the lower control algorithm of AVs and proposed a model named theory-data dual driven stochastically generative adversarial networks (TDS-GAN) to describe the CF behavior of HDVs based on experimental data from mixed traffic flow. Firstly, the experimental scenario and collected data were introduced. Secondly, two transfer functions for the lower control algorithm of AVs were compared. Then, to account for the stochasticity of HDVs, the idea of physics-informed deep learning (PIDL) was used to improve generative adversarial networks (GAN) and integrate it with two-dimensional intelligent driver model (2D-IDM). Finally, the effectiveness of the model was verified from both a micro prediction and a macro simulation perspective. The influence of AVs on the stability of mixed traffic flow at different Market Penetration Rate (MPR) was also observed. The results show that TDS-GAN can effectively describe the following behavior and stochasticity of HDVs. When combined with CF model of AVs, it can depict the evolution of mixed traffic flow more accurately. Additionally, AVs can improve traffic flow stability under different penetration rates, and the effect is more significant with higher MPR. However, the introduction of AVs may not necessarily be positive in terms of road capacity.

PID4LaTe: a physics-informed deep learning model for lake multi-depth temperature prediction

PID4LaTe: a physics-informed deep learning model for lake multi-depth temperature prediction

Phase-field modeling of fracture with physics-informed deep learning

We explore the potential of the deep Ritz method to learn complex fracture processes such as quasistatic crack nucleation, propagation, kinking, branching, and coalescence within the unified variational framework of phase-field modeling of brittle fracture. We elucidate the challenges related to the neural-network-based approximation of the energy landscape, and the ability of an optimization approach to reach the correct energy minimum, and we discuss the choices in the construction and training of the neural network which prove to be critical to accurately and efficiently capture all the relevant fracture phenomena. The developed method is applied to several benchmark problems and the results are shown to be in qualitative and quantitative agreement with the finite element solution. The robustness of the approach is tested by using neural networks with different initializations.

A physics-informed deep learning model to reconstruct turbulent wake from random sparse data

This study develops a flexible deep learning framework aimed at reconstructing the global turbulent wakes from the randomly distributed sparse data. The framework is based on a Generative Adversarial Networks where the generator utilizes U-Net architecture and a constraint module is integrated into the training process. It is designed to overcome challenges posed by the chaotic behavior of turbulent fields, randomness in sensor layouts, and sparse sensor numbers. The efficacy of the model is validated across three high-fidelity datasets, including laminar wake behind a circular cylinder, turbulent wake behind a circular cylinder, and turbulent wake behind a square cylinder. The proposed model demonstrates the ability to accurately reconstruct flow patterns of both turbulent and laminar wakes, even utilizing merely 0.043% of the data from the target flow field. The proposed model exhibits significant generalization capability, which means that the model has a nearly independence from the distributions of sensors and a robust adaptation across the inputs with unseen sensor numbers. Ablation studies elucidate the distinct and complementary roles of each module within the model. Additionally, the behavior of the bottleneck tensor is analyzed through visualization, including comparisons with the lift coefficient, quantitative analyses and dimensionality reduction. These visualizations confirm the ability of the model to extract distinctive phase information reliably from sparse data, thereby guiding the reconstruction of global flow patterns. These findings highlight the potential of the model for applications in fluid dynamics where data is collected in a variable manner.

A physics-informed deep learning description of Knudsen layer reactivity reduction

A physics-informed neural network (PINN) is used to evaluate the fast ion distribution in the hot spot of an inertial confinement fusion target. The use of tailored input and output layers to the neural network is shown to enable a PINN to learn the parametric solution to the Vlasov–Fokker–Planck equation in the absence of any synthetic or experimental data. As an explicit demonstration of the approach, the specific problem of Knudsen layer fusion yield reduction is treated. Here, the predictions from the Vlasov–Fokker–Planck PINN are used to provide a non-perturbative solution of the fast ion tail in the vicinity of the hot spot, thus allowing the spatial profile of the fusion reactivity to be evaluated for a range of collisionalities and hot spot conditions. Excellent agreement is found between the predictions of the Vlasov–Fokker–Planck PINN and the results from traditional numerical solvers with respect to both the energy and spatial distribution of fast ions and the fusion reactivity profile, demonstrating that the Vlasov–Fokker–Planck PINN provides an accurate and efficient means of determining the impact of Knudsen layer yield reduction across a broad range of plasma conditions.

A causal physics-informed deep learning formulation for groundwater flow modeling and climate change effect analysis

In this study, we propose and test a formulation for building causal physics-informed hybrid models over traditional physics-informed hybrid models (H-HBVo) and a convolutional neural network (1D-CNN) in groundwater level (GWL) modeling. Two types of models are built based on our formulation and named H-HBV and H-Lin, because they integrate two different hydrological models. The comparison is made in terms of performance and ability to learn cause-and-effect relationships between inputs and outputs. The novelty of this study lies in the CRC (Causal Relationship Constraints) conditions, derived from a mathematical development, that are imposed on the layers of the physics-informed hybrid model to force it to learn causal relationships. The results showed that the 1D-CNN algorithm performed slightly better than the hybrid algorithms, and that the H-Lin and H-HBV algorithms together achieved slightly better results than the traditional hybrid algorithm, H-HBVo. It is also obtained that the algorithms subjected to our formulations (H-Lin and H-HBV) are the ones with satisfactory causality properties and that the integration of domain knowledge is not sufficient to obtain causal and interpretable models within the framework of hybrid algorithms. Subsequently, an analysis of the effect of climate change is carried out for Quebec (Canada) using H-HBV models. The results show that the RCP and SSP scenarios give fairly similar results, i.e. an increase in GWL between 2011 and 2090 with median amplitudes varying between + 5.6 and + 7 cm. In some cases, the amplitude of increase can exceed between + 14 and + 23 cm, and even reach between + 35 and + 70 cm. The results of the climate change analysis are consistent with changes in vertical inflow and potential evapotranspiration, the input variables, for the different climate scenarios, confirming the robustness of our formulation.

Investigation of physics-informed deep learning for the prediction of parametric, three-dimensional flow based on boundary data

The placement of temperature sensitive and safety-critical components is crucial in the automotive industry. It is therefore inevitable, even at the design stage of new vehicles, that these components are assessed for potential safety issues. However, with increasing number of design proposals, risk assessment using Computational Fluid Dynamics (CFD) quickly becomes expensive. We therefore present a parameterized surrogate model for the prediction of three-dimensional flow fields in aerothermal vehicle simulations. The proposed physics-informed neural network (PINN) design is aimed at learning families of flow solutions according to a geometric variation. The novelty of our method compared to existing works in the field of PINN lies in the extension of parametric flow prediction to three-dimensional space by applying a mini-batch based Quasi-Newton optimization. We contribute a parametric minibatch training algorithm which enables the utilization of the large datasets necessary for the three-dimensional flow modeling. Further, we introduce a continuous resampling algorithm that allows to represent domain variations, while operating on one static dataset of reduced size. In scope of this work, we could show that our nondimensional, multivariate scheme can be efficiently extended to predict the velocity and pressure distribution in three-dimensional space for different design scenarios and geometric scales. Every feature of our methodology is tested individually and verified against conventional CFD simulations. Finally, we apply our proposed method in context of an exemplary real-world automotive application.