Physics-informed Deep Learning Research Articles

A thermodynamically consistent physics-informed deep learning material model for short fiber/polymer nanocomposites

This work proposes a physics-informed deep learning (PIDL)-based constitutive model for investigating the viscoelastic–viscoplastic behavior of short fiber-reinforced nanoparticle-filled epoxies under various ambient conditions. The deep-learning model is trained to enforce thermodynamic principles, leading to a thermodynamically consistent constitutive model. To accomplish this, a long short-term memory network is combined with a feed-forward neural network to predict internal variables required for characterizing the internal dissipation of the nanocomposite materials. In addition, another feed-forward neural network is used to indicate the free-energy function, which enables defining the thermodynamic state of the entire system. The PIDL model is initially developed for the three-dimensional case by generating synthetic data from a classical constitutive model. The model is then trained by extracting the data directly from cyclic loading–unloading experimental tests. Numerical examples show that the PIDL model can accurately predict the mechanical behavior of epoxy-based nanocomposites for different volume fractions of fibers and nanoparticles under various hygrothermal conditions.

Investigation of physics-informed deep learning for the prediction of parametric, three-dimensional flow based on boundary data

The placement of temperature sensitive and safety-critical components is crucial in the automotive industry. It is therefore inevitable, even at the design stage of new vehicles, that these components are assessed for potential safety issues. However, with increasing number of design proposals, risk assessment using Computational Fluid Dynamics (CFD) quickly becomes expensive. We therefore present a parameterized surrogate model for the prediction of three-dimensional flow fields in aerothermal vehicle simulations. The proposed physics-informed neural network (PINN) design is aimed at learning families of flow solutions according to a geometric variation. The novelty of our method compared to existing works in the field of PINN lies in the extension of parametric flow prediction to three-dimensional space by applying a mini-batch based Quasi-Newton optimization. We contribute a parametric minibatch training algorithm which enables the utilization of the large datasets necessary for the three-dimensional flow modeling. Further, we introduce a continuous resampling algorithm that allows to represent domain variations, while operating on one static dataset of reduced size. In scope of this work, we could show that our nondimensional, multivariate scheme can be efficiently extended to predict the velocity and pressure distribution in three-dimensional space for different design scenarios and geometric scales. Every feature of our methodology is tested individually and verified against conventional CFD simulations. Finally, we apply our proposed method in context of an exemplary real-world automotive application.

Deep learning reconstruction with uncertainty estimation for [formula omitted] photon interaction in fast scintillator detectors

This article presents a physics-informed deep learning method for the quantitative estimation of the spatial coordinates of gamma interactions within a monolithic scintillator, with a focus on Positron Emission Tomography (PET) imaging. A Density Neural Network approach is designed to estimate the 2-dimensional gamma photon interaction coordinates in a fast lead tungstate (PbWO4) monolithic scintillator detector. We introduce a custom loss function to estimate the inherent uncertainties associated with the reconstruction process and to incorporate the physical constraints of the detector.This unique combination allows for more robust and reliable position estimations and the obtained results demonstrate the effectiveness of the proposed approach and highlights the significant benefits of the uncertainties estimation. We discuss its potential impact on improving PET imaging quality and show how the results can be used to improve the exploitation of the model, to bring benefits to the application and how to evaluate the validity of the given prediction and the associated uncertainties. Importantly, our proposed methodology extends beyond this specific use case, as it can be generalized to other applications beyond PET imaging.

Physics-informed deep-learning applications to experimental fluid mechanics

High-resolution reconstruction of flow-field data from low-resolution and noisy measurements is of interest due to the prevalence of such problems in experimental fluid mechanics, where the measurement data are in general sparse, incomplete and noisy. Deep-learning approaches have been shown suitable for such super-resolution tasks. However, a high number of high-resolution examples is needed, which may not be available for many cases. Moreover, the obtained predictions may lack in complying with the physical principles, e.g. mass and momentum conservation. Physics-informed deep learning provides frameworks for integrating data and physical laws for learning. In this study, we apply physics-informed neural networks (PINNs) for super-resolution of flow-field data both in time and space from a limited set of noisy measurements without having any high-resolution reference data. Our objective is to obtain a continuous solution of the problem, providing a physically-consistent prediction at any point in the solution domain. We demonstrate the applicability of PINNs for the super-resolution of flow-field data in time and space through three canonical cases: Burgers’ equation, two-dimensional vortex shedding behind a circular cylinder and the minimal turbulent channel flow. The robustness of the models is also investigated by adding synthetic Gaussian noise. Furthermore, we show the capabilities of PINNs to improve the resolution and reduce the noise in a real experimental dataset consisting of hot-wire-anemometry measurements. Our results show the adequate capabilities of PINNs in the context of data augmentation for experiments in fluid mechanics.

Surface current prediction based on a physics-informed deep learning model

Sea surface current is an important factor that affects various aspects of oceanic climate, energy, and shipping. Traditional methods for simulating current based on ocean dynamic equations are computationally complex, while existing data-driven deep learning methods overlook the constraints of actual physical processes, limiting the further improvement of prediction accuracy. In this paper, we address the issue of sea surface current prediction by proposing a deep learning model that incorporates physical processes. The model combines physical information with deep learning techniques to overcome the limitations of low computational efficiency in numerical models and the lack of interpretability in data-driven deep learning methods. This integration allows us to improve the prediction accuracy of sea surface current. Furthermore, we propose a weighted loss function into the model to address the challenge of accurately predicting high-magnitude current. The research results demonstrate that our proposed prediction model for sea surface current outperforms the baseline model, particularly in predicting high-magnitude current. Simultaneously, the model demonstrates favorable generalization performance and lower data dependency. This paper provides a new and effective method and strategy for sea surface current prediction.

A physics-informed deep learning approach for solving strongly degenerate parabolic problems

AbstractIn recent years, Scientific Machine Learning (SciML) methods for solving Partial Differential Equations (PDEs) have gained increasing popularity. Within such a paradigm, Physics-Informed Neural Networks (PINNs) are novel deep learning frameworks for solving initial-boundary value problems involving nonlinear PDEs. Recently, PINNs have shown promising results in several application fields. Motivated by applications to gas filtration problems, here we present and evaluate a PINN-based approach to predict solutions to strongly degenerate parabolic problems with asymptotic structure of Laplacian type. To the best of our knowledge, this is one of the first papers demonstrating the efficacy of the PINN framework for solving such kind of problems. In particular, we estimate an appropriate approximation error for some test problems whose analytical solutions are fortunately known. The numerical experiments discussed include two and three-dimensional spatial domains, emphasizing the effectiveness of this approach in predicting accurate solutions.

Physics-informed deep learning with Kalman filter mixture for traffic state prediction

Accurate traffic forecasting is crucial for understanding and managing congestion for efficient transportation planning. However, conventional approaches often neglect epistemic uncertainty, which arises from incomplete knowledge across different spatiotemporal scales. This study addresses this challenge by introducing a novel methodology to establish dynamic spatiotemporal correlations that captures the unobserved heterogeneity in travel time through distinct peaks in probability density functions, guided by physics-based principles. We propose an innovative approach to modifying both prediction and correction steps of the Kalman Filter (KF) algorithm by leveraging established spatiotemporal correlations. Central to our approach is the development of a novel deep learning model called the Physics Informed-Graph Convolutional Gated Recurrent Neural Network (PI-GRNN). Functioning as the state-space model within the KF, the PI-GRNN exploits established correlations to construct dynamic adjacency matrices that utilize the inherent structure and relationships within the transportation network to capture sequential patterns and dependencies over time. Furthermore, our methodology integrates insights gained from correlations into the correction step of the KF algorithm that helps in enhancing its correctional capabilities. This integrated approach proves instrumental in alleviating the inherent model drift associated with data-driven methods, as periodic corrections through update step of KF refine the predictions generated by PI-GRNN. To the best of our knowledge, this study represents a pioneering effort in integrating deep learning and KF algorithms in this unique symbiotic manner. Through extensive experimentation with real-world traffic data, we demonstrate the superior performance of our model compared to the benchmark approaches.

Effect of network architecture on physics-informed deep learning of the Reynolds-averaged turbulent flow field around cylinders without training data

Unsupervised physics-informed deep learning can be used to solve computational physics problems by training neural networks to satisfy the underlying equations and boundary conditions without labeled data. Parameters such as network architecture and training method determine the training success. However, the best choice is unknown a priori as it is case specific. Here, we investigated network shapes, sizes, and types for unsupervised physics-informed deep learning of the two-dimensional Reynolds-averaged flow around cylinders. We trained mixed-variable networks and compared them to traditional models. Several network architectures with different shape factors and sizes were evaluated. The models were trained to solve the Reynolds-averaged Navier-Stokes equations incorporating Prandtl’s mixing length turbulence model. No training data were deployed to train the models. The superiority of the mixed-variable approach was confirmed for the investigated high Reynolds number flow. The mixed-variable models were sensitive to the network shape. For the two cylinders, differently deep networks showed superior performance. The best fitting models were able to capture important flow phenomena such as stagnation regions, boundary layers, flow separation, and recirculation. We also encountered difficulties when predicting high Reynolds number flows without training data.

Evolutionary probability density reconstruction of stochastic dynamic responses based on physics-aided deep learning

Probability density evolution is the vital probabilistic information for stochastic dynamic system. However, it may face big challenges when using numerical methods to solve the generalized probability density evolution equation (GDEE) of complicated real-world system under complex excitations. Recently, physics-informed deep learning has gained popularity in solving partial differential equations due to the superior approximation and generalization capabilities, which opens a promising intelligent way to solve the GDEE. In this work, a mesh-free learning model, i.e., Phy-EPDNN, is proposed for evolutionary probability density (EPD) reconstruction, where a normalized GDEE is derived and embedded in the developed network as the prior physical knowledge. The computational domain can be customized by users on demand, in which the GDEE is not required to be solved in the whole computational domain as that in numerical methods. There is no need of boundary and initial conditions for the proposed model. A data augmentation method is also proposed to obtain sufficient collection data for supervised learning. Three specially selected analytical models and two practical scenarios verify the potential applicability of the proposed general framework for EPD reconstruction. Parametric analysis is performed to discuss the influence of the major network parameters on reconstruction performance.

Landet: an efficient physics-informed deep learning approach for automatic detection of anatomical landmarks and measurement of spinopelvic alignment

Purpose:An efficient physics-informed deep learning approach for extracting spinopelvic measures from X-ray images is introduced and its performance is evaluated against manual annotations.Methods:Two datasets, comprising a total of 1470 images, were collected to evaluate the model’s performance. We propose a novel method of detecting landmarks as objects, incorporating their relationships as constraints (LanDet). Using this approach, we trained our deep learning model to extract five spine and pelvis measures: Sacrum Slope (SS), Pelvic Tilt (PT), Pelvic Incidence (PI), Lumbar Lordosis (LL), and Sagittal Vertical Axis (SVA). The results were compared to manually labelled test dataset (GT) as well as measures annotated separately by three surgeons.Results:The LanDet model was evaluated on the two datasets separately and on an extended dataset combining both. The final accuracy for each measure is reported in terms of Mean Absolute Error (MAE), Standard Deviation (SD), and R Pearson correlation coefficient as follows: [SS∘:3.7(2.7),R=0.89]\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$[SS^\\circ : 3.7 (2.7), R = 0.89]$$\\end{document}, [PT∘:1.3(1.1),R=0.98],[PI∘:4.2(3.1),R=0.93],[LL∘:5.1(6.4),R=0.83],[SVA(mm):2.1(1.9),R=0.96]\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$[PT^\\circ : 1.3 (1.1), R = 0.98], [PI^\\circ : 4.2 (3.1), R = 0.93], [LL^\\circ : 5.1 (6.4), R=0.83], [SVA(mm): 2.1 (1.9), R = 0.96]$$\\end{document}. To assess model reliability and compare it against surgeons, the intraclass correlation coefficient (ICC) metric is used. The model demonstrated better consistency with surgeons with all values over 0.88 compared to what was previously reported in the literature.Conclusion:The LanDet model exhibits competitive performance compared to existing literature. The effectiveness of the physics-informed constraint method, utilized in our landmark detection as object algorithm, is highlighted. Furthermore, we addressed the limitations of heatmap-based methods for anatomical landmark detection and tackled issues related to mis-identifying of similar or adjacent landmarks instead of intended landmark using this novel approach.

Physics-Informed Graph Neural Networks for Water Distribution Systems

Water distribution systems (WDS) are an integral part of critical infrastructure which is pivotal to urban development. As 70% of the world's population will likely live in urban environments in 2050, efficient simulation and planning tools for WDS play a crucial role in reaching UN's sustainable developmental goal (SDG) 6 - "Clean water and sanitation for all". In this realm, we propose a novel and efficient machine learning emulator, more precisely, a physics-informed deep learning (DL) model, for hydraulic state estimation in WDS. Using a recursive approach, our model only needs a few graph convolutional neural network (GCN) layers and employs an innovative algorithm based on message passing. Unlike conventional machine learning tasks, the model uses hydraulic principles to infer two additional hydraulic state features in the process of reconstructing the available ground truth feature in an unsupervised manner. To the best of our knowledge, this is the first DL approach to emulate the popular hydraulic simulator EPANET, utilizing no additional information. Like most DL models and unlike the hydraulic simulator, our model demonstrates vastly faster emulation times that do not increase drastically with the size of the WDS. Moreover, we achieve high accuracy on the ground truth and very similar results compared to the hydraulic simulator as demonstrated through experiments on five real-world WDS datasets.

A physics-informed deep learning liquid crystal camera with data-driven diffractive guidance

Whether in the realms of computer vision, robotics, or environmental monitoring, the ability to monitor and follow specific targets amidst intricate surroundings is essential for numerous applications. However, achieving rapid and efficient target tracking remains a challenge. Here we propose an optical implementation for rapid tracking with negligible digital post-processing, leveraging an all-optical information processing. This work combines a diffractive-based optical nerual network with a layered liquid crystal electrical addressing architecture, synergizing the parallel processing capabilities inherent in light propagation with liquid crystal dynamic adaptation mechanism. Through a one-time effort training, the trained network enable accurate prediction of the desired arrangement of liquid crystal molecules as confirmed through numerical blind testing. Then we establish an experimental camera architecture that synergistically combines an electrically-tuned functioned liquid crystal layer with materialized optical neural network. With integrating the architecture into optical imaging path of a detector plane, this optical computing camera offers a data-driven diffractive guidance, enabling the identification of target within complex backgrounds, highlighting its high-level vision task implementation and problem-solving capabilities.

Physics-informed deep learning to quantify anomalies for real-time fault mitigation in 3D printing

In 3D printing processes, there are many thermal stress related defects that can have a significant negative impact on the shape and size of the structure. Such anomalies in the heat transfer of the printing process need to be detected at an early stage. Understanding heat transfer is crucial, and simulation models can offer insights while reducing the need for costly experiments. Traditional numerical solvers for heat transfer can be complex to adapt to diverse printed part geometries, and their reliance on predefined mathematical models limits their flexibility. Our physics-informed deep learning (PIDL) approach eliminates the need for discretization, simplifying the analysis of complex geometries and enabling automation. The drawback of parametric PIDL is their scalability for high-dimensional problems. Computational time, energy and cost of training prevent real-time analysis. It often takes only a few seconds to print a single layer. We can show an energy efficient transfer and training strategy to reduce the computational effort of PIDL significantly. The approach is able to quantify relevant effects of thermal stresses and mitigate errors during selective laser melting (SLM). To this end, heat transfer is modelled, simulated and analysed using high-dimensional data obtained from printing experiments with different geometries of metal components. The proposed method is applied to the solving forward problem of heat transfer prediction. The governing results are based on the heat equation, which is integrated into a deep neural network (DNN).

Data-efficient surrogate modeling using meta-learning and physics-informed deep learning approaches

This paper proposes physics-informed meta-learning-based surrogate modeling (PI-MLSM), a novel approach that combines meta-learning and physics-informed deep learning to train surrogate models with limited labeled data. PI-MLSM consists of two stages: meta-learning and physics-informed task adaptation. The proposed approach is demonstrated to outperform other methods in four numerical examples while reducing errors in prediction and reliability analysis, exhibiting robustness, and requiring less labeled data during optimization. Moreover, compared to other approaches, the proposed approach exhibits better performance in solving out-of-distribution tasks. Although this paper acknowledges certain limitations and challenges, such as the subjective nature of physical information, it highlights the key contributions of PI-MLSM, including its effectiveness in solving a wide range of tasks and its ability in handling situations wherein physical laws are not explicitly known. Overall, PI-MLSM demonstrates potential as a powerful and versatile approach for surrogate modeling.

Physics-Informed Deep Learning Approach for Reintroducing Atomic Detail in Coarse-Grained Configurations of Multiple Poly(lactic acid) Stereoisomers.

Multiscale modeling of complex molecular systems, such as macromolecules, encompasses methods that combine information from fine and coarse representations of molecules to capture material properties over a wide range of spatiotemporal scales. Being able to exchange information between different levels of resolution is essential for the effective transfer of this information. The inverse problem of reintroducing atomistic degrees of freedom in coarse-grained (CG) molecular configurations is particularly challenging as, from a mathematical point of view, it is an ill-posed problem; the forward mapping from the atomistic to the CG description is typically defined via a deterministic operator ("one-to-one" problem), whereas the reversed mapping from the CG to the atomistic model refers to creating one representative configuration out of many possible ones ("one-to-many" problem). Most of the backmapping methods proposed so far balance accuracy, efficiency, and general applicability. This is particularly important for macromolecular systems with different types of isomers, i.e., molecules that have the same molecular formula and sequence of bonded atoms (constitution) but differ in the three-dimensional configurations of their atoms in space. Here, we introduce a versatile deep learning approach for backmapping multicomponent CG macromolecules with chiral centers, trained to learn structural correlations between polymer configurations at the atomistic level and their corresponding CG descriptions. This method is intended to be simple and flexible while presenting a generic solution for resolution transformation. In addition, the method is aimed to respect the structural features of the molecule, such as local packing, capturing therefore the physical properties of the material. As an illustrative example, we apply the model on linear poly(lactic acid) (PLA) in melt, which is one of the most popular biodegradable polymers. The framework is tested on a number of model systems starting from homopolymer stereoisomers of PLA to copolymers with randomly placed chiral centers. The results demonstrate the efficiency and efficacy of the new approach.

Amplitude/Phase Retrieval for Terahertz Holography With Supervised and Unsupervised Physics-Informed Deep Learning

Amplitude/Phase Retrieval for Terahertz Holography With Supervised and Unsupervised Physics-Informed Deep Learning

Modeling the anaerobic digestion of palm oil mill effluent via physics-informed deep learning

The modeling of anaerobic digestion (AD) plays a significant role in process monitoring and prediction. Typically, mechanistic models or machine learning are used to model the AD process, where the former relies on the first-principle knowledge, while the latter learns the trend in data. In this study, models of AD were developed using data obtained from two palm oil mill effluent (POME) AD industrial plants, here they are denoted as plant A and plant B. The mechanistic model ADM1-R4 was first used to simulate POME AD to predict 3 AD outputs – methane, carbon dioxide, and effluent concentration. However, moderate prediction accuracy was observed, this could be due to highly dynamic environments in industrial plants and the incomplete knowledge of the mechanistic model where only 3 input features were considered. Then, artificial neural networks (ANNs) were trained using simulated data from the variational autoencoder for training and making the same predictions on the test data. ANNs were more inclusive as they included 6 or 7 input features and were found to make predictions with higher accuracy. Moreover, different ANN architectures were also investigated. Nevertheless, ANNs were time-consuming to train. Hence, physics-informed neural networks, namely ADM1-R4-NN, were further developed by embedding the information of mechanistic equations into the loss function of ANN. Overall, ADM1-R4-NN outperformed ANN and ADM1-R4, showing higher training efficiency (vs ANN) and testing accuracy (vs ANN and ADM1-R4). For plant A, the best prediction of ADM1-R4-NN resulted in R2 values of 0.87–0.95, while for plant B, the R2 was 0.81–0.88. Finally, feature importance analysis was conducted, where hydraulic retention time and mass loading were found to be the top influencing factors, while pretreatment temperature was a significant factor as well. The developed ADM1-R4-NN is a promising model for POME AD, and this method has the potential to be applied to other wastewater AD processes.

Physics-informed Deep Learning to Solve 2D Electromagnetic Scattering Problems

The utilization of physics-informed deep learning (PI-DL) methodologies provides an approach to augment the predictive capabilities of deep learning (DL) models by constraining them with known physical principles. We utilize a PI-DL model called the deep operator network (DeepONet) to solve two-dimensional (2D) electromagnetic (EM) scattering problems. Numerical results demonstrate that the discrepancy between the DeepONet and conventional method of moments (MoM) is small, while maintaining computational efficiency.

Physics-informed deep learning for multi-species membrane separations

Conventional continuum models for ion transport across polyamide membranes require solving partial differential equations (PDEs). These models typically introduce a host of assumptions and simplifications to improve the computational tractability of existing solvers. As a consequence of these constraints, conventional models struggle to generalize predictive performance to new unseen conditions. Deep learning has recently shown promise in alleviating many of these concerns, making it a promising avenue for surrogate models that can replace conventional PDE-based approaches. In this work, we develop a physics-informed deep learning model to predict ion transport across diverse membrane types. The proposed architecture leverages neural differential equations in conjunction with classical closure models as inductive biases directly encoded into the neural framework. The neural methods are pre-trained on simulated data from continuum models and fine-tuned on independent experiments to learn multi-ionic rejection behaviour. We also harness the attention mechanism, commonly observed in language modelling, to learn and infer key paired transport relationships. Gaussian noise augmentations from experimental uncertainty estimates are also introduced into the measured data to improve robustness and generalization. We study the neural framework’s performance relative to conventional PDE-based methods, and also compare the use of hard/soft inductive bias constraints on prediction accuracy. Lastly, we compare our approach to other competitive deep learning architectures and illustrate strong agreement with experimental measurements across all studied datasets.

Explicit Physics-Informed Deep Learning for Computer-Aided Diagnostic Tasks in Medical Imaging

DNN-based systems have demonstrated unprecedented performance in terms of accuracy and speed over the past decade. However, recent work has shown that such models may not be sufficiently robust during the inference process. Furthermore, due to the data-driven learning nature of DNNs, designing interpretable and generalizable networks is a major challenge, especially when considering critical applications such as medical computer-aided diagnostics (CAD) and other medical imaging tasks. Within this context, a line of approaches incorporating prior knowledge domain information into deep learning methods has recently emerged. In particular, many of these approaches utilize known physics-based forward imaging models, aimed at improving the stability and generalization ability of DNNs for medical imaging applications. In this paper, we review recent work focused on such physics-based or physics-prior-based learning for a variety of imaging modalities and medical applications. We discuss how the inclusion of such physics priors to the training process and/or network architecture supports their stability and generalization ability. Moreover, we propose a new physics-based approach, in which an explicit physics prior, which describes the relation between the input and output of the forward imaging model, is included as an additional input into the network architecture. Furthermore, we propose a tailored training process for this extended architecture, for which training data are generated with perturbed physical priors that are also integrated into the network. Within the scope of this approach, we offer a problem formulation for a regression task with a highly nonlinear forward model and highlight possible useful applications for this task. Finally, we briefly discuss future challenges for physics-informed deep learning in the context of medical imaging.