Physics-informed Machine Learning Research Articles

Efficient Bayesian Physics Informed Neural Networks for inverse problems via Ensemble Kalman Inversion

Bayesian Physics Informed Neural Networks (B-PINNs) have gained significant attention for inferring physical parameters and learning the forward solutions for problems based on partial differential equations. However, the overparameterized nature of neural networks poses a computational challenge for high-dimensional posterior inference. Existing inference approaches, such as particle-based or variance inference methods, are either computationally expensive for high-dimensional posterior inference or provide unsatisfactory uncertainty estimates. In this paper, we present a new efficient inference algorithm for B-PINNs that uses Ensemble Kalman Inversion (EKI) for high-dimensional inference tasks. By reframing the setup of B-PINNs as a traditional Bayesian inverse problem, we can take advantage of EKI's key features: (1) gradient-free, (2) computational complexity scales linearly with the dimension of the parameter spaces, and (3) rapid convergence with typically O(100) iterations. We demonstrate the applicability and performance of the proposed method through various types of numerical examples. We find that our proposed method can achieve inference results with informative uncertainty estimates comparable to Hamiltonian Monte Carlo (HMC)-based B-PINNs with a much reduced computational cost. These findings suggest that our proposed approach has great potential for uncertainty quantification in physics-informed machine learning for practical applications.

Physics-Informed Machine Learning for metal additive manufacturing

Physics-Informed Machine Learning for metal additive manufacturing

Optimizing a Physics-Informed Machine Learning Model for Pulsatile Shear-Thinning Channel Flow

Optimizing a Physics-Informed Machine Learning Model for Pulsatile Shear-Thinning Channel Flow

Designing a set of criteria for evaluating artificial neural networks trained with physics-based data to replicate molecular dynamics and other particle method trajectories

This article presents an in-depth analysis and evaluation of artificial neural networks (ANNs) when applied to replicate trajectories in molecular dynamics (MD) simulations or other particle methods. This study focuses on several architectures—feedforward neural networks (FNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), time convolutions (TCs), self-attention (SA), graph neural networks (GNNs), neural ordinary differential equation (ODENets), and an example of physics-informed machine learning (PIML) model—assessing their effectiveness and limitations in understanding and replicating the underlying physics of particle systems. Through this analysis, this paper introduces a comprehensive set of criteria designed to evaluate the capability of ANNs in this context. These criteria include the minimization of losses, the permutability of particle indices, the ability to predict trajectories recursively, the conservation of particles, the model’s handling of boundary conditions, and its scalability. Each network type is systematically examined to determine its strengths and weaknesses in adhering to these criteria. While, predictably, none of the networks fully meets all criteria, this study extends beyond the simple conclusion that only by integrating physics-based models into ANNs is it possible to fully replicate complex particle trajectories. Instead, it probes and delineates the extent to which various neural networks can “understand” and interpret aspects of the underlying physics, with each criterion targeting a distinct aspect of this understanding.

Constraint free physics-informed machine learning for micromagnetic energy minimization

We introduce a novel method for micromagnetic energy minimization which uses physics-informed neural networks to find a magnetic configuration which minimizes the Gibbs Free energy functional without the need of any constraint optimization framework. The Cayley transform is applied to a neural network to assure that the model output lives on the Lie group of rotation matrices SO(3). For the stray field computation we use the splitting ansatz of Garcia-Cervera and Roma together with a hard constraint extreme learning machine in combination with a Taylor series approximation for very accurate evaluation of the single layer potential which only requires a very coarse discretization of the surface. Further, we present a modeling framework for constructive solid geometry which uses R-functions to exactly satisfy essential boundary conditions arising in the course of stray field computation. This framework can be applied to many other areas of interest. Our method shows promising results on the NIST μMAG Standard Problem #3, and is also applied to compute the demagnetization process of a hard magnetic Nd2Fe14B cube.

Unit Operation and Process Modeling with Physics-Informed Machine Learning

Unit Operation and Process Modeling with Physics-Informed Machine Learning

Machine Learning in Petrology: State-of-the-Art and Future Perspectives

Abstract This article reports on the state-of-the-art and future perspectives of machine learning (ML) in petrology. To achieve this goal, it first introduces the basics of ML, including definitions, core concepts, and applications. Then, it starts reviewing the state-of-the-art of ML in petrology. Established applications mainly concern the so-called data-driven discovery and involve specific tasks like clustering, dimensionality reduction, classification, and regression. Among them, clustering and dimensionality reduction have been demonstrated to be valuable for decoding the chemical record stored in igneous and metamorphic phases and to enhance data visualization, respectively. Classification and regression tasks find applications, for example, in petrotectonic discrimination and geo-thermobarometry, respectively. The main core of the manuscript consists of depicting emerging trends and the future directions of ML in petrological investigations. I propose a future scenario where ML methods will progressively integrate and support established petrological methods in automating time-consuming and repetitive tasks, improving current models, and boosting discovery. In this framework, promising applications include (1) the acquisition of new multimodal petrologic data; (2) the development of data fusion techniques, physics-informed ML models, and ML-supported numerical simulations; and (3) the continuous exploration of the ML potential in petrology. To boost the contribution of ML in petrology, our main challenges are: (1) to improve the ability of ML models to capture the complexity of petrologic processes, (2) progressively link ML algorithms with the physical and thermodynamic nature of the investigated problems, and (3) to start a collaborative effort among researchers coming from different disciplines, both in research and teaching.

Unsupervised Denoising and Super-Resolution of Vascular Flow Data by Physics-Informed Machine Learning.

We present an unsupervised deep learning method to perform flow denoising and super-resolution without high-resolution labels. We demonstrate the ability of a single model to reconstruct three-dimensional stenosis and aneurysm flows, with varying geometries, orientations, and boundary conditions. Ground truth data was generated using computational fluid dynamics, and then corrupted with multiplicative Gaussian noise. Auto-encoders were used to compress the representations of the flow domain geometry and the (possibly noisy and low-resolution) flow field. These representations were used to condition a physics-informed neural network. A physics-based loss was implemented to train the model to recover lost information from the noisy input by transforming the flow to a solution of the Navier-Stokes equations. Our experiments achieved mean squared errors in the true flow reconstruction of O(1.0 × 10-4), and root mean squared residuals of O(1.0 × 10-2) for the momentum and continuity equations. Our method yielded correlation coefficients of 0.971 for the hidden pressure field and 0.82 for the derived wall shear stress field. By performing point-wise predictions of the flow, the model was able to robustly denoise and super-resolve the field to 20× the input resolution.

Neural Oscillators for Generalization of Physics-Informed Machine Learning

A primary challenge of physics-informed machine learning (PIML) is its generalization beyond the training domain, especially when dealing with complex physical problems represented by partial differential equations (PDEs). This paper aims to enhance the generalization capabilities of PIML, facilitating practical, real-world applications where accurate predictions in unexplored regions are crucial. We leverage the inherent causality and temporal sequential characteristics of PDE solutions to fuse PIML models with recurrent neural architectures based on systems of ordinary differential equations, referred to as neural oscillators. Through effectively capturing long-time dependencies and mitigating the exploding and vanishing gradient problem, neural oscillators foster improved generalization in PIML tasks. Extensive experimentation involving time-dependent nonlinear PDEs and biharmonic beam equations demonstrates the efficacy of the proposed approach. Incorporating neural oscillators outperforms existing state-of-the-art methods on benchmark problems across various metrics. Consequently, the proposed method improves the generalization capabilities of PIML, providing accurate solutions for extrapolation and prediction beyond the training data.

SNN-PDE: Learning Dynamic PDEs from Data with Simplicial Neural Networks

Dynamics of many complex systems, from weather and climate to spread of infectious diseases, can be described by partial differential equations (PDEs). Such PDEs involve unknown function(s), partial derivatives, and typically multiple independent variables. The traditional numerical methods for solving PDEs assume that the data are observed on a regular grid. However, in many applications, for example, weather and air pollution monitoring delivered by the arbitrary located weather stations of the National Weather Services, data records are irregularly spaced. Furthermore, in problems involving prediction analytics such as forecasting wildfire smoke plumes, the primary focus may be on a set of irregular locations associated with urban development. In recent years, deep learning (DL) methods and, in particular, graph neural networks (GNNs) have emerged as a new promising tool that can complement traditional PDE solvers in scenarios of the irregular spaced data, contributing to the newest research trend of physics informed machine learning (PIML). However, most existing PIML methods tend to be limited in their ability to describe higher dimensional structural properties exhibited by real world phenomena, especially, ones that live on manifolds. To address this fundamental challenge, we bring the elements of the Hodge theory and, in particular, simplicial convolution defined on the Hodge Laplacian to the emerging nexus of DL and PDEs. In contrast to conventional Laplacian and the associated convolution operation, the simplicial convolution allows us to rigorously describe diffusion across higher order structures and to better approximate the complex underlying topology and geometry of the data. The new approach, Simplicial Neural Networks for Partial Differential Equations (SNN PDE) offers a computationally efficient yet effective solution for time dependent PDEs. Our studies of a broad range of synthetic data and wildfire processes demonstrate that SNN PDE improves upon state of the art baselines in handling unstructured grids and irregular time intervals of complex physical systems and offers competitive forecasting capabilities for weather and air quality forecasting.

Social Physics Informed Diffusion Model for Crowd Simulation

Crowd simulation holds crucial applications in various domains, such as urban planning, architectural design, and traffic arrangement. In recent years, physics-informed machine learning methods have achieved state-of-the-art performance in crowd simulation but fail to model the heterogeneity and multi-modality of human movement comprehensively. In this paper, we propose a social physics-informed diffusion model named SPDiff to mitigate the above gap. SPDiff takes both the interactive and historical information of crowds in the current timeframe to reverse the diffusion process, thereby generating the distribution of pedestrian movement in the subsequent timeframe. Inspired by the well-known social physics model, i.e., Social Force, regarding crowd dynamics, we design a crowd interaction encoder to guide the denoising process and further enhance this module with the equivariant properties of crowd interactions. To mitigate error accumulation in long-term simulations, we propose a multi-frame rollout training algorithm for diffusion modeling. Experiments conducted on two real-world datasets demonstrate the superior performance of SPDiff in terms of both macroscopic and microscopic evaluation metrics. Code and appendix are available at https://github.com/tsinghua-fib-lab/SPDiff.

Developing a hybrid data-driven and informed model for prediction and mitigation of agricultural nitrous oxide flux hotspots

Nitrous oxide (N2O) is one of the most significant contributors to greenhouse forcing and is the biggest contributor to ozone depletion in the 21st century, and roughly 70% of anthropogenic nitrous oxide emissions are from agriculture and soil management. Agricultural nitrous oxide emissions are shown to spike during hotspot events, and according to the data used in this study, over 78% of nitrous oxide flux occurred during just 15% of the recorded data points. Due to the complex biogeochemical processes governing nitrous oxide formation, machine learning and process-based models often fail to predict agricultural nitrous oxide flux. A novel informed neural network was developed that combined the trainability of neural networks with the rigorous differential equation-based framework of process-based models. Differential equations that explained the variability of various nitrogen-containing compounds in soil were derived, and integrated into the network loss. The informed model explained ∼85% of variation in the data and had an F1 score of 0.75, a marked improvement over the classical model explaining ∼30% of variation and having a score of 0.53. The informed network was also able to perform exceptionally well with only small subsets of the training data, having an F1 score of 0.41 with only 25% of training data. The model not only shows great promise in the remarkably accurate prediction of these hotspots but also serves as a potential new paradigm for physics-informed machine learning techniques in environmental and agricultural sciences.

Transfer learning and pretraining enhanced physics-informed machine learning for closed-circuit reverse osmosis modeling

Closed-circuit reverse osmosis (CCRO) is a widely concerned batch-type desalination process that exhibits dynamic, multi-mode, and cyclic behavior. This study provides a novel physics-informed machine learning method that integrate pretraining and transfer learning (PT-TL) to construct spatiotemporal model of the CCRO process. In this model, two types of networks are specifically tailored to approximate the latent solutions of the closed-circuit and flushing modes within each running cycle. To facilitate long-time integration of partial differential equations in the closed-circuit mode, time-adaptive decomposition is utilized in parameter transfer learning to identify appropriate sequence partitioning and accelerate the learning process. During the pretraining step, a coarse-grained model is constructed by adjusting the linear initial conditions of the flushing mode to capture time-varying characteristics. The integration of PT-TL with physics-informed machine learning not only reduces training time by over 50 % but also demonstrates comparable predictive ability to traditional numerical methods.

Physics-informed machine learning for battery degradation diagnostics: A comparison of state-of-the-art methods

Monitoring the health of lithium-ion batteries’ internal components as they age is crucial for optimizing cell design and usage control strategies. However, quantifying component-level degradation typically involves aging many cells and destructively analyzing them throughout the aging test, limiting the scope of quantifiable degradation to the test conditions and duration. Fortunately, recent advances in physics-informed machine learning (PIML) for modeling and predicting the battery state of health demonstrate the feasibility of building models to predict the long-term degradation of a lithium-ion battery cell’s major components using only short-term aging test data by leveraging physics. In this paper, we present four approaches for building physics-informed machine learning models and comprehensively compare them, considering accuracy, complexity, ease-of-implementation, and their ability to extrapolate to untested conditions. We delve into the details of each physics-informed machine learning method, providing insights specific to implementing them on small battery aging datasets. Our study utilizes long-term cycle aging data from 24 implantable-grade lithium-ion cells subjected to varying temperatures and C-rates over four years. This paper aims to facilitate the selection of an appropriate physics-informed machine learning method for predicting long-term degradation in lithium-ion batteries, using short-term aging data while also providing insights about when to choose which method for general predictive purposes.

Physics-informed machine learning in cyber-attack detection and resilient control of chemical processes

With the integration of internet of things (IoT) devices, cloud computing, and other digital technologies into chemical processes, the complexity and stealthiness of cyber-attacks have increased. To mitigate the impact of sensor cyber-attacks in chemical processes, this work presents a framework that develops physics-informed machine learning (PIML)-based detectors and resilient controllers for improving closed-loop performance of nonlinear system under cyber-attacks. The PIML detector is constructed through a customized loss function that integrates the domain knowledge of cyber-attacks into the training process. Additionally, upon detection of attacks, a knowledge-guided extended Kalman filter is developed to provide estimated states for resilient control prior to replacement by redundant sensors. A chemical process example is used to illustrate the application of the proposed PIML-based detection and resilient control methods to handle cyber-attacks.

Bending information of nanocomposites-reinforced microplate subjected to transient loading: Introducing physics-informed machine learning algorithm for solving the transient problem

Due to the importance of microplate structure in some related industries especially in microchips, transient dynamics analysis of microplate has got a lot of attention, recently. For this issue, in the current work, for the first time, transient dynamics analysis of microplate reinforced by graphene nanoplatelets composites (GPCs) using both mathematical modeling and physics-informed machine learning algorithm is presented. In the mathematical modeling section, the Halpin-Tsai correlation statement, and the role of mixture are used to simulate the composite structure. The microsystem is modeled using the classical and deviatoric components of the symmetric couple stress tensor. The transient dynamic loading with three different distribution patterns is studied to present the transient reaction of the presented microsystem to external shock. For solving the equations, a higher-order finite element model (HOFEM) with time-dependent nodal displacements, interpolation functions, and inverting the Laplace transform are used. In the machine learning section, the physics-informed machine learning algorithm is used to train and test the dataset of the mathematical modeling section. After training and testing, it is demonstrated that the physics-informed machine learning algorithm method with low computational cost can be used instead of the presented higher-order finite element model. The results show that, in comparison to alternative physics-informed machine learning algorithm configurations, having four hidden layers with the necessary number of neurons results in the highest prediction accuracy. Finally, some applicable suggestions for improving the stability of the nanocomposites-reinforced microplate under dynamical load will be presented for related industries.

Advancing material property prediction: using physics-informed machine learning models for viscosity

In materials science, accurately computing properties like viscosity, melting point, and glass transition temperatures solely through physics-based models is challenging. Data-driven machine learning (ML) also poses challenges in constructing ML models, especially in the material science domain where data is limited. To address this, we integrate physics-informed descriptors from molecular dynamics (MD) simulations to enhance the accuracy and interpretability of ML models. Our current study focuses on accurately predicting viscosity in liquid systems using MD descriptors. In this work, we curated a comprehensive dataset of over 4000 small organic molecules’ viscosities from scientific literature, publications, and online databases. This dataset enabled us to develop quantitative structure–property relationships (QSPR) consisting of descriptor-based and graph neural network models to predict temperature-dependent viscosities for a wide range of viscosities. The QSPR models reveal that including MD descriptors improves the prediction of experimental viscosities, particularly at the small data set scale of fewer than a thousand data points. Furthermore, feature importance tools reveal that intermolecular interactions captured by MD descriptors are most important for viscosity predictions. Finally, the QSPR models can accurately capture the inverse relationship between viscosity and temperature for six battery-relevant solvents, some of which were not included in the original data set. Our research highlights the effectiveness of incorporating MD descriptors into QSPR models, which leads to improved accuracy for properties that are difficult to predict when using physics-based models alone or when limited data is available.Graphical

Leave No Photon Behind: Artificial Intelligence in Multiscale Physics of Photocatalyst and Photoreactor Design.

Although solar fuels photocatalysis offers the promise of converting carbon dioxide directly with sunlight as commercially scalable solutions have remained elusive over the past few decades, despite significant advancements in photocatalysis band-gap engineering and atomic site activity. The primary challenge lies not in the discovery of new catalyst materials, which are abundant, but in overcoming the bottlenecks related to material-photoreactor synergy. These factors include achieving photogeneration and charge-carrier recombination at reactive sites, utilizing high mass transfer efficiency supports, maximizing solar collection, and achieving uniform light distribution within a reactor. Addressing this multi-dimensional problem necessitates harnessing machine learning techniques to analyze real-world data from photoreactors and material properties. In this perspective, the challenges are outlined associated with each bottleneck factor, review relevant data analysis studies, and assess the requirements for developing a comprehensive solution that can unlock the full potential of solar fuels photocatalysis technology. Physics-informed machine learning (or Physics Neural Networks) may be the key to advancing this important area from disparate data towards optimal reactor solutions.

Microstructure-Sensitive Deformation Modeling and Materials Design with Physics-Informed Neural Networks

Microstructure-sensitive materials design has become popular among materials engineering researchers in the last decade because it allows the control of material performance through the design of microstructures. In this study, the microstructure is defined by an orientation distribution function. A physics-informed machine learning approach is integrated into microstructure design to improve the accuracy, computational efficiency, and explainability of microstructure-sensitive design. When data generation is costly and numerical models need to follow certain physical laws, machine learning models that are domain-aware perform more efficiently than conventional machine learning models. Therefore, a new paradigm called the physics-informed neural network (PINN) is introduced in the literature. This study applies the PINN to microstructure-sensitive modeling and inverse design to explore the material behavior under deformation processing. In particular, we demonstrate the application of PINN to small-data problems driven by a crystal plasticity model that needs to satisfy the physics-based design constraints of the microstructural orientation space. For the first problem, we predict the microstructural texture evolution of copper during a tensile deformation process as a function of initial texturing and strain rate. The second problem aims to calibrate the crystal plasticity parameters of the Ti-7Al alloy by solving an inverse design problem to match the PINN-predicted final texture prediction and the experimental data.

Optimisation of quantitative brain diffusion-relaxation MRI acquisition protocols with physics-informed machine learning

Diffusion-relaxation MRI aims to extract quantitative measures that characterise microstructural tissue properties such as orientation, size, and shape, but long acquisition times are typically required. This work proposes a physics-informed learning framework to extract an optimal subset of diffusion-relaxation MRI measurements for enabling shorter acquisition times, predict non-measured signals, and estimate quantitative parameters.In vivo and synthetic brain 5D-Diffusion-T1-T2∗-weighted MRI data obtained from five healthy subjects were used for training and validation, and from a sixth participant for testing. One fully data-driven and two physics-informed machine learning methods were implemented and compared to two manual selection procedures and Cramér–Rao lower bound optimisation.The physics-informed approaches could identify measurement-subsets that yielded more consistently accurate parameter estimates in simulations than other approaches, with similar signal prediction error. Five-fold shorter protocols yielded error distributions of estimated quantitative parameters with very small effect sizes compared to estimates from the full protocol. Selected subsets commonly included a denser sampling of the shortest and longest inversion time, lowest echo time, and high b-value.The proposed framework combining machine learning and MRI physics offers a promising approach to develop shorter imaging protocols without compromising the quality of parameter estimates and signal predictions.