Types Of Partial Differential Equations Research Articles

An Interpretable Approach to the Solutions of High-Dimensional Partial Differential Equations

In recent years, machine learning algorithms, especially deep learning, have shown promising prospects in solving Partial Differential Equations (PDEs). However, as the dimension increases, the relationship and interaction between variables become more complex, and existing methods are difficult to provide fast and interpretable solutions for high-dimensional PDEs. To address this issue, we propose a genetic programming symbolic regression algorithm based on transfer learning and automatic differentiation to solve PDEs. This method uses genetic programming to search for a mathematically understandable expression and combines automatic differentiation to determine whether the search result satisfies the PDE and boundary conditions to be solved. To overcome the problem of slow solution speed caused by large search space, we propose a transfer learning mechanism that transfers the structure of one-dimensional PDE analytical solution to the form of high-dimensional PDE solution. We tested three representative types of PDEs, and the results showed that our proposed method can obtain reliable and human-understandable real solutions or algebraic equivalent solutions of PDEs, and the convergence speed is better than the compared methods. Code of this project is at https://github.com/grassdeerdeer/HD-TLGP.

A physics-informed deep learning framework for modeling of coronary in-stent restenosis.

Machine learning (ML) techniques have shown great potential in cardiovascular surgery, including real-time stenosis recognition, detection of stented coronary anomalies, and prediction of in-stent restenosis (ISR). However, estimating neointima evolution poses challenges for ML models due to limitations in manual measurements, variations in image quality, low data availability, and the difficulty of acquiring biological quantities. An effective in silico model is necessary to accurately capture the mechanisms leading to neointimal hyperplasia. Physics-informed neural networks (PINNs), a novel deep learning (DL) method, have emerged as a promising approach that integrates physical laws and measurements into modeling. PINNs have demonstrated success in solving partial differential equations (PDEs) and have been applied in various biological systems. This paper aims to develop a robust multiphysics surrogate model for ISR estimation using the physics-informed DL approach, incorporating biological constraints and drug elution effects. The model seeks to enhance prediction accuracy, provide insights into disease progression factors, and promote ISR diagnosis and treatment planning. A set of coupled advection-reaction-diffusion type PDEs is constructed to track the evolution of the influential factors associated with ISR, such as platelet-derived growth factor (PDGF), the transforming growth factor- (TGF- ), the extracellular matrix (ECM), the density of smooth muscle cells (SMC), and the drug concentration. The nature of PINNs allows for the integration of patient-specific data (procedure-related, clinical and genetic, etc.) into the model, improving prediction accuracy and assisting in the optimization of stent implantation parameters to mitigate risks. This research addresses the existing gap in predictive models for ISR using DL and holds the potential to enhance patient outcomes through predictive risk assessment.

Machine learning for a class of partial differential equations with multi-delays based on numerical Gaussian processes

Delay partial differential equations (PDEs) are widely utilized in many fields, such as climate prediction and epidemiology. But observation data in real world is often noisy and discrete. And in order to expand the applications of delay PDEs, we consider numerical Gaussian processes to solve these models. In this paper, numerical Gaussian processes for predicting the latent solution of a type of delay PDEs with multi-delays are investigated, and various delay PDEs are studied, including problems governed by variable-order fractional order operators and nonlinear operators, so as to adapt to the needs of practical applications. Numerical Gaussian processes are very good at fitting latent solution of PDEs, when all observation data is noisy and discontinuous. And the methodology can clearly quantify the uncertainty of the predicted solution. For complex boundaries controlled by ODEs, we consider mixed boundary conditions of delay PDEs in this paper. And we also apply Runge-Kutta methods to enhance the prediction accuracy of these problems. Finally, we design seven numerical examples to investigate the efficiency of NGPs and how the noisy data influences the solution of our studied problems.

A novel Eulerian model based on central moments to simulate age and reactivity continua interacting with mixing processes

Abstract. In geoscientific models, simulating the properties associated with particles in a continuum can serve many scientific purposes, and this has commonly been addressed using Lagrangian models. As an alternative approach, we present an Eulerian method here: diffusion–advection–reaction type partial differential equations are derived for centralized moments, which can describe the distribution of properties associated with chemicals in reaction–transport models. When the property is age, the equations for centralized moments (unlike non-central moments) do not require terms to account for aging, making this method suitable for modeling age tracers. The properties described by the distributions may also represent kinetic variables affecting reaction rates. In practical applications, continuous distributions of ages and reactivities are resolved to simulate organic matter mineralization in surficial sediments, where macrofaunal and physical mixing processes typically dominate transport. In test simulations, mixing emerged as the predominant factor shaping reactivity and age distributions. Furthermore, the applications showcase the method's aptitude for modeling continua in mixed environments while also highlighting practical considerations and challenges.

Numerical simulation and analysis of Airy's-type equation

Abstract In this article, we propose a novel new iteration method and homotopy perturbation method (HPM) along with the Elzaki transform to compute the analytical and semi-analytical approximations of fractional Airy’s-type partial differential equations (FAPDEs) subjected to specific initial conditions. A convergent series solution form with easily commutable coefficients is used to examine and compare the performance of the suggested methods. Using Maple graphical method analysis, the behavior of the estimated series results at various fractional orders ς \varsigma and its modeling in two-dimensional (2D) and three-dimensional (3D) spaces are compared with actual results. Also, detailed descriptions of the physical and geometric implications of the calculated graphs in 2D and 3D spaces are provided. As a result, the obtained solutions of FAPDEs that are subject to particular initial values quite closely match the exact solutions. In this way, to solve FAPDEs quickly, the proposed approaches are considered to be more accurate and efficient.

Frobenius methods for analytic second order linear partial differential equations

The main subject of this text is the study of analytic second order linear partial differential equations. We aim to solve the classical equations and some more, in the real or complex analytical case. This is done by introducing methods inspired by the Frobenius method for second order linear ordinary differential equations. We introduce a notion of Euler type partial differential equation. To such a PDE we associate an indicial conic, which is an affine plane curve of degree two. Then comes the concept of regular singularity and finally convergence theorems, which must necessarily take into account the type of PDE (parabolic, elliptical or hyperbolic) and a nonresonance condition. This condition gives a new geometric interpretation of the original condition between the roots of the original Frobenius theorem for second order ODEs. The interpretation is something like, a certain reticulate has or not vertices on the indexical conic. Finally, we retrieve the solution of all the classical PDEs by this method (heat diffusion, wave propagation and Laplace equation), and also increase the class of those that have explicit algorithmic solution to far beyond those admitting separable variables. The last part of the text is dedicated to the construction of PDE models for the classical ODEs like Airy, Legendre, Laguerre, Hermite and Chebyshev by two different means. One model is based on the requirement that the restriction of the PDE to lines through the origin must be the classical ODE model. The second is based on the idea of having symmetries on the PDE model and imitating the ODE model. We study these PDEs and obtain their solutions, obtaining for the framework of PDEs some of the classical results, like existence of polynomial solutions (Laguerre, Hermite and Chebyshev polynomials).

Discovering a reaction–diffusion model for Alzheimer’s disease by combining PINNs with symbolic regression

Misfolded tau proteins play a critical role in the progression and pathology of Alzheimer’s disease. Recent studies suggest that the spatio-temporal pattern of misfolded tau follows a reaction–diffusion type equation. However, the precise mathematical model and parameters that characterize the progression of misfolded protein across the brain remain incompletely understood. Here, we use deep learning and artificial intelligence to discover a mathematical model for the progression of Alzheimer’s disease using longitudinal tau positron emission tomography from the Alzheimer’s Disease Neuroimaging Initiative database. Specifically, we integrate physics informed neural networks (PINNs) and symbolic regression to discover a reaction–diffusion type partial differential equation for tau protein misfolding and spreading. First, we demonstrate the potential of our model and parameter discovery on synthetic data. Then, we apply our method to discover the best model and parameters to explain tau imaging data from 46 individuals who are likely to develop Alzheimer’s disease and 30 healthy controls. Our symbolic regression discovers different misfolding models f(c) for two groups, with a faster misfolding for the Alzheimer’s group, f(c)=0.23c3−1.34c2+1.11c, than for the healthy control group, f(c)=−c3+0.62c2+0.39c. Our results suggest that PINNs, supplemented by symbolic regression, can discover a reaction–diffusion type model to explain misfolded tau protein concentrations in Alzheimer’s disease. We expect our study to be the starting point for a more holistic analysis to provide image-based technologies for early diagnosis, and ideally early treatment of neurodegeneration in Alzheimer’s disease and possibly other misfolding-protein based neurodegenerative disorders.

Improved ParaDiag via low-rank updates and interpolation

This work is concerned with linear matrix equations that arise from the space-time discretization of time-dependent linear partial differential equations (PDEs). Such matrix equations have been considered, for example, in the context of parallel-in-time integration leading to a class of algorithms called ParaDiag. We develop and analyze two novel approaches for the numerical solution of such equations. Our first approach is based on the observation that the modification of these equations performed by ParaDiag in order to solve them in parallel has low rank. Building upon previous work on low-rank updates of matrix equations, this allows us to make use of tensorized Krylov subspace methods to account for the modification. Our second approach is based on interpolating the solution of the matrix equation from the solutions of several modifications. Both approaches avoid the use of iterative refinement needed by ParaDiag and related space-time approaches in order to attain good accuracy. In turn, our new algorithms have the potential to outperform, sometimes significantly, existing methods. This potential is demonstrated for several different types of PDEs.

A spatial local method for solving 2D and 3D advection-diffusion equations

PurposeThis study aims to derive a novel spatial numerical method based on multidimensional local Taylor series representations for solving high-order advection-diffusion (AD) equations.Design/methodology/approachThe parabolic AD equations are reduced to the nonhomogeneous elliptic system of partial differential equations by utilizing the Chebyshev spectral collocation method (ChSCM) in the temporal variable. The implicit-explicit local differential transform method (IELDTM) is constructed over two- and three-dimensional meshes using continuity equations of the neighbor representations with either explicit or implicit forms in related directions. The IELDTM yields an overdetermined or underdetermined system of algebraic equations solved in the least square sense.FindingsThe IELDTM has proven to have excellent convergence properties by experimentally illustrating both h-refinement and p-refinement outcomes. A distinctive feature of the IELDTM over the existing numerical techniques is optimizing the local spatial degrees of freedom. It has been proven that the IELDTM provides more accurate results with far fewer degrees of freedom than the finite difference, finite element and spectral methods.Originality/valueThis study shows the derivation, applicability and performance of the IELDTM for solving 2D and 3D advection-diffusion equations. It has been demonstrated that the IELDTM can be a competitive numerical method for addressing high-space dimensional-parabolic partial differential equations (PDEs) arising in various fields of science and engineering. The novel ChSCM-IELDTM hybridization has been proven to have distinct advantages, such as continuous utilization of time integration and optimized formulation of spatial approximations. Furthermore, the novel ChSCM-IELDTM hybridization can be adapted to address various other types of PDEs by modifying the theoretical derivation accordingly.

Dispersion analysis of SPH as a way to understand its order of approximation

Smoothed Particle Hydrodynamics (SPH) is a numerical method to solve dynamical partial differential equations (PDE). The basis of the method is a ¡¡kernel-based¿¿ way to compute the spatial derivatives of a function whose values are given in moving irregularly located nodes (Lagrangian particles). Accuracy of the SPH is determined by independent parameters — the shape of the kernel, the kernel size h, the distance between the particles Δx. Constructing high-order SPH-schemes for different types of PDE is a state-of-the-art problem of computational mathematics.For the classical SPH-approximation of one-dimensional hyperbolic equations (isothermal gas dynamics) we found that the order of approximation of smooth solution correlates to the dispersion properties of the method. To this end we analyzed the dispersion relation for the approximation and found analytical representation of the numerical wave phase velocity. Moreover, for the first time, the order of approximation with respect to Δx/h was confirmed in computational experiments on a dynamic problem of sound wave propagation. For two kernels with 2 and 4 continuum derivatives, the second and the fourth order of approximation, respectively, was found. This finding may be generalized as follows. The solution error in the one-dimensional case for a quasi-uniformly located particles has the form O((hλ)η+(Δxh)ξ), where ξ is a parameter determined by the shape of kernel (its smoothness, i.e. the number of continuum derivatives), η is a parameter that does not depend on the shape of kernel (for classical non-negative kernels η=2), λ is the wavelength.Our results indicates that to develop high-order SPH-schemes for hyperbolic equations besides improving the order of approximation with respect to h/λ one need to ensure the order of approximation with respect to Δx/h. To this end kernels of which smoothness is at least 4 are necessary.

Theoretical Study of the Effect of Permeability Tensor upon Drainage of Soils

the influence of permeability tensor upon drainage of anisotropic soils under ponded water and steady recharge (rainfall) is theoretically investigated. Tensorial permeability has led to the formulation of mixed type partial differential equations. Since there is no analytical solution to this problem, the formulation is therefore solved numerically by the method of finite elements. The finite element formulation is implemented into a computer model which can be applied to any problem of seepage under steady state conditions. Two different example problems representing two different flow conditions under full anisotropy have been studied. Results of the model for the isotropic case were checked against exact mathematical solutions derived analytically for isotropic soils and found to be accurate which indicates that using this model for anisotropic soils is safe and sound

Physics-informed neural network based on a new adaptive gradient descent algorithm for solving partial differential equations of flow problems

Physics-informed neural network (PINN) is an emerging technique for solving partial differential equations (PDEs) of flow problems. Due to the advantage of low computational cost, the gradient descent algorithms coupled with the weighted objectives method are usually used to optimize loss functions in the PINN training. However, the interaction mechanisms between gradients of loss functions are not fully clarified, leading to poor performances in loss functions optimization. For this, an adaptive gradient descent algorithm (AGDA) is proposed based on the interaction mechanisms analyses and then validated by analytical PDEs and flow problems. First, the interaction mechanisms of loss functions gradients in the PINN training based on the traditional Adam optimizer are analyzed. The main factors responsible for the poor performances of the Adam optimizer are identified. Then, a new AGDA optimizer is developed for the PINN training by two modifications: (1) balancing the magnitude difference of loss functions gradients and (2) eliminating the gradient directions conflict. Finally, three types of PDEs (elliptic, hyperbolic, and parabolic) and four viscous incompressible flow problems are selected to validate the proposed algorithm. It is found that to reach the specified accuracy, the required training time of the AGDA optimizer is about 16%–90% of the Adam optimizer and 41%–64% of the PCGrad optimizer, and the demanded number of iterations is about 10%–68% of the Adam optimizer and 38%–77% of the PCGrad optimizer. Therefore, the PINN method coupled with the AGDA optimizer is a more efficient and robust technique for solving partial differential equations of flow problems.

Reliable extrapolation of deep neural operators informed by physics or sparse observations

Deep neural operators can learn nonlinear mappings between infinite-dimensional function spaces via deep neural networks. As promising surrogate solvers of partial differential equations (PDEs) for real-time prediction, deep neural operators such as deep operator networks (DeepONets) provide a new simulation paradigm in science and engineering. Pure data-driven neural operators and deep learning models, in general, are usually limited to interpolation scenarios, where new predictions utilize inputs within the support of the training set. However, in the inference stage of real-world applications, the input may lie outside the support, i.e., extrapolation is required, which may result to large errors and unavoidable failure of deep learning models. Here, we address this challenge of extrapolation for deep neural operators. First, we systematically investigate the extrapolation behavior of DeepONets by quantifying the extrapolation complexity, via the 2-Wasserstein distance between two function spaces and propose a new strategy of bias–variance trade-off for extrapolation with respect to model capacity. Subsequently, we develop a complete workflow, including extrapolation determination, and we propose five reliable learning methods that guarantee a safe prediction under extrapolation by requiring additional information—the governing PDEs of the system or sparse new observations. The proposed methods are based on either fine-tuning a pre-trained DeepONet or multifidelity learning. We demonstrate the effectiveness of the proposed framework for various types of parametric PDEs. Our systematic comparisons provide practical guidelines for selecting a proper extrapolation method depending on the available information, desired accuracy, and required inference speed.

Noise-driven bifurcations in a nonlinear Fokker–Planck system describing stochastic neural fields

The existence and characterisation of noise-driven bifurcations from the spatially homogeneous stationary states of a nonlinear, non-local Fokker–Planck type partial differential equation describing stochastic neural fields is established. The resulting theory is extended to a system of partial differential equations modelling noisy grid cells. It is shown that as the noise level decreases, multiple bifurcations from the homogeneous steady state occur. Furthermore, the shape of the branches at a bifurcation point is characterised locally. The theory is supported by a set of numerical illustrations of the condition leading to bifurcations, the patterns along the corresponding local bifurcation branches, and the stability of the homogeneous state and the most prevalent pattern: the hexagonal one.

Unsupervised Legendre–Galerkin Neural Network for Solving Partial Differential Equations

In recent years, machine learning methods have been used to solve partial differential equations (PDEs) and dynamical systems, leading to the development of a new research field called scientific machine learning, which combines techniques such as deep neural networks and statistical learning with classical problems in applied mathematics. In this paper, we present a novel numerical algorithm that uses machine learning and artificial intelligence to solve PDEs. Based on the Legendre-Galerkin framework, we propose an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">unsupervised machine learning</i> algorithm that learns <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">multiple instances</i> of the solutions for different types of PDEs. Our approach addresses the limitations of both data-driven and physics-based methods. We apply the proposed neural network to general 1D and 2D PDEs with various boundary conditions, as well as convection-dominated <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">singularly perturbed PDEs</i> that exhibit strong boundary layer behavior.

On the Limitations of Physics-Informed Deep Learning: Illustrations Using First-Order Hyperbolic Conservation Law-Based Traffic Flow Models

Since its introduction in 2017, physics-informed deep learning (PIDL) has garnered growing popularity in understanding the systems governed by physical laws in terms of partial differential equations (PDEs). However, empirical evidence points to the limitations of PIDL for learning certain types of PDEs. In this paper, we (a) present the challenges in training PIDL architecture, (b) contrast the performance of PIDL architecture in learning a first order scalar hyperbolic conservation law and its parabolic counterpart, (c) investigate the effect of training data sampling, which corresponds to various sensing scenarios in traffic networks, (d) comment on the implications of PIDL limitations for traffic flow estimation and prediction in practice. Case studies present the contrast in PIDL results between learning the traffic flow model (LWR PDE) and its diffusive variation. The outcome indicates that PIDL experiences significant challenges in learning the hyperbolic LWR equation due to the non-smoothness of its solution. Conversely, the architecture with parabolic PDE, augmented with the diffusion term, leads to the successful reassembly of the density data even with the shockwaves present. The paper concludes by providing a discussion on recent assessments of reasons behind the challenge PIDL encounters with hyperbolic PDEs and the corresponding mitigation strategies.

Studying the solution of the system of differential equations obtained in simulation of gas-lift process by the relaxation method

Using the relation method for the system of first order, two-dimensional hyperbolic type partial differential equations describing the motion of gas liquid gas mixture corresponding to gas-lift process of oil production in an annular and lifting pipe, we consider a boundary condition problem. In this problem the existence a boundary condition problem. In this problem the existence of the solution with respect to the equations of motion is studied and it is shown that when constructing the solution of this problem by means of boundary conditions, it is impossible to determine the coefficients of positive degrees of the parameter ε. For this reason, the solution is sought in the form of a series using negative degrees of the parameter ε.

Finite line method for solving high-order partial differential equations in science and engineering

In this paper, a completely new numerical method, named Finite Line Method (FLM), is proposed for solving general linear and non-linear high-order partial differential equations (PDEs) in science as well as engineering problems in heat conduction and mechanics. In this method, the computational domain is discretized into a number of collocation points as in the free element method (Gao et al., 2019; Liu et al., 2019), and at each collocation point a set of straight or curved lines crossing the point is formed, which is represented by a number of nodes distributed over lines. The shape functions for each line are constructed using the Lagrange interpolation formulation and the first and high order spatial partial derivatives of the shape functions with respect to global coordinates are derived through an ingenious technique. The derived partial derivatives can be directly substituted into the governing differential equations and related boundary conditions to set up the discretized system of equations.FLM is a type of collocation method, not needing any integration to establish the solution scheme. And since the Lagrange interpolation formulation is used to construct the shape functions, arbitrarily high order lines can be easily formulated for solving any types of linear and non-linear high order PDEs and engineering problems. A number of numerical examples for PDEs, heat conduction and solid mechanics problems are given to demonstrate the correctness and stability of the proposed method.

Adaptive Learning Rate Residual Network Based on Physics-Informed for Solving Partial Differential Equations

Recently, Physics-informed neural networks (PINNs) have been widely applied to solving various types of partial differential equations (PDEs) such as Poisson equation, Klein–Gordon equation, and diffusion equation. However, it is difficult to obtain higher accurate solutions, especially at the boundary due to the gradient imbalance of different loss terms for the PINN model. In this work, an adaptive learning rate residual network algorithm based on physics-informed (adaptive-PIRN) is proposed to overcome this limitation of the PINN model. In the adaptive-PIRN model, an adaptive learning rate technique is introduced to adaptively configure appropriate weights to the residual loss of the governing equation and the loss of initial/boundary conditions (I/BCs) by utilizing gradient statistics, which can alleviate gradient imbalance of different loss terms in PINN. Besides, based on the idea of ResNet, the “short connection” technique is used in adaptive-PIRN model, which can ensure that the original information is identically mapped. This structure has stronger expressive capabilities than fully connected neural networks and can avoid gradient disappearance. Finally, three different types of PDE are conducted to demonstrate predictive accuracy of our model. In addition, it is clearly observed from the results that the adaptive-PIRN can balance the gradient of loss items to a great extent, which improves the effectiveness of this network.

Solving the 2D and 3D nonlinear inverse source problems of elliptic type partial differential equations by a homogenization function method

AbstractIn this article, the inverse source problems of 2D and 3D elliptic type nonlinear partial differential equations are resolved. For this purpose, a family of single‐parameter homogenization functions that automatically meet the given boundary conditions are deduced and employed as the bases to expand the solution. We solve a linear algebraic equations system which satisfies the over‐specified Neumann boundary condition to obtain the unspecified coefficients, and then the solution in the entire domain is permitted. Taking the solution into the governing equation, the unknown source function can be determined quickly. The present novel method is verified to be an accurate, effective, and robust scheme which is without solving nonlinear equations and iterations, and the additional data used are quite economical.