Solving nuclear Dirac Woods-Saxon potential with the physics-informed neural network

A physics‐informed neural network (PINN) that combines deep learning with physics is studied to solve the nonlinear Schrödinger equation for learning nonlinear dynamics in fiber optics. A systematic investigation and comprehensive verification on PINN for multiple physical effects in optical fibers is carried out, including dispersion, self‐phase modulation, and higher‐order nonlinear effects. Moreover, both the special case (soliton propagation) and general case (multipulse propagation) are investigated and realized with PINN. In existing studies, PINN is mainly effective for a single scenario. To overcome this problem, the physical parameters (i.e., pulse peak power and amplitudes of subpulses) are hereby embedded as additional input parameter controllers, which allow PINN to learn the physical constraints of different scenarios and perform good generalizability. Furthermore, PINN exhibits better performance than the data‐driven neural network using much less data, and its computational complexity (in terms of number of multiplications) is much lower than that of the split‐step Fourier method. The results show that PINN is not only an effective partial differential equation solver, but also a prospective technique to advance the scientific computing and automatic modeling in fiber optics.

Physics-informed neural networks (PINNs) have recently emerged as an important and ground-breaking technique in scientific machine learning for numerous applications including in optical fiber communications. However, the vanilla/baseline version of PINNs is prone to fail under certain conditions because of the nature of the physics-based regularization term in its loss function. The use of this unique regularization technique results in a highly complex non-convex loss landscape when visualized. This leads to failure modes in PINN-based modeling. The baseline PINN works very well as an optical fiber model with relatively simple fiber parameters and for uncomplicated transmission tasks. Yet, it struggles when the modeling task becomes relatively complex, reaching very high error, for example, numerous modeling tasks/scenarios in soliton communication and soliton pulse development in special fibers such as erbium-doped dispersion compensating fibers. We implement two methods to circumvent the limitations caused by the physics-based regularization term to solve this problem, namely, the so-called scaffolding technique for PINN modeling and the progressive block learning PINN modeling strategy to solve the nonlinear Schrödinger equation (NLSE), which models pulse propagation in an optical fiber. This helps PINN learn more accurately the dynamics of pulse evolution and increases accuracy by two to three orders of magnitude. We show in addition that this error is not due to the depth or architecture of the neural network but a fundamental issue inherent to PINN by design. The results achieved indicate a considerable reduction in PINN error for complex modeling problems, with accuracy increasing by up to two orders of magnitude.

One-dimensional ice shelf hardness inversion: Clustering behavior and collocation resampling in physics-informed neural networks

Physics-informed neural networks (PINNs) and computational fluid dynamics (CFD) have both demonstrated an ability to derive accurate hemodynamics if boundary conditions (BCs) are known. Unfortunately, patient-specific BCs are often unknown, and assumptions based upon previous investigations are used instead. High speed angiography (HSA) may allow extraction of these BCs due to the high temporal fidelity of the modality. We propose to investigate whether PINNs using convection and Navier-Stokes equations with BCs derived from HSA data may allow for extraction of accurate hemodynamics in the vasculature. Imaging data generated from in vitro 1000 fps HSA, as well as simulated 1000 fps angiograms generated using CFD were utilized for this study. Calculations were performed on a 3D lattice comprised of 2D projections temporally stacked over the angiographic sequence. A PINN based on an objective function comprised of the Navier-Stokes equation, the convection equation, and angiography-based BCs was used for estimation of velocity, pressure and contrast flow at every point in the lattice. Imaging-based PINNs show an ability to capture such hemodynamic phenomena as vortices in aneurysms and regions of rapid transience, such as outlet vessel blood flow within a carotid artery bifurcation phantom. These networks work best with small solution spaces and high temporal resolution of the input angiographic data, meaning HSA image sequences represent an ideal medium for such solution spaces. The study shows the feasibility of obtaining patient-specific velocity and pressure fields using an assumption-free data driven approach based purely on governing physical equations and imaging data.

A complete Physics-Informed Neural Network-based framework for structural topology optimization

Several recent works in scientific machine learning have revived interest in the application of neural networks to partial differential equations (PDEs). A popular approach is to aggregate the residual form of the governing PDE and its boundary conditions as soft penalties into a composite objective/loss function for training neural networks, which is commonly referred to as physics-informed neural networks (PINNs). In the present study, we visualize the loss landscapes and distributions of learned parameters and explain the ways this particular formulation of the objective function may hinder or even prevent convergence when dealing with challenging target solutions. We construct a purely data-driven loss function composed of both the boundary loss and the domain loss. Using this data-driven loss function and, separately, a physics-informed loss function, we then train two neural network models with the same architecture. We show that incomparable scales between boundary and domain loss terms are the culprit behind the poor performance. Additionally, we assess the performance of both approaches on two elliptic problems with increasingly complex target solutions. Based on our analysis of their loss landscapes and learned parameter distributions, we observe that a physics-informed neural network with a composite objective function formulation produces highly non-convex loss surfaces that are difficult to optimize and are more prone to the problem of vanishing gradients.

The physics-informed neural network (PINN) is an effective alternative method for solving differential equations that do not require grid partitioning, making it easy to implement. In this study, using automatic differentiation techniques, the PINN method is employed to solve differential equations by embedding prior physical information, such as boundary and initial conditions, into the loss function. The differential equation solution is obtained by minimizing the loss function. The PINN method is trained using the Adam algorithm, taking the differential equations of motion in structural dynamics as an example. The time sample set generated by the Sobol sequence is used as the input, while the displacement is considered the output. The initial conditions are incorporated into the loss function as penalty terms using automatic differentiation techniques. The effectiveness of the proposed method is validated through the numerical analysis of a two-degree-of-freedom system, a four-story frame structure, and a cantilever beam. The study also explores the impact of the input samples, the activation functions, the weight coefficients of the loss function, and the width and depth of the neural network on the PINN predictions. The results demonstrate that the PINN method effectively solves the differential equations of motion of damped systems. It is a general approach for solving differential equations of motion.

A gradient-enhanced physics-informed neural networks method for the wave equation

This paper explores the difficulties in solving partial differential equations (PDEs) using physics-informed neural networks (PINNs). PINNs use physics as a regularization term in the objective function. However, a drawback of this approach is the requirement for manual hyperparameter tuning, making it impractical in the absence of validation data or prior knowledge of the solution. Our investigations of the loss landscapes and backpropagated gradients in the presence of physics reveal that existing methods produce non-convex loss landscapes that are hard to navigate. Our findings demonstrate that high-order PDEs contaminate backpropagated gradients and hinder convergence. To address these challenges, we introduce a novel method that bypasses the calculation of high-order derivative operators and mitigates the contamination of backpropagated gradients. Consequently, we reduce the dimension of the search space and make learning PDEs with non-smooth solutions feasible. Our method also provides a mechanism to focus on complex regions of the domain. Besides, we present a dual unconstrained formulation based on Lagrange multiplier method to enforce equality constraints on the model's prediction, with adaptive and independent learning rates inspired by adaptive subgradient methods. We apply our approach to solve various linear and non-linear PDEs.

Optimization tasks are essential in modern engineering fields such as chip design, spacecraft trajectory determination, and reactor scenario development. Recently, machine learning applications, including deep reinforcement learning (RL) and genetic algorithms (GA), have emerged in these real-world optimization tasks. We introduce a new machine learning-based optimization scheme that incorporates physics with the operational objectives. This physics-informed neural network (PINN) could find the optimal path in well-defined systems with less exploration and also was capable of obtaining narrow and unstable solutions that have been challenging with bottom-up approaches like RL or GA. Through an objective function that integrates governing laws, constraints, and goals, PINN enables top-down searches for optimal solutions. In this study, we showcase the PINN applications to various optimization tasks, ranging from inverting a pendulum, determining the shortest-time path, to finding the swingby trajectory. Through this, we discuss how PINN can be applied in the tasks with different characteristics.

Subsurface imaging techniques with improved resolution is the industry's pressing challenge. In this paper, we present a novel Machine Learning based data processing method to generate accurate seismic profiles. Through this method, the processing of humongous acoustic data and subsequent inference of subsurface properties is significantly expedited. The work being presented is an application of a recently developed novel class of algorithms called the Physics Informed Neural Networks (PINNs). The method has been proven in its efficacy for approximating the solutions to Partial Differential Equations (PDE) governing complex physical phenomena and in data-driven discovery of the underlying PDEs such as the Navier-Stokes system. Herein, we have followed a two-stage procedure. Stage one is the accurate prediction of spatiotemporal propagation of a seismic pulse through a simulated domain resembling thesubsurface. For this we developed a simulator to generate vast amounts of virtual seismic data. Dense Deep Neural Networks (DNN) were used to overfit the data so that it approximates the solution to the wave equations. This was followed by the data-driven discovery of local velocities in different reflector layers. In the second stage, we have extended the algorithm to infer the real subsurface velocity profile based on the data collected at receiver positions during an actual seismic survey from Volve, a field on the Norwegian continental shelf. Moreover, we have bench-marked and established the performance of various models in their ability to learn the receiver data obtained from a simulated 2D seismic survey. The results are promising as the network chosen can learn the receiver data with a minimal number of iterations and with near-perfect accuracy. The models were further tested in their ability to represent solutions of the wave equation in the spatiotemporal domain and in their ability to represent the actual subsurface velocity profiles. The novelty of this paper lies in the overlap of a nascent-stage deep learning tool, called PINNs, with seismic data processing to expedite and improvise high-resolution subsurface imaging. For this, various other uncommon developments were done such as that of adapting PINN as a dynamic wave function approximator. Moreover, the effectiveness of the proposed method is numerically demonstrated by the excellent accuracy obtained when validated against the velocity profile from the field as well as the velocity profile from the simulator.

PINN training using biobjective optimization: The trade-off between data loss and residual loss

A physics-informed learning approach to Bernoulli-type free boundary problems

With the increases in computational power and advances in machine learning, data-driven learning-based methods have gained significant attention in solving PDEs. Physics-informed neural networks (PINNs) have recently emerged and succeeded in various forward and inverse PDE problems thanks to their excellent properties, such as flexibility, mesh-free solutions, and unsupervised training. However, their slower convergence speed and relatively inaccurate solutions often limit their broader applicability in many science and engineering domains. This paper proposes a new kind of data-driven PDEs solver, physics-informed cell representations (PIXEL), elegantly combining classical numerical methods and learning-based approaches. We adopt a grid structure from the numerical methods to improve accuracy and convergence speed and overcome the spectral bias presented in PINNs. Moreover, the proposed method enjoys the same benefits in PINNs, e.g., using the same optimization frameworks to solve both forward and inverse PDE problems and readily enforcing PDE constraints with modern automatic differentiation techniques. We provide experimental results on various challenging PDEs that the original PINNs have struggled with and show that PIXEL achieves fast convergence speed and high accuracy. Project page: https://namgyukang.github.io/PIXEL/

Interpolation of aliased seismic data constitutes a key step in a seismic processing workflow to obtain high-quality velocity models and seismic images. Building on the idea of describing seismic wavefields as a superposition of local plane waves, we propose to interpolate seismic data by using a physics informed neural network (PINN). In the proposed framework, two feed-forward neural networks are jointly trained using the local plane wave differential equation as well as the available data as two terms in the objective function: a primary network assisted by positional encoding is tasked with reconstructing the seismic data, whereas an auxiliary, smaller network estimates the associated local slopes. Results on synthetic and field data validate the effectiveness of the proposed method in handling aliased (coarsely sampled) data and data with large gaps. Our method compares favorably against a classic least-squares inversion approach regularized by the local plane-wave equation as well as a PINN-based approach with a single network and precomputed local slopes. We find that introducing a second network to estimate the local slopes, whereas at the same time interpolating the aliased data enhances the overall reconstruction capabilities and convergence behavior of the primary network. Moreover, an additional positional encoding layer embedded as the first layer of the wavefield network confers to the network the ability to converge faster, improving the accuracy of the data term.