Unsupervised spectra information extraction using physics-informed neural networks in the presence of non-linearities and multi-agent problems.

Predicting positon solutions of a family of nonlinear Schrödinger equations through deep learning algorithm

Waterflooding has a long history as a successful development strategy in oil recovery, yet maximizing its potential through optimized strategies remains a significant challenge. Traditionally, identifying the most effective waterflooding designs requires extensive numerical simulations, which can be computationally demanding. This study introduces a comprehensive framework that employs a Physics-Informed Neural Network (PINN) to optimize waterflooding designs for enhancing oil recovery. Specifically, the PINN incorporates fluid dynamics principles into deep learning algorithms and serves as a rapid surrogate method to effectively predicts oil production across a range of waterflooding scenarios. Furthermore, an optimization technique is designed to fine-tune injection designs, thereby optimizing oil recovery. Experiments on 2D synthetic and 3D Brugge benchmark cases demonstrate that the PINN model achieves higher accuracy compared to a pure data-driven neural network. Using the PINN surrogate, a genetic algorithm quickly searches the injection parameter space for optimal oil production. The optimized strategies are validated using the full numerical simulator, confirming the feasibility of the proposed approach. Overall, the integration of domain knowledge into deep learning not only improves the generalise ability of the pure data-driven model but also provides insightful physical interpretations for engineers.

Biofluid mechanics play an important role in the study of the mechanism of cardiovascular diseases and in the development of new implants. For the assessment of hydrodynamic parameters, experimental methods as well as in-silico approaches can be used, such as particle image velocimetry (PIV) and Deep Learning, respectively. Challenges for PIV are the optical access to the region of interest, and time consumption for measuring and post-processing analysis in particular for three dimensional flow. To overcome these limitations state-of-the-art deep learning algorithms could be utilized to augment spatially coarse resolved flow fields. In this study, we demonstrate the use of Physics Informed Neural Networks (PINN) to augment PIV measurement data. To demonstrate a combined workflow, we investigate the flow of a Newtonian fluid through a simplified aneurysm under laminar conditions. Generation of synthetic PIV particle images of a single measurement plane and the corresponding PIV vector calculations were performed as the basis for the PINN algorithm. Based on the Navier-Stokes equations the PINN reconstructs the entire 3D flow field and pressure distribution inside the aneurysm. We observed qualitative agreements between ground through data and PINN predictions. Nevertheless, there are substantial differences in the quantitative, locally resolved comparison of the flow metrics, despite the generally tendency for the PINN algorithm to correctly augment the flow field.

Physics-informed neural networks for parametric compressible Euler equations

In recent years, successful applications of deep learning (DL) have inspired scientists to explore the possibilities of applying DL approaches to modeling scientific problems. Existing studies have revealed that to bake the physics into the DL makes a good supplement to the traditional numerical methods (e.g., finite element, finite volume method) which primarily rely on partial differential equations (PDEs). While DL models are ordinarily trained in a purely data-driven manner, integrating physics into them for simulating scientific problems has several benefits such as (i) physics constraints could regularize the over-parameterized model and hence mitigate the overfitting issue commonly seen in DL; (ii) physics information could also effectively reduce the amount of data needed for training the model; (iii) the resultant physics-informed DL models feature better interpretability and generalizability compared with the conventional black-box model. Furthermore, the powerful expressiveness of the deep network, guaranteed by the universal approximation theorem, makes it a suitable approximator for the solution to a physical system. In this dissertation, we develop two different DL architectures (or approaches), one being continuous scheme-based while the other discrete scheme-based, that leverage physics knowledge for modeling scientific problems. Through comprehensive numerical experiments, we demonstrate the proposed models can be used in solving general PDEs, establishing predictive data-driven models for dynamical systems, identifying the parameters in governing PDEs or even discovering the entire governing PDEs of dynamical systems from scarce and noisy measurements. The continuous model roots on the physics-informed neural network (PINN) which uses a fully connected neural network (FCNN) to approximate the physical fields of a system globally. This model is mesh-free as the residual of the physics (e.g., PDEs, initial/boundary values) is evaluated on a set of collocation points within the physical domain. Several applications including the forward simulations, data-driven simulations and solving inverse problems are presented to exemplify the advantages of PINN over traditional numerical methods. However, the original PINN suffers from inaccurate initial/boundary values due to the weak enforcement of the initial/boundary conditions (I/BCs). To overcome this issue, we propose an improved PINN model by utilizing multiple deep neural networks (DNNs) to construct the solution. Through a DNN pre-trained to represent the initial/boundary values, the approximated solution would obey the given I/BCs forcibly. With several numerical examples, we show that the improved PINN is characterized with much better accuracy on the I/BCs. Though the PINN shows great promise in data-driven modeling and solving inverse problems, some inherent limitations of PINN still exist, such as (i) the solution might lacks fine-scale details due to the global approximation of FCNN; (ii) high computational expense caused by the FCNN it roots on; (iii) incapability to incorporate existing PDE terms (e.g., $\Delta u$) into the network architecture. To overcome these drawbacks, this dissertation also proposes a discrete model - Physics-encoded Recurrent Convolutional Neural Network (PeRCNN) which recurrently updates the solution (or state variable) for time marching. Specifically, it utilizes convolutional neural network (CNN) to capture the spatial patterns of the solution while the recurrent network mimics the forward Euler scheme (or Runge-Kutta scheme) in numerical methods. PeRCNN is a mesh-based and discrete model due to the discretization in time and spatial dimension. The local connectivity of CNN makes PeRCNN more computationally efficient. In addition, the coercive encoding mechanism of physics in PeRCNN, fundamentally different from the PINN relying on soft penalty, ensures the network to rigorously obey given physics. The proposed PeRCNN is successfully applied to solving general PDEs, the data-driven modeling of dynamical systems and the data-driven discovery of governing PDEs from scarce and noisy measurements. Comparisons with the state-of-the-art DL models demonstrate that the proposed PeRCNN possesses excellent computational efficiency, accuracy and generalizability. --Author's abstract

The numerical approximation of solutions to the compressible Euler and Navierstokes equations is a crucial but challenging task with relevance in various fields of science and engineering. Recently, methods from deep learning have been successfully employed for solving partial differential equations by incorporating the equations into a loss function that is minimized during the training of a neural network. This approach yields a so-called physics-informed neural network which does not rely on a classical discretization and can address parametric problems in a straightforward manner. Therefore, it avoids characteristic difficulties of traditional approaches, such as finite volume methods. This has raised the question, whether physics-informed neural networks may be a viable alternative to conventional methods for computational fluid dynamics. Here, we show a new physics-informed neural network training procedure to approximately solve the two-dimensional compressible Euler equations, which makes use of artificial dissipation during the training process. We demonstrate how additional dissipative terms help to avoid unphysical results and how the additional numerical viscosity can be reduced during training while iterating towards a solution. Furthermore, we showcase how this approach can be combined with parametric boundary conditions. Our results highlight the appearance of unphysical results when solving compressible flows with physics-informed neural networks and offer a new approach to overcome this problem. We therefore expect that the presented methods enable the application of physics-informed neural networks for previously difficult to solve problems.

Deep learning has achieved remarkable success in diverse applications; however, its use in solving partial differential equations (PDEs) has emerged only recently. Here, we present an overview of physics-informed neural networks (PINNs), which embed a PDE into the loss of the neural network using automatic differentiation. The PINN algorithm is simple, and it can be applied to different types of PDEs, including integro-differential equations, fractional PDEs, and stochastic PDEs. Moreover, from an implementation point of view, PINNs solve inverse problems as easily as forward problems. We propose a new residual-based adaptive refinement (RAR) method to improve the training efficiency of PINNs. For pedagogical reasons, we compare the PINN algorithm to a standard finite element method. We also present a Python library for PINNs, DeepXDE, which is designed to serve both as an educational tool to be used in the classroom as well as a research tool for solving problems in computational science and engineering. Specifically, DeepXDE can solve forward problems given initial and boundary conditions, as well as inverse problems given some extra measurements. DeepXDE supports complex-geometry domains based on the technique of constructive solid geometry and enables the user code to be compact, resembling closely the mathematical formulation. We introduce the usage of DeepXDE and its customizability, and we also demonstrate the capability of PINNs and the user-friendliness of DeepXDE for five different examples. More broadly, DeepXDE contributes to the more rapid development of the emerging scientific machine learning field.

Physics-informed neural networks (PINNs) have shown promising potential for solving partial differential equations (PDEs) using deep learning. However, PINNs face training difficulties for evolutionary PDEs, particularly for dynamical systems whose solutions exhibit multi-scale or turbulent behavior over time. The reason is that PINNs may violate the temporal causality property since all the temporal features in the PINNs loss are trained simultaneously. This paper proposes to use implicit time differencing schemes to enforce temporal causality, and use transfer learning to sequentially update the PINNs in space as surrogates for PDE solutions in different time frames. The evolving PINNs are better able to capture the varying complexities of the evolutionary equations, while only requiring minor updates between adjacent time frames. Our method is theoretically proven to be convergent if the time step is small and each PINN in different time frames is well-trained. In addition, we provide state-of-the-art (SOTA) numerical results for a variety of benchmarks for which existing PINNs formulations may fail or be inefficient. We demonstrate that the proposed method improves the accuracy of PINNs approximation for evolutionary PDEs and improves efficiency by a factor of 4–40x. The code is available at https://github.com/SiqiChen9/TL-DPINNs.

In recent engineering applications using deep learning, physics-informed neural network (PINN) is a new development as it can exploit the underlying physics of engineering systems. The novelty of PINN lies in the use of partial differential equations (PDE) for the loss function. Most PINNs are implemented using automatic differentiation (AD) for training the PDE loss functions. A lesser well-known study is the use of finite difference method (FDM) as an alternative. Unlike an AD based PINN, an immediate benefit of using a FDM based PINN is low implementation cost. In this paper, we propose the use of finite difference method for estimating the PDE loss functions in PINN. Our work is inspired by computational analysis in electromagnetic systems that traditionally solve Laplace’s equation using successive over-relaxation. In the case of Laplace’s equation, our PINN approach can be seen as taking the Laplacian filter response of the neural network output as the loss function. Thus, the implementation of PINN can be very simple. In our experiments, we tested PINN on Laplace’s equation and Burger’s equation. We showed that using FDM, PINN consistently outperforms non-PINN based deep learning. When comparing to AD based PINNs, we showed that our method is faster to compute as well as on par in terms of error reduction.

In recent years great interest has risen towards surrogate reservoir models based on data-driven methodologies with the purpose of speeding up reservoir management decisions. In this work, a Physics Informed Neural Network (PINN) based on a Capacitance Resistance Model (CRM) has been developed and tested on a synthetic and on a real dataset to predict the production of oil reservoirs under waterflooding conditions. CRMs are simple models based on material balance that estimate the liquid production as a function of injected water and bottom hole pressure. PINNs are Artificial Neural Networks (ANNs) that incorporate prior physical knowledge of the system under study to regularize the network. A PINN based on a CRM is obtained by including the residual of the CRM differential equations in the loss function designed to train the neural network on the historical data. During training, weights and biases of the network and parameters of the physical equations, such as connectivity factors between wells, are updated with the backpropagation algorithm. To investigate the effectiveness of the novel methodology on waterflooded scenarios, two test cases are presented: a small synthetic one and a real mature reservoir. Results obtained with PINN are compared with respect to CRM and ANN alone. In the synthetic case CRM and PINN give slightly better quality history matches and predictions than ANN. The connectivity factors estimated by CRM and PINN are very similar and correctly represent the underlying geology. In the real case PINN gives better quality history matches and predictions than ANN, and both significantly outperform CRM. Even though the CRM formulation is too simple to predict the complex behavior of a real reservoir, the CRM based regularization contributes to improving the PINN predictions quality compared to the purely data-driven ANN model. The connectivity factors estimated by CRM and PINN are not in agreement. However, the latter method provided results closer to our understanding of the flooding process after many years of operations and data analysis. All considered, PINN outperformed both CRM and ANN in terms of predictivity and interpretability, effectively combining strengths from both methodologies. The presented approach does not require the construction of a 3D model since it learns directly from production data, while preserving physical consistency. Moreover, it represents a computationally inexpensive alternative to traditional full-physics reservoir simulations which could have vast applications for problems requiring many forward evaluations, like the optimization of water allocation for mature reservoirs.

Physics-informed neural networks (PINNs) have recently been developed as a novel solution approach for physical problems governed by partial differential equations (PDEs). Compared to purely data-driven methods, PINNs have the advantage of embedding physical constraints in the training process, thus increasing their reliability. Compared to traditional numerical methods for PDEs, PINNs have the advantage of being “meshless”; they are in general less accurate and more computationally expensive, but also more suitable to sparse-data assimilation and to inverse modelling, which is increasing their popularity in many scientific fields. However, hydraulic applications of PINNs in the context of free-surface flows are still in their infancy. In this work, the effectiveness of PINNs to model one-dimensional free-surface flows over non-horizontal bottom is tested. The governing PDEs are the shallow water equations (SWEs), which represent the mass and momentum conservation in free-surface flows. The inclusion of a spatially variable topography in a meshless method such as PINNs is not trivial. Here, the idea of solving the augmented system of SWEs with topography is exploited. Augmentation consists in treating the bed elevation as a conserved variable (together with water depth and unit discharge) and adding a fictitious equation to the system, which states that this variable is constant in time (i.e., its time derivative is null), while it can be variable in space (its space derivative is included in the bed slope source term). In this way, bed elevation can be easily provided with other initial conditions, and the fixed-bed constraint preserves its value in time. Different cases of unsteady flows with flat and non-flat bottom are considered, and the accuracy obtained using PINNs with augmented SWEs is checked by comparing PINNs predictions with analytical solutions. Results show that a fair accuracy for depth and velocity can be obtained, even for some challenging test cases such as the dam-break over a bottom step and the planar flow over a parabolic basin (Thacker’s test case). Moreover, it is shown that, if PINNs are applied to a case with horizontal bottom, for which topography is not strictly necessary, similar accuracy and computational time are obtained when PINNs solve standard SWEs or augmented SWEs. It can therefore be concluded that the augmentation of SWEs is a simple but promising strategy to deal with flows over complex bathymetries using PINNs, which paves the way for applications to flows over more realistic topographies.

We propose a new approach to the solution of the wave propagation and full waveform inversions (FWIs) based on a recent advance in deep learning called physics‐informed neural networks (PINNs). In this study, we present an algorithm for PINNs applied to the acoustic wave equation and test the method with both forward models and FWI case studies. These synthetic case studies are designed to explore the ability of PINNs to handle varying degrees of structural complexity using both teleseismic plane waves and seismic point sources. PINNs' meshless formalism allows for a flexible implementation of the wave equation and different types of boundary conditions. For instance, our models demonstrate that PINN automatically satisfies absorbing boundary conditions, a serious computational challenge for common wave propagation solvers. Furthermore, a priori knowledge of the subsurface structure can be seamlessly encoded in PINNs' formulation. We find that the current state‐of‐the‐art PINNs provide good results for the forward model, even though spectral element or finite difference methods are more efficient and accurate. More importantly, our results demonstrate that PINNs yield excellent results for inversions on all cases considered and with limited computational complexity. We discuss the current limitations of the method with complex velocity models as well as strategies to overcome these challenges. Using PINNs as a geophysical inversion solver offers exciting perspectives, not only for the full waveform seismic inversions, but also when dealing with other geophysical datasets (e.g., MT, gravity) as well as joint inversions because of its robust framework and simple implementation.

Two-phase flow rate estimation of liquid and gas flow through wellhead chokes is essential for determining and monitoring production performance from oil and gas reservoirs at specific well locations. Liquid flow rate (QL) tends to be nonlinearly related to these influencing variables, making empirical correlations unreliable for predictions applied to different reservoir conditions and favoring machine learning (ML) algorithms for that purpose. Recent advances in deep learning (DL) algorithms make them useful for predicting wellhead choke flow rates for large field datasets and suitable for wider application once trained. DL has not previously been applied to predict QL from a large oil field. In this study, 7245 multi-well data records from Sorush oil field are used to compare the QL prediction performance of traditional empirical, ML and DL algorithms based on four influencing variables: choke size (D64), wellhead pressure (Pwh), oil specific gravity (γo) and gas–liquid ratio (GLR). The prevailing flow regime for the wells evaluated is critical flow. The DL algorithm substantially outperforms the other algorithms considered in terms of QL prediction accuracy. The DL algorithm predicts QL for the testing subset with a root-mean-squared error (RMSE) of 196 STB/day and coefficient of determination (R2) of 0.9969 for Sorush dataset. The QL prediction accuracy of the models evaluated for this dataset can be arranged in the descending order: DL > DT > RF > ANN > SVR > Pilehvari > Baxendell > Ros > Glbert > Achong. Analysis reveals that input variable GLR has the greatest, whereas input variable D64 has the least relative influence on dependent variable QL.

The Industrial Internet of Things has grown significantly in recent years. While implementing industrial digitalization, automation, and intelligence introduced a slew of cyber risks, the complex and varied industrial Internet of Things environment provided a new attack surface for network attackers. As a result, conventional intrusion detection technology cannot satisfy the network threat discovery requirements in today’s Industrial Internet of Things environment. In this research, the authors have used reinforcement learning rather than supervised and unsupervised learning, because it could very well improve the decision‐making ability of the learning process by integrating abstract thinking of complete understanding, using deep knowledge to perform simple and nonlinear transformations of large‐scale original input data into higher‐level abstract expressions, and using learning algorithm or learning based on feedback signals, in the lack of guiding knowledge, which is based on the trial‐and‐error learning model, from the interaction with the environment to find the best good solution. In this respect, this article presents a near‐end strategy optimization method for the Industrial Internet of Things intrusion detection system based on a deep reinforcement learning algorithm. This method combines deep learning’s observation capability with reinforcement learning’s decision‐making capability to enable efficient detection of different kinds of cyberassaults on the Industrial Internet of Things. In this manuscript, the DRL‐IDS intrusion detection system is built on a feature selection method based on LightGBM, which efficiently selects the most attractive feature set from industrial Internet of Things data; when paired with deep learning algorithms, it effectively detects intrusions. To begin, the application is based on GBM’s feature selection algorithm, which extracts the most compelling feature set from Industrial Internet of Things data; then, in conjunction with the deep learning algorithm, the hidden layer of the multilayer perception network is used as the shared network structure for the value network and strategic network in the PPO2 algorithm; and finally, the intrusion detection model is constructed using the PPO2 algorithm and ReLU (R). Numerous tests conducted on a publicly available data set of the Industrial Internet of Things demonstrate that the suggested intrusion detection system detects 99 percent of different kinds of network assaults on the Industrial Internet of Things. Additionally, the accuracy rate is 0.9%. The accuracy, precision, recall rate, F1 score, and other performance indicators are superior to those of the existing intrusion detection system, which is based on deep learning models such as LSTM, CNN, and RNN, as well as deep reinforcement learning models such as DDQN and DQN.

Condition monitoring of rotating shafts is essential for ensuring the reliability and optimal performance of machinery in diverse industries. In this context, as industrial systems become increasingly complex, the need for efficient data processing techniques is paramount. Deep learning has emerged as a dominant approach due to its capacity to capture intricate data patterns and relationships. However, a prevalent challenge lies in the black-box nature of many deep learning algorithms, which often operate without adhering to the underlying physical characteristics intrinsic to the studied phenomena. To address this limitation and enhance the fusion of data-driven methodologies with the fundamental physics of the system under study, this paper leverages physics-informed neural networks (PINNs). Specifically, a simple but realistic numerical case study of an extended Jeffcott rotor model, encompassing damping effects and anisotropic supports for a more comprehensive modelling, is considered. PINNs are used for the estimation of five parameters that characterize the health state of the system. These parameters encompass the radial and angular position of the static unbalance due to the disk installed on the shaft, the stiffness along the principal axes of elasticity, and the non-rotating damping coefficient. The estimation is conducted solely by exploiting the displacement signals from the centre of the disk and, to showcase the efficacy and precision provided by this novel methodology, various scenarios involving different constant rotational speeds are examined. Additionally, the impact of noisy input data is also taken into account within the analysis and the performance is compared to that of traditional optimization algorithms used for parameters estimation.