Physical Inconsistencies Research Articles

A Fluid Flow‐Based Deep Learning (FFDL) Architecture for Subsurface Flow Systems With Application to Geologic CO2 Storage

AbstractPrediction of the spatial‐temporal dynamics of the fluid flow in complex subsurface systems, such as geologic storage, is typically performed using advanced numerical simulation methods that solve the underlying governing physical equations. However, numerical simulation is computationally demanding and can limit the implementation of standard field management workflows, such as model calibration and optimization. Standard deep learning models, such as RUNET, have recently been proposed to alleviate the computational burden of physics‐based simulation models. Despite their powerful learning capabilities and computational appeal, deep learning models have important limitations, including lack of interpretability, extensive data needs, weak extrapolation capacity, and physical inconsistency that can affect their adoption in practical applications. We develop a Fluid Flow‐based Deep Learning (FFDL) architecture for spatial‐temporal prediction of important state variables in subsurface flow systems. The new architecture consists of a physics‐based encoder to construct physically meaningful latent variables, and a residual‐based processor to predict the evolution of the state variables. It uses physical operators that serve as nonlinear activation functions and imposes the general structure of the fluid flow equations to facilitate its training with data pertaining to the specific subsurface flow application of interest. A comprehensive investigation of FFDL, based on a field‐scale geologic storage model, is used to demonstrate the superior performance of FFDL compared to RUNET as a standard deep learning model. The results show that FFDL outperforms RUNET in terms of prediction accuracy, extrapolation power, and training data needs.

PGTransNet: a physics-guided transformer network for 3D ocean temperature and salinity predicting in tropical Pacific

Accurately predicting the spatio-temporal evolution trends and long-term dynamics of three-dimensional ocean temperature and salinity plays a crucial role in monitoring climate system changes and conducting fundamental oceanographic research. Numerical models are the most prevalent of the traditional approaches, which are often too complex and lack of generality. Recently, with the rise of AI, many data-driven methods are proposed. However, most of them take no consideration of natural physical laws that may cause issues of physical inconsistency among different variables. In this paper, we proposed PGTransNet, a novel physics-guided transformer network for 3D Ocean temperature and salinity forecasting. This model is based on Vision Transformer, and to enhance the performance we have three aspects of improvements. Firstly, we design a loss function that deliveries the physical relationship among temperature, salinity and density by fusing the Thermodynamic Equation. Secondly, to capture global and long-term dependencies effectively, we add the Pacific Decadal Oscillation (PDO) and North Pacific Gyre Oscillation (NPGO) in the embedding layer. Thirdly, we adopted the Laplacian sparse positional encodings to alleviate the artifacts caused by high-norm tokens. The former two are the core components to leverage the physical information. Finally, to comprehensively evaluate PGTransnet, we conduct rich experiments in metrics RMSE, Anomoly Correlation Coefficients, Bias and physical consistency. Our proposal demonstrates higher prediction accuracy with fast convergence, and the metrics and visualizations show that our model is insensitive to hyperparameter tuning, ensuring better generalization and adherence to physical consistency. Moreover, as observed from the spatial distribution of the anomaly correlation coefficient, the model exhibits higher forecasting accuracy for coastal and marginal sea regions.

Current progress in subseasonal-to-decadal prediction based on machine learning

The application of machine learning (ML) techniques to climate science has received significant attention, particularly in the field of climate predictions, ranging from sub-seasonal to decadal time scales. This paper reviews recent progress of ML techniques employed in climate phenomena prediction and the enhancement of dynamic forecast models, which provide valuable insights into the great potentials of ML techniques to improve climate prediction capabilities with reduced computational time and resource consumption. This paper also discusses several major challenges in the application of ML to climate prediction, including the scarcity of datasets, physical inconsistency, and lack of model transparency and interpretability. Additionally, this paper sheds light on how climate change impacts ML model training and prediction, and explores three key areas with potential breakthroughs: large-scale climate models, knowledge discovery driven by ML, and hybrid dynamical-statistical models, underscoring the important role of the integration of “ML and dynamical models” in building a bridge between the artificial intelligence and climate science.

Prediction of hydrate formation boundaries in pure water and salt/alcohol containing systems based on prior knowledge and artificial intelligence

As an efficient and clean energy source, natural gas plays a crucial role in optimizing the global energy consumption structure. However, the formation and accumulation of hydrates in pipelines during the exploitation and transportation of natural gas greatly jeopardize the production safety. Accurately predicting hydrate formation boundaries remains challenging due to the complex interation among various influencing factors. Although mechanistic models provide insights, they often fall short in accuracy. Conversely, machine learning models, while promising, may face limitations related to small sample sizes and a lack of physical interpretability. To overcome these limitations, this study proposed a physically guided neural network (PGNN) based on the Chen-Guo model, which integrated thermodynamic mechanisms with a neural network framework. This proposed model utilized pressure calculated from the Chen-Guo model as an input variable and incorporates physical inconsistencies into its loss function. A comparative analysis using literature data that PGNN achieved superior prediction accuracy, with an R2 value of 0.9768 for gas hydrate formation pressure prediction. The integration of thermodynamic mechanisms enhanced the prediction accuracy, as evidenced by an increase in the R2 value by 0.036, and a reduction in the MSE value by 5.56. Furthermore, the PGNN maintained a high level of prediction accuracy, ranging from 0.96 to 0.98, even with limited sample sizes, thus confirming its applicability and stability. This study validated the feasibility of PGNN for predicting gas hydrate formation conditions and offered insights into hydrate-based applications and hydrate management strategies.

Parameter estimation of PEM fuel cells using metaheuristic algorithms

Developing an accurate model for precise parameter estimation is crucial for the design of polymer electrolyte membrane fuel cells (PEMFC). While many studies can meet the evaluation criteria for parameter estimation, there is a significant physical inconsistency when applying the Generalized Steady State of Electrochemical Model (GSSEM) prediction in the optimization framework. This study aims to address this inconsistency by adding necessary constraints to the optimization model. It employs the Artificial Hummingbird Algorithm (AHA) and Lévy flight improvement to find unknown parameters by minimizing the sum of squared errors (SSE). The proposed method has been rigorously tested with seven PEMFC cases, demonstrating its robustness and superior accuracy compared to alternative approaches. Notably, the cathode charge transfer coefficient exhibits zero relative percent difference (RPD) between the two sources, confirming its physical consistency. Furthermore, the proposed constrained optimization model lays the groundwork for achieving consistent parameter estimation results for PEMFC when utilizing GSSEM.

Optical fibers with a frequency-dependent Kerr nonlinearity: Theory and applications

This review provides a detailed discussion of both the mathematical treatment and the impact of a frequency-dependent Kerr nonlinearity on the propagation of short pulses in optical fibers. We revisit the theoretical framework required to deal with the frequency dependence of the nonlinear response without incurring any physical inconsistencies, such as the non-conservation of the photon number. Then, we point out the role of the zero-nonlinearity wavelength, its interplay with the zero-dispersion wavelength, and their influence on evolution of optical pulses in optical fibers, specifically by looking at soliton propagation and the ensuing generation of Cherenkov radiation. Finally, by means of a space–time analogy involving the collision of a weak control pulse and an intense soliton, we describe an all-optical switching scheme in the presence of a zero-nonlinearity wavelength within a photon-conserving framework.

Discrete-continuum-discrete approach for the modeling of the dynamic behavior of 2D lattice systems

This work presents a new methodology to model elastic lattice systems through a two-step approach that permits to reliably capture its dynamic behavior with a lower computational cost than modeling the lattice explicitly. The first step consists of a non-standard continualization accounting for scale effects. Several methods are explored to derive new continuum models, whose dispersive and vibrational behaviors are compared to that of the lattice, considered as a reference. Non-classical models with micro-inertia reveal high accuracy, not presenting physical inconsistencies. The second step follows a FEM spatial discretization of the developed continua, accounting for micro inertia terms in the mass matrix. Finally, the FEM formulation allows the use of element sizes larger (up to four times) than the physical length scale of the lattice system, thus significantly reducing the computational cost while maintaining accuracy and enabling a versatile application to materials, geometries and boundary conditions. The methodology is tested here for a 2D system with displacements in the plane, but can be extended to other lattice typologies as well.

An Integer-Fractional Gradient Algorithm for Back Propagation Neural Networks

This paper proposes a new optimization algorithm for backpropagation (BP) neural networks by fusing integer-order differentiation and fractional-order differentiation, while fractional-order differentiation has significant advantages in describing complex phenomena with long-term memory effects and nonlocality, its application in neural networks is often limited by a lack of physical interpretability and inconsistencies with traditional models. To address these challenges, we propose a mixed integer-fractional (MIF) gradient descent algorithm for the training of neural networks. Furthermore, a detailed convergence analysis of the proposed algorithm is provided. Finally, numerical experiments illustrate that the new gradient descent algorithm not only speeds up the convergence of the BP neural networks but also increases their classification accuracy.

Interpretable chiller fault diagnosis based on physics-guided neural networks

Chillers are critical for building system energy efficiency and occupant comfort. Various methods have been developed for diagnosing chiller faults, with deep learning being a common choice due to its strong end-to-end learning capability. But in the absence of physical knowledge, deep learning models cannot provide a diagnosis outcome in accordance with the mechanisms behind faults, resulting in models lacking interpretability. In this regard, this paper proposes an interpretable chiller fault diagnosis method based on physics-guided neural networks. Through a comprehensive analysis of the chiller’s working process and fault evolution, a key physical indicator related to the condenser fouling fault is derived. Then a corresponding physical inconsistency loss function is developed to construct a condenser fouling fault-oriented diagnostic model. The proposed method has been demonstrated to be effective in reducing physical inconsistency while retaining diagnostic performance. The proposed method is conducive to applying the neural network approach in real operational scenarios and can serve as a reference for future research.

Toward Optimal Robot Machining Considering the Workpiece Surface Geometry in a Task-Oriented Approach

Robot workpiece machining is interesting in industry as it offers some advantages, such as higher flexibility in comparison with the conventional approach based on CNC technology. However, in recent years, we have been facing a strong progressive shift to custom-based manufacturing and low-volume/high-mix production, which require a novel approach to automation via the employment of collaborative robotics. However, collaborative robots feature only limited motion capability to provide safety in cooperation with human workers. Thus, it is highly necessary to perform more detailed robot task planning to ensure its feasibility and optimal performance. In this paper, we deal with the problem of studying kinematic robot performance in the case of such manufacturing tasks, where the robot tool is constrained to follow the machining path embedded on the workpiece surface at a prescribed orientation. The presented approach is based on the well-known concept of manipulability, although the latter suffers from physical inconsistency due to mixing different units of linear and angular velocity in a general 6 DOF task case. Therefore, we introduce the workpiece surface constraint in the robot kinematic analysis, which enables an evaluation of its available velocity capability in a reduced dimension space. Such constrained robot kinematics transform the robot’s task space to a two-dimensional surface tangent plane, and the manipulability analysis may be limited to the space of linear velocity only. Thus, the problem of physical inconsistency is avoided effectively. We show the theoretical derivation of the proposed method, which was verified by numerical experiments.

Physics-Constrained Deep Learning Postprocessing of Temperature and Humidity

Abstract Weather forecasting centers currently rely on statistical postprocessing methods to minimize forecast error. This improves skill but can lead to predictions that violate physical principles or disregard dependencies between variables, which can be problematic for downstream applications and for the trustworthiness of postprocessing models, especially when they are based on new machine learning approaches. Building on recent advances in physics-informed machine learning, we propose to achieve physical consistency in deep learning–based postprocessing models by integrating meteorological expertise in the form of analytic equations. Applied to the postprocessing of surface weather in Switzerland, we find that constraining a neural network to enforce thermodynamic state equations yields physically consistent predictions of temperature and humidity without compromising performance. Our approach is especially advantageous when data are scarce, and our findings suggest that incorporating domain expertise into postprocessing models allows the optimization of weather forecast information while satisfying application-specific requirements. Significance Statement Postprocessing is a widely used approach to reduce forecast error using statistics, but it may lead to physical inconsistencies. This outcome can be problematic for trustworthiness and downstream applications. We present the first machine learning–based postprocessing method intentionally designed to strictly enforce physical laws. Our framework improves physical consistency without sacrificing performance and suggests that human expertise can be incorporated into postprocessing models via analytic equations.

Physics-Guided Adversarial Machine Learning for Aircraft Systems Simulation

In the context of aircraft system performance assessment, deep learning technologies allow us to quickly infer models from experimental measurements, with less detailed system knowledge than usually required by physics-based modeling. However, this inexpensive model development also comes with new challenges regarding model trustworthiness. This article presents a novel approach, physics-guided adversarial machine learning (ML), which improves the confidence over the physics consistency of the model. The approach performs, first, a physics-guided adversarial testing phase to search for test inputs revealing behavioral system inconsistencies, while still falling within the range of foreseeable operational conditions. Then, it proceeds with a physics-informed adversarial training to teach the model the system-related physics domain foreknowledge through iteratively reducing the unwanted output deviations on the previously uncovered counterexamples. Empirical evaluation on two aircraft system performance models shows the effectiveness of our adversarial ML approach in exposing physical inconsistencies of both the models and in improving their propensity to be consistent with physics domain knowledge.

Novel low-order continuum models for the dynamic behaviour of microstructured plates based on a beam-grid lattice

In this paper, several non-classical continuum models are developed by applying different continualization techniques to a 2D beam-grid lattice. The purpose of these new models is to capture the size-effects that arise in certain dynamic problems, and which are not accounted by the Classic Kirchhoff plate model. All these models are evaluated via dispersion and natural frequency analyses, considering the discrete model as a reference. Interestingly, the non-standard models derived do not present any physical inconsistency, and capture the size-effect through space–time cross derivatives with low spatial order, thus not needing extra boundary conditions, an important feature ignored in other works.

Physics-added neural networks: An image-based deep learning for material printing system

Drop-on-demand material printing condition optimization with empirical knowledge based on the trial-and-error method and implementation of computational fluid dynamics (CFD) requires a significant amount of time and costs. In addition, current state-of-the-art neural network algorithms can reduce such time and cost by predicting droplet formation over a temporal domain; however, insufficient data and physical inconsistencies across output results limit their extrapolative prediction performance. In this study, we performed multi-GPU-implemented CFD simulations to collect large-scale training data, and we propose a novel algorithm for embedding physics knowledge in neural networks: physics-added neural networks. Integrating an image-to-image deep learning algorithm with our novel physics-embedding module, we obtain more consistent physical characteristics of the material printing process, such as mass and energy. Furthermore, by guiding the neural network to adhere to mass and energy conservation laws, the possible solution space of neural network training is restricted, which enhances convergence stability and decreases training time.

Reliable Thermal-Physical Modeling of Lithium-Ion Batteries: Consistency between High-Frequency Impedance and Ion Transport

Among lithium-ion battery diagnostic tests, electrochemical impedance spectroscopy, being highly informative on the physics of battery operation within limited testing times, deserves a prominent role in the identification of model parameters and the interpretation of battery state. Nevertheless, a reliable physical simulation and interpretation of battery impedance spectra is still to be addressed, due to its intrinsic complexity. An improved methodology for the calibration of a state-of-the-art physical model is hereby presented, focusing on high-energy batteries, which themselves require a careful focus on the high-frequency resistance of the impedance response. In this work, the common assumption of the infinite conductivity of the current collectors is questioned, presenting an improved methodology for simulating the pure resistance of the cell. This enables us to assign the proper contribution value to current collectors’ resistance and, in turn, not to underestimate electrolyte conductivity, thereby preserving the physical relation between electrolyte conductivity and diffusivity and avoiding physical inconsistencies between impedance spectra and charge–discharge curves. The methodology is applied to the calibration of the model on a commercial sample, demonstrating the reliability and physical consistency of the solution with a set of discharge curves, EIS, and a dynamic driving cycle under a wide range of operating conditions.

Artificial Intelligence Applications in Electric Distribution Systems: Post-Pandemic Progress and Prospect

Advances in machine learning and artificial intelligence (AI) techniques bring new opportunities to numerous intractable tasks for operation and control in modern electric distribution systems. Nevertheless, AI applications for such grids as cyber-physical systems encounter multifaceted challenges, e.g., high requirements for the quality and quantity of training data, data efficiency, physical inconsistency, interpretability, and privacy concerns. This paper provides a systematic overview of the state-of-the-art AI methodologies in the post-pandemic era, represented by transfer learning, deep attention mechanism, graph learning, and their combination with reinforcement learning and physics-guided neural networks. Dedicated research efforts on harnessing such recent advances, including power flow, state estimation, voltage control, topology identification, and line parameter calibration, are categorized and investigated in detail. Revolving around the characteristics of distribution system operation and integration of distributed energy resources, this paper also illuminates prospects and challenges typified by the privacy, explainability, and interpretability of such AI applications in smart grids. Finally, this paper attempts to shed light on the deeper and broader prospects in the realm of smart distribution grids by interoperating them with smart building and transportation electrification

Sensing prior constraints in deep neural networks for solving exploration geophysical problems

One of the key objectives in geophysics is to characterize the subsurface through the process of analyzing and interpreting geophysical field data that are typically acquired at the surface. Data-driven deep learning methods have enormous potential for accelerating and simplifying the process but also face many challenges, including poor generalizability, weak interpretability, and physical inconsistency. We present three strategies for imposing domain knowledge constraints on deep neural networks (DNNs) to help address these challenges. The first strategy is to integrate constraints into data by generating synthetic training datasets through geological and geophysical forward modeling and properly encoding prior knowledge as part of the input fed into the DNNs. The second strategy is to design nontrainable custom layers of physical operators and preconditioners in the DNN architecture to modify or shape feature maps calculated within the network to make them consistent with the prior knowledge. The final strategy is to implement prior geological information and geophysical laws as regularization terms in loss functions for training the DNNs. We discuss the implementation of these strategies in detail and demonstrate their effectiveness by applying them to geophysical data processing, imaging, interpretation, and subsurface model building.

Multivariate bias correction of regional climate model boundary conditions

Improving modeling capacities requires a better understanding of both the physical relationship between the variables and climate models with a higher degree of skill than is currently achieved by Global Climate Models (GCMs). Although Regional Climate Models (RCMs) are commonly used to resolve finer scales, their application is restricted by the inherent systematic biases within the GCM datasets that can be propagated into the RCM simulation through the model input boundaries. Hence, it is advisable to remove the systematic biases in the GCM simulations prior to downscaling, forming improved input boundary conditions for the RCMs. Various mathematical approaches have been formulated to correct such biases. Most of the techniques, however, correct each variable independently leading to physical inconsistencies across the variables in dynamically linked fields. Here, we investigate bias corrections ranging from simple to more complex techniques to correct biases of RCM input boundary conditions. The results show that substantial improvements in model performance are achieved after applying bias correction to the boundaries of RCM. This work identifies that the effectiveness of increasingly sophisticated techniques is able to improve the simulated rainfall characteristics. An RCM with multivariate bias correction, which corrects temporal persistence and inter-variable relationships, better represents extreme events relative to univariate bias correction techniques, which do not account for the physical relationship between the variables.

Bounding Variance and Skewness of Fluctuations in Nonlinear Dynamical Systems with Stochastic Thermodynamics*

Fluctuations arising in nonlinear dissipative systems (diode, transistors, chemical reaction, etc.) subject to an external drive (voltage, chemical potential, etc.) are well known to elude any simple characterisation such as the fluctuation-dissipation theorem (also called Johnson-Nyquist law, or Einstein's law in specific contexts). Using results from stochastic thermodynamics, we show that the variance of these fluctuations exceeds the variance predicted by a suitably extended version of Johnson-Nyquist's formula, by an amount that is controlled by the skewness (third moment) of the fluctuations. As a consequence, symmetric fluctuations necessarily obey the extended Johnson-Nyquist formula. This shows the physical inconsistency of Gaussian approximation for the noise arising in some nonlinear models, such as MOS transistors or chemical reactions. More generally, this suggests the need for a stochastic nonlinear systems theory that is compatible with the teachings of thermodynamics.

One-dimensional study of boundary effects and damage diffusion for regularized damage models

Local Continuum Damage Mechanics models cannot represent the entire degradation process in materials exhibiting strain-softening behaviors. It is well known that the rate equilibrium problem becomes ill-posed when softening occurs, and an infinity of solutions exists. From a numerical point of view, finite element analyses suffer from mesh-dependent results. Non-local models are generally used to regularize the structural response and recover objectivity. However, some physical inconsistencies can be observed in numerical results, e.g., damage diffusion over large damaged bands and damage attraction on the boundary of the computational domain. Non-local formulations with evolving interactions may better describe the damaging process and overcome these issues. This paper uses the so-called spalling test to underline the main drawbacks and advantages of several regularized models with constant and evolving non-local interactions. Concerning non-local formulations with constant interactions, attention is focused on the integral non-local formulation on the internal variable of the constitutive model and an implicit gradient damage formulation. Regarding formulations with evolving non-local interactions, attention is focused on a “stress-based integral non-local” approach and the so-called “eikonal non-local” approach. In this latter case, both its integral-type and gradient-type variants are considered.