Pyphysdisc: co-evolutionary symbolic regression with adaptive smoothing windows for autonomous physical law discovery from noisy data

Learning dynamics from coarse/noisy data with scalable symbolic regression

Nonlinear dynamical systems are omnipresent in nature, commonly seen in many disciplines such as physics, biology, chemistry, climate science, and engineering. Discovering the analytical expressions of underlying physics that govern the systems from their measurement data is essential for understanding the behaviors of the observed nonlinear dynamics, which might be complex, or even chaotic. Directly distilling the nonlinear governing equations from limited and noisy measurement data has long been vital but challenging. To tackle this fundamental issue, this dissertation introduces a novel Physics-informed Spline Learning (PiSL) framework to discover parsimonious governing equations for nonlinear dynamics, based on sparsely sampled noisy data. Specifically, splines are employed to locally interpolate the dynamics and perform analytical differentiation, feeding the discovery of underlying equations in form of either a linear interpolation of candidate terms or a symbolic-activated neural network model. The physics residual in turn informs the spline learning. The synergy between splines and discovered governing equations produces great robustness against high-level data sparsity and noise. Subsequently, a hybrid sparsity-promoting alternating direction optimization strategy is developed for fine-tuning the coefficients with a sparsity enforcement approach to obtain a parsimonious structure of discovered governing equations. The effectiveness and supremacy of the proposed PiSL architectures have been demonstrated by several numerical and experimental examples, in comparison with two state-of-the-art methods serving as baselines. Meanwhile, this dissertation re-envisions the data-driven nonlinear dynamics discovery tasks by casting them into symbolic regression problems. Under this scheme, a symbolic regressor is established to identify the differential equations that best describe the underlying governing physical laws without fixed forms of expressions as a starting point, leveraging great flexibility (i.e, free combination of mathematical operations and symbols) in model selection. This dissertation develops an innovative Symbolic Reinforcement Learning (SRL) machine to discover the mathematical structure of equations based on sparse and noisy measurement data. The central concept is to (i) interpret mathematical operations and variables by symbols following certain grammar rules, (ii) establish the symbolic reasoning of equations via expression trees, and (iii) develop an environment for the reinforcement learning (RL) agent to explore optimal expression trees based on data. In particular, the RL agent is able to obtain an optimistic computational policy through the traversal of expression trees, featuring the one that maps to the optimal arithmetic expression of the underlying equation. The robustness of the SRL machine is demonstrated by examples of symbolic regression and the identification of nonlinear dynamics (both numerically and experimentally). Salient features of the proposed framework include search flexibility and enforcement of parsimony for discovered equations. It offers a new perspective to finding interpretable and generalizable symbolic models for facilitating cross-disciplinary data-driven scientific discovery. Moreover, this dissertation presents an amelioration to the proposed RL-based symbolic regressor, the Symbolic Physics Learner (SPL) machine. The SPL machine employs Monte Carlo tree search (MCTS) algorithm, which is featured by a sound mathematical underpinning for the trade-off between exploration and exploitation. A few adjustments to the conventional MCTS algorithm are made to better fit the symbolic regression and nonlinear dynamics discovery problems. Consequently, the SPL machine is capable of efficiently uncovering the best path to formulate the complex mathematical expressions and governing equations of the target dynamical system. The efficacy and superiority of the PSL machine are demonstrated by numerical examples including the classic Nguyen symbolic regression benchmarks, the tasks of physics law discovery from experiment-measured data, and the nonlinear dynamics discovery experiments, in comparison with state-of-the-art methods.--Author's abstract

Abstract. Weighing lysimeters yield the most precise and realistic measures for evapotranspiration (ET) and precipitation (P), which are of great importance for many questions regarding soil and atmospheric sciences. An increase or a decrease of the system mass (lysimeter plus seepage) indicates P or ET. These real mass changes of the lysimeter system have to be separated from measurement noise (e.g., caused by wind). A promising approach to filter noisy lysimeter data is (i) to introduce a smoothing routine, like a moving average with a certain averaging window, w, and then (ii) to apply a certain threshold value, δ, accounting for measurement accuracy, separating significant from insignificant weight changes. Thus, two filter parameters are used, namely w and δ. In particular, the time-variable noise due to wind as well as strong signals due to heavy precipitation pose challenges for such noise-reduction algorithms. If w is too small, data noise might be interpreted as real system changes. If w is too wide, small weight changes in short time intervals might be disregarded. The same applies to too small or too large values for δ. Application of constant w and δ leads either to unnecessary losses of accuracy or to faulty data due to noise. The aim of this paper is to solve this problem with a new filter routine that is appropriate for any event, ranging from smooth evaporation to strong wind and heavy precipitation. Therefore, the new routine uses adaptive w and δ in dependence on signal strength and noise (AWAT – adaptive window and adaptive threshold filter). The AWAT filter, a moving-average filter and the Savitzky–Golay filter with constant w and δ were applied to real lysimeter data comprising the above-mentioned events. The AWAT filter was the only filter that could handle the data of all events very well. A sensitivity study shows that the magnitude of the maximum threshold value has practically no influence on the results; thus only the maximum window width must be predefined by the user.

Correcting model misspecification in physics-informed neural networks (PINNs)

Physics-constrained bayesian neural network for fluid flow reconstruction with sparse and noisy data

Bayesian physics-informed extreme learning machine for forward and inverse PDE problems with noisy data

This presentation gives an introduction to the topic of event generators in particle physics.The emphasis is on the physics aspects that have to be considered in the construction of a generator, and what lessons we h a v e learned from comparisons with data.A brief survey of existing generators is also included.As illustration, a few topics of current i n terest are covered in a bit more detail: QCD uncertainties in W mass determinations and p/ physics.

Intelligent computational techniques for physical object properties discovery, detection, and prediction: A comprehensive survey

In this paper, we present a stepwise genetic programming algorithm to perform regression on a large number of noisy data, the purpose of which is to find a mathematical model for samples. To obtain an accurate statistical model of noisy sample points, discrete cosine transform was inserted into a standard GP algorithm. The energy-compaction property of DCT makes it very suitable for accelerating the implement of the standard GP algorithm and dealing with the noisy data samples. We tested the proposed algorithm with benchmark instances and compared it with several popular other algorithms. The experimental results have shown that the proposed algorithm is a powerful tool in finding optimal solutions.

1-D coupled surface flow and transport equations revisited via the physics-informed neural network approach

For a variety of applications within power systems, the precision of data acquisition is of paramount importance. However, the actual data may be corrupted by noise in the process of measurement or transmission, and the accuracy of dynamic security assessment (DSA) will be affected. In light of the poor interpretability exhibited by traditional machine learning (ML) methods in denoising, a physics-informed denoising model (PIDM) for dynamic data recovery is proposed. The differential equations of physical models in power systems are employed to guide the training of PIDM. They are transformed into physical constraints and subsequently incorporated into the loss function of stacked denoising autoencoder (SDAE) to cleanse noisy data. By integrating the powerful learning capabilities of ML with the rigorous constraints of physical laws, the noisy data recovered by PIDM can better satisfy the dynamic equations. Consequently, a more pronounced denoising effect can be achieved. The improvement of the PIDM over common ML-based models is explored when dealing with the noisy data with varying degrees of interference or those of unexpected faults. The effectiveness is validated through simulation results in IEEE 39-bus system and the East China power grid. The results show that this method can reduce the total mean square error (MSE) of the recovery of noisy data to at least 65.27% of that of the traditional methods under the same conditions. In addition to demonstrating superior denoising performance, the generalization capability under diverse noise conditions is also deemed excellent.

B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data

This paper presents a novel method to perform regression on a finite sample of noisy data. The purpose is to obtain a mathematical model for data which is both reliable and valid, yet the analytical expression is not restricted to any particular form. To obtain a statistical model of the noisy data set we use symbolic regression with pseudorandom number generators. We begin by describing symbolic regression and our implementation of this technique using genetic programming (GP) and gene expression programming (GEP). We present some results for symbolic regression on computer generated and real financial data sets in the final part of this paper.

Automated learning of interpretable models with quantified uncertainty

Physics seeks to uncover the laws of Nature and express them through mathematical equations . Despite the vast diversity of natural phenomena, physical equations exhibit structural regularities that set them apart from arbitrary mathematical expressions. While principles such as dimensional analysis have long guided the formulation of physical models, the exploration of more subtle statistical patterns within the equations of physics remains an open question. Here, by analysing four corpora of physics equations and applying advanced implicit-likelihood techniques, we find that the frequency of mathematical operators follows an exponential decay law, in contrast to Zipf's power law for word frequencies in natural languages. This reveals a statistical meta-law of physics, possibly reflecting a combination of communication efficiency and constraints imposed by Nature itself. The meta-law offers practical benefits for symbolic regression by drastically narrowing down the space of physically plausible expressions. More broadly, it may inform the development of language models that can generate coherent mathematical representations, advancing the automation of physical law discovery. This article is part of the discussion meeting issue 'Symbolic regression in the physical sciences'.