Comparison Between Bayesian and Maximum Entropy Analyses of Flow Networks†

Steven Waldrip,Robert Niven

doi:10.3390/e19020058

Abstract

We compare the application of Bayesian inference and the maximum entropy (MaxEnt) method for the analysis of ﬂow networks, such as water, electrical and transport networks. The two methods have the advantage of allowing a probabilistic prediction of ﬂow rates and other variables, when there is insufﬁcient information to obtain a deterministic solution, and also allow the effects of uncertainty to be included. Both methods of inference update a prior to a posterior probability density function (pdf) by the inclusion of new information, in the form of data or constraints. The MaxEnt method maximises an entropy function subject to constraints, using the method of Lagrange multipliers,to give the posterior, while the Bayesian method ﬁnds its posterior by multiplying the prior with likelihood functions incorporating the measured data. In this study, we examine MaxEnt using soft constraints, either included in the prior or as probabilistic constraints, in addition to standard moment constraints. We show that when the prior is Gaussian,both Bayesian inference and the MaxEnt method with soft prior constraints give the same posterior means, but their covariances are different. In the Bayesian method, the interactions between variables are applied through the likelihood function, using second or higher-order cross-terms within the posterior pdf. In contrast, the MaxEnt method incorporates interactions between variables using Lagrange multipliers, avoiding second-order correlation terms in the posterior covariance. The MaxEnt method with soft prior constraints, therefore, has a numerical advantage over Bayesian inference, in that the covariance terms are avoided in its integrations. The second MaxEnt method with soft probabilistic constraints is shown to give posterior means of similar, but not identical, structure to the other two methods, due to its different formulation.

Highlights

The analysis of flow rates on networks is required for the design and monitoring of electrical, water, sewer, irrigation, fire suppression, drainage, oil, gas and any other networks through which fluids or energy are transported
In the Bayesian method, the interactions between variables are applied through the likelihood function, using second or higher-order cross-terms within the posterior pdf
The maximum entropy (MaxEnt) method with soft prior constraints, has a numerical advantage over Bayesian inference, in that the covariance terms are avoided in its integrations

Summary

Introduction

The analysis of flow rates on networks is required for the design and monitoring of electrical, water, sewer, irrigation, fire suppression, drainage, oil, gas and any other networks through which fluids or energy are transported. Bayes’ theorem is used to estimate the flows, pressures and pipeline characteristics as time progresses, using data obtained through real-time monitoring of the pipeline in a few locations As this method requires the solution of a partial differential equation which incorporates time and uncertainty, its computational cost is high and is restricted to small networks; in the example, a single pipe is analysed. Giffin and Caticha’s method [31,32,33] to obtain Bayes’ rule using MaxEnt requires the relative entropy function to be defined over the model parameters and the data. The current authors have compared the probability distributions of quasi-Newton rules obtained when inferring the Jacobian or Hessian using Bayesian inference [34,35] and the MaxEnt method [36] In both methods, the same Gaussian prior was used.

Bayesian Analysis

Formulation

Solution and Comparison to Bayesian Solution

Findings

Discussion