Posterior Variance Research Articles

Efficient Learning for Selecting Important Nodes in Random Network

In this article, we consider the problem of selecting important nodes in a random network, where the nodes connect to each other randomly with certain transition probabilities. The node importance is characterized by the stationary probabilities of the corresponding nodes in a Markov chain defined over the network, as in Google's PageRank. Unlike a deterministic network, the transition probabilities in a random network are unknown but can be estimated by sampling. Under a Bayesian learning framework, we apply the first-order Taylor expansion and normal approximation to provide a computationally efficient posterior approximation of the stationary probabilities. In order to maximize the probability of correct selection, we propose a dynamic sampling procedure, which uses not only posterior means and variances of certain interaction parameters between different nodes, but also the sensitivities of the stationary probabilities with respect to each interaction parameter. Numerical experiment results demonstrate the superiority of the proposed sampling procedure.

Bayesian inference for Pareto distribution with prior conjugate and prior non conjugate

The purpose of this study is to determine the best estimator for estimating the shape parameters of the Pareto distribution with the known scale parameter. Estimation of these parameters is done by using the Gamma distribution as the prior distribution of the conjugate and the Uniform distribution as the non-conjugate prior distribution. A comparison of the two prior distributions is done through simulation studies with various sample sizes. The best estimator net is a method that produces the smallest posterior variance, absolute bias, and Bayes confidence interval. This study proves that the Bayes estimator by using the prior conjugate distribution produces all indicators of the goodness of the model with a smaller value than the non-conjugate prior distribution. Thus it can be concluded that the estimator with prior conjugate will produce a better predictive value than prior non-conjugate.

Adaptive experiment design for probabilistic integration

Probabilistic integration is a Bayesian inference technique for numerical integration, and has received much attention in the community of scientific and engineering computations. The most appealing advantages are the ability to improve the integration accuracy by making full use of the spatial correlation information among the design points, and the treatment of discretization error as a source of epistemic uncertainty being explicitly propagated to the integration results. This paper aims to develop an adaptive algorithm for further improving the efficiency and accuracy of the probabilistic integration when it is applied to the time-consuming computer simulators. A learning function is first extracted from the posterior variance of the integration and is shown to be especially useful for identifying the design point, by adding which to the training data set, the most reduction of the posterior variance of integration can be achieved. Based on this learning function, an adaptive experiment design algorithm is then developed for actively producing optimal design points. Results of the experiment tests and engineering application show that, with the same number of design points, the developed design strategy always produce more accurate and robust integration results, than the three kinds of commonly used random sampling design strategies (i.e., Monte Carlo design, Latin-hypercube design and Sobol sequence).

Multi-Goal Prior Selection: A Way to Reconcile Bayesian and Classical Approaches for Random Effects Models

The two-level normal hierarchical model has played an important role in statistical theory and applications. In this article, we first introduce a general adjusted maximum likelihood method for estimating the unknown variance component of the model and the associated empirical best linear unbiased predictor of the random effects. We then discuss a new idea for selecting prior for the hyperparameters. The prior, called a multi-goal prior, produces Bayesian solutions for hyperparmeters and random effects that match (in the higher order asymptotic sense) the corresponding classical solution in linear mixed model with respect to several properties. Moreover, we establish for the first time an analytical equivalence of the posterior variances under the proposed multi-goal prior and the corresponding parametric bootstrap second-order mean squared error estimates in the context of a random effects model.

Rapid Forecast Calibration Using Nonlinear Simulation Regression With Localization

This article, written by JPT Technology Editor Judy Feder, contains highlights of paper SPE 193845, “Rapid Forecast Calibration Using Nonlinear Simulation Regression With Localization,” by Jincong He, SPE, Wenyue Sun, and Xian-Huan Wen, SPE, Chevron, prepared for the 2019 SPE Reservoir Simulation Conference, Galveston, Texas, 10–11 April. The paper has not been peer reviewed. The industry increasingly relies on forecasts from reservoir models for reservoir management and decision making. However, because forecasts from reservoir models carry large uncertainties, calibrating them as soon as data come in is crucial. Traditional probabilistic history matching remains time-consuming because it needs to calibrate the models before using them to update probability distributions (S-curves) of quantities of interest (QOIs). This paper presents a direct forecast method called simulation regression with localization (SRL), which is able to calibrate the forecast with observed data without calibrating the model. Introduction Production forecasts from reservoir-simulation models can be affected by the various uncertainties present in the subsurface, such as those in the geological characterization and rock and fluid properties. Calibrating uncertain QOIs, such as production-forecast or key subsurface parameters, to observation data is a crucial element in the context of closed-loop reservoir management. Traditionally, this task is accomplished by a two-step approach, as depicted in Fig. 1a. First, a probabilistic history-matching process needs to be carried out to condition the simulation models to the data. Probabilistic history matching generates an ensemble of reservoir models that characterize the posterior uncertainty given the measurement data (e.g., historical well-test and production data), as opposed to deterministic history matching, in which only one model that best matches the measurement data is generated. In the second step, reservoir simulations, subject to a given reservoir operation plan, are performed on the generated posterior samples, or models, to estimate the posterior distribution of the QOIs. There is rich literature on various probabilistic history-matching methods, and the complete paper cites and assesses several examples. However, the authors state that, although much progress has been made in probabilistic history matching, the resulting procedures still are generally time-consuming for realistic systems, and may not provide accurate sampling of posterior distribution when strong nonlinearity and non-Gaussianity exist. Proposed Work Flow The complete paper explores an alternative forecasting framework, as shown in Fig. 1b. The authors propose using SRL for direct forecasting to derive the posterior distributions for QOIs given the measurement data. As with other direct forecasting methods, the first step of SRL is to perform an ensemble of simulation runs according to the previous distributions of the uncertainties and construct a training data set for the QOIs and measurement data. Then, in SRL, a regression model is built on this training data set to estimate the mean value of the posterior distribution. Depending on the case at hand, this regression model can be as simple as a linear regression model, or as complex as a deep-learning model. The authors found that the quadratic partial least square (QPLS) regression model offers flexibility to handle a wide range of problems. However, the posterior variances of the QOIs are estimated by a localization process wherein the mismatch of the regression model for samples closed to observed data is used to estimate the posterior variance. Finally, the entire posterior distribution (in terms of the cumulative distribution function; i.e., the S-curve) can be estimated by scaling the prior S-curve with the updated mean and variance.

Reformulation of Bayesian Geostatistical Approach on Principal Components

AbstractIn this note, we reformulate Bayesian geostatistical inverse approach based on principal component analysis of the spatially correlated parameter field to be estimated. The unknown parameter field is described by a latent‐variable model as a realization of projections on its principal component axes. The reformulated geostatistical approach (RGA) achieves substantial dimensionality reduction by estimating the latent variable of projections on truncated principal components instead of directly estimating the parameter field. We provide solutions for best estimates and posterior variances for linear and quasi‐linear inverse problems. The number of normal equations to be solved is reduced to k+p, where k is the number of retained principal components and p is the number of drifts, both independent of the number of observations. To determine the Jacobian matrix for quasi‐linear problems, the number of forward model runs in each iteration is reduced to k+p+1. There is no need to evaluate the Jacobian matrix in terms of the unknown parameter field. RGA unifies the problem setup and computational techniques for large‐dimensional inverse problems introduced previously, which are now naturally built in the reformulated framework. RGA is more efficient and scalable for both large‐dimensional inverse problems and problems with a massive volume of observations. Moreover, conditional realizations of the parameter field can be conveniently generated by generating conditional realizations of latent variables on truncated principal component axes. We also relate the new approach to the classical geostatistical approach formulas. Large‐dimensional hydraulic tomography problems are used to demonstrate the application of the reformulated approach.

Active learning polynomial chaos expansion for reliability analysis by maximizing expected indicator function prediction error

AbstractAssessing the failure probability of complex aeronautical structure is a difficult task in presence of uncertainties. In this paper, active learning polynomial chaos expansion (PCE) is developed for reliability analysis. The proposed method firstly assigns a Gaussian Process (GP) prior to the model response, and the covariance function of this GP is defined by the inner product of PCE basis function. Then, we show that a PCE model can be derived by the posterior mean of the GP, and the posterior variance is obtained to measure the local prediction error as Kriging model. Also, the expectation of the prediction variance is derived to measure the overall accuracy of the obtained PCE model. Then, a learning function, named expected indicator function prediction error (EIFPE), is proposed to update the design of experiment of PCE model for reliability analysis. This learning function is developed under the framework of the variance‐bias decomposition. It selects new points sequentially by maximizing the EIFPE that considers both the variance and bias information, and it provides a dynamic balance between global exploration and local exploitation. Finally, several test functions and engineering applications are investigated, and the results are compared with the widely used Kriging model combined with U and expected feasibility function learning function. Results show that the proposed method is efficient and accurate for complex engineering applications.

Extra-Wide Lane Ambiguity Resolution and Validation for a Single Epoch Based on the Triple-Frequency BeiDou Navigation Satellite System

The ambiguity resolution (AR) and validation of the global navigation satellite system (GNSS) have been challenging tasks for some decades. Considering the reliability problem of extra-wide-lane (EWL) ambiguity in the triple-carrier ambiguity resolution (TCAR), a method for validating the reliability of the EWL ambiguity using a single epoch was proposed for the BeiDou Navigation Satellite System (BDS). For the initial EWL ambiguity, obtained using a rounding estimator with a geometry-free (GF) model, the double-difference ionospheric delay was first estimated to construct a relative positioning model with an initial fixed ambiguity. Second, based on the theory of gross error detection and the AR characteristics of EWL, the second-best ambiguity candidate was constructed. Finally, among the two sets of ambiguities, the one with the smaller posterior variance was taken as the reliable ambiguity. The study showed that, for a single epoch, when only one or two satellites had incorrect ambiguities, the AR success rate after ambiguity validation and correction could reach 100% for medium baselines. For long baselines, due to the increase of atmospheric error, the validation was affected to some extent. However, the AR success rates for two long baselines increased from 96.82% and 98.44% to 98.80% and 99.67%, respectively.

On the Bayesian Analysis of Constant-Stress Life Test Model Under Type-Ii Censoring

In this article, a constant-stress partially accelerated life test model of high reliability products and materials with type-II censored data from the linear failure rate distribution is considered. Two different methods of estimation likelihood and Bayesian are used to estimate the unknown parameters of the model. The maximum likelihood estimates of the model parameters are obtained using the Newton–Raphson technique. The posterior means and posterior variances are derived under the squared error loss function using Lindley’s approximation procedure. The advantages of this approximation are discussed. Monte Carlo simulations are made for comparing and evaluating the performance of the proposed methods of estimation.

A GREY MODEL FOR SHORT-TERM PREDICTION OF THE IONOSPHERIC TEC

Abstract. Accurate prediction of TEC can significantly improve the accuracy of navigation and positioning, therefore TEC observation and prediction has become a hot spot in ionospheric research. TEC has the characteristics of nonlinearity and non-stationarity, that cannot accurately describe this change by analytic expressions. Through the analysis of TEC content changes at the same time for several consecutive days in different seasons, it can be concluded that the TEC change at the same time in a short period is relatively stable, the overall monotonous change trend has a certain correlation. Since the grey model performs better in the prediction of a small amount of data and has a high accuracy in the prediction of time series with monotonous changes, it is used in the prediction of the same time and point-to-point short-term prediction of TEC. The accuracy of the grey model is verified by the Posterior variance ratio, the Little error probability test and the relation grade. The residual correction is made for the prediction results with low prediction accuracy, by further establishing the GM (1,1) model of residual values, and the original prediction results being compensated and refined by the residual GM (1,1) model. The experimental results show that the improved model is more accurate than the grey prediction model and can reflect the changing characteristics of ionospheric TEC.

Probabilistic H2-norm estimation via Gaussian process system identification

We present a method for data-based estimation of the H2-norm of a linear time-invariant system from input-output data in a probabilistic setting by employing the recent advances in Gaussian process system identification using stable-spline kernels. Advantages of this starting point include that the norm can be estimated for the continuous-time system and over infinite horizon, even though only a finite number of measurements are available. We approximate the H2-norm distribution as Gaussian, whose expectation can even be obtained analytically, while we use a numerical scheme based on Gaussian process quadrature for the variance. Not only do we utilize the posterior variance of the Gaussian process to derive an error estimate for the H2-norm, but also to tune the estimation by optimizing the input sequence. The performance of the developed scheme is thoroughly evaluated in simulation.

Sampling Properties of the Bayesian Posterior Mean With an Application to WALS Estimation

Many statistical and econometric learning methods rely on Bayesian ideas, often applied or reinterpreted in a frequentist setting. Two leading examples are shrinkage estimators and model averaging estimators, such as weighted-average least squares (WALS). In many instances, the accuracy of these learning methods in repeated samples is assessed using the variance of the posterior distribution of the parameters of interest given the data. This may be permissible when the sample size is large because, under the conditions of the Bernstein--von Mises theorem, the posterior variance agrees asymptotically with the frequentist variance. In finite samples, however, things are less clear. In this paper we explore this issue by first considering the frequentist properties (bias and variance) of the posterior mean in the important case of the normal location model, which consists of a single observation on a univariate Gaussian distribution with unknown mean and known variance. Based on these results, we derive new estimators of the frequentist bias and variance of the WALS estimator in finite samples. We then study the finite-sample performance of the proposed estimators by a Monte Carlo experiment with design derived from a real data application about the effect of abortion on crime rates.

Randomization and Reweighted $\ell_1$-Minimization for A-Optimal Design of Linear Inverse Problems

We consider optimal design of PDE-based Bayesian linear inverse problems with infinite-dimensional parameters. We focus on the A-optimal design criterion, defined as the average posterior variance ...

Prediction method of pavement performance based on same dimension gray recurrence dynamic model

In order to accurately grasp the change trend of asphalt pavement performance index, taking the rutting depth index (RDI) as an example, we establish a gray recurrence dynamic model with equivalent dimension which can use the new data effectively and dynamically. The model is used to predict the indexes such as pavement condition index (PCI), driving quality index (RQI) and skidding resistance index (SRI). The results show that in step 3, the minimum error probabilities of RDI, PCI, RQI and SRI are all 1, and the posterior prescription variance ratios are: 0.111 7, 1, 0.065 4, 1, 0.201 8, and 0.113 0, respectively. It is proved that with the increase of recursive steps, the accuracy of the prediction result of the model becomes higher, and the error becomes less, which shows that the method can accurately predict the road performance.

On the positivity and magnitudes of Bayesian quadrature weights

This article reviews and studies the properties of Bayesian quadrature weights, which strongly affect stability and robustness of the quadrature rule. Specifically, we investigate conditions that are needed to guarantee that the weights are positive or to bound their magnitudes. First, it is shown that the weights are positive in the univariate case if the design points locally minimise the posterior integral variance and the covariance kernel is totally positive (e.g. Gaussian and Hardy kernels). This suggests that gradient-based optimisation of design points may be effective in constructing stable and robust Bayesian quadrature rules. Secondly, we show that magnitudes of the weights admit an upper bound in terms of the fill distance and separation radius if the RKHS of the kernel is a Sobolev space (e.g. Matérn kernels), suggesting that quasi-uniform points should be used. A number of numerical examples demonstrate that significant generalisations and improvements appear to be possible, manifesting the need for further research.

Abundance estimation from multiple data types for group-living animals: An example using dhole (Cuon alpinus)

Large carnivores are declining globally and require baseline population estimates for management, however large-scale population estimation is problematic for species without unique natural marks. We used camera trap records of dhole Cuon alpinus, a group-living species, from three national parks in Thailand as a case study in which we develop integrated likelihood models to estimate abundance incorporating two different data sets, count data and detection/non-detection data. We further investigated relative biases of the models using different proportions of data with lower versus higher quality and assessed parameter identifiability. The simulations indicated that the relative bias on average across 24 tested scenarios was 2% with a 95% chance that the simulated data sets obtained the true animal abundances. We found that bias was high (>10%) when sampling 60 sites with only 5 sampling occasions. We tested four additional scenarios with varying proportions of count data. Our model tolerated the use of relatively low proportions of the higher quality count data, but below 10% the results began to show bias (>6%). Data cloning indicated that the parameters were identifiable with all posterior variances shrinking to near zero. Our model demonstrates the benefits of combining data from multiple studies even with different data types. Furthermore, the approach is not limited to camera trap data. Detection/non-detection data from track surveys or counts from transects could also be combined. Particularly, our model is potentially useful for assessing populations of rare species where large amounts of by-catch datasets are available.

Gaussian Mean Field Regularizes by Limiting Learned Information.

Variational inference with a factorized Gaussian posterior estimate is a widely-used approach for learning parameters and hidden variables. Empirically, a regularizing effect can be observed that is poorly understood. In this work, we show how mean field inference improves generalization by limiting mutual information between learned parameters and the data through noise. We quantify a maximum capacity when the posterior variance is either fixed or learned and connect it to generalization error, even when the KL-divergence in the objective is scaled by a constant. Our experiments suggest that bounding information between parameters and data effectively regularizes neural networks on both supervised and unsupervised tasks.

Phase Retrieval From Quantized Measurements via Approximate Message Passing

In this letter, the problem of sparse signal reconstruction from quantized noisy magnitudes y = Q (|z + w| + n) is studied, where z = Ax, Q(·) denotes a quantizer, and x is the sparse signal. According to expectation propagation, the abovementioned problem can be solved by exchanging extrinsic information between the standard linear model (SLM) module and the minimum mean square error (MMSE) module. For the MMSE module, exploiting the fact that the likelihood is only a function of the absolute value of z, the magnitude and phase information of z are decoupled, and only the posterior means and variances of the magnitude of z are calculated. While for the SLM module, the approximate message passing (AMP) is used. To obtain the closed-form expression during the iteration, a novel approximation is adopted. We refer to the resulting algorithm as generalized AMP (Gr-AMP) based quantized phase retrieval algorithm. Finally, several numerical simulations are conducted to demonstrate the performances of the proposed approach.

Gaussian process variance reduction by location selection

A key issue in Gaussian Process modeling is to decide on the locations where measurements are going to be taken. A good set of observations will provide a better model. Current state of the art selects such a set so as to minimize the posterior variance of the Gaussian process by exploiting submodularity. We propose two techniques, a Gradient Descent procedure and an heuristic algorithm to iteratively improve an initial set of observations so as to minimize the posterior variance directly. Results show the clear improvements that can be obtain under different settings.

Estimating the Expected Value of Sample Information across Different Sample Sizes Using Moment Matching and Nonlinear Regression

Background. The expected value of sample information (EVSI) determines the economic value of any future study with a specific design aimed at reducing uncertainty about the parameters underlying a health economic model. This has potential as a tool for trial design; the cost and value of different designs could be compared to find the trial with the greatest net benefit. However, despite recent developments, EVSI analysis can be slow, especially when optimizing over a large number of different designs. Methods. This article develops a method to reduce the computation time required to calculate the EVSI across different sample sizes. Our method extends the moment-matching approach to EVSI estimation to optimize over different sample sizes for the underlying trial while retaining a similar computational cost to a single EVSI estimate. This extension calculates the posterior variance of the net monetary benefit across alternative sample sizes and then uses Bayesian nonlinear regression to estimate the EVSI across these sample sizes. Results. A health economic model developed to assess the cost-effectiveness of interventions for chronic pain demonstrates that this EVSI calculation method is fast and accurate for realistic models. This example also highlights how different trial designs can be compared using the EVSI. Conclusion. The proposed estimation method is fast and accurate when calculating the EVSI across different sample sizes. This will allow researchers to realize the potential of using the EVSI to determine an economically optimal trial design for reducing uncertainty in health economic models. Limitations. Our method involves rerunning the health economic model, which can be more computationally expensive than some recent alternatives, especially in complex models.