Bayesian Posterior Distribution Research Articles

Bayesian bootstrapping for symmetric distributions

In this paper, we describe a Bayesian nonparametric approach to make inference for a spherically symmetric distribution. We consider a Dirichlet invariant process prior on the set of all spherically symmetric distributions and we derive the Dirichlet invariant process posterior. Indeed, our approach is an extension of the Dirichlet invariant process to a spherically symmetric distribution. In addition, we obtain the Dirichlet invariant process posterior for the infinite transformation group and we prove that it approaches the mixtures of Dirichlet processes. Moreover, we develop our approach to obtain the Bayesian nonparametric posterior distribution for functionals of the distribution's support when the support is symmetric with respect to the parallel lines of axes. This suggests a Bayesian nonparametric bootstrapping scheme. The estimates can be derived based on posterior averaging. Then, our simulation results demonstrate that our suggested bootstrapping technique improves the accuracy of the estimates.

A comparison of estimators for the network autocorrelation model based on observed social networks

Network autocorrelation models (NAMs) are widely used to study a response variable of interest among subjects embedded within a network. Although the NAM is highly useful for studying such networked observational units, several simulation studies have raised concerns about point estimation. Specifically, these studies have consistently demonstrated a negative bias of maximum likelihood estimators (MLEs) of the network effect parameter. However, in order to gain a practical understanding of point estimation in the NAM, these findings need to be expanded in three important ways. First, these simulation studies are based on relatively simple network generative models rather than observed networks, thereby leaving as an open question how realistic network topologies may affect point estimation in practice. Second, although there has been strong work done in developing two-stage least squares estimators as well as Bayesian estimators, only the MLE has received extensive attention in the literature, thus leaving practitioners in question as to best practices. Third, the performance of these estimators need to be compared using both bias and variance, as well as the coverage rate of each estimator's corresponding confidence or credible interval. In this paper we describe a simulation study which aims to overcome these shortcomings in the following way. We first fit real social networks using the exponential random graph model and used the Bayesian predictive posterior distribution to generate networks with realistic topologies. We then compared the performance of the three different estimators mentioned above.

Bayesian model inversion using stochastic spectral embedding

In this paper we propose a new sampling-free approach to solve Bayesian model inversion problems that is an extension of the previously proposed spectral likelihood expansions (SLE) method. Our approach, called stochastic spectral likelihood embedding (SSLE), uses the recently presented stochastic spectral embedding (SSE) method for local spectral expansion refinement to approximate the likelihood function at the core of Bayesian inversion problems.We show that, similar to SLE, this approach results in analytical expressions for key statistics of the Bayesian posterior distribution, such as evidence, posterior moments and posterior marginals, by direct post-processing of the expansion coefficients. Because SSLE and SSE rely on the direct approximation of the likelihood function, they are in a way independent of the computational/mathematical complexity of the forward model. We further enhance the efficiency of SSLE by introducing a likelihood specific adaptive sample enrichment scheme.To showcase the performance of the proposed SSLE, we solve three problems that exhibit different kinds of complexity in the likelihood function: multimodality, high posterior concentration and high nominal dimensionality. We demonstrate how SSLE significantly improves on SLE, and present it as a promising alternative to existing inversion frameworks.

Hierarchical Bayesian approach for improving weights for solving multi-objective route optimization problem

The weighted sum method is a simple and widely used technique that scalarizes multiple conflicting objectives into a single objective function. It suffers from the problem of determining the appropriate weights corresponding to the objectives. This paper proposes a novel Hierarchical Bayesian model based on multinomial distribution and Dirichlet prior to refine the weights for solving such multi-objective route optimization problems. The model and methodologies revolve around data obtained from a small-scale pilot survey. The method aims at improving the existing methods of weight determination in the field of Intelligent Transport Systems as data driven choice of weights through appropriate probabilistic modelling ensures, on an average, much reliable results than non-probabilistic techniques. Application of this model and methodologies to simulated as well as real data sets revealed quite encouraging performances with respect to stabilizing the estimates of weights. Generation of weights using the proposed Bayesian methodology can be used to develop a bona-fide Bayesian posterior distribution for the optima, thus properly and coherently quantifying the uncertainty about the optima.

PAC-Bayes Bounds on Variational Tempered Posteriors for Markov Models.

Datasets displaying temporal dependencies abound in science and engineering applications, with Markov models representing a simplified and popular view of the temporal dependence structure. In this paper, we consider Bayesian settings that place prior distributions over the parameters of the transition kernel of a Markov model, and seek to characterize the resulting, typically intractable, posterior distributions. We present a Probably Approximately Correct (PAC)-Bayesian analysis of variational Bayes (VB) approximations to tempered Bayesian posterior distributions, bounding the model risk of the VB approximations. Tempered posteriors are known to be robust to model misspecification, and their variational approximations do not suffer the usual problems of over confident approximations. Our results tie the risk bounds to the mixing and ergodic properties of the Markov data generating model. We illustrate the PAC-Bayes bounds through a number of example Markov models, and also consider the situation where the Markov model is misspecified.

Intensity of Respiratory Cortical Arousals Is a Distinct Pathophysiologic Feature and Is Associated with Disease Severity in Obstructive Sleep Apnea Patients.

Background: We investigated whether the number, duration and intensity of respiratory arousals (RA) on C3-electroencephalographic (EEG) recordings correlate with polysomnography (PSG)-related disease severity in obstructive sleep apnea (OSA) patients. We also investigated if every patient might have an individual RA microstructure pattern, independent from OSA-severity. Methods: PSG recordings of 20 OSA patients (9 female; age 27–80 years) were analyzed retrospectively. Correlation coefficients were calculated between RA microstructure (duration, EEG-intensity) and RA number and respiratory disturbance index (RDI), oxygen desaturation index (ODI) and arousal index (AI). Intraclass correlations (ICC) for both RA duration and intensity were calculated. Sleep stage-specific and apnea- and hypopnea-specific analyses were also done. The probability distributions of duration and intensity were plotted, interpolated with a kernel which fits the distribution. A Bayesian posterior distribution analysis and pair-wise comparisons of each patient with all other 19 patients were performed. Results: Of the analyzed 2600 RA, strong positive correlations were found between average RA intensity and both RDI and AI. The number of PSG-recorded RA was strongly positively correlated with RDI. Significant correlations between average RA intensity in REM, NREM2 and NREM3 sleep stages and total ODI were identified. No sleep stage-specific correlations of arousal microstructure with age, sex, RDI or AI were identified. Although between-subjects ICC values were <0.25, within-subject ICC values were all >0.7 (all p < 0.05). While apnea-related RA duration did not differ from hypopnea-related RA duration, RA intensity was significantly higher (p = 0.00135) in hypopneas than in apneas. A clear individual pattern of arousal duration for each patient was made distinct. For arousal intensity, a Gaussian distribution was identified in most patients. The Bayesian statistics regarding the arousal microstructure showed significant differences between each pair of patients. Conclusions: Each individual patient with OSA might have an individual pattern of RA intensity and duration indicating a distinct individual pathophysiological feature. Arousal intensity was significantly higher in hypopneic than in apneic events and may be related causally to the diminished (compared to apneas) respiratory distress associated with hypopneas. RA intensity in REM, NREM2 and NREM3 strongly correlated with ODI.

Efficient multiscale imaging of subsurface resistivity with uncertainty quantification using ensemble Kalman inversion

SUMMARYElectrical resistivity tomography (ERT) is widely used to image the Earth’s subsurface and has proven to be an extremely useful tool in application to hydrological problems. Conventional smoothness-constrained inversion of ERT data is efficient and robust, and consequently very popular. However, it does not resolve well sharp interfaces of a resistivity field and tends to reduce and smooth resistivity variations. These issues can be problematic in a range of hydrological or near-surface studies, for example mapping regolith-bedrock interfaces. While fully Bayesian approaches, such as those using Markov chain Monte Carlo sampling, can address the above issues, their very high computation cost makes them impractical for many applications. Ensemble Kalman inversion (EKI) offers a computationally efficient alternative by approximating the Bayesian posterior distribution in a derivative-free manner, which means only a relatively small number of ‘black-box’ model runs are required. Although common limitations for ensemble Kalman filter-type methods apply to EKI, it is both efficient and generally captures uncertainty patterns correctly. We propose the use of a new EKI-based framework for ERT which estimates a resistivity model and its uncertainty at a modest computational cost. Our EKI framework uses a level-set parametrization of the unknown resistivity to allow efficient estimation of discontinuous resistivity fields. Instead of estimating level-set parameters directly, we introduce a second step to characterize the spatial variability of the resistivity field and infer length scale hyperparameters directly. We demonstrate these features by applying the method to a series of synthetic and field examples. We also benchmark our results by comparing them to those obtained from standard smoothness-constrained inversion. Resultant resistivity images from EKI successfully capture arbitrarily shaped interfaces between resistivity zones and the inverted resistivities are close to the true values in synthetic cases. We highlight its readiness and applicability to similar problems in geophysics.

Sequential ensemble transform for Bayesian inverse problems

We present the Sequential Ensemble Transform (SET) method, an approach for generating approximate samples from a Bayesian posterior distribution. The method explores the posterior distribution by solving a sequence of discrete optimal transport problems to produce a series of transport plans which map prior samples to posterior samples. We prove that the sequence of Dirac mixture distributions produced by the SET method converges weakly to the true posterior as the sample size approaches infinity. Furthermore, our numerical results indicate that, when compared to standard Sequential Monte Carlo (SMC) methods, the SET approach is more robust to the choice of Markov mutation kernels and requires less computational efforts to reach a similar accuracy when used to explore complex posterior distributions. Finally, we describe adaptive schemes that allow to completely automate the use of the SET method.

PNS22 Network Meta-Analysis: A Comparison of Bayesian and Frequentist Approaches to Synthesis, and Evaluation of Their Suitability for Decision-Making

The aim was to compare the suitability of Bayesian and frequentist approaches to network meta-analysis (NMA), particularly around informing clinical guidelines and to provide a narrative comparison of methods. Inferences made around NMA estimates vary depending on the methodology adopted, which may alter the interpretation of results and conclusions drawn. Previous literature has shown that whilst the direction of results may be consistent across approaches, the magnitude of estimates has the potential to differ greatly. Frequentist methodology adopted in a recently published NMA evaluating treatments for moderate-to-severe psoriasis was compared with a Bayesian approach; a method advocated by the National Institute for Health and Care Excellence Decision Support Unit. An evaluation of both methods was undertaken, comparing key NMA concepts and assessing how this might impact results, as this could have implications when embedding synthesised evidence in an economic evaluation. The interpretation of results differs between approaches; Bayesian methods allow for the utilisation of prior distributions and direct probability statements can be made around Bayesian estimates, whereas frequentist methods are based on repeated sampling and p-values to inform conclusions. NMA in a Bayesian context can incorporate external information to quantify the amount of between-study heterogeneity, particularly when there is paucity of data. Use of Bayesian posterior distributions allows uncertainty to be propagated through probabilistic sensitivity analyses, preserving correlation between parameters; whereas frequentist methods rely on summary estimates. NMA is a valuable tool, however differences in statistical approaches can present challenges around the interpretation and use of NMA results, particularly when guiding the clinical decision-making process. Transparent reporting of limitations and validation of assumptions underpinning both methodologies are essential. Unsurprisingly, given the flexibility of NMA within a Bayesian framework, it is the preferred approach by many health technology assessment bodies. A quantitative comparison is planned to assess key differences between approaches.

Fusion positioning algorithm of indoor WiFi and bluetooth based on discrete mathematical model

Aiming at the problem of limited beacon coverage and low positioning accuracy when indoor WiFi and Bluetooth are positioned separately, the fusion positioning algorithm of indoor WiFi and Bluetooth based on discrete mathematical model is studied. The position fingerprint method is used to realize WiFi indoor positioning through two stages of “off-line/training” and “on-line positioning”. The intensity of Bluetooth is obtained through Gauss distribution model. The intensity of Bluetooth signal and the relationship between the distance corresponding to Bluetooth and signal intensity are used to obtain the positions of multiple groups of points to Bluetooth nodes. And then the final positioning coordinates of users are obtained by centroid algorithm. The prior location results and distribution are obtained by multi-source prior information fusion between WiFi and Bluetooth. The optimal Bayesian posterior distribution density function is used to estimate the coordinate deviation, which is used to correct the fusion positioning results and obtain the optimal estimation of the coordinates of WiFi and Bluetooth fusion positioning. The experimental results show that the algorithm can locate indoor volunteers with a positioning accuracy of more than 98% and a positioning time of less than 5 s, which has a high positioning performance.

Do School Districts Affect NYC House Prices? Identifying Border Differences Using a Bayesian Nonparametric Approach to Geographic Regression Discontinuity Designs

What is the premium on house price for a particular school district? To estimate this in New York City we use a novel implementation of a geographic regression discontinuity design (GeoRDD) built from Gaussian processes regression (kriging) to model spatial structure. With a GeoRDD, we specifically examine price differences along borders between “treatment” and “control” school districts. GeoRDDs extend RDDs to multivariate settings; location is the forcing variable and the border between school districts constitutes the discontinuity threshold. We first obtain a Bayesian posterior distribution of the price difference function, our nominal treatment effect, along the border. We then address nuances of having a functional estimand defined on a border with potentially intricate topology, particularly when defining and estimating causal estimands of the local average treatment effect (LATE). We test for nonzero LATE with a calibrated hypothesis test with good frequentist properties, which we further validate using a placebo test. Using our methodology, we identify substantial differences in price across several borders. In one case, a border separating Brooklyn and Queens, we estimate a statistically significant 20% higher price for a house on the more desirable side. We also find that geographic features can undermine some of these comparisons. Supplementary materials for this article, including a standardized description of the materials available for reproducing the work, are available as an online supplement.

Rifampin-accompanied antibiotic regimens in the treatment of prosthetic joint infections: a frequentist and Bayesian meta-analysis of current evidence.

Prosthetic joint infections cause serious morbidity and mortality among joint arthroplasty patients. Rifampin-accompanied antibiotic regimens are recommended for gram-positive infections. This study aimed to combine current evidence supporting the rifampin supplement to an effective antibiotic in the treatment of prosthetic joint infections. We conducted a random-effects meta-analysis with frequentist and Bayesian approaches. A total of 13 studies, all observational, were included in the final analysis. The predominant bacteria in eight, two, and three studies were Staphylococcus spp., Propionibacterium spp., and Streptococcus spp., respectively. We pooled data from 568 patients in the staphylococcus subset (OR, 1.18; 95% CIs, [0.76; 1.82]; I2 = 23%) and data from 80 patients in the propionibacterium subset (REM OR, 1.61; 95% CIs [0.58; 4.47]; I2 = 0%). Both were insignificant with little heterogeneity. We pooled data from 483 patients in the streptococcus subset; the pooled estimate in this subset favored the use of rifampin supplemented regimens (1.84; [0.90; 3.76]) with moderate to high unaccounted heterogeneity (I2 = 57%). Bayesian random-effects models produced a posterior probability density indicating that future studies will not favor rifampin supplementation in Staphylococcus infections (μ, 0.074; τ, 0.570; 89% HPD, [- 0.48; 0.54]). Bayesian posterior distribution in the Streptococcus subset displayed a tendency toward rifampin supplementation. Studies had a substantial selection bias. Available evidence did not encourage rifampin-accompanied regimens for staphylococcal infections.

Confidence intervals with maximal average power

We propose a frequentist testing procedure that maintains a defined coverage and is optimal in the sense that it gives maximal power to detect deviations from a null hypothesis when the alternative to the null hypothesis is sampled from a pre-specified distribution (the prior distribution). Selecting a prior distribution allows to tune the decision rule. This leads to an increased power, if the true data generating distribution happens to be compatible with the prior. It comes at the cost of losing power, if the data generating distribution or the observed data are incompatible with the prior. We illustrate the proposed approach for a binomial experiment, which is sufficiently simple such that the decision sets can be illustrated in figures, which should facilitate an intuitive understanding. The potential beyond the simple example will be discussed: the approach is generic in that the test is defined based on the likelihood function and the prior only. It is comparatively simple to implement and efficient to execute, since it does not rely on Minimax optimization. Conceptually it is interesting to note that for constructing the testing procedure the Bayesian posterior probability distribution is used.

Fuzzy Linear regression based on approximate Bayesian computation

Fuzzy linear regression with crisp inputs and fuzzy output data constitutes an important modeling problem. Basic strategies used to solve this problem, i.e., the possibilistic method and the least squares method, together with their extensions, have some drawbacks. The possibilistic methods put emphasis on an inclusion property while the least squares methods focus on a central tendency property. Therefore, many researchers work on combining these two methods to obtain a better performance. In this paper, in contrast to most existing techniques which treat fuzzy linear regression as an optimization problem, we set the problem of constructing a fuzzy linear regression model in Bayesian statistics and propose a new fuzzy linear regression method based on approximate Bayesian computation (ABC). The method applies the likelihood-free inference algorithm ABC to generate independent samples of unknown model coefficients from Bayesian posterior distribution. This overcomes difficulty of defining likelihood function in fuzzy environment. By adjusting a prior distribution and a threshold of the ABC algorithm, the proposed approach can flexibly balance the inclusion property of the possibilistic methods and the central tendency property of the least squares methods. The convergence property of the proposed ABC algorithm is verified by a numerical example. Two measuring criteria, i.e., a distance metric and a degree of fitting index, which indicate the central tendency property and the inclusion property, respectively, are introduced to evaluate the quality of regression results. Three numerical examples are applied to show the performances of the proposed method. The numerical results are also compared with those obtained by some classical and recently proposed approaches. Additionally, a practical engineering application example is used to illustrate effectiveness of the proposed method.

Addressing uncertainty in extreme rainfall intensity for semi-arid urban regions: case study of Delhi, India

Classical approaches are used to develop rainfall intensity duration frequency curves for the estimation of design rainfall intensities corresponding to various return periods. The study modelled extreme rainfall intensities at different durations and compared the classical Gumbel and generalized extreme value (GEV) distributions in semi-arid urban region. The model and parameter uncertainties are translated to uncertainties in design storm estimates. A broader insight emerges that rainfall extremes in 1 h and 3 h are sensitive to the choice of frequency analysis (GEV in this case) and helps address anticipated intensification of extreme events for short duration at urban local scale. In comparison with Gumbel, GEV predicts higher extreme rainfall intensity corresponding to various return periods and duration (for 1-h duration the increase in extreme rainfall intensity is from 27 to 33% for return periods 10 years and higher, 3-h and 50-year return period—20%, 3-h and 100-year return period—20.6%, 24 h at similar return periods—10%). The Bayesian posterior distribution has a calibration effect on the GEV predictions and reduces the upper range of uncertainty in the GEV probability model prediction from a range of 16–31% to 10–28.4% for return period varying from 10 to 50 year for 1-h storms. In geographically similar areas these extreme intensities may be used to prepare for the rising flash flood risks.

Quantifying uncertainties in neutron- α scattering with chiral nucleon-nucleon and three-nucleon forces

Background: Modern ab initio theory combined with high-quality nucleon-nucleon (NN) and three-nucleon (3N) interactions from chiral effective field theory (EFT) can provide a predictive description of low-energy light-nuclei reactions relevant for astrophysics and fusion-energy applications. However, the high cost of computations has so far impeded a complete analysis of the uncertainty budget of such calculations. Purpose: Starting from NN potentials up to fifth order (N4LO) combined with leading-order 3N forces, we study how the order-by-order convergence of the chiral expansion and confidence intervals for the 3N contact and contact-plus-one-pion-exchange low-energy constants (cE and cD) contribute to the overall uncertainty budget of many-body calculations of neutron-He elastic scattering. Methods: We compute structure and reaction observables for three-, four- and five-nucleon systems within the ab initio frameworks of the no-core shell model an no-core shell model with continuum. Using a small set of design runs, we construct a Gaussian process model (GPM) that acts as a statistical emulator for the theory. With this, we gain insight into how uncertainties in the 3N low-energy constants propagate throughout the calculation and determine the Bayesian posterior distribution of these parameters with Markov-Chain Monte-Carlo.

Calibrate, emulate, sample

Many parameter estimation problems arising in applications can be cast in the framework of Bayesian inversion. This allows not only for an estimate of the parameters, but also for the quantification of uncertainties in the estimates. Often in such problems the parameter-to-data map is very expensive to evaluate, and computing derivatives of the map, or derivative-adjoints, may not be feasible. Additionally, in many applications only noisy evaluations of the map may be available. We propose an approach to Bayesian inversion in such settings that builds on the derivative-free optimization capabilities of ensemble Kalman inversion methods. The overarching approach is to first use ensemble Kalman sampling (EKS) to calibrate the unknown parameters to fit the data; second, to use the output of the EKS to emulate the parameter-to-data map; third, to sample from an approximate Bayesian posterior distribution in which the parameter-to-data map is replaced by its emulator. This results in a principled approach to approximate Bayesian inference that requires only a small number of evaluations of the (possibly noisy approximation of the) parameter-to-data map. It does not require derivatives of this map, but instead leverages the documented power of ensemble Kalman methods. Furthermore, the EKS has the desirable property that it evolves the parameter ensemble towards the regions in which the bulk of the parameter posterior mass is located, thereby locating them well for the emulation phase of the methodology. In essence, the EKS methodology provides a cheap solution to the design problem of where to place points in parameter space to efficiently train an emulator of the parameter-to-data map for the purposes of Bayesian inversion.

Confidence distributions and empirical Bayes posterior distributions unified as distributions of evidential support

While empirical Bayes methods thrive in the presence of the thousands of simultaneous hypothesis tests in genomics and other large-scale applications, significance tests and confidence intervals are considered more appropriate for small numbers of tested hypotheses. Indeed, for fewer hypotheses, there is more uncertainty in empirical Bayes estimates of the prior distribution. Confidence intervals have been used to propagate the uncertainty in the prior to empirical Bayes inference about a parameter, but only by combining a Bayesian posterior distribution with a confidence distribution. Combining distributions of both types has also been used to combine empirical Bayes methods and confidence intervals for estimating a parameter of interest. To clarify the foundational status of such combinations, the concept of an evidential model is proposed. In the framework of evidential models, both Bayesian posterior distributions and confidence distributions are special cases of evidential support distributions. Evidential support distributions, by quantifying the sufficiency of the data as evidence, leverage the strengths of Bayesian posterior distributions and confidence distributions for cases in which each type performs well and for cases benefiting from the combination of both. Evidential support distributions also address problems of bioequivalence, bounded parameters, and the lack of a unique confidence distribution.

Reducing estimation risk using a Bayesian posterior distribution approach: Application to stress testing mortgage loan default

We propose a new stress testing method to model coefficient uncertainty in addition to macroeconomic stress. Based on U.S. mortgage loan data, we model the probability of default at account level using discrete time hazard analysis. We employ both the frequentist and Bayesian methods in parameter estimation and default rate (DR) stress testing. By applying the Bayesian parameter posterior distribution, which includes all ranges of possible parameter estimates, obtained in the Bayesian approach to simulating the DR distribution, we reduce the estimation risk coming from employing point estimates in stress testing. Since estimation risk, a commonly neglected source of risk, is addressed in our method, we obtain more prudential forecasts of credit losses. We find that the simulated DR distribution obtained using the Bayesian approach with the parameter posterior distribution has a standard deviation 10.7 times as large as that using the frequentist approach with parameter mean estimates. Moreover, the 99% value at risk (VaR) using the Bayesian posterior distribution approach is around 6.5 times the VaR at the same probability level using the point estimate approach.

Bayesian mean–variance analysis: optimal portfolio selection under parameter uncertainty

The paper solves the problem of optimal portfolio choice when the parameters of the asset returns distribution, for example the mean vector and the covariance matrix, are unknown and have to be estimated by using historical data on asset returns. Our new approach employs the Bayesian posterior predictive distribution which is the distribution of future realizations of asset returns given the observable sample. The parameters of posterior predictive distributions are functions of the observed data values and, consequently, the solution of the optimization problem is expressed in terms of data only and does not depend on unknown quantities. By contrast, the optimization problem of the traditional approach is based on unknown quantities which are estimated in the second step, and lead to a suboptimal solution. We also derive a very useful stochastic representation of the posterior predictive distribution whose application not only gives the solution of the considered optimization problem, but also provides the posterior predictive distribution of the optimal portfolio return which can be used to construct a prediction interval. A Bayesian efficient frontier, the set of optimal portfolios obtained by employing the posterior predictive distribution, is constructed as well. Theoretically and using real data we show that the Bayesian efficient frontier outperforms the sample efficient frontier, a common estimator of the set of optimal portfolios which is known to be overoptimistic.