Bayesian Standpoint Research Articles

Information About the Moments or the Likelihood Model Parameters? A Chicken and Egg Problem

ABSTRACTThis article compares the information content of a sample for two competing Bayesian approaches. One approach follows Dennis Lindley's Bayesian standpoint, where one begins by formulating a prior for a parameter related to the problem in question and incorporates a likelihood to transition to a posterior. This contrasts with the usual Bayesian approach, where one starts with a likelihood model, formulates a prior distribution for its parameters, and derives the corresponding posterior. In both cases, the sample information content is measured using the difference between the prior and posterior entropies. We investigate this contrast in the context of learning about the moments of a variable. The maximum entropy principle is used to construct the likelihood model consistent with the given moment parameters. This likelihood model is then combined with the prior information on the parameters to derive the posterior. The model parameters are the Lagrange multipliers for the moment constraints. A prior for the moments induces a prior for the model parameters; however, the data provides differing amounts of information about them. The results obtained for several problems show that the information content using the two formulations can differ significantly. Additional information measures are derived to assess the effects of operating environments on the lifetimes of system components.

Weighted inverse gamma innovation for the structure learning of DAGs

AbstractIn cancer causal analysis, graphical models can help to identify genomic changes using the relations between proteins and phosphoproteins. In particular, directed acyclic graphs (DAGs) are effective methods to analyze intricate structures of dependence and causal connections among variables. From a Bayesian standpoint, DAG structural learning is a crucial goal in genomics studies since it enables the discovery of dependent relationships that contribute to the comprehension of variable behavior. However, we benefit from Bayesian DAG learning when dealing with small sample cases such as genomics and large networks because it promotes sparsity in the graphs, integrates prior knowledge, and crucially considers the uncertainty in the graph structure. On the other hand, modeling a proper prior distribution plays a vital role in correct posterior learning. Our work recommends using a weighted prior distribution for Gaussian DAG structure learning to improve graphical metrics. Simulation studies show that our proposal is superior to the existing one. We give the network configuration using the weighted prior distribution in a cancer data analysis.

Confirmation by analogy

This paper proposes a framework for representing in Bayesian terms the idea that analogical arguments of various degrees of strength may provide inductive support to yet untested scientific hypotheses. On this account, contextual information plays a crucial role in determining whether, and to what extent, a given similarity or dissimilarity between source and target may confirm an empirical hypothesis over a rival one. In addition to showing confirmation by analogy compatible with the adoption of a Bayesian standpoint, the proposal outlined in this paper reveals a close agreement between the fulfillment of Hesse’s (Models and analogies in science, University of Notre Dame Press, 1963) criteria for analogical arguments capable of inductive support and the attribution of confirmatory power by the lights of Bayesian confirmation theory. In this sense, the Bayesian representation not only enriches a framework, Hesse’s, of enduring relevance for understanding scientific activity, but may offer something akin to a proof of concept of it.

Line spectral estimation: Generalized bilinear modeling and hybrid inference method

• New algorithm of hybrid inference for line spectral estimation in multi-snapshot context. • Computationally efficient, only cubic complexity per iteration. • Comprehensively effective, approaching the Cramer-Rao bound. This paper studies line spectral estimation, a classical signal processing problem, under the new assumption of lower-resolution quantization and multiple snapshots. We take a Bayesian standpoint to reformulate the problem as regression in the generalized bilinear model, where the two matrices to estimate are stochastic and structurally correlated. We then propose a hybrid inference method that matches the BiG-AMP [1, 2], a high efficient bilinear regressor, with belief propagation [3], the classical sum-product message passing algorithm. To estimate the model order, we further incorporate the hybrid method into the expectation-maximization [4] framework. The algorithm obtained is then verified via extensive Monte Carlo simulations, in which the proposed algorithm comprehensively outperforms the two state-of-the-art methods, MVALSE [5] and MVALSE-EP [6], while keeping its computational complexity at a relatively low level.

Analysis of 20th century surface air temperature using linear dynamical modes.

A Bayesian Linear Dynamical Mode (LDM) decomposition method is applied to isolate robust modes of climate variability in the observed surface air temperature (SAT) field. This decomposition finds the optimal number of internal modes characterized by their own time scales, which enter the cost function through a specific choice of prior probabilities. The forced climate response, with time dependence estimated from state-of-the-art climate-model simulations, is also incorporated in the present LDM decomposition and shown to increase its optimality from a Bayesian standpoint. On top of the forced signal, the decomposition identifies five distinct LDMs of internal climate variability. The first three modes exhibit multidecadal scales, while the remaining two modes are attributable to interannual-to-decadal variability associated with El Niño-Southern oscillation; all of these modes contribute to the secular climate signal-the so-called global stadium wave-missing in the climate-model simulations. One of the multidecadal LDMs is associated with Atlantic multidecadal oscillation. The two remaining slow modes have secular time scales and patterns exhibiting regional-to-global similarities to the forced-signal pattern. These patterns have a global scale and contribute significantly to SAT variability over the Southern and Pacific Oceans. In combination with low-frequency modulation of the fast LDMs, they explain the vast majority of the variability associated with interdecadal Pacific oscillation. The global teleconnectivity of the secular climate modes and their possible crucial role in shaping the forced climate response are the two key dynamical questions brought about by the present analysis.

Visual Binary Stars with Partially Missing Data: Introducing Multiple Imputation in Astrometric Analysis

Partial measurements of relative position are a relatively common event during the observation of visual binary stars. However, these observations are typically discarded when estimating the orbit of a visual pair. In this article we present a novel framework to characterize the orbits from a Bayesian standpoint, including partial observations of relative position as an input for the estimation of orbital parameters. Our aim is to formally incorporate the information contained in those partial measurements in a systematic way into the final inference. In the statistical literature, an imputation is defined as the replacement of a missing quantity with a plausible value. To compute posterior distributions of orbital parameters with partial observations, we propose a technique based on Markov chain Monte Carlo with multiple imputation. We present the methodology and test the algorithm with both synthetic and real observations, studying the effect of incorporating partial measurements in the parameter estimation. Our results suggest that the inclusion of partial measurements into the characterization of visual binaries may lead to a reduction in the uncertainty associated to each orbital element, in terms of a decrease in dispersion measures (such as the interquartile range) of the posterior distribution of relevant orbital parameters. The extent to which the uncertainty decreases after the incorporation of new data (either complete or partial) depends on how informative those newly incorporated measurements are. Quantifying the information contained in each measurement remains an open issue.

Bayesian Regularization for Graphical Models With Unequal Shrinkage

ABSTRACTWe consider a Bayesian framework for estimating a high-dimensional sparse precision matrix, in which adaptive shrinkage and sparsity are induced by a mixture of Laplace priors. Besides discussing our formulation from the Bayesian standpoint, we investigate the MAP (maximum a posteriori) estimator from a penalized likelihood perspective that gives rise to a new nonconvex penalty approximating the ℓ0 penalty. Optimal error rates for estimation consistency in terms of various matrix norms along with selection consistency for sparse structure recovery are shown for the unique MAP estimator under mild conditions. For fast and efficient computation, an EM algorithm is proposed to compute the MAP estimator of the precision matrix and (approximate) posterior probabilities on the edges of the underlying sparse structure. Through extensive simulation studies and a real application to a call center data, we have demonstrated the fine performance of our method compared with existing alternatives. Supplementary materials for this article are available online.

A Bayesian asymmetric logistic model of factors underlying team success in top‐level basketball in Spain

This paper analyses the factors underlying the victories and defeats of the Spanish basketball teams Real Madrid and Barcelona in the national league, ACB. The following research questions were addressed: (a) Is it possible to identify the factors underlying these results? (b) Can knowledge of these factors increase the probability of winning and thus help coaches take better decisions? We analysed 80 and 79 games played in the 2013–2014 season by Real Madrid and Barcelona, respectively. Logistic regression analysis was performed to predict the probability of the team winning. The models were estimated by standard (frequentist) and Bayesian methods, taking into account the asymmetry of the data, that is, the fact that the database contained many more wins than losses. Thus, the analysis consisted of an asymmetric logistic regression. From the Bayesian standpoint, this model was considered the most appropriate, as it highlighted relevant factors that might remain undetected by standard logistic regression. The prediction quality of the models obtained was tested by application to the results produced in the following season (2014–2015). Again, asymmetric logistic regression achieved the best results. In view of the study findings, we make various practical recommendations to improve decision making in this field. In short, asymmetric logistic regression is a valuable tool that can help coaches improve their game strategies.

A Variant of AIC Based on the Bayesian Marginal Likelihood

We propose information criteria that measure the prediction risk of a predictive density based on the Bayesian marginal likelihood from a frequentist point of view. We derive criteria for selecting variables in linear regression models, assuming a prior distribution of the regression coefficients. Then, we discuss the relationship between the proposed criteria and related criteria. There are three advantages of our method. First, this is a compromise between the frequentist and Bayesian standpoints because it evaluates the frequentist’s risk of the Bayesian model. Thus, it is less influenced by a prior misspecification. Second, the criteria exhibits consistency when selecting the true model. Third, when a uniform prior is assumed for the regression coefficients, the resulting criterion is equivalent to the residual information criterion (RIC) of Shi and Tsai (J. R. Stat. Soc. Ser. B 64, 237–252 2002).

The Variability and Interpretation of Earthquake Source Mechanisms in The Geysers Geothermal Field From a Bayesian Standpoint Based on the Choice of a Noise Model

AbstractMoment tensor (MT) inversion studies of events in The Geysers geothermal field mostly focused on microseismicity and found a large number of earthquakes with significant non‐double‐couple (non‐DC) seismic radiation. Here we concentrate on the largest events in the area in recent years using a hierarchical Bayesian MT inversion. Initially, we show that the non‐DC components of the MT can be reliably retrieved using regional waveform data from a small number of stations. Subsequently, we present results for a number of events and show that accounting for noise correlations can lead to retrieval of a lower isotropic (ISO) component and significantly different focal mechanisms. We compute the Bayesian evidence to compare solutions obtained with different assumptions of the noise covariance matrix. Although a diagonal covariance matrix produces a better waveform fit, inversions that account for noise correlations via an empirically estimated noise covariance matrix account for interdependences of data errors and are preferred from a Bayesian point of view. This implies that improper treatment of data noise in waveform inversions can result in fitting the noise and misinterpreting the non‐DC components. Finally, one of the analyzed events is characterized as predominantly DC, while the others still have significant non‐DC components, probably as a result of crack opening, which is a reasonable hypothesis for The Geysers geothermal field geological setting.

Bayesian Inference and Model Assessment for Spatial Point Patterns Using Posterior Predictive Samples

Spatial point pattern data describes locations of events observed over a given domain, with the number of and locations of these events being random. Historically, data analysis for spatial point patterns has focused on rejecting complete spatial randomness and then on fitting a richer model specification. From a Bayesian standpoint, the literature is growing but primarily considers versions of Poisson processes, focusing on specifications for the intensity. However, the Bayesian literature on, e.g., clustering or inhibition processes is limited, primarily attending to model fitting. There is little attention given to full inference and scant with regard to model adequacy or model comparison. The contribution here is full Bayesian analysis, implemented through generation of posterior point patterns using composition. Model features, hence broad inference, can be explored through functions of these samples. The approach is general, applicable to any generative model for spatial point patterns. The approach is also useful in considering model criticism and model selection both in-sample and, when possible, out-of-sample. Here, we adapt or extend familiar tools. In particular, for model criticism, we consider Bayesian residuals, realized and predictive, along with empirical coverage and prior predictive checks through Monte Carlo tests. For model choice, we propose strategies using predictive mean square error, empirical coverage, and ranked probability scores. For simplicity, we illustrate these methods with standard models such as Poisson processes, log-Gaussian Cox processes, and Gibbs processes. The utility of our approach is demonstrated using a simulation study and two real datasets.

A Review and Comparison of Age–Period–Cohort Models for Cancer Incidence

Age-period-cohort models have been used to examine and forecast cancer incidence and mortality for over three decades. However, the fitting and interpretation of these models requires great care because of the well-known identifiability problem that exists; given any two of age, period, and cohort, the third is determined. In this paper, we review the identifiability problem and models that have been proposed for analysis, from both frequentist and Bayesian standpoints. A number of recent analyses that use age-period-cohort models are described and critiqued before data on cancer incidence inWashington State are analyzed with various models, including a new Bayesian approach based on an identifiable parameterization. © Institute of Mathematical Statistics, 2016.

Rationality, the Bayesian standpoint, and the Monty-Hall problem.

The Monty-Hall Problem (MHP) has been used to argue against a subjectivist view of Bayesianism in two ways. First, psychologists have used it to illustrate that people do not revise their degrees of belief in line with experimenters' application of Bayes' rule. Second, philosophers view MHP and its two-player extension (MHP2) as evidence that probabilities cannot be applied to single cases. Both arguments neglect the Bayesian standpoint, which requires that MHP2 (studied here) be described in different terms than usually applied and that the initial set of possibilities be stable (i.e., a focusing situation). This article corrects these errors and reasserts the Bayesian standpoint; namely, that the subjective probability of an event is always conditional on a belief reviser's specific current state of knowledge.

Changing the Direction of Sleep Medicine: Business can Boom, but it is Not as Usual

Changing the Direction of Sleep Medicine: Business can Boom, but it is Not as Usual

The best of both worlds: a joint modeling approach for the assessment of change across repeated measurements

The usefulness of Bayesian methods in estimating complex statistical models is undeniable. From a Bayesian standpoint, this paper aims to demonstrate the capacity of Bayesian methods and propose a comprehensive model combining both a measurement model (e.g., an item response model, IRM) and a structural model (e.g., a latent variable model, LVM). That is, through the incorporation of the probit link and Bayesian estimation, the item response model can be introduced naturally into a latent variable model. The utility of this IRM-LVM comprehensive framework is investigated with a real data example and promising results are obtained, in which the data drawn from part of the British Social Attitudes Panel Survey 1983-1986 reveal the attitude toward abortion of a representative sample of adults aged 18 or older living in Great Britain. The application of IRMs to responses gathered from repeated assessments allows us to take the characteristics of both item responses and measurement error into consideration in the analysis of individual developmental trajectories, and helps resolve some difficult modeling issues commonly encountered in developmental research, such as small sample sizes, multiple discretely scaled items, many repeated assessments, and attrition over time.

Assessing Robustness of Intrinsic Tests of Independence in Two-Way Contingency Tables

For testing nested hypotheses from a Bayesian standpoint, a desirable condition is that the prior for the alternative model concentrates mass around the smaller, or null, model. For testing independence in contingency tables, the intrinsic priors satisfy this requirement. Furthermore, the degree of concentration of the priors is controlled by a discrete parameter, t, the training sample size, which plays an important role in the resulting answer. In this article we report on the robustness of the tests of independence for small or moderate sample sizes in contingency tables with respect to intrinsic priors with different degrees of concentration around the null. We compare these tests to frequentist tests and other robust Bayes tests. For large sample sizes, robustness is achieved because the intrinsic Bayesian tests are consistent. Examples using real and simulated data are given. Supplemental materials (technical details and data sets) are available online.

Speaking Stata: I. J. Good and Quasi-Bayes Smoothing of Categorical Frequencies

I. J. Good (1916–2009) was a prolific scientist who contributed to many fields, mostly from a Bayesian standpoint. This column explains his idea of quasi-Bayes (a.k.a. pseudo-Bayes) estimation or smoothing of categorical frequencies in a contingency table, which is especially useful as a way of dealing with awkward sampling or random zeros. It shows how the method can be implemented, almost calculator-style, using a combination of Stata and Mata. Convenience commands qsbayesi and qsbayes are also introduced.

Grounding Bohmian mechanics in weak values and bayesianism

Bohmian mechanics (BM) is a popular interpretation of quantum mechanics (QM) in which particles have real positions. The velocity of a point x in configuration space is defined as the standard probability current j(x) divided by the probability density P(x). However, this ‘standard’ j is in fact only one of infinitely many that transform correctly and satisfy . In this paper, I show that a particular j is singled out if one requires that j be determined experimentally as a weak value, using a technique that would make sense to a physicist with no knowledge of QM. This ‘naively observable’ j seems the most natural way to define j operationally. Moreover, I show that this operationally defined j equals the standard j, so, assuming , one obtains the dynamics of BM. It follows that the possible Bohmian paths are naively observable from a large enough ensemble. Furthermore, this justification for the Bohmian law of motion singles out x as the hidden variable, because (for example) the analogously defined momentum current is in general incompatible with the evolution of the momentum distribution. Finally I discuss how, in this setting, the usual quantum probabilities can be motivated from a Bayesian standpoint, via the principle of indifference.

Model-based clustering for longitudinal data

A model-based clustering method is proposed for clustering individuals on the basis of measurements taken over time. Data variability is taken into account through non-linear hierarchical models leading to a mixture of hierarchical models. We study both frequentist and Bayesian estimation procedures. From a classical viewpoint, we discuss maximum likelihood estimation of this family of models through the EM algorithm. From a Bayesian standpoint, we develop appropriate Markov chain Monte Carlo (MCMC) sampling schemes for the exploration of target posterior distribution of parameters. The methods are illustrated with the identification of hormone trajectories that are likely to lead to adverse pregnancy outcomes in a group of pregnant women.

BAYESIAN IDENTIFICATION OF OUTLIERS AND CHANGE-POINTS IN MEASUREMENT ERROR MODELS

The problem of outlier and change-point identification has received considerable attention in traditional linear regression models from both, classical and Bayesian standpoints. In contrast, for the case of regression models with measurement errors, also known as error-in-variables models, the corresponding literature is scarce and largely focused on classical solutions for the normal case. The main object of this paper is to propose clustering algorithms for outlier detection and change-point identification in scale mixture of error-in-variables models. We propose an approach based on product partition models (PPMs) which allows one to study clustering for the models under consideration. This includes the change-point problem and outlier detection as special cases. The outlier identification problem is approached by adapting the algorithms developed by Quintana and Iglesias [32] for simple linear regression models. A special algorithm is developed for the change-point problem which can be applied in a more general setup. The methods are illustrated with two applications: (i) outlier identification in a problem involving the relationship between two methods for measuring serum kanamycin in blood samples from babies, and (ii) change-point identification in the relationship between the monthly dollar volume of sales on the Boston Stock Exchange and the combined monthly dollar volumes for the New York and American Stock Exchanges.