Prior-data Conflict Research Articles

Bias away from the null due to miscounted outcomes? A case study on the TORCH trial

Count outcomes occur in virtually all disciplines, such as medicine, epidemiology or biology, but they often contain error. Recently, it has been shown that self-reported numbers of exacerbations of Chronic Obstructive Pulmonary Disease patients can be considerably miscounted. Motivated by this result, we reanalysed data from the Towards a Revolution in Chronic Obstructive Pulmonary Disease Health trial, a large randomized controlled trial with the self-reported number of exacerbations of Chronic Obstructive Pulmonary Disease patients as outcome. To adjust for miscounting error in the response of Poisson and (zero-inflated) negative binomial models, we introduce novel, general methodology. The key idea is to formulate a zero-inflated negative binomial model to capture the error mechanism. This parametric approach automatically circumvents drawbacks of previously suggested methodology that treats miscounted outcomes in the misclassification framework. Prior information for the response error model parameters was elicited from validation data of an external study and adaptively weighted to account for potential prior-data conflict. The results of the Bayesian hierarchical modelling approach indicated that the treatment effect has been overestimated in the original study. However, closer inspection revealed that this unexpected result was an artefact of an unaccounted time dependency of the treatment effect.

Addressing prior-data conflict with empirical meta-analytic-predictive priors in clinical studies with historical information

ABSTRACTA common question in clinical studies is how to use historical data from earlier studies, leveraging relevant information into the design and analysis of a new study. Bayesian approaches are particularly well-suited to this task, with their natural ability to borrow strength across data sources. In this paper, we propose an eMAP approach for incorporating historical data into the analysis of clinical studies, and we discuss an application of this method to the analysis of observational safety studies for a class of products for patients with hemophilia A. The eMAP prior approach is flexible and robust to prior-data conflict. We conducted simulations to compare the frequentist operating characteristics of three approaches under different prior-data conflict assumptions and sample size scenarios.

Adaptive prior weighting in generalized regression.

The prior distribution is a key ingredient in Bayesian inference. Prior information on regression coefficients may come from different sources and may or may not be in conflict with the observed data. Various methods have been proposed to quantify a potential prior-data conflict, such as Box's p-value. However, there are no clear recommendations how to react to possible prior-data conflict in generalized regression models. To address this deficiency, we propose to adaptively weight a prespecified multivariate normal prior distribution on the regression coefficients. To this end, we relate empirical Bayes estimates of prior weight to Box's p-value and propose alternative fully Bayesian approaches. Prior weighting can be done for the joint prior distribution of the regression coefficients or-under prior independence-separately for prespecified blocks of regression coefficients. We outline how the proposed methodology can be implemented using integrated nested Laplace approximations (INLA) and illustrate the applicability with a Bayesian logistic regression model for data from a cross-sectional study. We also provide a simulation study that shows excellent performance of our approach in the case of prior misspecification in terms of root mean squared error and coverage. Supplementary Materials give details on software implementation and code and another application to binary longitudinal data from a randomized clinical trial using a Bayesian generalized linear mixed model.

Strategies on Using Prior Information When Assessing Adverse Events

Assessing safety in drug development naturally incorporates an accumulation of knowledge as we progress from one study to another in a clinical development program. Ideally, it is the early clinical trial data that give us the greatest opportunity to leverage relevant historical information into the design and analysis of later phase clinical trials. While Bayesian methods would appear to provide an ideal framework for assessing safety in this context, concerns regarding the formulation and prespecification of a prior have limited their use in practice. More specifically, when information from previous studies is used to form a prior, an implicit assumption of exchangeability is made. However, the possibility of nonexchangeability, which could lead to conflict between prior and the data, cannot be ruled out. Based on these challenges, in this article we outline a number of strategies based on using simple Bayesian methods to assess safety concerns related to a prespecified adverse event. Three approaches to forming a prior distribution will be examined: (i) a single informative conjugate prior; (ii) a meta-analytic-predictive prior (MAP), which comprises of a mixture of conjugate priors; and (iii) a robust mixture prior, involving a combination of either the single conjugate prior or MAP with a noninformative prior. In the final case, when prior-data conflict arises between the historical informative prior and the data collected from the concurrent study, the addition of a noninformative prior serves as an automatic corrective feature. These methods will be illustrated with a motivating example involving the development of a new drug/delivery device for the treatment of agitation in schizophrenia. In addition, a simulation study will examine the performance of each approach to prior specification.

Measuring statistical evidence using relative belief

A fundamental concern of a theory of statistical inference is how one should measure statistical evidence. Certainly the words “statistical evidence,” or perhaps just “evidence,” are much used in statistical contexts. It is fair to say, however, that the precise characterization of this concept is somewhat elusive. Our goal here is to provide a definition of how to measure statistical evidence for any particular statistical problem. Since evidence is what causes beliefs to change, it is proposed to measure evidence by the amount beliefs change from a priori to a posteriori. As such, our definition involves prior beliefs and this raises issues of subjectivity versus objectivity in statistical analyses. This is dealt with through a principle requiring the falsifiability of any ingredients to a statistical analysis. These concerns lead to checking for prior-data conflict and measuring the a priori bias in a prior.

Addressing potential prior-data conflict when using informative priors in proof-of-concept studies.

Bayesian methods are increasingly used in proof-of-concept studies. An important benefit of these methods is the potential to use informative priors, thereby reducing sample size. This is particularly relevant for treatment arms where there is a substantial amount of historical information such as placebo and active comparators. One issue with using an informative prior is the possibility of a mismatch between the informative prior and the observed data, referred to as prior-data conflict. We focus on two methods for dealing with this: a testing approach and a mixture prior approach. The testing approach assesses prior-data conflict by comparing the observed data to the prior predictive distribution and resorting to a non-informative prior if prior-data conflict is declared. The mixture prior approach uses a prior with a precise and diffuse component. We assess these approaches for the normal case via simulation and show they have some attractive features as compared with the standard one-component informative prior. For example, when the discrepancy between the prior and the data is sufficiently marked, and intuitively, one feels less certain about the results, both the testing and mixture approaches typically yield wider posterior-credible intervals than when there is no discrepancy. In contrast, when there is no discrepancy, the results of these approaches are typically similar to the standard approach. Whilst for any specific study, the operating characteristics of any selected approach should be assessed and agreed at the design stage; we believe these two approaches are each worthy of consideration.

Bayesian inference in two‐arm trials using relative belief ratios

Various methodologies proposed for some inference problems associated with two-arm trails are known to suffer from difficulties, as documented in Senn (2001). We propose an alternative Bayesian approach to these problems that deals with these difficulties through providing an explicit measure of statistical evidence and the strength of this evidence. Bayesian methods are often criticized for their intrinsic subjectivity. We show how these concerns can be dealt with through assessing the bias induced by a prior model checking and checking for prior-data conflict.

Inference after checking multiple Bayesian models for data conflict and applications to mitigating the influence of rejected priors

Two major approaches have developed within Bayesian statistics to address uncertainty in the prior distribution and in the rest of the model. First, methods of model checking, including those assessing prior-data conflict, determine whether the posterior resulting from the model is adequate for purposes of inference and estimation or other decision-making. A potential drawback of this approach is that it provides little guidance for inference in the event that the model is found to be inadequate, that is, in conflict with the data. Second, the robust Bayes approach determines the sensitivity of inferences and decisions to the prior distribution and other model assumptions. This approach includes rules for making decisions on the basis of a set of posterior distributions corresponding to the set of reasonable model assumptions. Drawbacks of the second approach include the inability to criticize the set of models and the lack of guidance for specifying such a set. Those two approaches to model uncertainty are combined into a two-stage procedure in order to overcome each of their limitations. The first stage checks each model within a large class of models to assess which models are in conflict with the data and which are adequate for purposes of data analysis. The resulting set of adequate models is then used in the second stage either for summarizing a combined posterior such as a maximum-entropy posterior or for inference according to decision rules of the robust Bayes approach and of imprecise probability more generally. This proposed framework is illustrated by the application of a class of hierarchical models to a simple data set.

Robust meta-analytic-predictive priors in clinical trials with historical control information.

Historical information is always relevant for clinical trial design. Additionally, if incorporated in the analysis of a new trial, historical data allow to reduce the number of subjects. This decreases costs and trial duration, facilitates recruitment, and may be more ethical. Yet, under prior-data conflict, a too optimistic use of historical data may be inappropriate. We address this challenge by deriving a Bayesian meta-analytic-predictive prior from historical data, which is then combined with the new data. This prospective approach is equivalent to a meta-analytic-combined analysis of historical and new data if parameters are exchangeable across trials. The prospective Bayesian version requires a good approximation of the meta-analytic-predictive prior, which is not available analytically. We propose two- or three-component mixtures of standard priors, which allow for good approximations and, for the one-parameter exponential family, straightforward posterior calculations. Moreover, since one of the mixture components is usually vague, mixture priors will often be heavy-tailed and therefore robust. Further robustness and a more rapid reaction to prior-data conflicts can be achieved by adding an extra weakly-informative mixture component. Use of historical prior information is particularly attractive for adaptive trials, as the randomization ratio can then be changed in case of prior-data conflict. Both frequentist operating characteristics and posterior summaries for various data scenarios show that these designs have desirable properties. We illustrate the methodology for a phase II proof-of-concept trial with historical controls from four studies. Robust meta-analytic-predictive priors alleviate prior-data conflicts ' they should encourage better and more frequent use of historical data in clinical trials.

Consistency of Bayesian Estimates for the Sum of Squared Normal Means with a Normal Prior

We consider the problem of estimating the sum of squared means when the data (x1,...,xn) are independent values with xi ∼ N(θi, 1) and θ1, θ2... are a priori i.i.d. N(0, σ2) with σ2 known. This example has posed difficulties for many approaches to inference. We examine the consistency properties of several estimators derived from Bayesian considerations. We prove that a particular Bayesian estimate (LRSE) is consistent in a wider set of circumstances than other Bayesian estimates like the posterior mean and mode. We show that the LRSE is either equal to the positive part of the UMVUE or differs from it with a relative error no greater than 2/n. We also prove a consistency result for interval estimation and discuss checking for prior-data conflict. While it can be argued that the choice of the N(0,σ2) prior is inappropriate when σ2 is chosen large to reflect noninformativity, this argument is not applicable when σ2 is chosen to reflect knowledge about the unknowns. As such it is important to show that there are consistent Bayesian estimation procedures using this prior.

Weak Informativity and the Information in One Prior Relative to Another

A question of some interest is how to characterize the amount of information that a prior puts into a statistical analysis. Rather than a general characterization, we provide an approach to characterizing the amount of information a prior puts into an analysis, when compared to another base prior. The base prior is considered to be the prior that best reflects the current available information. Our purpose then is to characterize priors that can be used as conservative inputs to an analysis relative to the base prior. The characterization that we provide is in terms of a priori measures of prior-data conflict.

A limit result for the prior predictive applied to checking for prior-data conflict

We consider checking for prior-data conflict in a Bayesian analysis via a tail probability based on the prior predictive distribution. We establish the appropriateness of this measure in the sense that the limiting value of the tail probability measures the extent to which the true value of the parameter is a surprising value from the prior.

Imprecision and Prior-Data Conflict in Generalized Bayesian Inference

A great advantage of imprecise probability models over models based on precise, traditional probabilities is the potential to reflect the amount of knowledge they stand for. Consequently, imprecise probability models promise to offer a vivid tool for handling situations of prior-data conflict in (generalized) Bayesian inference. In this paper we consider a general class of recently studied imprecise probability models, including the Imprecise Dirichlet Model under prior information, and more generally the framework of Quaeghebeur and de Cooman for imprecise inference in canonical exponential families. We demonstrate that such models, in their originally proposed form, prove to be insensitive to the extent of prior-data conflict. We propose an extension reestablishing the natural relationship between knowledge and imprecision: the higher the discrepancy between the observed sample and what was expected from prior knowledge, the higher the imprecision in the posterior, producing cautious inferences if, and only if, caution is needed. Our approach is illustrated by some examples and simulation results.

Diagnostics of prior-data agreement in applied Bayesian analysis

This article focused on the definition and the study of a binary Bayesian criterion which measures a statistical agreement between a subjective prior and data information. The setting of this work is concrete Bayesian studies. It is an alternative and a complementary tool to the method recently proposed by Evans and Moshonov, [M. Evans and H. Moshonov, Checking for Prior-data conflict, Bayesian Anal. 1 (2006), pp. 893–914]. Both methods try to help the work of the Bayesian analyst, from preliminary to the posterior computation. Our criterion is defined as a ratio of Kullback–Leibler divergences; two of its main features are to make easy the check of a hierarchical prior and be used as a default calibration tool to obtain flat but proper priors in applications. Discrete and continuous distributions exemplify the approach and an industrial case study in reliability, involving the Weibull distribution, is highlighted.

Comment: Bayesian Checking of the Second Levels of Hierarchical Models

We discuss the methods of Evans and Moshonov [Bayesian Analysis 1 (2006) 893–914, Bayesian Statistics and Its Applications (2007) 145–159] concerning checking for prior-data conflict and their relevance to the method proposed in this paper.

Checking for prior-data conflict

Inference proceeds from ingredients chosen by the analyst and data. To validate any inferences drawn it is essential that the inputs chosen be deemed appropriate for the data. In the Bayesian context these inputs consist of both the sampling model and the prior. There are thus two possibilities for failure: the data may not have arisen from the sampling model, or the prior may place most of its mass on parameter values that are not feasible in light of the data (referred to here as prior-data conflict). Failure of the sampling model can only be fixed by modifying the model, while prior-data conflict can be overcome if sufficient data is available. We examine how to assess whether or not a prior-data conflict exists, and how to assess when its effects can be ignored for inferences. The concept of prior-data conflict is seen to lead to a partial characterization of what is meant by a noninformative prior or a noninformative sequence of priors.

Quasi-Bayesian Analysis Using Imprecise Probability Assessments And The Generalized Bayes’ Rule

The generalized Bayes’ rule (GBR) can be used to conduct ‘quasi-Bayesian’ analyses when prior beliefs are represented by imprecise probability models. We describe a procedure for deriving coherent imprecise probability models when the event space consists of a finite set of mutually exclusive and exhaustive events. The procedure is based on Walley’s theory of upper and lower prevision and employs simple linear programming models. We then describe how these models can be updated using Cozman’s linear programming formulation of the GBR. Examples are provided to demonstrate how the GBR can be applied in practice. These examples also illustrate the effects of prior imprecision and prior-data conflict on the precision of the posterior probability distribution.

Statistical Reasoning with Imprecise Probabilities.

Part 1 Reasoning and behaviour: interpretations of probability beliefs and behaviour inference and decision reasoning and rationality assessment strategies survey of related work. Part 2 Coherent previsions: possibility probability currency upper and lower previsions avoiding sure loss coherence basic properties of coherent previsions coherent probabilities linear previsions and additive probabilities examples of coherent previsions interpretations of prevision and probability objections to behavioural theories of probability. Part 3 Extensions, envelopes and decisions: natural extension extension from a field lower envelopes of linear previsions linear extension invariant linear previsions compactness and extreme points of M(P) desirability and preference equivalent models for beliefs decision making. Part 4 Assessment and elicitation: a general elicitation procedure finitely-generated models and simplex representations steps in assessment process classificatory probability comparative probability other types of assessment. Part 5 The importance of imprecision: uncertainty, indeterminacy and imprecision sources from imprecision information from Bernoulli trials prior-data conflict Bayesian noninformative prioirs indecision axioms of precision practical reasons for precision Bayesian sensitivity analysis second-order probabilities fuzzy sets maximum entropy the Dempster-Shafer theory of belief functions. Part 6 Conditional previsions: updated and contingent previsions separate coherence coherence with unconditional previsions the generalized Bayes rule coherence axioms examples of conditional previsions extension of conditional and marginal previsions conglomerability countable additivity conditioning on events of probability zero updating beliefs. Part 7 Coherent statistical models: general concepts of coherence sampling models coherence of sampling model and posterior previsions inferences from improper priors confidence intervals and relevant subsets proper prior previsions standard Bayesian inference inferences from imprecise priors joint prior previsions. Part 8 Statistical reasoning: a general theroy of natural extension extension to prior previsions extension to predictive previsions extension to posterior previsions posteriors for imprecise sampling models the likelihood principle. Part 9 Structural judgements: independent events independent experiments constructing joint previsions from independent marginals permutability exchangeability Robust Bernoulli models structural judgements. Appendices: verifying coherence N-coherence win and place betting on horses topological structure and L and P separating hyperplane theorems desirability upper and lower variances operational measuremnet procedures the World Cup football experiment regular extension W-coherence.