Abstract
The prior probabilities of true outcomes for scientific replication have to be uniform by definition. This is because for replication, a study’s observations are regarded as samples taken from the set of possible outcomes of an ideally large continuation of that study. (The sampling is not done directly from some source population.) Therefore, each possible outcome is based on the same ideally large number of observations so that all possible outcomes for that study have the same prior probability. The calculation methods were demonstrated on a spreadsheet with simulated data on the distribution of people with an imaginary genetic marker. Binomial distributions are used to illustrate the concepts to avoid the effects of potentially misleading assumptions. Uniform prior probabilities allow a frequentist posterior probability distribution of a study result’s replication to be calculated conditional solely on the study’s observations. However, they can be combined with prior data or Bayesian prior distributions. If the probability distributions are symmetrical then the frequentist posterior probability of a true result that is equal to or more extreme than a null hypothesis will be the same as the one-sided P-value. This is an idealistic probability of replication within a specified range based on an assumption of perfect study method reproducibility. It can be used to estimate a realistic probability of replication by taking into account the probability of non-reproducible methods or subjects. A probability of replication will be lower if the subsequent outcome is a narrower range corresponding to a specified statistical significance, this being a more severe test. The frequentist posterior probability of replication may be easier than the P-value for non-statisticians to understand and to interpret.
Highlights
There is currently a crisis of confidence in the results of research in the medical, biological and social sciences because a higher than expected proportion of results are failing to be replicated [1]
The natural tendency of a scientist is to wish to estimate the probability that the outcome of a perfect version of a study based on an unlimited number of observations will fall within a specified range that includes the original study result, replicating it
The probability that the true outcome lies within any range is calculated by adding all the discrete probabilities within that range. (In the case of continuous distributions, this is achieved by normalising the likelihood probability densities and integrating them.) The range can have an upper and lower bound (e.g. 40% to 60%) in an analogous way to a credibility or confidence interval
Summary
There is currently a crisis of confidence in the results of research in the medical, biological and social sciences because a higher than expected proportion of results are failing to be replicated [1]. Replacing one-sided P-values with frequentist posterior probabilities. Many scientists and their students find statistical principles difficult. Perhaps this is due to a missing link that causes controversy amongst statisticians about how probability theory should be used when interpreting scientific data. The natural tendency of a scientist is to wish to estimate the probability that the outcome of a perfect version of a study based on an unlimited number of observations will fall within a specified range that includes the original study result, replicating it. It is widely assumed that this cannot be done based on a study’s data alone because the prior probabilities of such outcomes are unknown (e.g. in populations of people) and can only be guessed at. It is well recognised that if the prior probabilities of possible true values during random sampling are assumed to be uniform, according to Bayes’ rule, the probability of the null hypothesis or something more extreme will equal the one-sided P-value [3, 4, 5, 6]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
