Simulation-based Tests: Basic Ideas

Abstract

The merits of tests are usually discussed by considering their behaviour under both null and alternative hypotheses. In the former situation, attention is drawn to the problem of devising a decision rule that permits the probability of rejecting a true null hypothesis to be controlled, at least approximately, for example, (1.21) of Chapter 1. In the latter situation, the probability of detecting a departure from the null hypothesis, that is, the power of the test, is emphasized. Given the decision rule, these probabilities are implied by the sampling distributions of test statistics under null and alternative hypotheses, respectively. As discussed in the previous chapter, the exact form of a sampling distribution under the null hypothesis can sometimes be derived for certain tests, under very restrictive assumptions. However, it is much more common in econometrics to admit that the assumptions required for exact knowledge are not satisfied in many cases of practical relevance and instead to use approximate sampling distributions that are asymptotically valid under relatively weak assumptions. A matter of real concern to the applied worker is then the quality of the asymptotically justified approximation to the sampling distribution of the test statistic that is being calculated.