A Warning on the Use of Chi-Squared Statistics with Frequency Tables with Small Expected Cell Counts

Abstract

Abstract When applied to frequency tables with small expected cell counts, Pearson chi-squared test statistics may be asymptotically inconsistent even in cases in which a satisfactory chi-squared approximation exists for the distribution under the null hypothesis. This problem is particularly important in cases in which the number of cells is large and the expected cell counts are quite variable. To illustrate this bias of the chi-squared test, this article considers the Pearson chi-squared test of the hypothesis that the cell probabilities for a multinomial frequency table have specified values. In this case, the expected value and variance of the Pearson chi-square may be evaluated under both the null and alternative hypotheses. When the number of cells is large, normal approximations and discrete Edgeworth expansions may also be used to assess the size and power of the Pearson chi-squared test. These analyses show that unless all cell probabilities are equal, it is possible to select a significance lev...