On the singularity of the covariance matrix for estimates of multinomial proportions

Abstract

It is well known that the covariance matrix for the multinomial distribution is singular and, therefore, does not have a unique inverse. If, however, any row and corresponding column are removed, the reduced matrix is nonsingular and the unique inverse has a closed form. We elucidate some of the properties of the multinomial covariance matrix and its reduced forms. We state and prove a theorem that gives insight into the singularity and its removal. Based on these results, we establish that the covariance matrix for the multinomial distribution is positive semidefinite and that the reduced matrix is positive definite. In addition, we show that the determinant of the reduced matrix is invariant to the particular row and column that are removed. Goodness-of-fit statistics, including Pearson's chi-square, and justification of the degrees of freedom follow from the multivariate central limit theorem once the singularity is removed.