An identity for the Fisher information and Mahalanobis distance

Abstract

Consider a mixture problem consisting of k classes. Suppose we observe an s-dimensional random vector X whose distribution is specified by the relations P ( X ∈ A | Y = i ) = P i ( A ) , where Y is an unobserved class identifier defined on { 1 , … , k } , having distribution P ( Y = i ) = p i . Assuming the distributions P i having a common covariance matrix, elegant identities are presented that connect the matrix of Fisher information in Y on the parameters p 1 , … , p k , the matrix of linear information in X, and the Mahalanobis distances between the pairs of P's. Since the parameters are not free, the information matrices are singular and the technique of generalized inverses is used. A matrix extension of the Mahalanobis distance and its invariant forms are introduced that are of interest in their own right. In terms of parameter estimation, the results provide an independent of the parameter upper bound for the loss of accuracy by esimating p 1 , … , p k from a sample of X ′ s, as compared with the ideal estimator based on a random sample of Y ′ s.