Maximum likelihood estimation of magnitude-squared multiple and ordinary coherence

Abstract

The probability density function (pdf) of estimated magnitude-squared ordinary or multiple coherence has been known for some time, and considerable use has been made of it in the ordinary coherence case in producing unbiased estimates. Apparently only recently has the pdf been used—and only for ordinary coherence—to formulate a maximum likelihood estimation scheme. This paper improves and corrects this scheme, and extends it to the multiple coherence case. Computational procedures are described. It is shown that the first and second derivatives of the probability density function can be written very simply in terms of the probability density function itself. As a result the iterative scheme for maximum likelihood estimation can be expressed in much simpler terms than done heretofore. A way of treating small maximum likelihood ordinary coherence estimates stated in the recent study is shown to be false. Illustrative plots are given showing the relationship between the standard multiple and ordinary coherence estimates, and maximum likelihood counterparts. An expression for the mean of the standard multiple coherence estimate is given in terms of generalized hypergeometric series, and is shown to reduce to the correct form for ordinary coherence. A simple recursive formula is given for computing the mean. An interpretation of the maximum likelihood estimate of multiple coherence derived from a given value of the standard estimate is that it is the value of the multiple coherence which makes the standard estimate into approximately the mean of the probability density function.