Statistics of Leakage-Influenced Squared Coherence Estimated by Bartlett's and Welch's Procedures

Abstract

Numerical simulations are applied to investigate the statistics of squared coherence. Stochastic variations of the pa- rameter are generated from random harmonic processes in- volved in a noise-contaminated single-input-single-output lin- ear system. Expressions are presented for bias, variance, and probability density functions of squared-coherence estimates, as functions of true squared coherence, degrees of freedom, and leakage parameter. It is shown both theoretically, and by simulations, that leakage effects give rise to significant bias, whereas the variance of the estimates does not seem to be sys- tematically affected. The influence of various spectral window selections is also included. The above-mentioned bias feature occurs especially at large numbers of degrees of freedom and squared-coherence values close to one, and may dominate the conventional bias resulting from a finite number of statistical realizations. The paper derives analytical expressions quanti- tatively, showing the effect of leakage on squared coherence. A method to minimize the power leakage, by tracking the leakage parameter from both random and deterministic sinusoidal pro- cesses, is also proposed. Eventually, it is shown on a semiem- pirical basis that leakage only influences the squared-coherence spectrum marginally for a selected narrow-banded bivariate autoregressive process of second order.