A new nonparametric method for variance estimation and confidence interval construction for Spearman's rank correlation

Abstract

Spearman's rank correlation, ρ s, has become one of the most widely used nonparametric statistical techniques. However, explicit formulas for the finite sample variance of its point estimate, ρ ̂ s , are generally not available, except under special conditions, and the estimation of this variance from observed data remains a challenging statistical problem. In this paper, we show that ρ ̂ s can be calculated from a two-way contingency table with categories defined by the bivariate ranks. We note that this table has the “empirical bivariate quantile-partitioned” (EBQP) distribution ( Borkowf et al., 1997), and hence ρ ̂ s belongs to the class of statistics with distributions derived from the EBQP distribution. The study of ρ ̂ s provides an opportunity to extend large sample EBQP methods to handle the special challenges posed by statistics calculated from EBQP tables defined by bivariate ranks. We present extensive simulations to study the estimation of the sample variance of ρ ̂ s and the coverage of confidence intervals for this measure. We compare these results for the EBQP method with those for the bootstrap and jackknife algorithms. We illustrate the use of these nonparametric methods on two data sets, Spearman's original data set and an example from nutritional epidemiology. These results demonstrate that standard EBQP methods can be successfully adapted for the estimation of the sample variance of ρ ̂ s . They also suggest that EBQP methods should be used to estimate the sample variances of other nonparametric statistics calculated from bivariate ranks, such as Kendall's tau.