A Regression Perspective on Generalized Distance Covariance and the Hilbert–Schmidt Independence Criterion

Abstract

In a seminal paper, Sejdinovic et al. (Ann. Statist. 41 (2013) 2263–2291) showed the equivalence of the Hilbert–Schmidt Independence Criterion (HSIC) and a generalization of distance covariance. In this paper, the two notions of dependence are unified with a third prominent concept for independence testing, the “global test” introduced in (J. R. Stat. Soc. Ser. B. Stat. Methodol. 68 (2006) 477–493). The new viewpoint provides novel insights into all three test traditions, as well as a unified overall view of the way all three tests contrast with classical association tests. As our main result, a regression perspective on HSIC and generalized distance covariance is obtained, allowing such tests to be used with nuisance covariates or for survival data. Several more examples of cross-fertilization of the three traditions are provided, involving theoretical results and novel methodology. To illustrate the difference between classical statistical tests and the unified HSIC/distance covariance/global tests we investigate the case of association between two categorical variables in depth.