A Kernel Measure of Dissimilarity between M Distributions

Abstract

Given M ≥ 2 distributions defined on a general measurable space, we introduce a nonparametric (kernel) measure of multi-sample dissimilarity (KMD) — a parameter that quantifies the difference between the M distributions. The population KMD, which takes values between 0 and 1, is 0 if and only if all the M distributions are the same, and 1 if and only if all the distributions are mutually singular. Moreover, KMD possesses many properties commonly associated with f-divergences such as the data processing inequality and invariance under bijective transformations. The sample estimate of KMD, based on independent observations from the M distributions, can be computed in near linear time (up to logarithmic factors) using k-nearest neighbor graphs (for k ≥ 1 fixed). We develop an easily implementable test for the equality of M distributions based on the sample KMD that is consistent against all alternatives where at least two distributions are not equal. We prove central limit theorems for the sample KMD, and provide a complete characterization of the asymptotic power of the test, as well as its detection threshold. The usefulness of our measure is demonstrated via real and synthetic data examples; our method is also implemented in an R package.