A Novel Non-parametric Two-Sample Test on Imprecise Observations

Abstract

In kernel non-parametric two-sample test, we aim to determine whether two sets of precise observations (i.e., samples) are from the same distribution based on a selected kernel. However, in real world, precise observations may be unavailable. For example, readings on an analogue measurement equipment are not precise numbers but intervals since there is only a finite number of decimals available. Hence, we consider a new and more realistic problem setting—two-sample test on imprecise observations. We show that the test power of existing kernel two- sample tests will drop significantly if they do not take care of the vagueness of the imprecise observations, and to this end, we propose a fuzzy-based maximum mean discrepancy (F-MMD), a powerful two-sample test on imprecise observations. F-MMD is based on a novel fuzzy-based kernel function that can measure the discrepancy between two imprecise observations. This novel kernel function takes care of the vagueness of the imprecise observations and its parameters are optimized to maximize the approximate test power of F-MMD. Experiments demonstrate that F-MMD significantly outperforms competitive two-sample test methods when facing imprecise observations.