Jackknife approach to the estimation of mutual information

Abstract

Quantifying the dependence between two random variables is a fundamental issue in data analysis, and thus many measures have been proposed. Recent studies have focused on the renowned mutual information (MI) [Reshef DN, et al. (2011) Science 334:1518-1524]. However, "Unfortunately, reliably estimating mutual information from finite continuous data remains a significant and unresolved problem" [Kinney JB, Atwal GS (2014) Proc Natl Acad Sci USA 111:3354-3359]. In this paper, we examine the kernel estimation of MI and show that the bandwidths involved should be equalized. We consider a jackknife version of the kernel estimate with equalized bandwidth and allow the bandwidth to vary over an interval. We estimate the MI by the largest value among these kernel estimates and establish the associated theoretical underpinnings.