Abstract
A new, non–parametric and binless estimator for the mutual information of a d–dimensional random vector is proposed. First of all, an equation that links the mutual information to the entropy of a suitable random vector with uniformly distributed components is deduced. When d = 2 this equation reduces to the well known connection between mutual information and entropy of the copula function associated to the original random variables. Hence, the problem of estimating the mutual information of the original random vector is reduced to the estimation of the entropy of a random vector obtained through a multidimensional transformation. The estimator we propose is a two–step method: first estimate the transformation and obtain the transformed sample, then estimate its entropy. The properties of the new estimator are discussed through simulation examples and its performances are compared to those of the best estimators in the literature. The precision of the estimator converges to values of the same order of magnitude of the best estimator tested. However, the new estimator is unbiased even for larger dimensions and smaller sample sizes, while the other tested estimators show a bias in these cases.
Highlights
The measure of multivariate association, that is, of the association between groups of components of a general d-dimensional random vector X = (X1, ..., Xd ), is a topic of increasing interest in a series of application contexts
We first deduce an equation that links the mutual information between groups of components of a d-dimensional random vector to the entropy of the so called linkage function [19], that reduces to the copula function [20] in dimension d = 2
We propose here a method to estimate the mutual information (MI) of a d-dimensional random vector defined in Equation (6) by means of a random sample drawn from the corresponding d-dimensional joint distribution
Summary
We propose here a new and simple estimator for the mutual information in its general multidimensional definition To accomplish this aim, we first deduce an equation that links the mutual information between groups of components of a d-dimensional random vector to the entropy of the so called linkage function [19], that reduces to the copula function [20] in dimension d = 2. We first deduce an equation that links the mutual information between groups of components of a d-dimensional random vector to the entropy of the so called linkage function [19], that reduces to the copula function [20] in dimension d = 2 In this way the problem of estimating mutual information is reduced to the estimation of the entropy of a suitably transformed sample of the same dimensions as the original random vector.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
