Abstract
Several types of entropy estimators exist in the information theory literature. Most of these estimators explicitly involve estimating the density of the available data samples before computing the entropy. However, the entropy-estimator using sample spacing avoids this intermediate step and computes the entropy directly using the order-statistics. In this paper, we extend our horizon beyond Shannon's definition of entropy and analyze the entropy estimation performance at higher orders of alpha, using Renyi's generalized entropy estimator. We show that the estimators for higher orders of alpha better approximate the true entropy for an exponential family of distributions. Practical application of this estimator is demonstrated by computing mutual information between functionally coupled systems. During the estimation process, the joint distributions are decomposed into sum of their marginals by using linear ICA
Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
