Estimation of Information Measures for Power-Function Distribution in Presence of Outliers and Their Applications

Abstract

Entropy measurement plays an important role in the field of information theory. Furthermore, the estimation of entropy is an important issue in statistics and machine learning. This study estimated the Rényi and q-entropies of a power-function distribution in the presence of s outliers using classical and Bayesian procedures. In the classical method, the maximum likelihood estimators of the entropies were obtained and their performance was assessed through a numerical study. In the Bayesian method, the Bayesian estimators of the entropies under uniform and gamma priors were acquired based on different loss functions. The Bayesian estimators were computed empirically using a Monte Carlo simulation based on the Gibbs sampling algorithm. The simulated datasets were analyzed to investigate the accuracy of the estimates. The study results showed that the precision of the maximum likelihood and Bayesian estimates of both entropies improved with increasing the sample size and the number of outliers. The absolute biases and the mean squared errors of the estimates in the presence of outliers exceeded those of the corresponding estimates in the homogenous case (no-outliers). Furthermore, the Bayesian estimates of the Rényi and q-entropies under the squared error loss function were preferable to the other Bayesian estimates in a majority of the cases. Finally, analysis results of real data examples were consistent with those of the simulated data.