Estimation of a continuous distribution on the real line by discretization methods

Abstract

For an unknown continuous distribution on the real line, we consider the approximate estimation by discretization. There are two methods for discretization. The first method is to divide the real line into several intervals before taking samples (“fixed interval method”). The second method is to divide the real line using the estimated percentiles after taking samples (“moving interval method”). In either method, we arrive at the estimation problem of a multinomial distribution. We use (symmetrized) f-divergence to measure the discrepancy between the true distribution and the estimated distribution. Our main result is the asymptotic expansion of the risk (i.e., expected divergence) up to the second-order term in the sample size. We prove theoretically that the moving interval method is asymptotically superior to the fixed interval method. We also observe how the presupposed intervals (fixed interval method) or percentiles (moving interval method) affect the asymptotic risk.