Estimating the probability mass of unobserved support in random sampling

Abstract

The problem of estimating the probability mass of the support of a distribution not observed in random sampling is considered in the case where the distribution is discrete. An example of a situation in which the problem arises is that of species sampling: suppose that one wishes to determine the species of fish native to a body of water and that, after repeated sampling, one identifies a certain number of species. The problem is to estimate the proportion of the fish population belonging to the unobserved species. Since it is a rare event, ideas from large deviation theory play a role in answering the question. The result depends on the underlying distribution, which is unknown in general. Methods similar to nonparametric bootstrapping are therefore used to prove a limit theorem and obtain a confidence interval for the rate function.