Abstract
A classical problem in statistics is estimating the expected coverage of a sample, which has had applications in gene expression, microbial ecology, optimization, and even numismatics. Here we consider a related extension of this problem to random samples of two discrete distributions. Specifically, we estimate what we call the dissimilarity probability of a sample, i.e., the probability of a draw from one distribution not being observed in draws from another distribution. We show our estimator of dissimilarity to be a -statistic and a uniformly minimum variance unbiased estimator of dissimilarity over the largest appropriate range of . Furthermore, despite the non-Markovian nature of our estimator when applied sequentially over , we show it converges uniformly in probability to the dissimilarity parameter, and we present criteria when it is approximately normally distributed and admits a consistent jackknife estimator of its variance. As proof of concept, we analyze V35 16S rRNA data to discern between various microbial environments. Other potential applications concern any situation where dissimilarity of two discrete distributions may be of interest. For instance, in SELEX experiments, each urn could represent a random RNA pool and each draw a possible solution to a particular binding site problem over that pool. The dissimilarity of these pools is then related to the probability of finding binding site solutions in one pool that are absent in the other.
Highlights
An inescapable problem in microbial ecology is that a sample from an environment typically does not observe all species present in that environment
We study an estimator of dissimilarity probability similar to Robbins’ and Starr’s statistic for discovery probability
Our statistic in ^h(k) is the U-statistic associated with the kernel
Summary
An inescapable problem in microbial ecology is that a sample from an environment typically does not observe all species present in that environment. In [14], this problem has 1been recently linked to the concepts of coverage probability (i.e. the probability that a member from the environment is represented in the sample) and the closely related discovery or unobserved probability (i.e. the probability that a previously unobserved species is seen with another random observation from that environment). Following Robbins [6] and in more generality Starr [7], an unbiased estimator of the expected discovery probability of a sample of size n is r{1 Xr k{1 nzr. :N (k,nzr), ð1Þ k~1 k where N(k,nzr) is the number of species observed exactly k-times in a sample with replacement of size (nzr).
Sign-in/Register to view highlights & summary 
Published Version (
Free)
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call