Estimating life-time distribution by observing population continuously

Abstract

Often in a real world system with a fairly large population, members are not individually traceable for various reasons. As a result, the relationship between a member’s behavior and the system’s behavior is quite hard to understand. In this paper, we focus on a fundamental problem in such a system: the relationship between its population size and the life-time distribution for its members. We answer two questions: (A)If the life-time distribution is known and the times when members join are observable, how do we best estimate the population size?(B)If the population size can be observed accurately, how do we estimate the unknown life-time distribution for members? In the paper we focus on (B), using the results of (A) as a basis.We model the system as a G/GI/∞ queue with incomplete information, where jobs, once entering the queue, are no longer tracked. With this model, the population size is the number of jobs in the queue and the life times of members are the service times of the jobs. The problem (A) is to estimate the number of jobs in the queue, with known arrival times and a known service-time distribution. We show that, in terms of mean square error, the best deterministic estimator for the (stochastic) number of jobs in the system can be constructed using the survival function of the service-time distribution. The problem (B) is to estimate the unknown service-time distribution in a G/GI/∞ queue where the number of jobs are observable. We demonstrate that the service-time distribution can be inferred indirectly from continuous observations of the number of jobs in the queue, and then propose a few easy-to-implement algorithms. Using only a limited amount of memory, these on-line, streaming algorithms continuously refine their results which, over time, converge to the true service-time distribution.