Infinite Discrete Sources

Abstract

Agent 00111 was happiest taking cases that had only a finite number of outcomes. These were well-behaved in terms of pricing and computation. Even if the finite number of outcomes were extremely large, the Shannon-McMillan Theorem ( Theorem 1.2a.) often made the costing tractable. Sometimes, however, there were not a finite number of possibilities but two other possibilities. The number of outcomes could be countably or uncountably infinite. In Chapter 4, we consider infinite discrete distributions. For example, Agent 00111 may be asked to estimate how many days he would spend researching the true state of a country’s secret committees and to justify the time in terms of the uncertainty eliminated. In such cases, Agent 00111’s department could theoretically spend from here to eternity on the project, since the outcomes (in days) are countably infinite. In practice, this did not bother Agent 00111 too much—an obvious point of diminishing returns and increasing boredom caused him to quit after a period of time. However, as his scientists pointed out to him, just suppose that each day he were on a project he obtained more information than the last; how tempting it would be to stay.