Finite Sample Space Research Articles

Admissibility: Survey of a Concept in Progress

Summary This paper surveys the study of admissibility in statistical decision theory. Reviewed are: Stein's necessary and sufficient admissibility condition and its extensions; Brown's heuristic method of determining admissibility; a differential inequality of the statistical estimation problem; admissible estimators of functions of normal parameters; complete class theorems in hypotheses testing; and problems with finite sample spaces. The connection between admissibility and variational problems of mathematical physics is stressed.

Probability and Its Applications for Engineers

Preliminaries of probability finite sample spaces two or more events random variable, distribution function and expected value functions of a random variable two or more random variables statistics quality control - control charts tolerancing, error analysis and parameter uncertainty reliability engineering, K.C. Kapur random processes and congestion decision trees. Appendices: factorial, gamma function and binomial theorem moments by Taylor Series expansion tables answers to problems.

Discussion on the Meeting on the Gibbs Sampler and Other Markov Chain Monte Carlo Methods

Discussion on the Meeting on the Gibbs Sampler and Other Markov Chain Monte Carlo Methods

Rates of Convergence for Data Augmentation on Finite Sample Spaces

We consider a version of the data augmentation algorithm of Tanner and Wong, which is a special case of the Gibbs sampler. Using ideas from Harris recurrence, we derive quantitative, a priori bounds on the number of iterations required to achieve convergence. Our analysis involves relating the Markov chain to an associated dynamical system.

A Counter-Example Against the Likelihood Principle

SUMMARY A counter-example against the likelihood principle is strengthened and shown to hold also for a finite sample space.

The principle of maximum entropy and Boltzmann type kinetic equations

The problem of the paper is the possibility of a dynamical justification of the principle of maximum entropy in the sense of a dynamical semigroup of open systems. It has been shown that, under the assumption of a convex dynamical semigroup defined on discrete and finite probability distributions (a finite sample space), this principle cannot be realized. This is possible, however, for non-linear dynamical semigroups for some random variables called p-collision-type variables in analogy to the Boltzmann 2-collision problem.

Stein's Paradox is Impossible in Problems with Finite Sample Space

The use of admissible procedures in each of several problems may be inadmissible when the problems are combined, in the sense of summed loss. Thus apparently irrelevant information can sometimes be used to reduce risk over the entire parameter space. This is known as Stein's Paradox. We prove here that this cannot occur when the sample spaces for the problems are finite.

A Complete Class Theorem for Statistical Problems with Finite Sample Spaces

This paper contains a complete class theorem (Theorem 3.2) which applies to most statistical estimation problems having a finite sample space. This theorem also applies to many other statistical problems with finite sample spaces. The description of this complete class involves a stepwise algorithm. At each step of the process it is necessary to construct the Bayes procedures in a suitably modified version of the original problem. The complete class is a minimal complete class if the loss function is strictly convex. Some examples are given to illustrate the application of this complete class theorem. Among these is a new result concerning the estimation of the parameters of a multinomial distribution under a normalized quadratic loss function. (See Example 4.5).

A Systematic Approach to Teaching Counting Formulae

In a first course in statistics or probability, the student encounters problems of calculation of probabilities in finite sample spaces. Generally, this requires learning some basic counting formulae; and it is at this point where many students become confused. Frequently, confusion results from absence of a systematic presentation of four basic counting formulae. The aim of the present note is to suggest a systematic approach based on defining each counting formula as a function of two variables with two levels for each variable. Besides, each formula is interpreted in terms of the same physical problem, namely, the distribution of balls into cells. Consider the problem of finding the number of distinct ways by which r balls can be distributed into n cells. The number of distinct distributions depends on two 2-level variables. The first variable is whether the balls are distinguishable or not; or equivalently, whether the order (in which the balls appear in the cells) matters or does not matter. Let the levels of this first variable be denoted by O (order matters or distinguishable) and 0 (not O). The second variable is whether a cell may receive more than one ball or not; or equivalently, whether replacement (or repetition) is allowed or not. Let the levels of this second variable be denoted by R (with replacement) and R (without replacement). Now, the number of distinct distributions may be denoted by nCr (n choose r), and this should be represented as a bivariate function of the two 2-level variables mentioned above. This leads to four counting functions denoted by nCr (RO), nCr (RO), nCr (RO) and nCr (RO). The corresponding formulae are: nCr (RO) = n', (1)

A Conditional Probability Table for Finite Sample Spaces

A Conditional Probability Table for Finite Sample Spaces

Information and Sufficient Sub-fields

This paper is the result of an attempt to clarify and improve some results in the theory of statistical information. The term information is used to denote different things in different contexts. First of all, there is Shannon's information, $-\sum p_i \log p_i$, defined for probability distributions on a finite sample space; this measures, in an esthetically satisfactory way, the entropy or amount of uncertainty in a distribution. Then there is Wiener's information, $\int f(x) \log f(x) dx$, defined for an absolutely continuous distribution on the line (or in $n$-space); it was introduced by Wiener, with an acknowledgment to von Neumann, as a "reasonable measure" of the amount of information, having the property of being "the negative of the quantity usually defined as entropy in similar situations" ([10], p. 76). Finally, there is "information of one probability distribution $P$ with respect to another $Q$," commonly known as Kullback-Leibler information. On a finite sample space, this has the form $\sum p_i \log (p_i/q_i) = - \sum p_i \log q_i - (- \sum p_i \log p_i)$, and thus has some relationship to entropy; note that the second term, which is the entropy of $\{p_i\}$, is the minimum of the first expression over all distributions $\{q_i\}$. An interesting idea due to Gelfand, Kolmogorov and Yaglom [3] establishes a connection between the Kullback-Leibler information for a finite probability space and that for any space: If $P, Q$ are probability measures on a measurable space $(\Omega, \mathscr{F}), P \ll Q$, and $\{A_i, i = 1, \cdots, n\}$ is any finite measurable partition of $\Omega$, then the supremum of $\sum_i \log \lbrack P(A_i)/Q(A_i)\rbrack P(A_i)$ over all finite measurable partitions is $\int_\Omega \log (dP/dQ)dP$. The only published proof of this result seems to be that due to Kallianpur [5], which uses martingale theory. In Section 1, we shall obtain a rather simple direct proof of this result (Theorem 1.1) and extend it to the case where $Q$ is any $\sigma$-finite measure (Theorem 1.2). Wiener's information is then seen to be the supremum of $\sum \log \lbrack P(A_i)/Q(A_i)\rbrack P(A_i)$ over countable partitions, with $Q =$ Lebesgue measure. Section 2 will be concerned with Kullback-Leibler information. We shall define conditional information relative to a sub-field, establish a relation between this conditional information and sufficiency of the sub-field (Theorem 2.2), and also show that this conditional information equals the difference between information contained in the field and that in the sub-field (Theorem 2.3). These are extensions of results obtained by Kullback and Leibler in a somewhat limited context.

Invariance of Probabilities in Finite Sample Spaces Under Stochastic Operations

Invariance of Probabilities in Finite Sample Spaces Under Stochastic Operations

Invariance of Probabilities in Finite Sample Spaces Under Stochastic Operations

Invariance of Probabilities in Finite Sample Spaces Under Stochastic Operations