Iterative Proportional Fitting Procedure Research Articles

Alternating minimization and Boltzmann machine learning

Training a Boltzmann machine with hidden units is appropriately treated in information geometry using the information divergence and the technique of alternating minimization. The resulting algorithm is shown to be closely related to gradient descent Boltzmann machine learning rules, and the close relationship of both to the EM algorithm is described. An iterative proportional fitting procedure for training machines without hidden units is described and incorporated into the alternating minimization algorithm.

Computing the maximum-entropy extension of given discrete probability distributions

The maximum-entropy extension of given discrete probability distributions over a “fully reducible” set class admits a closed-form expression, which makes its computation direct. In the remaining cases the computation of the maximum-entropy extension is carried out using the standard procedure of iterative proportional fitting, whose costs can often be reduced by applying the graph-theoretical techniques of reduction and decomposition. By making joint use of these techniques, we present a computation policy which limits the application of the iterative proportional fitting procedure to computing the marginals of a maximum-entropy distribution corresponding to the nonseparable components (in the graph-theoretical sense) of the underlying set class.

Duality ofIProjections and Maximum Likelihood Estimation for Log-Linear Models Under Cone Constraints

Abstract Important order restrictions and general log-linear models for multinomial experiments can often be expressed by requiring that the vector composed of the logs of the probabilities fall within a closed convex cone. Here we exhibit a duality relationship (by way of a Fenchel duality theorem) between these types of problems and projections in I-divergence geometry. We show that many cone-constrained maximum likelihood estimation problems are exactly equivalent to an I projection onto a translation of the negative polar cone. This duality relationship permits the concise characterization of cone-restricted maximum likelihood estimates and the use of iterative algorithms proposed by Csiszar (1975) and Dykstra (1985b), which are analogous to the iterative proportional-fitting procedure. Computational aspects of maximum likelihood estimates are discussed, and a relationship with least squares projections onto isotonic cones is presented. We discuss examples in detail to exhibit the usefulness and power...

On the construction of approximate multi-factor designs from given marginals using the iterative proportional fitting procedure

For multifactor designs based on linear models, the information matrix generally depends on a certain set of marginal tables created from the design itself. This note considers the problems of whether a set of marginal tables is consistent, in that a design exists that can yield them, and of calculating such a design when at least one does exist. The results are obtained by direct analogy with the problem of maximum likelihood estimation in longlinear models for categorical data.

Maximum Entropy Convex Decompositions of Doubly Stochastic and Nonnegative Matrices

Two maximum entropy convex decompositions are computed with the use of the iterative proportional fitting procedure. First, a doubly stochastic version of a 5 × 5 British social mobility table is represented as the sum of 120 5 × 5 permutation matrices. The most heavily weighted permutations display a bandwidth form, indicative of relatively strong movements within social classes and between neighboring classes. Then the mobility table itself is expressed as the sum of 6 985 5 × 5 transportation matrices—possessing the same row and column sums as the mobility table. A particular block-diagonal structure is evident in the matrices assigned the greatest weight. The methodology can be applied as well to the representation of other nonnegative matrices in terms of their extreme points, and should be extendable to higher-order mathematical structures—for example, operators and functions.

Analysis of contingency table data on torus mandibularis using a log linear model.

AbstractThe hypothesized sexual difference in the incidence of torus mandibularis in Eskimoid groups, as well as age and group differences, was studied in two Northwest Territories Eskimo groups and in an Aleut group (examined by Moorrees). The data were analyzed using statistical methods new to the field of anthropology. The method analyzes data sampled from a multinomial distribution. A hierarchy of log linear models are fitted to the cell counts of a contingency table. An iterative proportional fitting procedure is used to obtain expected cell counts under each log linear model. The “goodness of fit” of each model is tested by the log likelihood ratio. This statistic can be partitioned into additive components such that differences between models can be tested. In this way a “best” model, from the hierarchy of models, is chosen. Among these three groups of Eskimos, the incidence of torus mandibularis was not affected by sex but was affected by age and was different between the three groups.

An Iterative Procedure for Estimation in Contingency Tables

Deming and Stephan (1940) first proposed the use of an iterative proportional fitting procedure to estimate cell probabilities in a contingency table subject to certain marginal constraints. In this paper we first relate this procedure to a variety of sources and a variety of statistical problems. We then describe the procedure geometrically for two-way contingency tables using the concepts presented in Fienberg (1968). This geometrical description leads to a rather simple proof of the convergence of the iterative procedure. We conclude the paper with a discussion of extensions to multi-dimensional tables and to tables with some zero entries.