Exponential Family Of Probability Distributions Research Articles

A Unified Model of Complex Specified Information

A mathematical theory of complex specified information is introduced which unifies several prior methods of computing specified complexity. Similar to how the exponential family of probability distributions have dissimilar surface forms yet share a common underlying mathematical identity, we define a model that allows us to cast Dembski’s semiotic specified complexity, Ewert et al.’s algorithmic specified complexity, Hazen et al.’s functional information, and Behe’s irreducible complexity into a common mathematical form. Adding additional constraints, we introduce canonical specified complexity models, for which one-sided conservation bounds are given, showing that large specified complexity values are unlikely under any given continuous or discrete distribution and that canonical models can be used to form statistical hypothesis tests, by bounding tail probabilities for arbitrary distributions.

Hierarchical modeling of graphs using modular decomposition

Hierarchical modeling of graphs using modular decomposition

The Vertex Reinforced Jump Process and a random Schrödinger operator on finite graphs

We introduce a new exponential family of probability distributions, which can be viewed as a multivariate generalization of the Inverse Gaussian distribution. Considered as the potential of a random Schr\"odinger operator, this exponential family is related to the random field that gives the mixing measure of the Vertex Reinforced Jump Process (VRJP), and hence to the mixing measure of the Edge Reinforced Random Walk (ERRW), the so-called magic formula. In particular, it yields by direct computation the value of the normalizing constants of these mixing measures, which solves a question raised by Diaconis. The results of this paper are instrumental in [Sabot-Zeng,2015], where several properties of the VRJP and the ERRW are proved, in particular a functional central limit theorem in transient regimes, and recurrence of the 2-dimensional ERRW.

Sample Size Determination Within the Scope of Conditional Maximum Likelihood Estimation with Special Focus on Testing the Rasch Model.

This paper refers to the exponential family of probability distributions and the conditional maximum likelihood (CML) theory. It is concerned with the determination of the sample size for three groups of tests of linear hypotheses, known as the fundamental trinity of Wald, score, and likelihood ratio tests. The main practical purpose refers to the special case of tests of the class of Rasch models. The theoretical background is discussed and the formal framework for sample size calculations is provided, given a predetermined deviation from the model to be tested and the probabilities of the errors of the first and second kinds.

Geometry of deformed exponential families: Invariant, dually-flat and conformal geometries

An information-geometrical foundation is established for the deformed exponential families of probability distributions. Two different types of geometrical structures, an invariant geometry and a flat geometry, are given to a manifold of a deformed exponential family. The two different geometries provide respective quantities such as deformed free energies, entropies and divergences. The class belonging to both the invariant and flat geometries at the same time consists of exponential and mixture families. Theq-families are characterized from the viewpoint of the invariant and flat geometries. The q-exponential family is a unique class that has the invariant and flat geometries in the extended class of positive measures. Furthermore, it is the only class of which the Riemannian metric is conformally connected with the invariant Fisher metric.

On φ-Families of Probability Distributions

We generalize the exponential family of probability distributions. In our approach, the exponential function is replaced by a φ-function, resulting in a φ-family of probability distributions. We show how φ-families are constructed. In a φ-family, the analogue of the cumulant-generating function is a normalizing function. We define the φ-divergence as the Bregman divergence associated to the normalizing function, providing a generalization of the Kullback–Leibler divergence. A formula for the φ-divergence where the φ-function is the Kaniadakis κ-exponential function is derived.

Geometry of q-Exponential Family of Probability Distributions

The Gibbs distribution of statistical physics is an exponential family of probability distributions, which has a mathematical basis of duality in the form of the Legendre transformation. Recent studies of complex systems have found lots of distributions obeying the power law rather than the standard Gibbs type distributions. The Tsallis q-entropy is a typical example capturing such phenomena. We treat the q-Gibbs distribution or the q-exponential family by generalizing the exponential function to the q-family of power functions, which is useful for studying various complex or non-standard physical phenomena. We give a new mathematical structure to the q-exponential family different from those previously given. It has a dually flat geometrical structure derived from the Legendre transformation and the conformal geometry is useful for understanding it. The q-version of the maximum entropy theorem is naturally induced from the q-Pythagorean theorem. We also show that the maximizer of the q-escort distribution is a Bayesian MAP (Maximum A posteriori Probability) estimator.

Matrix Nearness Problems with Bregman Divergences

This paper discusses a new class of matrix nearness problems that measure approximation error using a directed distance measure called a Bregman divergence. Bregman divergences offer an important generalization of the squared Frobenius norm and relative entropy, and they all share fundamental geometric properties. In addition, these divergences are intimately connected with exponential families of probability distributions. Therefore, it is natural to study matrix approximation problems with respect to Bregman divergences. This article proposes a framework for studying these problems, discusses some specific matrix nearness problems, and provides algorithms for solving them numerically. These algorithms apply to many classical and novel problems, and they admit a striking geometric interpretation.

An Asymptotic Second-Order Lower Bound for the Bayes Risk of a Sequential Procedure

Consider the problem of estimating the difference of the means of two populations, where each population distribution is a member of the one-parameter exponential family of probability distributions. A Bayesian approach is adopted in which the mean difference is estimated under the squared error loss and the prior distributions are of the form proposed by Diaconis and Ylivisaker [Diaconis, P.; Ylivisaker, D. Conjugate priors for exponential families. Ann. of Statist. 1979, 6, 269–281]. The main result determines an asymptotic second-order lower bound for the Bayes risk of a sequential procedure that takes N observations from one population and t − N from the other population, and estimates the mean difference by the Bayes estimator, where N is determined according to a sequential design and t denotes the total number of observations sampled from both populations.

Geometry associated with normal distributions

It is known that manifolds of smooth families of probability distributions admit dualistic structures. S. Amari proposed Information Geometry, whose keywords are dualistic connections [1]. Among them the case of dual connections being flat is interesting. Many important families of probability distributions, e.g. exponential families admit flat dual connections. The notion of flat dual connections is the same with Hessian structures which have being developed from a different view point [9]-[12]. In this paper, for a linear mapping p of a domain Ω into the space of positive definite symmetric matrices we construct an exponential family of probability distributions [ p ( x ; θ , ώ)} on parametrized by θ e R, ω e Ω, and study a Hessian structure on R x Ω given by the exponential family. Such families contain n -dimensional normal distributions (Example 1) and a family of constant negative curvature (Example 2). In case of a Lie group acting on Ω, p is assumed to be equivariant. O.S. Rothaus and I. Satake studied such a linear mapping p for homogeneous convex cones [7] [8]. Using p we introduce a Hessian structure on a vector bundle over a compact hyperbolic affine manifold and prove a certain vanishing theorem (Theorem 2).

A p* primer: logit models for social networks

Abstract A major criticism of the statistical models for analyzing social networks developed by Holland, Leinhardt, and others [Holland, P.W., Leinhardt, S., 1977. Notes on the statistical analysis of social network data; Holland, P.W., Leinhardt, S., 1981. An exponential family of probability distributions for directed graphs. Journal of the American Statistical Association. 76, pp. 33–65 (with discussion); Fienberg, S.E., Wasserman, S., 1981. Categorical data analysis of single sociometric relations. In: Leinhardt, S. (Ed.), Sociological Methodology 1981, San Francisco: Jossey-Bass, pp. 156–192; Fienberg, S.E., Meyer, M.M., Wasserman, S., 1985. Statistical analysis of multiple sociometric relations. Journal of the American Statistical Association, 80, pp. 51–67; Wasserman, S., Weaver, S., 1985. Statistical analysis of binary relational data: Parameter estimation. Journal of Mathematical Psychology. 29, pp. 406–427; Wasserman, S., 1987. Conformity of two sociometric relations. Psychometrika. 52, pp. 3–18] is the very strong independence assumption made on interacting individuals or units within a network or group. This limiting assumption is no longer necessary given recent developments on models for random graphs made by Frank and Strauss [Frank, O., Strauss, D., 1986. Markov graphs. Journal of the American Statistical Association. 81, pp. 832–842] and Strauss and Ikeda [Strauss, D., Ikeda, M., 1990. Pseudolikelihood estimation for social networks. Journal of the American Statistical Association. 85, pp. 204–212]. The resulting models are extremely flexible and easy to fit to data. Although Wasserman and Pattison [Wasserman, S., Pattison, P., 1996. Logit models and logistic regressions for social networks: I. An introduction to Markov random graphs and p*. Psychometrika. 60, pp. 401–426] present a derivation and extension of these models, this paper is a primer on how to use these important breakthroughs to model the relationships between actors (individuals, units) within a single network and provides an extension of the models to multiple networks. The models for multiple networks permit researchers to study how groups are similar and/or how they are different. The models for single and multiple networks and the modeling process are illustrated using friendship data from elementary school children from a study by Parker and Asher [Parker, J.G., Asher, S.R., 1993. Friendship and friendship quality in middle childhood: Links with peer group acceptance and feelings of loneliness and social dissatisfaction. Developmental Psychology. 29, pp. 611–621].

Memorial: Nikolai Nikolaevitch Chentsov

Memorial: Nikolai Nikolaevitch Chentsov

Sequential estimation of the mean of an exponential family in three stages

Abstract This paper provides a three-stage sequential procedure for estimating the mean of an exponential family of probability distributions with a weighted squared error loss and a fixed cost for each observation. The procedure is shown to be asymptotically optimal to second order, for a general class of estimators of the mean.

Natural exponential families of probability distributions and exponential-polynomial approximation

We consider expansions of regular functions in series with respect to infinite systems of Dirichlet polynomials, i.e., combinations of polynomials and exponents. We show that for every natural exponential family of probability measures on a real line and an infinite sequence of complex numbers, whose real parts belong to the natural parameter space of the above family, one can associate an expansion of any sufficiently regular function with respect to certain Dirichlet polynomials. We give an explicit formula for the remainder term in this expansion. It turns out to be an expectation of convolution of one of the above Dirichlet polynomials with certain differential operator of the function. The Dirichlet polynomials in question are combinations of exponents and Appel polynomials generated by members of the exponential family of distributions. Dirichlet series and expansions with respect to Appel polynomials are particular cases of the above expansions.

An Asymptotic Lower Bound for the Local Minimax Regret in Sequential Point Estimation

Let $\Omega$ be an interval and let $F_\omega, \omega \in \Omega,$ denote a one-parameter exponential family of probability distributions on $\mathscr{R} = (-\infty, \infty),$ each of which has a finite mean $\theta,$ depending on some unknown parameter $\omega \in \Omega.$ The main results of this paper determine an asymptotic lower bound for the local minimax regret, under a general smooth loss function and for a general class of estimators of $\theta.$ This bound is obtained by first determining the limit of the Bayes regret and then maximizing with respect to the prior distribution of $\omega.$

On the Fokker-Planck equation with time-dependent drift and diffusion coefficients and its exponential solutions

The systems described by the Fokker-Planck equation (FPE) with time-dependent drift and diffusion coefficients are considered. It is shown that for periodic in time drift and diffusion there exists a periodic solution of the FPE such that any other solution of this equation tends to the periodic one fort→∞. Some method of solution of the FPE using elements of exponential family of probability distributions is proposed. GeneralisedH-theorem for exponential solutions of the FPE is fulfiled. Stability properties of exponential solutions are investigated.

An Exponential Family of Probability Distributions for Directed Graphs: Comment

An Exponential Family of Probability Distributions for Directed Graphs: Comment

An Exponential Family of Probability Distributions for Directed Graphs

Abstract Directed graph (or digraph) data arise in many fields, especially in contemporary research on structures of social relationships. We describe an exponential family of distributions that can be used for analyzing such data. A substantive rationale for the general model is presented, and several special cases are discussed along with some possible substantive interpretations. A computational algorithm based on iterative scaling procedures for use in fitting data is described, as are the results of a pilot simulation study. An example using previously reported empirical data is worked out in detail. An extension to multiple relationship data is discussed briefly.

An Exponential Family of Probability Distributions for Directed Graphs: Comment

An Exponential Family of Probability Distributions for Directed Graphs: Comment

An Exponential Family of Probability Distributions for Directed Graphs

Abstract Directed graph (or digraph) data arise in many fields, especially in contemporary research on structures of social relationships. We describe an exponential family of distributions that can be used for analyzing such data. A substantive rationale for the general model is presented, and several special cases are discussed along with some possible substantive interpretations. A computational algorithm based on iterative scaling procedures for use in fitting data is described, as are the results of a pilot simulation study. An example using previously reported empirical data is worked out in detail. An extension to multiple relationship data is discussed briefly.