A Toolbox for Refined Information-Theoretic Analyses with Applications

This paper contains two main contributions concerning the asymmetric broadcast channel. The first is an analysis of the exact random coding error exponents for both users, and the second is the derivation of universal decoders for both users. These universal decoders are certain variants of the maximum mutual information universal decoder, and they achieve the corresponding random coding exponents of optimal decoding. In addition, we introduce some lower bounds, which involve optimizations over very few parameters, unlike the original, exact exponents, which involve minimizations over auxiliary probability distributions. Numerical results for the binary symmetric broadcast channel show improvements over previously derived error exponents for the same model.

Let $\mathscr{K} = \{K_\lambda: \lambda \in \Lambda\}$ be a family of sampling distributions for the data $x$ on a sample space $\mathscr{X}$ which is indexed by a parameter $\lambda \in \Lambda,$ and let $\mathscr{F}$ be a family of priors on $\Lambda$. Then $\mathscr{F}$ is said to be conjugate for $\mathscr{K}$ if it is closed under sampling, that is, if the posterior distributions of $\lambda$ given the data $x$ belong to $\mathscr{F}$ for almost all $x$. In this paper, we set up a framework for the study of what we term the dual problem: for a given family of priors $\mathscr{F}$ (a subfamily of a general exponential family), find the class of sampling models $\mathscr{K}$ for which $\mathscr{F}$ is conjugate. In particular, we show that $\mathscr{K}$ must be a general exponential family dominated by some measure $Q$ on $(\mathscr{X}, B),$ where $B$ is the Borel field on $\mathscr{X}$. It is the class of such measures $Q$ that we investigate in this paper. We study its geometric features and general structure and apply the results to some familiar examples.

The main object of this paper is to investigate several general families of hyper-geometric polynomials and their associated multiple integral representations. By suitably specializing our main results, the corresponding integral representations are deduced for such familiar classes of hypergeometric polynomials as (for example) the generalized Bedient polynomials of the first and second kinds. Each of the integral representations, which are derived in this paper, may be viewed also as a linearization relationship for the product of two different members of the associated family of hypergeometric polynomials.

Motivated essentially by their potential for applications in the mathematical, physical, and statistical sciences, the object of this paper is to investigate several general families of hypergeometric polynomials and their associated multiple integral representations. By suitably specializing the main results presented here, the corresponding integral representations are derived for familiar simpler classes of hypergeometric polynomials such as (for example) the Lagrange polynomials, Shively’s pseudo-Laguerre polynomials, and generalized Bessel polynomials. Each of the integral representations derived in this paper may be also viewed as a linearization relationship for the product of two different members of the associated family of hypergeometric polynomials.

By using some integral representations for several Mathieu type series (see P.L. Butzer, T.K. Pogány, and H.M. Srivastava, A linear ODE for the Omega function associated with the Euler function E α(z) and the Bernoulli function B α(z), Appl. Math. Lett. 19 (2006), pp. 1073–1077; P. Cerone and C.T. Lenard, On integral forms of generalised Mathieu series, J. Inequal. Pure Appl. Math. 4 (5) (2003), Article 100, pp. 1–11 (electronic), T.K. Pogány; H.M. Srivastava and Ž. Tomovski, Some families of Mathieu a-series and alternating Mathieu a-series, Appl. Math. Comput. 173 (2006), pp. 69–108; H.M. Srivastava and Ž. Tomovski, Some problems and solutions involving Mathieu's series and its generalizations, J. Inequal. Pure Appl. Math. 5 (2) (2004), Article 45, pp. 1–13 (electronic); Ž. Tomovski, Integral representations of generalized Mathieu series via Mittag-Leffler type functions, Fract. Calc. Appl. Anal. 10 (2007), pp. 127–138.) via the Bessel function J ν of the first kind, the Gauss hypergeometric function 2 F 1, the generalized hypergeometric function p F q and the Fox–Wright generalization p Ψ q of the hypergeometric function p F q , a number of integral representations of the Laplace, Fourier, and Mellin types are derived here for certain general families of Mathieu type series. Some interesting corollaries and consequences of these integral representations are also considered.

Integral representations for the generalized Bedient polynomials and the generalized Cesàro polynomials

In this paper, we introduce two families of general bivariate distributions. We refer to these families as general bivariate exponential family and general bivariate inverse exponential family. Many bivariate distributions in the literature are members of the proposed families. Some properties of the proposed families are discussed, as well as a characterization associated with the stress-strength reliability parameter, R, is presented. Concerning R, the maximum likelihood estimators and a simple estimator with an explicit form depending on some marginal distributions are obtained in case of complete sampling. When the stress is censored at the strength, an explicit estimator of R is also obtained. The results obtained can be applied to a variety of bivariate distributions in the literature. A numerical illustration is applied on some well-known distributions. Finally a real data example is presented to fit one of the proposed models.

On the equivalence of some test criteria based on BAN estimators for the multivariate exponential family

We explore known integral representations of the logarithmic and power functions, and demonstrate their usefulness for information-theoretic analyses. We obtain compact, easily-computable exact formulas for several source and channel coding problems that involve expectations and higher moments of the logarithm of a positive random variable and the moment of order ρ>0 of a non-negative random variable (or the sum of i.i.d. positive random variables). These integral representations are used in a variety of applications, including the calculation of the degradation in mutual information between the channel input and output as a result of jamming, universal lossless data compression, Shannon and Rényi entropy evaluations, and the ergodic capacity evaluation of the single-input, multiple-output (SIMO) Gaussian channel with random parameters (known to both transmitter and receiver). The integral representation of the logarithmic function and its variants are anticipated to serve as a rigorous alternative to the popular (but non-rigorous) replica method (at least in some situations).

Following our careful analysis of the normal conditionals example in the last chapter and our brief mention of the exponential conditionals distribution in Chapter 2, it is natural to seek out more general results regarding distributions whose conditionals are posited to be members of quite general exponential families. Indeed the discussion leading up to Theorem 2.4, suggests that things should work well when conditionals are from exponential families. The key reference for the present chapter is Arnold and Strauss (1991). However, it should be mentioned that results due to Besag (1974) in a stochastic process setting anticipate some of the observations in this chapter.

By using some recently investigated fourier sine integral representations for the Mathieu type series (see [4]), new integral and series representations are derived here for certain general families of Mathieu type series.

A generalization of BIC for the general exponential family

This thesis includes three chapters in addition to an introductory chapter. Chapters two and three focus on generalized linear mixed models (GLMMs). In Chapter two, we study a popular GLMM, namely the logistic linear mixed model (LLMM). Using P\'{o}lya-Gamma latent variables, we construct an efficient two-block Gibbs sampler for LLMMs. Also, the geometric ergodicity of this sampler is established. Using numerical examples, we perform a comparison between the block Gibbs sampler and the full Gibbs sampler to demonstrate the performance of these algorithms. Finally, some useful properties of the P\'{o}lya-Gamma distributions are obtained. In Chapter three, we explore posterior propriety for Bayesian GLMMs. We provide necessary and sufficient conditions for posterior propriety for GLMMs with general exponential family as well as particularly for binomial and Poisson GLMMs. Compared to the existing results in the literature, our posterior propriety conditions remove some usual constraints, including those on the random effect design matrix. Examples are used to demonstrate how to check posterior propriety for binomial and Poisson GLMMs. In Chapter four, we study posterior propriety for spatial generalized linear mixed models (SGLMMs). We provide necessary conditions for posterior propriety for general exponential family SGLMMs. The sufficient conditions are given specifically for Poisson and binomial SGLMMs. The existing posterior propriety results for spatially correlated data do not focus on SGLMMs. Thus our work will allow practitioners of Bayesian SGLMMs to use improper priors with confidence.

Generalized linear models (GLMs) are generalization of the linear regression models, which allow fitting regression models to response variable that is non normal and follows a general exponential family. The aim of this study is to encourage and initiate the application of GLMs to predict the maternal and fetal blood lead level. The inverse Gaussian distribution with inverse quadratic link function is considered. Four main effects were significant in the prediction of the maternal blood lead level (pica, smoking of mother, dairy products intake of mother, calcium intake of mother), while in the prediction of the fetal blood lead level two main effects showed significance (dairy products intake of mother and hemoglobin of mother). Keywords: Generalized linear models, Exponential family, Inverse Gaussian distribution, Link functions

A variational norm associated with sets of computational units and used in function approximation, learning from data, and infinite-dimensional optimization is investigated. For sets Gk obtained by varying a vector y of parameters in a fixed-structure computational unit K(-,y) (e.g., the set of Gaussians with free centers and widths), upper and lower bounds on the G <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</sub> -variation norms of functions having certain integral representations are given, in terms of the £ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -norms of the weighting functions in such representations. Families of functions for which the two norms are equal are described.