Polynomial spectral decomposition of conditional expectation operators

Abstract

The spectral structure of conditional expectation operators underlies several useful notions in statistics and information theory. For instance, the second largest singular value of such an operator is known as maximal correlation, which has received considerable attention in the context of performing non-linear regression and analyzing contraction coefficients. Given source-channel pairs, we study the singular value decompositions of the corresponding conditional expectation operators, and derive necessary and sufficient conditions on the conditional moments that characterize when the conditional expectation operators have singular vectors that are orthogonal polynomials. Furthermore, we illustrate that conditional expectation operators constructed using well-known natural exponential families with quadratic variance functions and their conjugate priors have orthogonal polynomial singular vectors. In particular, the Gaussian source and Gaussian channel beget the Hermite case, the gamma source and Poisson channel beget the Laguerre case, and the beta source and binomial channel beget the Jacobi case.