Abstract
This paper derives the optimal test channel distribution and the complete characterization of the classical Gorbunov and Pinsker (1973), Gorbunov and Pinsker (1974) nonanticipatory epsilon entropy of multivariate Gaussian Markov sources with square-error fidelity, which remained an open problem since 1974. The paper also formulates a state dependent nonanticipatory epsilon entropy, in which past reproductions are available to the decoder and not to the encoder, the test channel is specified with respect to an auxiliary (state) random process, and the reproduction process is a causal function of past reproduction and the auxiliary random process. This variation is analogous to the Wyner and Ziv (1976) and Wyner (1978) rate distortion function (RDF), of memoryless sources. It is shown that the operational rate of zero-delay codes, with past reproductions available to the decoder but not to the encoder is bounded below by the state dependent nonanticipatory epsilon entropy rate. For the case of multivariate Gaussian Markov sources with square-error fidelity, the optimal test channel distribution and the complete characterization of the state dependent of nonanticipatory epsilon entropy are derived, and also shown that that the two nonanticipatory epsilon entropies coincide. The derivations are new; they are based on structural properties of the stochastic realizations of the reproduction process that induce the optimal test channel distributions. They are derived using, achievable lower bounds on information theoretic measures, properties of mean-square estimation theory, Hadamard’s inequality, and canonical correlation coefficients of a tuple of multivariate jointly Gaussian random processes. Applications of the nonanticipatory epsilon entropy and its state dependent variation are discussed to the areas of control of unstable Gaussian systems over limited memory channels, design of causal estimators for Gaussian Markov sources with a fidelity criterion, computation of the rate loss of causal and zero-delay codes of Gaussian Markov sources with respect to non-causal codes.
Highlights
This paper derives the optimal test channel distribution and the complete characterization of the classical Gorbunov and Pinsker [2], [3] nonanticipatory epsilon entropy of multivariate Gaussian Markov sources with square-error fidelity, which remained an open problem since 1974
The state dependent nonanticipatory epsilon entropy involves the state or auxiliary random variable (RV); the reproduction process Y n is analogous to the WynerZiv [4], [5] rate distortion function (RDF) with side information Y t−1 at the decoder but not the encoder
We show a structural property, which states that certain matrices that parametrize the realizations of the reproduction process and produce the optimal test channel distribution, admit spectral representations w.r.t. unitary matrices
Summary
{0, 1, 2, . . .}, N {1, 2, . . .}, Nn {1, . . . , n}, n ∈ N. (Ω, F ) −→ (X, B(X)), we denote by P X ∈ dx = PX (dx) ≡ P(dx) the probability measure induced by X on (X, B(X)) (i.e., probability distribution if X = Rn) Given another RV, Y : (Ω, F) −→ (Y, B(Y)) we denote by. Consider three random variables (RVs) X : Ω → X, Y : Ω → Y, and W : Ω → W defined on some probability space (Ω, F, P). We say that RVs (X, Y ) are conditionally independent given RV W if PX,Y |W = PX|W PY |W − a.s., (almost surely); the specification a.s. is often omitted We often denote this conditional independence by the Markov. ∈ R(nx+ny)×(nx+ny), The reader should distinguish the covariance matrices K(X,Y ) and KX,Y ∈ Rnx×ny
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
