Local and Networked Mean-square Estimation with High Dimensional Log-concave Noise

Abstract

We consider the problem of mean-square estimation of the state of a discrete time dynamical system having additive non-Gaussian noise. We assume that the noise has the structure that at each time instant, it is a projection of a fixed high-dimensional noise vector with a log-concave density. We derive conditions which guarantee that, as the dimension of this noise vector grows large, the optimal estimator of the problem with Gaussian noise becomes near-optimal for the problem with non-Gaussian noise. The results are derived by first showing an asymptotic Gaussian lower bound on the minimum error which holds even for nonlinear systems. The bound is asymptotically tight for linear systems. For nonlinear systems, this bound is tight provided and the noise has a strongly log-concave density with some additional structure. These results imply that the estimate obtained by employing the Gaussian estimator on the non-Gaussian problem satisfies an approximate orthogonality principle, and the difference between this estimate and the optimal estimate vanishes strongly in $L^{2}$ . For linear systems with high-dimensional log-concave noise of this structure, we get that the Kalman filter serves as a near-optimal estimator. A key ingredient in the proofs is a recent central limit theorem of Eldan and Klartag. The minimum mean-square error is in general neither weakly upper semicontinuous nor weakly lower semicontinuous. Our results show that in the setting we consider, it is weakly continuous. We then consider mean-square estimation over a noisy network, where the problem is to jointly design strategies for sensing and estimation to minimize a composite quadratic cost. These problems do not have a static information structure, whereby the previously derived results do not apply. We consider two different cases—when the estimation part of the problem corresponds to estimating the original source sensed by the sensor, and when this part corresponds to the estimation of the action of the sensor. In the former case, we show that results analogous to the above continue to hold. The latter case corresponds to a variant of Witsenhausen’s problem, wherein we show only partial results.