This article proposes a probabilistic framework for the design of robustly asymptotically stable moving-horizon estimators (MHE) for discrete-time nonlinear systems, and a mechanism to incorporate differential privacy in moving-horizon estimation. We formulate the moving-horizon estimator as an iterative proximal descent scheme in the space of probability measures with respect to the L <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -Wasserstein metric, which we name W <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> -MHE. We then investigate asymptotic stability and robustness properties of the W <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> -MHE against the backdrop of the classical notion of strong local observability. Motivated by applications where the measurement data used by the estimator is to be kept private, we then propose a mechanism to incorporate differential privacy in the estimation method, based on an entropy regularization of the MHE objective functional. In particular, we find sufficient bounds on the regularization parameter to achieve the desired level of differential privacy. We then demonstrate the performance of the W <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> -MHE in numerical simulations.