Langevin Monte Carlo Research Articles

Wireless Federated Langevin Monte Carlo: Repurposing Channel Noise for Bayesian Sampling and Privacy

Most works on federated learning (FL) focus on the most common frequentist formulation of learning whereby the goal is minimizing the global empirical loss. Frequentist learning, however, is known to be problematic in the regime of limited data as it fails to quantify epistemic uncertainty in prediction. Bayesian learning provides a principled solution to this problem by shifting the optimization domain to the space of distribution in the model parameters. {\color{black}This paper proposes a novel mechanism for the efficient implementation of Bayesian learning in wireless systems. Specifically, we focus on a standard gradient-based Markov Chain Monte Carlo (MCMC) method, namely Langevin Monte Carlo (LMC), and we introduce a novel protocol, termed Wireless Federated LMC (WFLMC), that is able to repurpose channel noise for the double role of seed randomness for MCMC sampling and of privacy preservation.} To this end, based on the analysis of the Wasserstein distance between sample distribution and global posterior distribution under privacy and power constraints, we introduce a power allocation strategy as the solution of a convex program. The analysis identifies distinct operating regimes in which the performance of the system is power-limited, privacy-limited, or limited by the requirement of MCMC sampling. Both analytical and simulation results demonstrate that, if the channel noise is properly accounted for under suitable conditions, it can be fully repurposed for both MCMC sampling and privacy preservation, obtaining the same performance as in an ideal communication setting that is not subject to privacy constraints.

Markov Chain Monte Carlo for Gaussian: A Linear Control Perspective

Drawing samples from a given target probability distribution is a fundamental task in many science and engineering applications. A commonly used method for sampling is the Markov chain Monte Carlo (MCMC) which simulates a Markov chain whose stationary distribution coincides with the target one. In this work, we study the convergence and complexity of MCMC algorithms from a dynamic system point of view. We focus on the special cases with Gaussian target distributions and provide a Lyapunov perspective to them using tools from linear control theory. In particular, we systematically analyze two popular MCMC algorithms: Langevin Monte Carlo (LMC) and kinetic Langevin Monte Carlo (KLMC). By applying Lyapunov theory we derive impressive complexity bounds to these algorithms: for LMC, our result is better than all existing results, and for KLMC, ours matches the best known bound. Our analysis also highlights subtle differences between sampling and optimization that could inform the more challenging task to sample from general distributions. Overall, our findings offer valuable insights for improving MCMC algorithms.

Bayesian Inversion of Time Domain Electromagnetic (TDEM) Synthetic Data using Langevin Monte Carlo (LMC)

The inversion of time domain electromagnetic (TDEM) data is an ill-posed problem. This makes the standard inversion procedure, which is to linearize the related objective function. Then take a deterministic approach to determine a solution that can minimize the said objective function, which has the potential to be trapped in a local minimum. In this study, we solved the problem of TDEM data inversion using the Bayesian framework by generating a sample from the posterior distribution. The posterior distribution contains information related to the uncertainty of the TDEM data and the results of the forward modeling formulation, and prior information about the subsurface parameter model. In conducting the sampling process, we use the Langevin Monte Carlo (LMC) algorithm, one of many gradient-based Markov chain Monte Carlo (MCMC) sampling algorithms. Bayesian inversion was performed on synthetic data generated through the forward modeling of several test models. We used a model with varying thickness and resistivity values for inversion. We also added a prior probability distribution related to the smoothness constraint between resistivity values in adjacent layers allowing sharp and smooth transitions.

Langevin monte carlo rendering with gradient-based adaptation

We introduce a suite of Langevin Monte Carlo algorithms for efficient photorealistic rendering of scenes with complex light transport effects, such as caustics, interreflections, and occlusions. Our algorithms operate in primary sample space, and use the Metropolis-adjusted Langevin algorithm (MALA) to generate new samples. Drawing inspiration from state-of-the-art stochastic gradient descent procedures, we combine MALA with adaptive preconditioning and momentum schemes that re-use previously-computed first-order gradients, either in an online or in a cache-driven fashion. This combination allows MALA to adapt to the local geometry of the primary sample space, without the computational overhead associated with previous Hessian-based adaptation algorithms. We use the theory of controlled Markov chain Monte Carlo to ensure that these combinations remain ergodic, and are therefore suitable for unbiased Monte Carlo rendering. Through extensive experiments, we show that our algorithms, MALA with online and cache-driven adaptation, can successfully handle complex light transport in a large variety of scenes, leading to improved performance (on average more than 3× variance reduction at equal time, and 7× for motion blur) compared to state-of-the-art Markov chain Monte Carlo rendering algorithms.

User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradient

In this paper, we study the problem of sampling from a given probability density function that is known to be smooth and strongly log-concave. We analyze several methods of approximate sampling based on discretizations of the (highly overdamped) Langevin diffusion and establish guarantees on its error measured in the Wasserstein-2 distance. Our guarantees improve or extend the state-of-the-art results in three directions. First, we provide an upper bound on the error of the first-order Langevin Monte Carlo (LMC) algorithm with optimized varying step-size. This result has the advantage of being horizon free (we do not need to know in advance the target precision) and to improve by a logarithmic factor the corresponding result for the constant step-size. Second, we study the case where accurate evaluations of the gradient of the log-density are unavailable, but one can have access to approximations of the aforementioned gradient. In such a situation, we consider both deterministic and stochastic approximations of the gradient and provide an upper bound on the sampling error of the first-order LMC that quantifies the impact of the gradient evaluation inaccuracies. Third, we establish upper bounds for two versions of the second-order LMC, which leverage the Hessian of the log-density. We provide non asymptotic guarantees on the sampling error of these second-order LMCs. These guarantees reveal that the second-order LMC algorithms improve on the first-order LMC in ill-conditioned settings.

Higher order Langevin Monte Carlo algorithm

A new (unadjusted) Langevin Monte Carlo (LMC) algorithm with improved rates in total variation and in Wasserstein distance is presented. All these are obtained in the context of sampling from a target distribution $\pi$ that has a density $\hat{\pi}$ on $\mathbb{R}^d$ known up to a normalizing constant. Moreover, $-\log \hat{\pi}$ is assumed to have a locally Lipschitz gradient and its third derivative is locally H\"{o}lder continuous with exponent $\beta \in (0,1]$. Non-asymptotic bounds are obtained for the convergence to stationarity of the new sampling method with convergence rate $1+ \beta/2$ in Wasserstein distance, while it is shown that the rate is 1 in total variation even in the absence of convexity. Finally, in the case where $-\log \hat{\pi}$ is strongly convex and its gradient is Lipschitz continuous, explicit constants are provided.

Riemann–Langevin Particle Filtering in Track-Before-Detect

Track-before-detect (TBD) is a powerful approach that consists in providing the tracker directly with the sensor measurements without any predetection. Due to the measurement model nonlinearities, online state estimation in TBD is most commonly solved via particle filtering. Existing particle filters for TBD do not incorporate measurement information in their proposal distribution. The Langevin Monte Carlo (LMC) is a sampling method whose proposal is able to exploit all available knowledge of the posterior (that is, both prior and measurement information). This letter synthesizes recent advances in differential-geometric LMC-based filtering to introduce its application to TBD. The benefits of LMC filtering in TBD are illustrated in a challenging low-noise scenario.

Sampling from a Log-Concave Distribution with Projected Langevin Monte Carlo

We extend the Langevin Monte Carlo (LMC) algorithm to compactly supported measures via a projection step, akin to projected stochastic gradient descent (SGD). We show that (projected) LMC allows to sample in polynomial time from a log-concave distribution with smooth potential. This gives a new Markov chain to sample from a log-concave distribution. Our main result shows in particular that when the target distribution is uniform, LMC mixes in $$\widetilde{O}(n^7)$$ steps (where n is the dimension). We also provide preliminary experimental evidence that LMC performs at least as well as hit-and-run, for which a better mixing time of $$\widetilde{O}(n^4)$$ was proved by Lovász and Vempala.

Annealed Importance Sampling for Neural Mass Models.

Neural Mass Models provide a compact description of the dynamical activity of cell populations in neocortical regions. Moreover, models of regional activity can be connected together into networks, and inferences made about the strength of connections, using M/EEG data and Bayesian inference. To date, however, Bayesian methods have been largely restricted to the Variational Laplace (VL) algorithm which assumes that the posterior distribution is Gaussian and finds model parameters that are only locally optimal. This paper explores the use of Annealed Importance Sampling (AIS) to address these restrictions. We implement AIS using proposals derived from Langevin Monte Carlo (LMC) which uses local gradient and curvature information for efficient exploration of parameter space. In terms of the estimation of Bayes factors, VL and AIS agree about which model is best but report different degrees of belief. Additionally, AIS finds better model parameters and we find evidence of non-Gaussianity in their posterior distribution.