Langevin Dynamics Method Research Articles

LL-Diff: Low-Light Image Enhancement Utilizing Langevin Sampling Diffusion

In this paper, we propose a new algorithm called LL-Diff, which is innovative compared to traditional augmentation methods in that it introduces the sampling method of Langevin dynamics. This sampling approach simulates the motion of particles in complex environments and can better handle noise and details in low-light conditions. We also incorporate a causal attention mechanism to achieve causality and address the issue of confounding effects. This attention mechanism enables us to better capture local information while avoiding over-enhancement. We have conducted experiments on the LOL-V1 and LOL-V2 datasets, and the results show that LL-Diff significantly improves computational speed and several evaluation metrics, demonstrating the superiority and effectiveness of our method for low-light image enhancement tasks. The code will be released on GitHub when the paper has been accepted.

Hyperspectral Image Denoising by Pixel-Wise Noise Modeling and TV-Oriented Deep Image Prior

Model-based hyperspectral image (HSI) denoising methods have attracted continuous attention in the past decades, due to their effectiveness and interpretability. In this work, we aim at advancing model-based HSI denoising, through sophisticated investigation for both the fidelity and regularization terms, or correspondingly noise and prior, by virtue of several recently developed techniques. Specifically, we formulate a novel unified probabilistic model for the HSI denoising task, within which the noise is assumed as pixel-wise non-independent and identically distributed (non-i.i.d) Gaussian predicted by a pre-trained neural network, and the prior for the HSI image is designed by incorporating the deep image prior (DIP) with total variation (TV) and spatio-spectral TV. To solve the resulted maximum a posteriori (MAP) estimation problem, we design a Monte Carlo Expectation–Maximization (MCEM) algorithm, in which the stochastic gradient Langevin dynamics (SGLD) method is used for computing the E-step, and the alternative direction method of multipliers (ADMM) is adopted for solving the optimization in the M-step. Experiments on both synthetic and real noisy HSI datasets have been conducted to verify the effectiveness of the proposed method.

Micro-scale simulation of adsorption-diffusion behaviors of nanoparticles onto mobile oil/water interface

A novel, hybrid pore-scale simulation method using Lattice-Boltzmann (LB) coupled with Langevin-Dynamics (LD) is proposed to investigate the physics of nanoparticles onto oil/water interface. Based on the LB method, the high-resolution characterization of oil-water interface is established. Then, in a fashion of discrete LB forcing source distribution, the LD method is introduced to characterize the effects of Brownian motion, thermal fluctuation-dissipation, multi-body hydrodynamics, and particle-particle interactions. By the means of the new LB-LD coupling model, the adsorption and diffusion characterization of nanoparticles onto oil/water interface are investigated. Moreover, by introducing the interference coefficient and non-equilibrium time, a modified Langmuir adsorption equation is first established by more accurately quantifying the adsorption characterization of nanoparticles and the consequent impacts onto the oil/water interfacial force (IFT), which cannot be accurately described by the classical Langmuir model. For the example of SiO2 nanoparticles adsorption, it is observed that small-size nanofluids with high concentration could accelerate the adsorption of nanoparticles. In addition, both the lateral and longitudinal diffusion coefficients of nanoparticles in the water phase and onto interface are obtained, and of which the underlying mechanisms are explained by introducing residual oil. The proposed simulation method provides valuable insights into how nanoparticles adsorb and diffuse onto oil/water interface and how oil/water IFT can be reduced.

Calculation of Concentration Distribution of Steady Flow Aerosol by Langevin Dynamics Method

Langevin dynamics (LD) method is a modelling technique using pseudo-random numbers to reproduce the acceleration of aerosol particles due to random collisions of gas molecules onto the particles. Although the LD method can reproduce the Brownian motion of particles and motion due to inertial force and/or external force, it is difficult to understand the behavior of aerosol, which is group of particles, from the information on the motion of individual particles. In this study, a novel calculation method was developed to obtain the concentration distribution of aerosol by accumulating the information of position of particles calculated by the LD method, and it was applied to the diffusion of aerosol flowing steadily in a cylindrical tube. The concentration distribution obtained by this method agreed well with that obtained by solving the conventional convective-diffusion equation. Furthermore, change in cup-mixing concentration of aerosol in axial direction calculated from the concentration distribution agreed well also with the penetration expressed by Gormley & Kennedy.

Deep Bayesian inference for seismic imaging with tasks

We use techniques from Bayesian inference and deep neural networks to translate uncertainty in seismic imaging to uncertainty in tasks performed on the image, such as horizon tracking. Seismic imaging is an ill-posed inverse problem because of bandwidth and aperture limitations, which are hampered by the presence of noise and linearization errors. Many regularization methods, such as transform-domain sparsity promotion, have been designed to deal with the adverse effects of these errors; however, these methods run the risk of biasing the solution and do not provide information on uncertainty in the image space and how this uncertainty impacts certain tasks on the image. A systematic approach is developed to translate uncertainty due to noise in the data to the confidence intervals of automatically tracked horizons in the image. The uncertainty in the seismic image is characterized by a convolutional neural network (CNN) that is used to reparameterize the image. To assess these uncertainties, samples are drawn from the posterior distribution of the CNN weights. Compared with traditional priors, it is argued in the literature that these CNNs introduce a flexible inductive bias that is a surprisingly good fit for a diverse set of problems, including medical imaging, compressive sensing, and diffraction tomography. The method of stochastic gradient Langevin dynamics is used to sample from the posterior distribution. This method is designed to handle large-scale Bayesian inference problems with computationally expensive forward operators as in seismic imaging. Aside from offering a robust alternative to the maximum a posteriori estimate that is prone to overfitting, access to these samples allows us to translate uncertainty in the image, due to noise in the data, to uncertainty on the tracked horizons. For instance, it admits estimates for the pointwise standard deviation on the image and for confidence intervals on its automatically tracked horizons.

Multi-variance replica exchange SGMCMC for inverse and forward problems via Bayesian PINN

Physics-informed neural network (PINN) has been successfully applied in solving a variety of nonlinear non-convex forward and inverse problems. However, the training is challenging because of the non-convex loss functions and the multiple optima in the Bayesian inverse problem. In this work, we propose a multi-variance replica exchange stochastic gradient Langevin dynamics method to tackle the challenge of the multiple local optima in the optimization and the challenge of the multiple modal posterior distribution in the inverse problem. Replica exchange methods are capable of escaping from the local traps and accelerating the convergence; two chains with different temperatures are designed where the low temperature chain aims for the local convergence, and the target of the high temperature chain is to travel globally and explore the whole loss function entropy landscape. However, it may not be efficient to solve mathematical inversion problems by using the vanilla replica method directly since the method doubles the computational cost in evaluating the forward solvers (likelihood functions) in the two chains. To address this issue, we propose to make different assumptions on the energy function estimation and this facilities one to use solvers of different fidelities in the likelihood function evaluation. More precisely, one can use a solver with low fidelity in the high temperature chain while using a solver with high fidelity in the low temperature chain. Our proposed method significantly lowers the computational cost in the high temperature chain, meanwhile preserving the accuracy and converging very fast. We give an unbiased estimate of the swapping rate and give an estimation of the discretization error of the scheme. To verify our idea, we design and solve four inverse problems which have multiple modes. The proposed method is also employed to train the Bayesian PINN to solve the forward and inverse problems; faster and more accurate convergence has been observed when compared to the stochastic gradient Langevin dynamics (SGLD) method and vanilla replica exchange methods.

Computer Simulations of Dynamic Response of Ferrofluids on an Alternating Magnetic Field with High Amplitude

The response of ferrofluids to a high-amplitude AC magnetic field is important for several applications including magnetic hyperthermia and biodetection. In computer simulations of the dynamic susceptibility of a ferrofluid outside the linear response region, there are several problems associated with the fact that an increase in the frequency of the AC field leads to the appearance of additional computational errors, which can even lead to unphysical results. In this article, we study the dependence of the computational error arising in the computer simulation of the dynamic susceptibility on the input parameters of the numerical algorithm: the length of the time step, the total number of computer simulation periods, and averaging period. Computer simulation is carried out using the Langevin dynamics method and takes Brownian rotational relaxation of magnetic particles and interparticle interactions into account. The reference theory [Yoshida T.; Enpuku K. Jap. J. Ap. Phys. 2009] is used to estimate computational error. As a result, we give practical recommendations for choosing the optimal input parameters of the numerical algorithm, which make it possible to obtain reliable results of the dynamic susceptibility of a ferrofluid in a high-amplitude AC field in a wide frequency range.

Tutorial: Langevin Dynamics methods for aerosol particle trajectory simulations and collision rate constant modeling

The Langevin Dynamics (LD) method (also known in the literature as Brownian Dynamics) is routinely used to simulate aerosol particle trajectories for transport rate constant calculations as well as to understand aerosol particle transport in internal and external fluid flows. This tutorial intends to explain the methodological details of setting up a LD simulation of a population of aerosol particles and to deduce rate constants from an ensemble of classical trajectories. We discuss the applicability and limitations of the translational Langevin equation to model the combined stochastic and deterministic motion of particles in fields of force or fluid flow. The drag force and stochastic “diffusion” force terms that appear in the Langevin equation are discussed elaborately, along with a summary of common forces relevant to aerosol systems (electrostatic, gravity, van der Waals, …); a commonly used first order and a fourth order Runge-Kutta time stepping schemes for linear stochastic ordinary differential equations are presented. A MATLAB® implementation of a LD computer code for simulating particle settling under gravity using the first order scheme is included for illustration. Scaling analysis of aerosol transport processes and the selection of timestep and domain size for trajectory simulations are demonstrated through two specific aerosol processes: particle diffusion charging and coagulation. Fortran® implementations of the first order and fourth order time-stepping schemes are included for simulating the 3D motion of a particle in a periodic domain. Potential applications and caveats to the usage of LD are included as a summary.

Dynamic PET Image Denoising Using Deep Image Prior Combined With Regularization by Denoising

The quantitative accuracy of positron emission tomography (PET) is affected by several factors, including the intrinsic resolution of the imaging system and inherently noisy data, which result in a low signal-to-noise ratio (SNR) of PET image. To address this problem, in this paper, we proposed a novel deep learning denoising framework aiming to enhance the quantitative accuracy of dynamic PET images via introduction of deep image prior (DIP) combined with Regularization by Denoising (RED), as such the method is labeled as DeepRED denoising. The network structure is based on encoder-decoder architecture and uses skip connections to combine hierarchical features to generate the estimated image. The network input can be random noise or other prior images (such as the patient’s own static PET image), avoiding the need of high quality noiseless images, which is limited in PET clinical practice due to high radiation dose. Based on simulated data and real patient data, the quantitative performance of the proposed method was compared with conventional Gaussian filtering (GF), non-local mean (NLM), block-matching and 3D filtering (BM3D), DIP and stochastic gradient Langevin dynamics (SGLD) method. Overall, the proposed method can outperform other conventional methods in substantial visual as well as quantitative accuracy improvements (in terms of noise versus bias performance) with and without prior images.

Influence of the magnetic nanoparticle coating on the magnetic relaxation time.

Colloidal systems consisting of monodomain superparamagnetic nanoparticles have been used in biomedical applications, such as the hyperthermia treatment for cancer. In this type of colloid, called a nanofluid, the nanoparticles tend to agglomeration. It has been shown experimentally that the nanoparticle coating plays an important role in the nanoparticle dispersion stability and biocompatibility. However, theoretical studies in this field are lacking. In addition, the ways in which the nanoparticle coating influences the magnetic properties of the nanoparticles are not yet understood. In order to fill in this gap, this study presents a numerical simulation model that elucidates how the nanoparticle coating affects the nanoparticle agglomeration tendency as well as the effective magnetic relaxation time of the system. To simulate the self-organization of the colloidal nanoparticles, a stochastic Langevin dynamics method was applied based on the effective Verlet-type algorithm. The Néel magnetic relaxation time was obtained via the Coffey method in an oblique magnetic field, adapted to the local magnetic field on a nanoparticle.

Pickering emulsions stabilized with magnetite, gold, and silica nanoparticles: Mathematical modeling and experimental study

Stability of O/W emulsions with magnetite/silica and gold/silica nanoparticles was investigated experimentally and by mathematical modeling using the Langevin dynamics method. Experimentally studied emulsions with magnetite/silica nanoparticles were stable to creaming and coalescence (at pH 8). Results of numerical modeling showed that silica and magnetite nanoparticles aggregated with the formation of chain-like heteroaggregates that were attached to the surface of oil droplets and located into the bulk of the aqueous phase. A gel-like network of aggregated particles was formed that prevented droplet creaming and coalescence.In emulsions with gold/silica nanoparticles, numerical simulations revealed that chain-like heteroaggregates of silica and gold nanoparticles existed for short periods. Over time, these heteroaggregates shrank, and oil droplets were covered with bulky protruding structures of aggregated nanoparticles. The heteroaggregates that were not attached to the oil droplets did not combine to form a gel-like network, so such emulsions were unstable to creaming. However, bulky heteroaggregates on the surface of oil droplets prevented them from approaching and fixed droplets at some distance from each other. This was in agreement with experimental results. The aqueous phase partly separated from emulsions with silica/gold nanoparticles. The oil phase fraction increased in the residual emulsion but was less than the dense packing of droplets.

Performance of the standard exchange-correlation functionals in predicting melting properties fully from first principles: Application to Al and magnetic Ni

We apply the efficient two-optimized references thermodynamic integration using Langevin dynamics method [Phys. Rev. B 96, 224202 (2017)] to calculate highly accurate melting properties of Al and magnetic Ni from first principles. For Ni we carefully investigate the impact of magnetism on the liquid and solid free energies including longitudinal spin fluctuations and the reverse influence of atomic vibrations on magnetic properties. We show that magnetic fluctuations are effectively canceling out for both phases and are thus not altering the predicted melting temperature. For both elements, the generalized gradient approximation (GGA) and the local-density approximation (LDA) are used for the exchange-correlation functional revealing a reliable ab initio confidence interval capturing the respective experimental melting point, enthalpy of fusion, and entropy of fusion.

Multi-level Monte Carlo methods for the approximation of invariant measures of stochastic differential equations

We develop a framework that allows the use of the multi-level Monte Carlo (MLMC) methodology (Giles in Acta Numer. 24:259–328, 2015. https://doi.org/10.1017/S096249291500001X) to calculate expectations with respect to the invariant measure of an ergodic SDE. In that context, we study the (over-damped) Langevin equations with a strongly concave potential. We show that when appropriate contracting couplings for the numerical integrators are available, one can obtain a uniform-in-time estimate of the MLMC variance in contrast to the majority of the results in the MLMC literature. As a consequence, a root mean square error of mathcal {O}(varepsilon ) is achieved with mathcal {O}(varepsilon ^{-2}) complexity on par with Markov Chain Monte Carlo (MCMC) methods, which, however, can be computationally intensive when applied to large datasets. Finally, we present a multi-level version of the recently introduced stochastic gradient Langevin dynamics method (Welling and Teh, in: Proceedings of the 28th ICML, 2011) built for large datasets applications. We show that this is the first stochastic gradient MCMC method with complexity mathcal {O}(varepsilon ^{-2}|log {varepsilon }|^{3}), in contrast to the complexity mathcal {O}(varepsilon ^{-3}) of currently available methods. Numerical experiments confirm our theoretical findings.

Multiscale method based on coupled lattice‐Boltzmann and Langevin‐dynamics for direct simulation of nanoscale particle/polymer suspensions in complex flows

SummaryA hybrid computational method coupling the lattice‐Boltzmann (LB) method and a Langevin‐dynamics (LD) method is developed to simulate nanoscale particle and polymer (NPP) suspensions in the presence of both thermal fluctuation and long‐range many‐body hydrodynamic interactions (HIs). Brownian motion of the NPP is explicitly captured by a stochastic forcing term in the LD method. The LD method is two‐way coupled to the nonfluctuating LB fluid through a discrete LB forcing source distribution to capture the long‐range HI. To ensure intrinsically linear scalability with respect to the number of particles, a Eulerian‐host algorithm for short‐distance particle neighbor search and interaction is developed and embedded to LB‐LD framework. The validity and accuracy of the LB‐LD approach are demonstrated through several sample problems. The simulation results show good agreements with theory and experiment. The LB‐LD approach can be favorably incorporated into complex multiscale computational frameworks for efficiently simulating multiscale multicomponent particulate suspension systems such as complex blood suspensions.

A leap-frog algorithm-based efficient unified thermostat scheme for molecular dynamics

An important challenge for modern molecular simulations is to develop robust algorithms for molecular dynamics (MD), which employ a time interval (i.e., step size) of evolution as large as possible to achieve high efficiency while maintaining accuracy for sampling the distribution of configuration and momentum, e.g., for the canonical ensemble where the number of particles ( N ), the volume ( V ), and the temperature ( T ) are constant. Starting from the velocity-Verlet algorithm, in the recent paper we have proposed a unified scheme—the middle scheme for constructing efficient thermostatting integrators for configurational sampling of the canonical ensemble. In the present paper we extend the middle scheme to derive new thermostatting integrators based on the leap-frog algorithm. The leap-frog middle scheme is able to recover the exact marginal distribution of configuration and that of momentum in the harmonic limit, regardless of the time interval as long as the evolution of the MD trajectory is stable. It is also proved that the leap-frog middle scheme leads to the same marginal distribution of configuration as the velocity- Verlet middle scheme does even for general anharmonic systems. As illustrated in numerical examples (such as a 1-dimensional quartic potential, liquid water at room temperature with or without bond-length constraints, etc.), while both middle schemes perform better than conventional approaches (e.g., the side scheme and the default Langevin dynamics method in AMBER) and can calculate configuration-dependent properties with the same accuracy by using a larger time interval for from at least 4 times to even an order of magnitude, the leap-frog middle scheme produces a more accurate marginal distribution of momentum than that the velocity-Verlet middle scheme does. The sampling efficiency analysis by employing the characteristic correlation time step number (of the potential or of the Hamiltonian) suggests that both middle schemes share the same efficiency in sampling the configuration space and lead to similar efficiency in sampling the whole phase space (i.e., both configuration and momentum). When both only-configuration- dependent and only-momentum-dependent physical observables are considered, we suggest that the leap-frog middle scheme should in principle offer a better approach. When we study properties involving cross-terms of configuration and momentum, e.g., the fluctuation of the Hamiltonian (related to the heat capacity) and the energy current and its fluctuation (related to the thermal conductivity), analytic results in the harmonic limit and numerical data for anharmonic systems indicate that which of the two middle schemes performs better depends on the system and on the specific property. Note that the leap-frog middle scheme is of first-order accuracy for sampling the whole phase space while the velocity-Verlet middle scheme is of second-order accuracy. It is reasonable to choose the velocity-Verlet middle scheme to perform simulations for such properties for general molecular systems. Because either the leap-frog or the velocity-Verlet algorithm is the default integrator in molecular dynamics simulation software packages, the two unified middle schemes can offer novel, simple, accurate, and efficient thermostatting integrators for performing MD (or path integral MD (PIMD)) for the classical (or quantum) canonical ensemble.

How does the packing density of a metal screen affect the mechanism for catching highly charged nanoparticles?

Highly charged aerosol nanoparticles of a molecular size (< 5 nm) exhibit complex behaviors caused by random (Brownian) motion and electrostatic interactions (image force) with solid surfaces. The classical single fiber theory, however, can only partially describe the complex filtration mechanism of highly charged nanoparticles. We began this study by employing the Langevin Dynamics (LD) Method to simulate the random motion and deposition of highly charged nanoparticles on a single fiber. The single fiber efficiencies predicted by the LD simulation did not agree with data from experiments using electrosprayed highly charged molecular ions. To explain this mismatch and search for other factors that could potentially affect the capturing mechanism, we investigated the influence of the mesh structure. While the image force exerted a significant influence as the packing density decreased, the influence tended to oppose the theoretical predictions. These results suggested that the fiber-fiber interaction in a wire screen with high packing density significantly influences the collection of multiply charged molecular ions. A new theoretical model describing the influence of inter-fiber interaction will be required to clarify how highly charged molecular ions are captured by image force.

Efficient approach to compute melting properties fully from ab initio with application to Cu

Applying thermodynamic integration within an ab initio-based free-energy approach is a state-of-the-art method to calculate melting points of materials. However, the high computational cost and the reliance on a good reference system for calculating the liquid free energy have so far hindered a general application. To overcome these challenges, we propose the two-optimized references thermodynamic integration using Langevin dynamics (TOR-TILD) method in this work by extending the two-stage upsampled thermodynamic integration using Langevin dynamics (TU-TILD) method, which has been originally developed to obtain anharmonic free energies of solids, to the calculation of liquid free energies. The core idea of TOR-TILD is to fit two empirical potentials to the energies from density functional theory based molecular dynamics runs for the solid and the liquid phase and to use these potentials as reference systems for thermodynamic integration. Because the empirical potentials closely reproduce the ab initio system in the relevant part of the phase space the convergence of the thermodynamic integration is very rapid. Therefore, the proposed approach improves significantly the computational efficiency while preserving the required accuracy. As a test case, we apply TOR-TILD to fcc Cu computing not only the melting point but various other melting properties, such as the entropy and enthalpy of fusion and the volume change upon melting. The generalized gradient approximation (GGA) with the Perdew-Burke-Ernzerhof (PBE) exchange-correlation functional and the local-density approximation (LDA) are used. Using both functionals gives a reliable ab initio confidence interval for the melting point, the enthalpy of fusion, and entropy of fusion.

Conformational motion correlations in the formation of polypeptide secondary structure in a viscous medium

The Langevin dynamics method and statistical correlation analysis were used to study the α-helical structure folding dynamics of the (Ala)50, (AlaGly)25, and (AlaGly)75 polypeptides depending on the viscosity of the medium. Friction forces that arise when the effective viscosity of the medium is similar to the viscosity of water were found to result in strong correlations between the backbone torsion angles. The polypeptides under study folded mainly to produce α-helical structures. A structure of two contacting α-helices that were approximately equal in length and had a loop between them was observed for a longer chain of 150 residues. A method to visualize the correlation matrix of the dihedral angles of a polypeptide chain was developed for analyzing the effects of the dynamic correlation of conformational degrees of freedom. The analysis of the dynamics of the correlation matrix showed that rotations involving angles of the same type (φ–φ and ψ–ψ) occur predominantly in one direction. Rotations invoving different angles (φ–ψ) occur predominantly in opposite directions, so that the total macromolecule does not rotate. A significant reduction in the effective viscosity of the medium disrupts the correlation and makes the rotations stochastic, thus distorting the formation of the regular (helical) structure. The effects of correlated conformational motions are consequences of viscous friction forces. This conclusion agrees with our previous results that outlined the principle of the minimum rate of energy dissipation and the equipartition of energy dissipation rate between conformational degrees of freedom.

Simulation of self-assembly of polyzwitterions into vesicles.

Using the Langevin dynamics method and a coarse-grained model, we have studied the formation of vesicles by hydrophobic polymers consisting of periodically placed zwitterion side groups in dilute salt-free aqueous solutions. The zwitterions, being permanent charge dipoles, provide long-range electrostatic correlations which are interfered by the conformational entropy of the polymer. Our simulations are geared towards gaining conceptual understanding in these correlated dipolar systems, where theoretical calculations are at present formidable. A competition between hydrophobic interactions and dipole-dipole interactions leads to a series of self-assembled structures. As the spacing d between the successive zwitterion side groups decreases, single chains undergo globule → disk → worm-like structures. We have calculated the Flory-Huggins χ parameter for these systems in terms of d and monitored the radius of gyration, hydrodynamic radius, spatial correlations among hydrophobic and dipole monomers, and dipole-dipole orientational correlation functions. During the subsequent stages of self-assembly, these structures lead to larger globules and vesicles as d is decreased up to a threshold value, below which no large scale morphology forms. The vesicles form via a polynucleation mechanism whereby disk-like structures form first, followed by their subsequent merger.

Consistency and fluctuations for stochastic gradient Langevin dynamics

Applying standard Markov chain Monte Carlo (MCMC) algorithms to large data sets is computationally expensive. Both the calculation of the acceptance probability and the creation of informed proposals usually require an iteration through the whole data set. The recently proposed stochastic gradient Langevin dynamics (SGLD) method circumvents this problem by generating proposals which are only based on a subset of the data, by skipping the accept-reject step and by using decreasing step-sizes sequence (δm)m≥0. We provide in this article a rigorous mathematical framework for analysing this algorithm. We prove that, under verifiable assumptions, the algorithm is consistent, satisfies a central limit theorem (CLT) and its asymptotic bias-variance decomposition can be characterized by an explicit functional of the step-sizes sequence (δm)m≥0. We leverage this analysis to give practical recommendations for the notoriously difficult tuning of this algorithm: it is asymptotically optimal to use a step-size sequence of the type δm = m-1/3, leading to an algorithm whose mean squared error (MSE) decreases at rate O(m-1/3).