Relative Entropy Method Research Articles

Relative Entropy and Optimization-Driven Coarse-Graining Methods in VOTCA.

We discuss recent advances of the VOTCA package for systematic coarse-graining. Two methods have been implemented, namely the downhill simplex optimization and the relative entropy minimization. We illustrate the new methods by coarse-graining SPC/E bulk water and more complex water-methanol mixture systems. The CG potentials obtained from both methods are then evaluated by comparing the pair distributions from the coarse-grained to the reference atomistic simulations. In addition to the newly implemented methods, we have also added a parallel analysis framework to improve the computational efficiency of the coarse-graining process.

Asymptotic analysis of Vlasov-type equations under strong local alignment regime

We consider the hydrodynamic limit of a collisionless and non-diffusive kinetic equation under strong local alignment regime. The local alignment is first considered by Karper, Mellet and Trivisa in [On strong local alignment in the kinetic Cucker–Smale model, in Hyperbolic Conservation Laws and Related Analysis with Applications, Springer Proceedings in Mathematics & Statistics, Vol. 49 (Springer, 2014), pp. 227–242], as a singular limit of an alignment force proposed by Motsch and Tadmor in [A new model for self-organized dynamics and its flocking behavior, J. Statist. Phys. 141 (2011) 923–947]. As the local alignment strongly dominates, a weak solution to the kinetic equation under consideration converges to the local equilibrium, which has the form of mono-kinetic distribution. We use the relative entropy method and weak compactness to rigorously justify the weak convergence of our kinetic equation to the pressureless Euler system.

Long-time Stability in Systems of Conservation Laws, Using Relative Entropy/Energy

We study the long-time stability of shock-free solutions of hyperbolic systems of conservation laws, under an arbitrarily large initial disturbance in L2∩ L∞. We use the relative entropy method, a robust tool which allows us to consider rough and large disturbances. We display practical examples in several space dimensions, for scalar equations as well as isentropic gas dynamics. For full gas dynamics, we use a trick from Chen [1], in which the estimate is made in terms of the relative mechanical energy instead of the relative mathematical entropy.

Asymptotic limits of Navier–Stokes equations with quantum effects

This paper is concerned with the combined incompressible limit and semiclassical limit of the weak solutions to the barotropic quantum Navier–Stokes equations of compressible flows. By using the relative entropy method, we show that for well-prepared initial data, the weak solutions of the compressible quantum Navier–Stokes model converge to the strong solution of the incompressible Navier–Stokes equations as long as the latter exists. Furthermore, the convergence rates are also obtained.

A Unique Relative Entropy‐Based Symmetrical Co‐occurrence Matrix Thresholding with Statistical Spatial Information

Thresholding based on gray-gray cooccurrence matrix is a local thresholding technique. Relative entropy is usually used to gauge the relative difference of uncertainties in two physical systems, and the relative entropy-based asymmetrical co-occurrence matrix thresholding has been applied successfully. We propose to construct symmetrical co-occurrence matrix with the statistical spatial information from the mean values in object and background regions of an image. In this way, a unique relative entropy-based symmetrical co-occurrence matrix thresholding method is derived. Computer-simulation results demonstrated the higher adaptability and efficiency of the proposed method, as compared with square distance based symmetrical co-ocurrence matrix thresholding, Otsu’s and relative entropy methods.

Asymptotic limit to a shock for BGK models using relative entropy method

We consider the hydrodynamic limit of a kinetic BGK model in one space dimension, to a scalar conservation law with a strict convex flux. In this paper, we provide sharp estimates for the asymptotic limit to shocks, where layers, called Knudsen layers, may appear. The used method is based on the relative entropy. It follows a previous work of Choi and Vasseur (arXiv:1210.7853 preprint) where similar sharp estimates were obtained for the inviscid limit of a viscous conservation laws to a shock.

The Inviscid Limit to a Contact Discontinuity for the Compressible Navier--Stokes--Fourier System Using the Relative Entropy Method

We consider the zero heat conductivity limit to a contact discontinuity for the mono-dimensional full compressible Navier-Stokes-Fourier system. The method is based on the relative entropy method, and do not assume any smallness conditions on the discontinuity, nor on the $BV$ norm of the initial data. It is proved that for any viscosity $\nu\geq0$, the solution of the compressible Navier-Stokes-Fourier system (with well prepared initial value) converges, when the heat conductivity $\kappa$ tends to zero, to the contact discontinuity solution to the corresponding Euler system. We obtain the decay rate $\kappa^{\frac12}$. It implies that the heat conductivity dominates the dissipation in the regime of the limit to a contact discontinuity. This is the first result, based on the relative entropy, of an asymptotic limit to a discontinuous solutions for a system.

Short-Time Stability of Scalar Viscous Shocks in the Inviscid Limit by the Relative Entropy Method

We consider inviscid limits to shocks for viscous scalar conservation laws in one space dimension, with strict convex fluxes. We show that we can obtain sharp estimates in $L^2$ for a class of large perturbations and for any bounded time interval. Those perturbations can be chosen big enough to destroy the viscous layer. This shows that the fast convergence to the shock does not depend on the fine structure of the viscous layers. This is the first application of the relative entropy method developed by N. Leger [Arch. Ration. Mech. Anal., 199 (2011), pp. 761--778] and N. Leger and A. Vasseur [Arch. Ration. Mech. Anal., 201 (2011), pp. 271--302] to the study of an inviscid limit to a shock.

Exploring the use of transformation group priors and the method of maximum relative entropy for Bayesian glaciological inversions

AbstractIce-sheet models can be used to forecast ice losses from Antarctica and Greenland, but to fully quantify the risks associated with sea-level rise, probabilistic forecasts are needed. These require estimates of the probability density function (PDF) for various model parameters (e.g. the basal drag coefficient and ice viscosity). To infer such parameters from satellite observations it is common to use inverse methods. Two related approaches are in use: (1) minimization of a cost function that describes the misfit to the observations, often accompanied by explicit or implicit regularization, or (2) use of Bayes’ theorem to update prior assumptions about the probability of parameters. Both approaches have much in common and questions of regularization often map onto implicit choices of prior probabilities that are made explicit in the Bayesian framework. In both approaches questions can arise that seem to demand subjective input. One way to specify prior PDFs more objectively is by deriving transformation group priors that are invariant to symmetries of the problem, and then maximizing relative entropy, subject to any additional constraints. Here we investigate the application of these methods to the derivation of priors for a Bayesian approach to an idealized glaciological inverse problem.

A class of nonergodic interacting particle systems with unique invariant measure

We consider a class of discrete $q$-state spin models defined in terms of a translation-invariant quasilocal specification with discrete clock-rotation invariance which have extremal Gibbs measures $\mu'_{\varphi}$ labeled by the uncountably many values of $\varphi$ in the one-dimensional sphere (introduced by van Enter, Opoku, K\"{u}lske [J. Phys. A 44 (2011) 475002, 11]). In the present paper we construct an associated Markov jump process with quasilocal rates whose semigroup $(S_t)_{t\geq0}$ acts by a continuous rotation $S_t(\mu'_{\varphi})=\mu'_{\varphi +t}$. As a consequence our construction provides examples of interacting particle systems with unique translation-invariant invariant measure, which is not long-time limit of all starting measures, answering an old question (compare Liggett [Interacting Particle Systems (1985) Springer], question four, Chapter one). The construction of this particle system is inspired by recent conjectures of Maes and Shlosman about the intermediate temperature regime of the nearest-neighbor clock model. We define our generator of the interacting particle system as a (noncommuting) sum of the rotation part and a Glauber part. Technically the paper rests on the control of the spread of weak nonlocalities and relative entropy-methods, both in equilibrium and dynamically, based on Dobrushin-uniqueness bounds for conditional measures.

A relative entropy method to measure non-exponential random data

Abstract This paper develops a relative entropy method to measure non-exponential random data in conjunction with fractional order moment, logarithmic moment and tail statistics of Mittag–Leffler distribution. The distribution of non-exponential random data follows neither the exponential distribution nor exponential decay. The proposed strategy is validated by analyzing the experiment data, which are generated by Monte Carlo method using Mittag–Leffler distribution. Compared with the traditional Shannon entropy, the relative entropy method is simple to be implemented, and its corresponding relative entropies approximated by the fractional order moment, logarithmic moment and tail statistics can easily and accurately detect the non-exponential random data.

Hydrodynamic limit of the kinetic Cucker–Smale flocking model

The hydrodynamic limit of a kinetic Cucker–Smale flocking model is investigated. The starting point is the model considered in [Existence of weak solutions to kinetic flocking models, SIAM Math. Anal.45 (2013) 215–243], which in addition to free transport of individuals and a standard Cucker–Smale alignment operator, includes Brownian noise and strong local alignment. The latter was derived in [On strong local alignment in the kinetic Cucker–Smale equation, in Hyperbolic Conservation Laws and Related Analysis with Applications (Springer, 2013), pp. 227–242] as the singular limit of an alignment operator first introduced by Motsch and Tadmor in [A new model for self-organized dynamics and its flocking behavior, J. Stat. Phys.141 (2011) 923–947]. The objective of this work is the rigorous investigation of the singular limit corresponding to strong noise and strong local alignment. The proof relies on a relative entropy method. The asymptotic dynamics is described by an Euler-type flocking system.

Future change of extreme temperature climate indices over East Asia with uncertainties estimation in the CMIP5

How well the climate models simulate extreme temperature over East Asia and how the extreme indices would change under anthropogenic global warming are investigated. The indices studied include hot days (HD), tropical nights (TN), growing degree days (GDD), and cooling degree days (CDD) in summer and heating degree days (HDD) and frost days (FD) in winter. The representative concentration pathway 4.5 (RCP 4.5) experiments for the period of 2075–2099 are compared with historical simulations for the period of 1979–2005 from 15 coupled models that are participated in phase 5 of the Coupled Model Intercomparison Project (CMIP5). To optimally estimate future change and its uncertainty, groups of best models are selected based on Taylor diagrams, relative entropy, and probability density function (PDF) methods previously suggested. Overall, the best models’ multi-model ensemble based on Taylor diagrams has the lowest errors in reproducing temperature extremes in the present climate among three methods. Selected best models in three methods tend to project considerably different changes in the extreme indices from each other, indicating that the selection of reliable models are of critical importance to reduce uncertainties. Three groups of best models show significant increase of summerbased indices but decrease of the winter-based indices. Over East Asia, the most significant increase is seen in the HD (336 ± 23.4% of current climate) and the most significant decrease is appeared in the HDD (82 ± 4.2%). It is suggested that the larger future change in the HD is found over in the Southeastern China region, probably due to a higher local maximum temperature in the present climate. All of the indices show the largest uncertainty over Southeastern China, particularly in the TN (~3.9 times as large as uncertainty over East Asia) and in the HD (~2.4). It is further noted that the TN reveals the largest uncertainty over three East Asian countries (~1.7 and 1.4 over Korea and Japan, respectively). These future changes in extreme temperature events have an important implication for energy-saving applications and human molarity in the future.

Hydrodynamic Limit of Mean Zero Condensing Zero Range Processes with Sub-Critical Initial Profiles

We prove the hydrodynamic limit of mean zero condensing Zero Range Processes with bounded local jump rate, for sub-critical initial profiles. The proof is based on H.T. Yau's relative entropy method and is made possible by a generalisation of the One Block Estimate.

Relative entropy method for fuzzy multiple attribute decision making and its application to software quality evaluation

To study the problem of multiple attribute decision making in which the decision making information values are triangular fuzzy number, a relative entropy decision making method for software qualit...

Reputation in the presence of noisy exogenous learning

This note extends Wiseman [6] to more general reputation games with exogenous learning. Using Gossner's [4] relative entropy method, we provide an explicit lower bound on all Nash equilibrium payoffs of the long-lived player. The lower bound shows that when the exogenous signals are sufficiently noisy and the long-lived player is patient, he can be assured of a payoff strictly higher than his minmax payoff.

Hybrid algorithm of minimum relative entropy-particle swarm optimization with adjustment parameters for gas source term identification in atmosphere

In order to identify the source term of gas emission in atmosphere, an improved hybrid algorithm combined with the minimum relative entropy (MRE) and particle swarm optimization (PSO) method was presented. Not only are the estimated source parameters obtained, but also the confidence intervals at some probability levels. If only the source strength was required to be determined, the problem can be viewed as a linear inverse problem directly, which can be solved by original MRE method successfully. When both source strength and location are unknown, the common gas dispersion model should be transformed to be a linear system. Although the transformed linear model has some differences from that in original MRE method, satisfied estimation results were still obtained by adding iteratively adaptive adjustment parameters in the MRE-PSO method. The dependence of the MRE-PSO method on prior information such as lower and upper bound, prior expected values and noises were also discussed. The results showed that the confidence intervals and estimated parameters are influenced little by the prior bounds and expected values, but the errors affect the estimation results greatly. The simulation and experiment verification results showed that the MRE-PSO method is able to identify the source parameters with satisfied results. Finally, the error model was probed and then it was added in the MRE-PSO method. The addition of error model improves the performance of the identification method. Therefore, the MRE-PSO method with adjustment parameters proposed in this paper is a potential good method to resolve inverse problem in atmosphere environment.

A COMBINATION WEIGHTING ALGORITHM USING RELATIVE ENTROPY FOR DOCUMENT CLUSTERING

This paper proposes a combination weighting algorithm using relative entropy for document clustering. Combination weighting is widely used in multiple attribute decision making (MADM) problem. However, there exist two difficulties to hinder the applications of combination weighting on document clustering. First, combination weighting is based on the integration of subjective weighting and objective weighting. However, there are so many attributes in documents that the subjective weights which rely on manual annotation by experts are impracticable. Secondly, a document data object might contain hundreds or even thousands of features. It is an extremely time-consuming task to calculate the combination weights. To address the issues, we suggest to simplify the combination weighting by not distinguishing subjective weight and objective weight. Meanwhile, we choose relative entropy method to reduce running time. In our algorithm, we obtain a combination weight set with 14 combination forms. The experiments on real document data show that both on the AC/PR/RE measures and the mutual information (MI) measure, the proposed CWRE-sIB algorithm is superior to the original sequential information bottleneck (sIB) algorithm and a series of weighting-sIB algorithms, which are built by applying a single weighting scheme to the original sIB algorithm.

Formula omitted] stability for entropy solutions of scalar conservation laws with strict convex flux

Here we consider the scalar convex conservation laws in one space dimension with strictly convex flux which is in C1. Existence, uniqueness and L1 contractivity were proved by Kružkov [14]. Using the relative entropy method, Leger showed that for a uniformly convex flux and for the shock wave solutions, the L2 norm of a perturbed solution relative to the shock wave is bounded by the L2 norm of the initial perturbation. Here we generalize the result to Lp norm for all 1⩽p<∞. Also we show that for the non-shock wave solution, Lp norm of the perturbed solution relative to the modified N-wave is bounded by the Lp norm of the initial perturbation for all 1⩽p<∞.

Reference-free damage detection for truss bridge structures by continuous relative wavelet entropy method

This article proposes a continuous relative wavelet entropy–based reference-free damage detection algorithm for truss bridge structures. Advantages of the proposed method are that (1) there is no need to measure dynamic response of pristine structures, in other words, the method is reference-free; (2) it is suitable for highly nonlinear and nonstationary random response data due to the multiresolution signal analysis feature of the continuous wavelet transform; and (3) it is sensitive to slight damage extents (i.e. 5%–10%) for the tested damage type (i.e. loosening of bolts). In order to demonstrate consistency and sensitivity of the proposed method, multiple experimental tests using a laboratory-size truss structure were mainly conducted for various damage scenarios and progressive damage states. The proposed continuous relative wavelet entropy–based reference-free damage detection algorithm showed reliable damage localization capabilities, and it is proven as an effective method compared to other damage detection methods that are dependent on the measurement signals from pristine structures. Due to the generality of the proposed method, applications to identify other types of damage based on different types of signals can be expected.