Entropy Convergence Research Articles

Using Entropy as the Convergence Criteria of Ant Colony Optimization and the Application at Gene Chip Data Analysis.

When Ant Colony Optimization algorithm (ACO) is adept at identifying the shortest path, the temporary solution is uncertain during the iterative process. All temporary solutions form a solution set. Where each solution is random. That is, the solution set has entropy. When the solution tends to be stable, the entropy also converges to a fixed value. Therefore, it was proposed in this paper that apply entropy as a convergence criterion of ACO. The advantage of the proposed criterion is that it approximates the optimal convergence time of the algorithm. In order to prove the superiority of the entropy convergence criterion, it was used to cluster gene chip data, which were sampled from patients of Alzheimer's Disease (AD). The clustering algorithm is compared with six typical clustering algorithms. The comparison shows that the ACO using entropy as a convergence criterion is of good quality. At the same time, applying the presented algorithm, we analyzed the clustering characteristics of genes related to energy metabolism and found that as AD occurs, the entropy of the energy metabolism system decreases; that is, the system disorder decreases significantly.

Novel insights into the problem of enthalpy and entropy convergence in thermal decomposition of coal slag using the data from non-isothermal kinetic measurements

This study provides insight into benefits of thermo-chemical conversion of coal slag as recovery process into value-added products. This research involves kinetic analysis of process conducted through non-isothermal thermal analysis measurements, with additional raw material characterization. Kinetic results showed that decomposition proceeds through two consecutive reactions steps (first one, including anorthite P1̅ → I1̅ phase transition, and then production of incongruent melting product (ternary system: CaO·Al2O3·2SiO2 (CAS2), where viscosity of slag changes), and second one including dehydration and formation of meta-muscovite, and subsequently, thermal disruption of muscovite de-hydroxylated phase, which proceeds with breaking of octahedral Al–O bonds), and one single-step reaction (attributed to CO-reduction of hematite to magnetite). Thermodynamic results showed an existence of physically meaningful isokinetic temperature (Tiso), which corresponds to active vibrational frequency of surroundings of SiO2 reaction site, manifested through Si‒O bond weakening by catalytic reaction of freed hydroxide ion (OH−). It was concluded that at temperature T = Tiso, the course of process loses its dependence on temperature and pressure, regulating changes between thermodynamic parameters, through enthalpy-entropy compensation (EEC) effect.

Convergence of Relative Entropy for Euler-Maruyama Scheme to Stochastic Differential Equations with Additive Noise.

For a family of stochastic differential equations driven by additive Gaussian noise, we study the asymptotic behaviors of its corresponding Euler-Maruyama scheme by deriving its convergence rate in terms of relative entropy. Our results for the convergence rate in terms of relative entropy complement the conventional ones in the strong and weak sense and induce some other properties of the Euler-Maruyama scheme. For example, the convergence in terms of the total variation distance can be implied by Pinsker's inequality directly. Moreover, when the drift is β(0<β<1)-Hölder continuous in the spatial variable, the convergence rate in terms of the weighted variation distance is also established. Both of these convergence results do not seem to be directly obtained from any other convergence results of the Euler-Maruyama scheme. The main tool this paper relies on is the Girsanov transform.

Research on the Source Convergence Diagnose Method of Monte Carlo Critical Calculation for Loosely Coupled System

In order to improve the reliability and accuracy of Monte Carlo program in the critical safety calculation of loosely coupled systems, an improved Shannon entropy diagnose method considering the distribution of fission sources and the statistical bias is proposed in this paper. The relative deviation of the distribution of fission sources between adjacent generations and the standard deviation of fission source is proposed. The standard deviation of fission source distribution during source iteration are used as the weight of fission source distribution. Finally, an improved Shannon entropy diagnose method considering the distribution of fission sources and the statistical bias is proposed. This improved Shannon entropy convergence index makes up for the shortcomings of traditional Shannon entropy for lacking the details of local fission source. The improved Shannon entropy convergence index is applied to the benchmark problem: loosely coupled solution Slabs. The results show that compared with the traditional Shannon entropy index, this improved Shannon entropy is more sensitive to the iterative convergence process and can more intuitively and accurately determine the convergence of fission source distribution accompanied by source iteration.

Probability Distributions Approximation via Fractional Moments and Maximum Entropy: Theoretical and Computational Aspects

In the literature, the use of fractional moments to express the available information in the framework of maximum entropy (MaxEnt) approximation of a distribution F having finite or unbounded positive support, has been essentially considered as a computational tool to improve the performance of the analogous procedure based on integer moments. No attention has been paid to two formal aspects concerning fractional moments, such as conditions for the existence of the maximum entropy approximation based on them or convergence in entropy of this approximation to F. This paper aims to fill this gap by providing proofs of these two fundamental results. In fact, convergence in entropy can be involved in the optimal selection of the order of fractional moments for accelerating the convergence of the MaxEnt approximation to F, to clarify the entailment relationships of this type of convergence with other types of convergence useful in statistical applications, and to preserve some important prior features of the underlying F distribution.

On the Convergence of α Schemes with Source Terms for Scalar Convex Conservation Laws

In this study, we use an extension of Yang’s convergence criterion [N. Jiang, On the wavewise entropy inequality for high-resolution schemes with source terms II: the fully discrete case] to show the entropy convergence of a class of fully discrete α schemes, now with source terms, for non-homogeneous scalar convex conservation laws in the one-dimensional case. The homogeneous counterparts (HCPs) of these schemes were constructed by S. Osher and S. Chakravarthy in the mid-1980s [A New Class of High Accuracy TVD Schemes for Hyperbolic Conservation Laws (1985), Very High Order Accurate TVD Schemes (1986)], and the entropy convergence of these methods, when m=2, was settled by the author [N. Jiang, The Convergence of α Schemes for Conservation Laws II: Fully-Discrete]. For semi-discrete α schemes, with or without source terms, the entropy convergence of these schemes was previously established (for m=2) by the author [N. Jiang, The Convergence of α Schemes for Conservation Laws I: Semi-Discrete Case].

Dynamic Pattern and Evolution Trend of the New Four Modernizations Synchronous Development in China: An Analysis Based on Panel Data from 31 Provinces

The New Four Modernizations (NFM) synchronous development strategy proposed by the Chinese government is an important part of Chinese modernization and is of great significance in promoting the sustainable and high-quality development of the social economy. This paper aims to provide a framework for analyzing the dynamic evolution pattern of China’s NFM synchronous development. We expounded on the theoretical basis of China’s NFM synchronous development strategy. Then, the entropy method and convergence model were used to analyze the dynamic pattern and evolution trend of the NFM synchronous development in China. The findings are as follows: (1) In terms of the current situation of NFM development, China’s new urbanization and industrialization were in the middle stage, while informationization and agricultural modernization were in the initial stage of rapid development. (2) The level of China’s NFM synchronous development continued to improve, but the problem of unbalanced, uncoordinated, and unsynchronized development remained prominent. There were great differences in the development levels of China’s new urbanization, industrialization, informationization, and agricultural modernization, which showed that there was a problem of unbalanced and unsynchronized development within China’s NFM. On the other hand, the synchronous development level of NFM in the eastern, central, and western regions of China showed a spatial pattern of decline although the eastern region was clearly higher than the central and western regions; this revealed that the regional development of NFM in China was unbalanced and uncoordinated. (3) China’s NFM synchronous development has significant σ-convergence and β-convergence, indicating that backward regions are catching up and regional differences will gradually narrow in the future. This study is helpful in understanding the current situation as well as the dynamic pattern and future evolution trend of NFM synchronous development in emerging countries such as China.

Entropic Central Limit Theorem for Order Statistics

It is well known that central order statistics exhibit a central limit behavior and converge to a Gaussian distribution as the sample size grows. This paper strengthens this known result by establishing an entropic version of the central limit theorem that ensures a stronger mode of convergence using the relative entropy. This upgrade in convergence is shown at the expense of extra regularity conditions, which can be considered as mild. To prove this result, ancillary results on order statistics are derived, which might be of independent interest. For instance, a rather general bound on the moments of order statistics, and an upper bound on the mean squared error of estimating the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p \in (0,1)$ </tex-math></inline-formula> -th quantile of an unknown cumulative distribution function, are derived. Finally, a discussion on the necessity of the derived conditions for convergence and on the rate of convergence and monotonicity of the relative entropy is provided.

Entropy-stable Gauss collocation methods for ideal magneto-hydrodynamics

In this paper, we present an entropy-stable Gauss collocation discontinuous Galerkin (DG) method on 3D curvilinear meshes for the GLM-MHD equations: the single-fluid magneto-hydrodynamics (MHD) equations with a generalized Lagrange multiplier (GLM) divergence cleaning mechanism. For the continuous entropy analysis to hold and to ensure Galilean invariance in the divergence cleaning technique, the GLM-MHD system requires the use of non-conservative terms.Traditionally, entropy-stable DG discretizations have used a collocated nodal variant of the DG method, also known as the discontinuous Galerkin spectral element method (DGSEM) on Legendre-Gauss-Lobatto (LGL) points. Recently, Chan et al. [1, “Efficient Entropy Stable Gauss Collocation Methods”. SIAM (2019)] presented an entropy-stable DGSEM scheme that uses Legendre-Gauss points (instead of LGL points) for conservation laws. Our main contribution is to extend the discretization technique of Chan et al. to the non-conservative GLM-MHD system.We provide a numerical verification of the entropy behavior and convergence properties of our novel scheme on 3D curvilinear meshes. Moreover, we test the robustness and accuracy of our scheme with a magneto-hydrodynamic Kelvin-Helmholtz instability problem. The numerical experiments suggest that the entropy-stable DGSEM on Gauss points for the GLM-MHD system is more accurate than the LGL counterpart.

Stability of Logarithmic Sobolev Inequalities Under a Noncommutative Change of Measure

We generalize Holley-Stroock's perturbation argument from commutative to quantum Markov semigroups. As a consequence, results on (complete) modified logarithmic Sobolev inequalities and logarithmic Sobolev inequalities for self-adjoint quantum Markov process can be used to prove estimates on the exponential convergence in relative entropy of quantum Markov systems which preserve a fixed state. This leads to estimates for the decay to equilibrium for coupled systems and to estimates for mixed state preparation times using Lindblad operators. Our techniques also apply to discrete time settings, where we show that the strong data processing inequality constant of a quantum channel can be controlled by that of a corresponding unital channel.

Entropy convergence in early bilinguals' syntactic packaging.

A core question in developmental and cognitive research concerns the way linguistic variation affects the acquisition process. Previous research on monolinguals suggests that children, but not adults, tend to regularize inconsistent input, resulting in reduced variation. Some recent claims explain regularization as a general bias linked to cognitive load. However, little is known about bilingual acquisition contexts where children naturally experience both increased variability and cognitive load. This study investigated the impact of between- and within-language variability in syntactic packaging (i.e., how semantic elements are mapped onto syntactic units) on simultaneous bilinguals’ acquisition of motion event encoding. In this domain, French is considered highly variable, in contrast to low variability demonstrated by English. Based on this crosslinguistic contrast, 96 English–French bilingual children (aged 4–11 years) and 96 age-matched monolinguals of each language described 32 animated cartoons showing complex motion events. Children’s variability of selected syntactic patterns was measured using the information-theoretical concept of entropy. Results indicated that bilingual children significantly reduced syntactic variation relative to monolingual peers, but only in French, the more variable language. Moreover, bilingual children converged in entropy levels across the two languages and patterned mid-way between respective monolinguals. These findings suggest that the cognitive load inherent in bilingualism is not sufficient to explain reduced linguistic variation. Instead, the asymmetric drop in entropy highlights the strong impact of crosslinguistic differences and thus underlines the importance of taking language-specific factors into account in theories of cognitive load.

Uniform Poincaré and logarithmic Sobolev inequalities for mean field particle systems

In this paper we consider a mean field particle systems whose confinement potentials have many local minima. We establish some explicit and sharp estimates of the spectral gap and logarithmic Sobolev constants uniform in the number of particles. The uniform Poincaré inequality is based on the work of Ledoux (In Séminaire de Probabilités, XXXV (2001) 167–194, Springer) and the uniform logarithmic Sobolev inequality is based on Zegarlinski’s theorem for Gibbs measures, both combined with an explicit estimate of the Lipschitz norm of the Poisson operator for a single particle from (J. Funct. Anal. 257 (2009) 4015–4033). The logarithmic Sobolev inequality then implies the exponential convergence in entropy of the McKean–Vlasov equation with an explicit rate, We need here weaker conditions than the results of (Rev. Mat. Iberoam. 19 (2003) 971–1018) (by means of the displacement convexity approach), (Stochastic Process. Appl. 95 (2001) 109–132; Ann. Appl. Probab. 13 (2003) 540–560) (by Bakry–Emery’s technique) or the recent work (Arch. Ration. Mech. Anal. 208 (2013) 429–445) (by dissipation of the Wasserstein distance).

Convergence criterion for quantum relative entropy and its use

A criterion and necessary conditions for the convergence (local continuity) of quantum relative entropy are obtained. Some applications of these results are considered. In particular, the preservation of the local continuity of quantum relative entropy under completely positive linear maps is established. Bibliography: 29 titles.

Bridging Gaussian Density Fluctuations from Microscopic to Macroscopic Volumes: Applications to Non-Polar Solute Hydration Thermodynamics.

The hydration of hydrophobic solutes is intimately related to the spontaneous formation of cavities in water through ambient density fluctuations. Information theory-based modeling and simulations have shown that water density fluctuations in small volumes are approximately Gaussian. For limiting cases of microscopic and macroscopic volumes, water density fluctuations are known exactly and are rigorously related to the density and isothermal compressibility of water. Here, we develop a theory—interpolated gaussian fluctuation theory (IGFT)—that builds an analytical bridge to describe water density fluctuations from microscopic to molecular scales. This theory requires no detailed information about the water structure beyond the effective size of a water molecule and quantities that are readily obtained from water’s equation-of-state—namely, the density and compressibility. Using simulations, we show that IGFT provides a good description of density fluctuations near the mean, that is, it characterizes the variance of occupancy fluctuations over all solute sizes. Moreover, when combined with the information theory, IGFT reproduces the well-known signatures of hydrophobic hydration, such as entropy convergence and solubility minima, for atomic-scale solutes smaller than the crossover length scale beyond which the Gaussian assumption breaks down. We further show that near hydrophobic and hydrophilic self-assembled monolayer surfaces in contact with water, the normalized solvent density fluctuations within observation volumes depend similarly on size as observed in the bulk, suggesting the feasibility of a modified version of IGFT for interfacial systems. Our work highlights the utility of a density fluctuation-based approach toward understanding and quantifying the solvation of non-polar solutes in water and the forces that drive them toward surfaces with different hydrophobicities.

Finite difference modified WENO schemes for hyperbolic conservation laws with non‐convex flux

AbstractThe Weighted Essentially Non‐Oscillatory (WENO) reconstruction provides higher‐order accurate solutions to hyperbolic conservation laws for convex flux. But it fails to capture composite structure in the case of non‐convex flux and converges to the wrong solution (Qiu &amp; Shu SIAM J Sci Comput. 2008;31:584‐607). In this article, we have developed a Modified WENO (MWENO) scheme in the finite difference framework, which can resolve the composite structure and ensures the entropic convergence. The MWENO reconstruction procedure involves the identification of the troubled‐cells, followed by the use of first‐order monotone modification in the troubled‐cells and employ the fifth‐order WENO reconstruction in the non‐troubled cells. A new troubled‐cell indicator is developed using the information of the smoothness indicator of the WENO reconstruction. Numerical experiments are performed for 1D and 2D test cases, which ensure the entropic convergence of the proposed schemes.

Free energy convergence in short- and long-length hydrophobic hydration

We show, through molecular dynamics, that the hydration free energy of model hydrocarbons, exhibits a convergence temperature in supercooled water, TG∗ ~ 255 K ± 10 K, around which, hydration entropy and enthalpy minima are observed. Further, although entropy convergence at TS∗ ~ 400 K ± 10 K is found to be violated by large non-spherical hydrocarbons, due to “folding”, a free energy maximum is still observed, opposite to hard spheres. Evidence is provided that the entropy and enthalpy minima are associated with shell-bulk water structural differences, also observed between low-density liquid and high-density liquid water, around TG∗, specifically, the occurrence of an orientational order maximum.

On the Diffusive-Mean Field Limit for Weakly Interacting Diffusions Exhibiting Phase Transitions

The objective of this article is to analyse the statistical behaviour of a large number of weakly interacting diffusion processes evolving under the influence of a periodic interaction potential. We focus our attention on the combined mean field and diffusive (homogenisation) limits. In particular, we show that these two limits do not commute if the mean field system constrained to the torus undergoes a phase transition, that is to say, if it admits more than one steady state. A typical example of such a system on the torus is given by the noisy Kuramoto model of mean field plane rotators. As a by-product of our main results, we also analyse the energetic consequences of the central limit theorem for fluctuations around the mean field limit and derive optimal rates of convergence in relative entropy of the Gibbs measure to the (unique) limit of the mean field energy below the critical temperature.

Residual entropy of twisted and helical ice nanotubes

To study the properties of ice nanotubes, the exact statistics of proton disorder are of interest. In this paper, a new version of the transfer-matrix method is applied to compute the number of defect-free configurations in twisted and helical ice nanotubes. For the calculation of the transfer matrices themselves, this new version of the transfer-matrix method uses small conditional matrices of size 2 x 2 or 4 x 4. For different types of nanotubes, the number of proton configurations in the unit cells of different lengths and the asymptotic values of the residual entropy are presented. For wide tubes, the convergence of the residual entropy to the entropy of the well-known two-dimensional ice model is discussed.

Exponential convergence in entropy and Wasserstein for McKean–Vlasov SDEs

The following type of exponential convergence is proved for (non-degenerate or degenerate) McKean–Vlasov SDEs: W2(μt,μ∞)2+Ent(μt|μ∞)≤ce−λtmin{W2(μ0,μ∞)2,Ent(μ0|μ∞)},t≥1, where c,λ>0 are constants, μt is the distribution of the solution at time t, μ∞ is the unique invariant probability measure, Ent is the relative entropy and W2 is the L2-Wasserstein distance. In particular, this type of exponential convergence holds for some (non-degenerate or degenerate) granular media type equations generalizing those studied in Carrillo et al. (2003) and Guillin et al. (0000) on the exponential convergence in a mean field entropy.

Convexity and robustness of the Rényi entropy

We study convexity properties of the Rényi entropy as function of $\alpha &gt;0$ on finite alphabets. We also describe robustness of the Rényi entropy on finite alphabets, and it turns out that the rate of respective convergence depends on initial alphabet. We establish convergence of the disturbed entropy when the initial distribution is uniform but the number of events increases to ∞ and prove that the limit of Rényi entropy of the binomial distribution is equal to Rényi entropy of the Poisson distribution.