Maximum Information Entropy Principle Research Articles

Synergy Prediction Model of Information Entropy Based on Zone Safety Degree and Stope Roof Weighting Step Analysis

During the underground mining of coal resources, the rock pressure emerges acutely and the mine geological disasters occur frequently. It is of great significance to grasp the manifestation law of rock pressure in time to guide the safety production and operation in the pit. In this research, the calculation equations and concept of information entropy based on zone safety degree are primarily defined, and the synergetic theory of the maximum information entropy principle is combined to put forward the synergy prediction model of information entropy based on zone safety degree. In the meantime, the synergy prediction model of information entropy based on zone safety degree is employed to calculate and predict the first weighting step and the periodic weighting step of the main roof of the 9203 working face of Hengsheng Coal Mine in China’s Shanxi Province, as well as verifying the validity and reliability of the synergy prediction model of information entropy based on zone safety degree by the comparison of similar simulation test results, which has presented a scientific basis for the effective control of rock pressure and roof management.

Synergetic Theory of Information Entropy Based on Failure Approach Index for Stability Analysis of Surrounding Rock System.

It is generally acknowledged that the stability evaluation of surrounding rock denotes nonlinear complex system engineering. In order to accurately and quantitatively assess the safety states of surrounding rock and provide a scientific basis for the prevention and control of surrounding rock stability, the analysis method of the synergetic theory of information entropy using the failure approach index has been proposed. By means of deriving the general relationship between the total two-dimensional plastic shear strain and the total three-dimensional plastic shear strain and obtaining the numerical limit analysis step of the plastic shear strain, the threshold value of the ultimate plastic shear strain can be determined, which has provided the key criterion for the calculation of the information entropy based on the failure approach index. In addition, combining with the synergetic theory of the principle of maximum information entropy, the evolution equation of the excavation step and information entropy based on the failure approach index of the surrounding rock system in underground mining space are established, and the equations of the general solution and particular solution as well as the expression of the destabilizing excavation step are given. To account for this, the method is applied to analyze the failure states of the floor surrounding rock after the mining of the 71 coal seam in Xutuan Coal Mine and involve the disturbance effect and stability control method of the underlying 72 coal seam roof from the macroscopic and microscopic aspects. Consequently, the validity of the analysis method of synergetic theory of information entropy based on the failure approach index has been verified, which presents an updated approach for the stability evaluation of surrounding rock systems that is of satisfactory capability and value in engineering applications.

Дискретная модель эволюции информации сложной системы вблизи точки фазового перехода

Landau's theory of phase transitions and Frenkel's theory of heterophase fluctuations, due to their simplicity and efficiency, have acquired the status of universal theories that have numerous applications in various branches of science and technology. One of the important scientific directions in the development of the theory is the study of changes in information in complex systems. To date, an urgent task is the development of multilevel mathematical models of the dynamics of the transition of a macroscopic system to a new phase state, based on the analysis of changes in Shannon information. The paper proposes a discrete model of the phase transition at the microscopic level. The discrete phase transition model differs from the models used in the Landau and Frenkel theory by a smaller scale, which leads to a greater dependence on random factors. The model is based on the theory of Markov processes and the principle of maximum information or information entropy. The main mathematical apparatus is the Kolmogorov equations. For clarity, Kolmogorov graphs are used. Work results. A stochastic model of the phase transition at the microlevel is constructed. The model remains adequate both near the phase transition point and far from it. An estimate of the intensity of the evolution of the entropy of the system is given. The dynamics of phase change during transitions of the first and second order is obtained. Conditions for phase equilibrium are obtained for phase transitions of the first and second kind. A stochastic interpretation of the phenomenon of hysteresis at the microscopic level is given. The results obtained are in accordance with the principles of Landau and Frenkel's theory. The results of the work have prospects for application in forecasting crises of complex systems based on the dynamics of information changes.

Bayesian post-processing of Monte Carlo simulation in reliability analysis

In reliability analysis with Monte Carlo simulation, the uncertainty about the probability of failure can be formally quantified through Bayesian statistics. Credible intervals for the probability of failure can be derived analytically. This paper gives a detailed overview of Bayesian post-processing for Monte Carlo simulation. We investigate the influence of different weakly-informative prior assumptions on the resulting credible intervals. On this basis, we recommend to use a prior distribution on the probability of failure that follows from the principle of maximum information entropy. We also show that even if no failure sample occurs in a Monte Carlo simulation, Bayesian post-processing still allows to deduce useful information about the probability of failure. The presented Bayesian post-processing strategy can also be applied if Monte Carlo simulation is used for reliability updating; i.e., to evaluate the probability of failure conditional on data or observations. We derive expectations for credible intervals for this case.

Distilling slow process probability density from fast random data

This paper addresses the following question: “Starting from discrete state data collected from a randomly excited dissipative nonlinear system, is it possible to directly distill the probability density of slowly varying processes hidden in it?” The answer turns out to be affirmative! This work provides a data-driven methodology to solve two successive sub-problems: extracting slow processes hidden in fast state data and identifying the probability density of slow process from the data of calculated slow process. The former resorts to a mathematical definition of slow process and an optimization algorithm, while the latter depends on the principle of maximum information entropy. The application and efficacy of this method are demonstrated by three typical nonlinear examples, i.e., two single-degree-of-freedom systems and one two-degree-of-freedom system. This method is equation-free, which circumvents stochastic differential equations and attending traditional methods. Thus, the proposed methodology is especially suitable to complex systems without derivation of associated accurate governing equations.

Agricultural drought vulnerability assessment and diagnosis based on entropy fuzzy pattern recognition and subtraction set pair potential

The assessment and diagnosis of agricultural drought vulnerability is the basis for the scientific control and prevention of agricultural drought risk. In order to quantitatively evaluate the regional agricultural drought vulnerability and effectively deal with the fuzziness and randomness between evaluation sample and evaluation grade, the entropy fuzzy pattern recognition model, which was coupled with maximum information entropy principle and fuzzy pattern recognition method, was applied to the evaluation of agricultural drought vulnerability. Furthermore, in order to find methods to reduce drought vulnerability, the subtraction set pair potential method was used to further identify the main influence factors of agricultural drought vulnerability. In addition, a case study was carried out in Bengbu City, China. The results showed that during the period from 2001 to 2010, the evaluation value of agricultural drought vulnerability in Bengbu dropped from grade IV to III, which meant that the tolerance of agricultural system to drought had been improved. Moreover, the main factors that affected the vulnerability in Bengbu were precipitation, water use efficiency and irrigation protection area rate. And the latter two were the main objects of regulation and control. Compared with the other three methods, the physical concept of drought vulnerability assessment model based on entropy fuzzy pattern recognition is more obvious, and the evaluation results are reasonable and credible. The drought vulnerability diagnosis method based on subtraction set pair potential can accurately identify key influence factor and provide measures to reduce drought vulnerability and improve drought risk management.

Self-consistent equations governing the dynamics of non-equilibrium binary colloidal systems

Most of the existing non-equilibrium theories are developed based on local-equilibrium assumptions and therefore encounter difficulty in addressing dynamical processes far-from-equilibrium. Herein, we present a set of dynamical equations to describe the dynamics of non-equilibrium binary colloidal system, which is derived by combining the Kramers equation with the maximum path information entropy principle. These equations, involving the local density, local momentum and local kinetic energy, are coupled with each other and eventually depend on the two-body probability distribution function, whose least-biased prediction is given by applying the maximum path information entropy principle. We show that the proposed dynamical governing equations are self-consistent, and can recover to the existing relevant theories upon various local equilibrium assumptions. The simplified forms of these equations are also discussed for several types of systems with geometrical symmetries. This work provides a theoretical framework at molecular level for investigating dynamical behaviors of multi-component systems far-from-equilibrium.

Stability of Random Processes with the 1/f α Spectrum

Extreme fluctuations have been modeled based on point nonpotential and potential systems of stochastic equations, in which white noise induces random processes with a power-law frequency dependence of the power spectra. The distribution of extreme fluctuations corresponds to the maximum of statistical entropy, which indicates their stability. The stability of fluctuation processes with 1/f α power spectra has been analyzed based on the principle of maximum information entropy.

Estimation of Precipitation Evolution from Desert to Oasis Using Information Entropy Theory: A Case Study in Tarim Basin of Northwestern China

The cold-wet effect of oasis improves the extreme natural conditions of the desert areas significantly. However, the relationship between precipitation and the width of oasis is challenged by the shortage of observed data. In this study, the evolution of annual precipitation from desert to oasis was explored by the model establishment and simulation in Tarim Basin of northwestern China. The model was developed from the principle of maximum information entropy, and was calibrated by the China Meteorological Forcing Dataset with a high spatial resolution of 0.1° from 1990 to 2010. The model performs well in describing the evolution of annual precipitation from the desert to oasis when the oasis is wide enough, and the R2 is generally more than 0.90 and can be up to 0.99. However, it fails to simulate the seasonal precipitation evolution because of the non-convergence solved by nonlinear fitting and the unfixed upper boundary condition solved by the least square method. Through the simulation with the parameters obtained from the nonlinear fitting, the basic patterns, four stages of precipitation evolution with the oasis width increasing, are revealed at annual scale, and the current stages of these oases are also uncovered. Therefore, the establishment of the model and the simulated results provide a deeper insight from the perspective of informatics to understand the regional precipitation evolution of the desert–oasis system. These results are not only helpful in desertification prevention, but also helpful in fusing multisource data, especially in extreme drought desert areas.

Determination of preferential exponent [formula omitted] in random processes with a [formula omitted] power spectrum

The stability of fluctuation processes with 1∕fα power spectra in systems of stochastic equations simulating coupled phase transitions has been analyzed with the use of the maximum information entropy principle. It is shown that in the class of processes described by a nonpotential system of model equations fluctuations with a 1/f power spectrum, i.e. with the exponent α=1, are preferential, Preferential for the class of potential systems under consideration are fluctuations with a 1∕fα spectrum at α=1.3. The spectral entropy of the random processes has been calculated. Calculation of the spectral entropy makes it possible to investigate the stability of random processes directly from the power spectra, without calculating the probability density functions. The dependence of the spectral entropies on the amplitude of white noise has a minimum whose position corresponds to the critical behavior.

A novel assessment of urbanization quality and its applications

By establishing a network with related indices in urban system as nodes, urbanization quality can be defined as the structural features of the network. According to maximum information entropy principle (MIEP), structural parameter ξ of the index network and its dynamic equation have been derived. The dynamic equation can be calculated by self-organizing feature map (SOFM) artificial neural network algorithm in MATLAB platform. In this way, a new index network model (INM) of urbanization quality assessment (UQA) is constructed. The new INM is applied to the UQA of 37 major cities in China. Values of ξ quantitatively give the rank of the cities, which show that Shenzhen, Beijing and Suzhou have the highest quality scores more than 10 both in 2010 and 2014, reflecting much higher urbanization quality than other cities. The following cities are Xiamen, Tianjin, Nanjing, Hangzhou etc., then Chengdu, Zhengzhou, Shenyang and Tangshan etc. The quality scores of Guiyang, Kunming and Chongqing etc. are the lowest. With Xi’an and Wenzhou as sample cities, the contributions of each index to the urbanization quality are simultaneously revealed. Based on the structural parameter ξ of each mesoscope indexes derived from the new INM, we use the analysis method of factor contribution rate to reveal the contributions of each index to the urbanization quality. By recognizing the key factors, which influence the urbanization quality, the new INM can guide us take targeted measures for the improvement of urbanization quality. Beyond the application cases given, we finally state that the new INM is very universal and advantageous, which can be conveniently applied to the UQA of many other cities.

Information and Self-Organization

The process of “self-organization” takes place in open and complex systems that acquire spatio-temporal or functional structures without specific ordering instructions from the outside. [...]

An information entropy model on clinical assessment of patients based on the holographic field of meridian

Meridian system is not only the basis of traditional Chinese medicine (TCM) method (e.g. acupuncture, massage), but also the core of TCM’s basic theory. This paper has introduced a new informational perspective to understand the reality and the holographic field of meridian. Based on maximum information entropy principle (MIEP), a dynamic equation for the holographic field has been deduced, which reflects the evolutionary characteristics of meridian. By using self-organizing artificial neural network as algorithm, the evolutionary dynamic equation of the holographic field can be resolved to assess properties of meridians and clinically diagnose the health characteristics of patients. Finally, through some cases from clinical patients (e.g. a 30-year-old male patient, an apoplectic patient, an epilepsy patient), we use this model to assess the evolutionary properties of meridians. It is proved that this model not only has significant implications in revealing the essence of meridian in TCM, but also may play a guiding role in clinical assessment of patients based on the holographic field of meridians.

Continuum representation of systems of dislocation lines: A general method for deriving closed-form evolution equations

Plasticity is governed by the evolution of, in general anisotropic, systems of dislocations. We seek to faithfully represent this evolution in terms of density-like variables which average over the discrete dislocation microstructure. Starting from T. Hochrainer's continuum theory of dislocations (CDD) (Hochrainer, 2015), we introduce a methodology based on the ‘Maximum Information Entropy Principle’ (MIEP) for deriving closed-form evolution equations for dislocation density measures of different order. These equations provide an optimum representation of the kinematic properties of systems of curved and connected dislocation lines with the information contained in a given set of density measures. The performance of the derived equations is benchmarked against other models proposed in the literature, using discrete dislocation dynamics simulations as a reference. As a benchmark problem we study dislocations moving in a highly heterogeneous, persistent-slip-band like geometry. We demonstrate that excellent agreement with discrete simulations can be obtained in terms of a very small number of averaged dislocation fields containing information about the edge and screw components of the total and excess (geometrically necessary) dislocation densities. From these the full dislocation orientation distribution which emerges as dislocations move through a channel-wall structure can be faithfully reconstructed.

Maximum information entropy principle and the interpretation of probabilities in statistical mechanics − a short review

In this paper an alternative approach to statistical mechanics based on the maximum information entropy principle (MaxEnt) is examined, specifically its close relation with the Gibbs method of ensembles. It is shown that the MaxEnt formalism is the logical extension of the Gibbs formalism of equilibrium statistical mechanics that is entirely independent of the frequentist interpretation of probabilities only as factual (i.e. experimentally verifiable) properties of the real world. Furthermore, we show that, consistently with the law of large numbers, the relative frequencies of the ensemble of systems prepared under identical conditions (i.e. identical constraints) actually correspond to the MaxEnt probabilites in the limit of a large number of systems in the ensemble. This result implies that the probabilities in statistical mechanics can be interpreted, independently of the frequency interpretation, on the basis of the maximum information entropy principle.

Predictive Statistical Mechanics and Macroscopic Time Evolution: Hydrodynamics and Entropy Production

In the previous papers (Kui\'{c} et al. in Found Phys 42:319-339, 2012; Kui\'{c} in arXiv:1506.02622, 2015), it was demonstrated that applying the principle of maximum information entropy by maximizing the conditional information entropy, subject to the constraint given by the Liouville equation averaged over the phase space, leads to a definition of the rate of entropy change for closed Hamiltonian systems without any additional assumptions. Here, we generalize this basic model and, with the introduction of the additional constraints which are equivalent to the hydrodynamic continuity equations, show that the results obtained are consistent with the known results from the nonequilibrium statistical mechanics and thermodynamics of irreversible processes. In this way, as a part of the approach developed in this paper, the rate of entropy change and entropy production density for the classical Hamiltonian fluid are obtained. The results obtained suggest the general applicability of the foundational principles of predictive statistical mechanics and their importance for the theory of irreversibility.

PRIOR AND POSTERIOR ROBUST STOCHASTIC PREDICTIONS FOR DYNAMICAL SYSTEMS USING PROBABILITY LOGIC

An overview is given of a powerful unifying probabilistic framework for treating modeling uncertainty, along with input uncertainty, when using dynamic models to predict the response of a system during its design or operation. This framework uses probability as a multivalued conditional logic for quantitative plausible reasoning in the presence of uncertainty due to incomplete information. The fundamental probability models that represent the system’s uncertain behavior are specified by the choice of a stochastic system model class: a set of input–output probability models for the system and a prior probability distribution over this set that quantifies the relative plausibility of each model. A model class can be constructed from a parametrized deterministic system model by stochastic embedding which utilizes Jaynes’ principle of maximum information entropy. Robust predictive analyses use the entire model class with the probabilistic predictions of each model being weighted by its prior probability, or if response data are available, by its posterior probability from Bayes’ theorem for the model class. Additional robustness to modeling uncertainty comes from combining the robust predictions of each model class in a set of competing candidates weighted by the prior or posterior probability of the model class, the latter being computed from Bayes’ theorem. This higher-level application of Bayes’ theorem automatically applies a quantitative Ockham razor that penalizes the data-fit of more complex model classes that extract more information from the data. Robust predictive analyses involve integrals over high-dimensional spaces that usually must be evaluated numerically by Laplace’s method of asymptotic approximation or by Markov chain Monte Carlo methods. These computational tools are demonstrated in an illustrative example involving the vertical dynamic response of a car being driven along a rough road.

A Bayesian approach to information fusion for evaluating the measurement uncertainty

The Bayesian approach to uncertainty evaluation is a classical example of the fusion of information from different sources. Basically, it is founded on both the knowledge about the measurement process and the influencing quantities and parameters. The knowledge about the measurement process is primarily represented by the so-called model equation, which forms the basic relationship for the fusion of all involved quantities. The knowledge about the influencing quantities and parameters is expressed by their degree of belief, i.e. appropriate probability density functions that usually are obtained by utilizing the principle of maximum information entropy and the Bayes theorem. Practically, the Bayesian approach to uncertainty evaluation is put into effect by employing numerical integration techniques, preferably Monte-Carlo methods. Compared to the ISO-GUM procedure, the Bayesian approach does not have any restrictions with respect to nonlinearities and calculation of confidence intervals.

Determination of the orientation distribution function in aligned single wall nanotube polymer nanocomposites by polarized Raman spectroscopy

This work focuses on the derivation of the orientation distribution function (ODF) for a uniaxial-axially symmetric system using polarized Raman spectroscopy. A numerical methodology is proposed to determine the ODF that is formulated in terms of Legendre polynomials and the principle of maximum information entropy. The ultimate goal is to quantify the alignment of single wall nanotubes (SWNTs) in a polymer matrix using the experimental information from the Raman intensity. Some of the mathematical and numerical steps in the determination of ODF, not clarified in the current literature, are shown in this work. The proposed numerical methodology to obtain the ODF is illustrated using an experimental case. Electric field–aligned SWNT-urethane dimethacrylate/1,6-hexanediol dimethacrylate nanocomposites are investigated at different processing conditions to bring forward factors that may enhance the alignment of SWNT inclusions in the polymer. ODF results indicate that the higher electric field frequencies produce a good alignment of the SWNT inclusions; a result also corroborated by optical microscopy imaging and electrical conductivity measurements.

Probability distribution of wave velocity in heterogeneous media due to random phase configuration

The paper deals with a class of heterogeneous isotropic elastic materials, which are composed of isotropic elastic components (phases) with the prescribed elastic moduli, mass densities and volume concentrations but random (uncertain) phase configurations. The elastic response of such materials is controlled by the effective bulk and shear moduli. The latter are known only as lying within the Hashin–Shtrikman (H–S) bounds, the uncertainty being caused by the above randomness. The paper presents a probabilistic approach to effective response of such materials, which is based on the principle of maximum information entropy. Explicit probability density functions are derived for the effective moduli and the low-frequency wave velocity within the bounds. The obtained solution complements the well-known deterministic approximations.