MaxEnt Principle Research Articles

Review of Fourth-Order Maximum Entropy Based Predictive Modeling and Illustrative Application to a Nuclear Reactor Benchmark: II. Best-Estimate Predicted Values and Uncertainties for Model Responses and Parameters

This work continues the review and illustrative application to energy systems of the “Fourth-Order Best-Estimate Results with Reduced Uncertainties Predictive Modeling” (4th-BERRU-PM) methodology. The 4th-BERRU-PM methodology uses the Maximum Entropy (MaxEnt) principle to incorporate fourth-order experimental and computational information, including fourth (and higher) order sensitivities of computed model responses with respect to model parameters. The 4th-BERRU-PM methodology yields the fourth-order MaxEnt posterior distribution of experimentally measured and computed model responses and parameters in the combined phase space of model responses and parameters. The 4th-BERRU-PM methodology encompasses fourth-order sensitivity analysis (SA) and uncertainty quantification (UQ), which were reviewed in the accompanying work (Part 1), as well as fourth-order data assimilation (DA) and model calibration (MC) capabilities, which will be reviewed and illustrated in this work (Part 2). The applicability of the 4th-BERRU-PM methodology to energy systems is illustrated by using the Polyethylene-Reflected Plutonium (acronym: PERP) OECD/NEA reactor physics benchmark, which is modeled using the linear neutron transport Boltzmann equation, involving 21,976 imprecisely known parameters. This benchmark is representative of “large-scale computations” such as those involved in the modeling of energy systems. The result (“response”) of interest for the PERP benchmark is the leakage of neutrons through the outer surface of this spherical benchmark, which can be computed numerically and measured experimentally. The impact of the high-order sensitivities of the response with respect to the PERP model parameters is quantified for “high-precision” parameters (2% standard deviations) and “typical-precision” parameters (5% standard deviations). Analyzing the best-estimate results with reduced uncertainties for the 1st—through 4th-order moments (mean values, covariance, skewness, and kurtosis) produced by the 4th-BERRU-PM methodology for the PERP benchmark indicates that, even for systems modeled by linear equations (e.g., the PERP benchmark), retaining only first-order sensitivities is insufficient for reliable predictive modeling (including SA, UQ, DA, and MC). At least second-order sensitivities should be retained in order to obtain reliable predictions.

Review of Fourth-Order Predictive Modeling and Illustrative Application to a Nuclear Reactor Benchmark. I. Typical High-Order Sensitivity and Uncertainty Analysis

This work (in two parts) will review the recently developed predictive modeling methodology called “4th-BERRU-PM” and its applicability to nuclear energy systems as exemplified by an illustrative application to the Polyethylene-Reflected Plutonium (acronym: PERP) OECD/NEA reactor physics benchmark. The acronym 4th-BERRU-PM designates the “Fourth-Order Best-Estimate Results with Reduced Uncertainties Predictive Modeling” methodology, which uses the Maximum Entropy (MaxEnt) principle to incorporate fourth-order experimental and computational information, including fourth (and higher) order sensitivities of computed model responses to model parameters, while yielding best-estimate results with reduced uncertainties for the first fourth-order moments (mean values, covariance, skewness, and kurtosis) of the optimally predicted posterior distribution of model results and calibrated model parameters. The 4th-BERRU-PM methodology encompasses the scopes of high-order sensitivity analysis (SA), uncertainty quantification (UQ), data assimilation (DA) and model calibration (MC), as will be illustrated in this work by means of the above-mentioned OECD/NEA reactor physics benchmark. This benchmark is modeled using the neutron transport Boltzmann equation involving 21,976 imprecisely known parameters, the solution of which is representative of “large-scale computations”. The model result (“response”) of interest is the leakage of neutrons through the outer surface of this spherical benchmark, which can be computed numerically and measured experimentally. Part 1 of this work illustrates the impact of high-order sensitivities, in conjunction with parameter standard deviations of various magnitudes, on the determination of the expected value and variance of the computed response in terms of the first four moments of the distribution of the uncertain model parameters. Part 2 of this work will illustrate the capabilities of the 4th-BERRU-PM methodology for combining computational and experimental information, up to and including forth-order sensitivities and distributional moments, for producing best-estimate values for the predicted responses and model parameters while reducing their accompanying uncertainties.

Towards systematic approach to boundary conditions in mixture and multiphasic incompressible models: Maximum Entropy principle estimate

The single continuum model formulation, no matter how complex the considered constitutive relations are, cannot describe important phenomena stemming from constituents’ interactions. In contrast, mixture theory is a successful framework for providing thermodynamically consistent governing equations in the bulk allowing for the inclusion of details in the material structure and interactions. Despite its ubiquitous applications, a fundamental open problem, a framework for the assessment of boundary conditions, persisted.Our objective is to relate these boundary conditions of mixtures to those of a single continuum and, hence, derive their possible form. To obtain such an estimation, we suggest using the Maximum Entropy (MaxEnt) principle yielding the least biased estimate (when measured by the entropy) of the values of the state variables on the more detailed level based on the knowledge of the state on the less detailed level. In the case of mixtures, the total mixture quantities represent the less detailed description, whereas the quantities related to each phase of the mixture represent the more detailed level, and the mapping (projection) connecting the two levels usually follows from the conservation of total mixture quantities. Therefore, once we have entropy on the detailed level and the aforementioned projection, from the MaxEnt principle, we get the least biased estimate of the decomposition of the total mixture state variables into variables corresponding to each constituent.These estimates can be used to obtain the interfacial conditions between two mixtures: we consider the decomposition of the total mixture quantities to partial quantities on both sides of the interface independently and match the mixture quantities at the interface using classical boundary conditions for a single phase. In this way, we may connect the well-developed theory for single continuum boundary conditions to the boundary conditions in mixtures. The generality of such an approach is discussed together with an explicit example of a binary system of ideal gases illustrating the whole procedure explicitly. The outlined MaxEnt procedure results in a decomposition of total pressure in accordance with Dalton’s law of partial pressures in a mixture of ideal gases. Finally, we apply the method numerically to a mixture of van der Waals gases demonstrating the versatility of the proposed framework. We obtain a nontrivial prediction which differs from the decomposition of mixture pressure according to Dalton’s law.

A maximum entropy approach for the modelling of car-sharing parking dynamics

The science of cities is a relatively new and interdisciplinary topic aimed at studying and characterizing the collective processes that shape the growth and dynamics of urban populations. Amongst other open problems, the forecast of mobility trends in urban spaces is a lively research topic that aims at assisting the design and implementation of efficient transportation policies and inclusive urban planning. To this end, many Machine-Learning models have been put forward to predict mobility patterns. However, most of them are not interpretable -as they build on complex hidden representations of the system configurations- or do not allow for model inspection, thus limiting our understanding of the underlying mechanisms driving the citizen’s daily routines. Here, we tackle this problem by building a fully interpretable statistical model that, incorporating only the minimum number of constraints, can predict different phenomena arising in the city. Using data on the movements of car-sharing vehicles in several Italian cities, we infer a model using the Maximum Entropy (MaxEnt) principle. The model allows for an accurate spatio-temporal prediction of car-sharing vehicles’ presence in different city areas and, thanks to its simple yet general formulation, to precisely perform anomaly detection (e.g., detect strikes and bad weather conditions from car-sharing data only). We compare the forecasting capabilities of our model with different state-of-the-art models explicitly made for time-series forecasting: SARIMA models and Deep Learning Models. We find that MaxEnt models are highly predictive, outperforming SARIMAs while having similar performances of deep Neural Networks - but with advantages of being more interpretable, more flexibile—i.e., they can be applied to different tasks- and being computationally efficient. Our results show that statistical inference might play a fundamental role in building robust and general models describing urban systems phenomena.

Random micropolar beams: response and identification

The objective of this paper is to provide a simplified procedure for the random analysis of axially functionally graded Timoshenko–Ehrenfest beams in the field of the linearized micropolar continuum. New approximated deterministic closed form solutions are proposed, which are also useful for the study of classic 1D structural elements characterized by defects distributed along their axes, bleeding phenomena and step geometric properties. Hence, the proposed results meet several technical applications. The Maximum Entropy (MaxEnt) principle and Monte Carlo (MC) simulations are used to solve the direct and inverse stochastic problems. In order to simplify the presentation, simply-supported and clamped macro- and micro-beams are considered. It is shown that the analysis of 1D micropolar models allows us to identify one of the material parameters related to 3D micropolar models.

Maximum-Entropy-Based Decision-Making Trial and Evaluation Laboratory and Its Application in Emergency Management

Decision-making trial and evaluation laboratory (DEMATEL) can visualize the structure of complex causal relationships, so it is widely used in decision making. One of the important steps in DEMATEL is normalization, and it has received much attention in recent years. Maximum entropy (MaxEnt) is a universal principle, and it is an effective tool for determining the amount of information existed in evidence. In this paper, the authors propose MaxEnt-based DEMATEL. The greatest contribution in this paper is the use of MaxEnt principle to determine the normalized direct influence matrix, which allows to obtain the normalized matrix with minimal information loss. The authors illustrate emergency management to show the superiority of the proposed method.

Performance of the quantum MaxEnt estimation in the presence of physical symmetries

When an informationally complete measurement is not available, the reconstruction of the density operator that describes the state of a quantum system can be obtained, in a reliable way, by adopting the maximum entropy principle (MaxEnt principle), as an additional criterion, to achieve the least biased estimation. In this paper, we study the performance of the MaxEnt method for quantum state estimation when there is prior information about symmetries of the unknown state. We explicitly describe how to work with this method in the most general case, and we present an algorithm that allows to improve the estimation of quantum states with arbitrary symmetries. Furthermore, we implement this algorithm to carry out numerical simulations estimating the density matrix of several three-qubit states of particular interest for quantum information tasks. We observed that, for most states, our approach allows to considerably reduce the number of independent measurements needed to obtain a sufficiently high fidelity in the reconstruction of the density matrix. Moreover, we analyze the performance of the method in realistic scenarios, showing that it is robust even when the effects of finite statistics and experimental noise are considered.

MaxEnt principle and reduced density matrix estimation

In this work we study the reduced density matrices of sublattices of fermionic, bosonic and spin lattice models. Firstly, we consider fermionic and bosonic lattice models, and we show that the reduced density matrix associated with a sublattice coincides with the state obtained by applying the maximum entropy principle under suitably chosen constraints. Secondly, for informationally incomplete scenarios, we considered spin lattice models. We study the performance of the MaxEnt method for estimating the reduced density matrix of sublattices of the lattice system. We find that the performance of the MaxEnt estimation improves not only with the number of measured observables (as expected), but also with the lattice length. In these cases, the MaxEnt solution can be considered, not as an exact solution, but as a good estimator.

Climate-induced range shifts of invasive species (Diaphorina citri Kuwayama).

The Asian citrus psyllid (ACP) Diaphorina citri Kuwayama (Hemiptera: Liviidae) is a destructive, invasive species that poses a serious threat to the citrus industry wherever it occurs. The psyllid vectors the phloem-limited bacteria 'Candidatus Liberibacter americanus' and 'Ca. L. asiaticus', causal agents of the incurable citrus greening disease or huanglongbing (HLB). It is essential to understand which regions and areas are suitable for colonization by ACP to formulate appropriate policy and preventive measures. Considering its biology and ecology, we used a machine learning algorithm based on the MaxEnt (Maximum Entropy) principle, to predict the potential global distribution of ACP using bioclimatic variables and elevation. The model predictions are consistent with the known distribution of ACP and also highlight the potential occurrence outside its current ecological range, that is, primarily in Africa, Asia and the Americas. The most important abiotic variables driving the global distribution of ACP were annual mean temperature, seasonality of temperature and annual precipitation. Our findings highlight the need for international collaboration in slowing the spread of invasive pests like D. citri. © 2022 Society of Chemical Industry.

Enhancing computational enzyme design by a maximum entropy strategy

Although computational enzyme design is of great importance, the advances utilizing physics-based approaches have been slow, and further progress is urgently needed. One promising direction is using machine learning, but such strategies have not been established as effective tools for predicting the catalytic power of enzymes. Here, we show that the statistical energy inferred from homologous sequences with the maximum entropy (MaxEnt) principle significantly correlates with enzyme catalysis and stability at the active site region and the more distant region, respectively. This finding decodes enzyme architecture and offers a connection between enzyme evolution and the physical chemistry of enzyme catalysis, and it deepens our understanding of the stability-activity trade-off hypothesis for enzymes. Overall, the strong correlations found here provide a powerful way of guiding enzyme design.

Markets as ecological networks: inferring interactions and identifying communities

AbstractFinancial markets are paradigmatic examples of complex systems and have been compared to ecological networks in which different species (firms) interact and co-evolve. A central object governing species dynamics in ecology is the community matrix, whose elements are closely related to pairwise interspecific interaction coefficients. Using this ecological analogy we propose a method, based on the Maximum Entropy (MaxEnt) principle, that allows us to infer candidates for an economic community matrix from time series data of market values. To assess the usefulness of this picture, we construct community matrices for a set of companies belonging to the Fortune 500 list and perform a community analysis on the resultant networks. This analysis shows these networks to strongly reflect the known industry groupings of the firms. We conclude therefore that our community matrices capture non-trivial information about the interaction of firms, not immediately apparent from the covariance of market values. We anticipate our approach being useful in elucidating further aspects of market structure, as well as forming the basis of forecasting market dynamics.

Optimal transportation theory for species interaction networks.

Observed biotic interactions between species, such as in pollination, predation, and competition, are determined by combinations of population densities, matching in functional traits and phenology among the organisms, and stochastic events (neutral effects).We propose optimal transportation theory as a unified view for modeling species interaction networks with different intensities of interactions. We pose the coupling of two distributions as a constrained optimization problem, maximizing both the system's average utility and its global entropy, that is, randomness. Our model follows naturally from applying the MaxEnt principle to this problem setting.This approach allows for simulating changes in species relative densities as well as to disentangle the impact of trait matching and neutral forces.We provide a framework for estimating the pairwise species utilities from data. Experimentally, we show how to use this framework to perform trait matching and predict the coupling in pollination and host–parasite networks.

Modeling biomass hydrothermal carbonization by the maximum information entropy criterion

The kinetics of biomass hydrothermal carbonization is modeled by the MaxEnt principle, without assuming a reaction network. Modeling is in good accordance with the experimental data concerning a broad range of biomass and reaction conditions.

Entropy and canonical ensemble of hybrid quantum classical systems

In this work we generalize and combine Gibbs and von Neumann approaches to build, for the first time, a rigorous definition of entropy for hybrid quantum-classical systems. The resulting function coincides with the two cases above when the suitable limits are considered. Then, we apply the MaxEnt principle for this hybrid entropy function and obtain the natural candidate for the hybrid canonical ensemble (HCE). We prove that the suitable classical and quantum limits of the HCE coincide with the usual classical and quantum canonical ensembles since the whole scheme admits both limits, thus showing that the MaxEnt principle is applicable and consistent for hybrid systems.

Reliability Evaluation of Multi-Mechanism Failure for Semiconductor Devices Using Physics-of-Failure Technique and Maximum Entropy Principle

The physics-of-failure (PoF) technique is a practical approach to evaluate the reliability of semiconductor devices. However, the PoF approaches are usually insufficient in dealing with multi-mechanism failure and fitting the Monte Carlo (MC) sampling data. In our study, we propose an improved reliability evaluation method based on PoF technique and maximum entropy (MaxEnt) principle. The PoF models are used to generate time-to-failure samples of the failure mechanisms. Cumulative damage rules and competing failure rules are adopted to deal with multi-point and multi-mechanism failure and generate lifetime samples of the device. And the lifetime samples are fitted by MaxEnt distributions through the proposed fitting algorithm. The numerical examples given in the paper indicate that the MaxEnt distributions can describe the samples well and have a competitive advantage in dealing with multi-peak samples. A case study about a semiconductor device with multi-mechanism failure is presented to explain the workflow of the proposed reliability evaluation approach. The results show that the proposed MaxEnt distributions can yield reliable reliability evaluation results compared with Weibull and Lognormal distributions in the multi-mechanism failure process.

Reply to “What Is the Maximum Entropy Principle? Comments on ‘Statistical Theory on the Functional Form of Cloud Particle Size Distributions’”

Abstract We welcome the opportunity to correct the misunderstandings and misinterpretations contained in Yano’s comment that led him to incorrectly state that Wu and McFarquhar misunderstood the maximum entropy (MaxEnt) principle. As correctly stated by Yano, the principle itself does not suffer from the problem of a lack of invariance. But, as restated in this reply and in Wu and McFarquhar, the commonly used Shannon–Gibbs entropy does suffer from a lack of invariance for coordinate transform when applied in continuous cases, and this problem is resolved by the use of the relative entropy. Further, it is restated that the Wu and McFarquhar derivation of the PSD form using MaxEnt is more general than the formulation by Yano and allows more constraints with any functional relations to be applied. The derivation of Yano is nothing new but the representation of PSDs in other variables.

Two-dimensional distribution of streamwise velocity in open channel flow using maximum entropy principle: Incorporation of additional constraints based on conservation laws

This study derived the vertical and transverse distribution of streamwise velocity in open channels using the information-theoretic concept involving the maximum entropy (MaxEnt) principle, moment constraints expressed by the conservation of mass, momentum, and energy and a hypothesis invoking the connection between probability and space domains through a generalized coordinate system. Explicit analytical solutions were obtained for velocity profiles using a non-perturbation approach along with Padé approximation. The convergence of the series solution was shown both theoretically and numerically. The Lagrange multipliers, arising through the application of MaxEnt principle, were obtained by minimizing a convex potential, and the quasi-Newton method along with BFGS scheme was applied for solving the unconstrained optimization problem. For applying the conservation laws, the momentum and energy coefficients were found to influence the velocity profile in a channel cross-section. With the expressions of these coefficients, the derived velocity equations were validated for a wide range of experimental and field data.

Vortex-induced Vibration Reliability Analysis in Cable-stayed Bridges: Wind and Aerodynamic Uncertainties

This paper proposes a vortex-induced vibration (VIV) reliability analysis method based on the maximum entropy (MaxEnt) principle in which constraints are specified in terms of fractional moments involving uncertain wind characteristics, structural properties, and bluff-body aerodynamics. The multiplicative dimensional reduction method (M-DRM) is used to calculate the fractional moments. A new threshold is obtained based on the stopping sight distance in the highway engineering technique standard, which is less strict than the China–Japan standard and the British code and thus better-suited to VIV reliability evaluation. A long-span cable-stayed bridge model is used to illustrate the accuracy and efficiency of the proposed method in comparison to Monte Carlo simulation (MCS) on three performance functions of VIV reliability. The results indicate that VIV maximum amplitude and wind speed failure probability are not sensitive to the selected performance function, but the effect of the wind attack angle on probability is significant. The M-DRM provides a workable alternative approach to analyzing the highly complex VIV reliability of long-span bridges.

Cyclic virtual test on wood furniture by Monte Carlo simulation: from compression behavior to connection modeling

Prediction of durability of wood product is a major challenge and an important goal for furniture industry. Numerical simulation based on approximation methods such as the finite element method (FEM) is an efficient and powerful tool to address this challenge while avoiding expensive experimental testing campaigns. Nevertheless, the strong heterogeneity of wood-based materials, the specific geometrical characteristics of wood-based structures (such as furniture that can often be represented as an assembly of beams, plates and/or shells) and the complex nonlinear 3D local behavior near the connections between structural parts may induce some difficulties in the numerical modeling and virtual testing of furniture for robust design purposes. Especially, when cyclic loading occurs, the behavior of junctions in furniture involves a local permanent strain that increases with the number of cycles and that can lead to an important gap potentially affecting the structural integrity of furniture. In this paper, we present an experimental campaign of cyclic compression tests carried out on spruce specimens. Theses specimens are cut out from a bunk bed and loaded under cyclic compression. The cyclic compression loading applied to the specimens leads to an evolution of the permanent strain during cycles that is modeled using a simple law describing the displacement gap as a function of the number of cycles. Considering the strong dispersion in the mechanical properties of wood-based materials and the variabilities induced by the experimental configuration, a stochastic modeling of the gap is proposed by having recourse to the maximum entropy (MaxEnt) principle in order to take into account the random uncertainties on the experimental setup and between the test specimens. The random mechanical response of a complex corner junction in a bunk bed under cyclic loading is then numerically simulated by using a Monte Carlo numerical simulation method as stochastic solver. This provides independent realizations of the random gap evolution (with respect to the number of cycles) in the bunk bed corner, allowing probabilistic quantities of interest related to the random gap, such as first- and second-order statistical moments (mean value, standard deviation) as well as confidence regions (with a given probability level), to be estimated.

Thermodynamic efficiency of contagions: a statistical mechanical analysis of the SIS epidemic model.

We present a novel approach to the study of epidemics on networks as thermodynamic phenomena, quantifying the thermodynamic efficiency of contagions, considered as distributed computational processes. Modelling SIS dynamics on a contact network statistical-mechanically, we follow the maximum entropy (MaxEnt) principle to obtain steady-state distributions and derive, under certain assumptions, relevant thermodynamic quantities both analytically and numerically. In particular, we obtain closed-form solutions for some cases, while interpreting key epidemic variables, such as the reproductive ratio of a SIS model, in a statistical mechanical setting. On the other hand, we consider configuration and free entropy, as well as the Fisher information, in the epidemiological context. This allowed us to identify criticality and distinct phases of epidemic processes. For each of the considered thermodynamic quantities, we compare the analytical solutions informed by the MaxEnt principle with the numerical estimates for SIS epidemics simulated on Watts-Strogatz random graphs.