Maximum Entropy Estimation Research Articles

Nonlinear manifold learning for model reduction in finite elastodynamics

Model reduction in computational mechanics is generally addressed with linear dimensionality reduction methods such as Principal Components Analysis (PCA). Hypothesizing that in many applications of interest the essential dynamics evolve on a nonlinear manifold, we explore here reduced order modeling based on nonlinear dimensionality reduction methods. Such methods are gaining popularity in diverse fields of science and technology, such as machine perception or molecular simulation. We consider finite deformation elastodynamics as a model problem, and identify the manifold where the dynamics essentially take place – the slow manifold – by nonlinear dimensionality reduction methods applied to a database of snapshots. Contrary to linear dimensionality reduction, the smooth parametrization of the slow manifold needs special techniques, and we use local maximum entropy approximants. We then formulate the Lagrangian mechanics on these data-based generalized coordinates, and develop variational time-integrators. Our proof-of-concept example shows that a few nonlinear collective variables provide similar accuracy to tens of PCA modes, suggesting that the proposed method may be very attractive in control or optimization applications. Furthermore, the reduced number of variables brings insight into the mechanics of the system under scrutiny. Our simulations also highlight the need of modeling the net effect of the disregarded degrees of freedom on the reduced dynamics at long times.

A Generalized Maximum Entropy Estimator to Simple Linear Measurement Error Model with a Composite Indicator

We extend the Simple Linear Measurement Error Model through the inclusion of a composite indicator by using the Generalized Maximum Entropy estimator. A Monte Carlo simulation study is proposed for comparing the performances of the proposed estimator to his counterpart the Ordinary Least Squares “Adjusted for attenuation.” The two estimators are compared in term of correlation with the true latent variable, standard error and root mean of squared error. Two illustrative case studies are reported in order to discuss the results obtained on the real data set, and relate them to the conclusions drawn via simulation study.

Statistical Equilibrium Models for Sparse Economic Networks

Real markets can be naturally represented as networks, and they share with other social networks the fundamental property of sparsity, whereby agents are connected by l = O (n) relationships. The exponential networks model introduced by Park and Newman can be extended in order to deal with this property. When compared with alternative statistical models of a given real network, this extended model provides a better statistical justification for the observed network values. Consequently, it provides more reliable maximum entropy estimates of partially known networks than previously known ME techniques.

Information Theoretic Models for Dependence Analysis And missing Data Estimation

In the present communication information theoretic dependence measure has been defined using maximum entropy principle, which measures amount of dependence among the attributes in a contingency table. A relation between information theoretic measure of dependence and Chi-square statistic has been discussed. A generalization of this information theoretic dependence measure has been also studied. In the end Yate’s method and maximum entropy estimation of missing data in design of experiment have been described and illustrated by considering practical problems with empirical data.

A bottom poor sensitive Gini coefficient and maximum entropy estimation of income distributions

A bottom poor sensitive Gini coefficient (pgini) is defined by replacing income observations with their reciprocal values in the Gini coefficient. The underlying true income share function can be derived approximately using the maximum entropy method given the pgini coefficient.

Particle hydrodynamics based on maximum entropy approximation

AbstractIn this contribution, we propose a new particle based scheme for computational hydrodynamics. In contrast to existing particle based approaches, we use a maximum entropy approximation scheme to construct corresponding shape functions. These allow first order consistency as well as an efficient realisation of essential boundary conditions. The scheme is relative simple from an algorithmic point of view and computations can be efficiently accelerated using graphic processing units. (© 2012 Wiley‐VCH Verlag GmbH & Co. KGaA, Weinheim)

An exploratory statistical approach to depression pattern identification

Depression is a complex phenomenon thought to be due to the interaction of biological, psychological and social factors. Currently depression assessment uses self-reported depressive symptoms but this is limited in the degree to which it can characterise the different expressions of depression emerging from the complex causal pathways that are thought to underlie depression. In this study, we aimed to represent the different patterns of depression with pattern values unique to each individual, where each value combines all the available information about an individual’s depression. We considered the depressed individual as a subsystem of an open complex system, proposed Generalized Information Entropy (GIE) to represent the general characteristics of information entropy of the system, and then implemented Maximum Entropy Estimates to derive equations for depression patterns. We also introduced a numerical simulation method to process the depression related data obtained by the Diamond Cohort Study which has been underway in Australia since 2005 involving 789 people. Unlike traditional assessment, we obtained a unique value for each depressed individual which gives an overall assessment of the depression pattern. Our work provides a novel way to visualise and quantitatively measure the depression pattern of the depressed individual which could be used for pattern categorisation. This may have potential for tailoring health interventions to depressed individuals to maximize health benefit.

Technical efficiency with state-contingent production frontiers using maximum entropy estimators

Although the theory of state-contingent production is well-established, the empirical implementation of this approach is still in an infancy stage. The possibility of finding a large number of states of nature, few observations per state and models affected by collinearity have led some researchers to claim the urgent need to develop robust estimation techniques. In this paper, we investigate the performance of some maximum entropy estimators to assess technical efficiency with state-contingent production frontiers. The methodological discussion and the simulation study provided in the paper reveal some of the potential of these estimators. Small mean squared error loss and small differences between the true and the estimated mean of technical efficiency show that the maximum entropy can be a powerful tool in the estimation of state-contingent production frontiers.

GRAVITY AND SPATIAL STRUCTURE: THE CASE OF INTERSTATE MIGRATION IN MEXICO*

ABSTRACTThe estimation of gravity models of internal (aggregate) place‐to‐place migration is plagued with endogeneity (omitted‐variable) biases if the unobserved effects of spatial structure are not accounted for. To address this econometric problem, this paper presents a more general specification of the gravity model, which allows for (bilateral) parameter heterogeneity across individual migration paths—along with (unilateral) origin‐ and destination‐specific effects. The resultant “three‐way fixed‐effects” (3FE) model is applied for an analysis of interstate migration in Mexico based on cross‐sectional data. To overcome parameter‐dimensionality problems (due to limited or incomplete information), the 3FE model is estimated using the Generalized Maximum Entropy (GME) estimator. The empirical implications of this new modeling strategy are illustrated by contrasting the 3FE‐GME estimates with those for the traditional and two‐way fixed‐effects (2FE) models. The former are far more plausible and intuitively interpretable than their traditional and 2FE counterparts, with parameter estimates changing in expected directions. The (average) effect of the migrant stock is markedly smaller than usually estimated, providing a more realistic measure of network‐induced migration. Migration outflows from centrally located origins have significantly steeper distance decay. Path‐specific distance effects exhibit directional asymmetries and spatial similarities.

Generalization of the k-moment method using the maximum entropy principle. Application to the NBKM and full spectrum SLMB gas radiation models

The k-moment method is generalized by applying the maximum entropy principle to get several estimates of the k-distribution function on any kind of spectral interval as a function of the first two moments of the absorption coefficient. Corresponding formulations of the blackbody weighted band averaged transmission function of a gaseous uniform path are obtained. Different constraints involving the first and second order positive, first order negative and logarithmic moments are introduced together with a physical meaning whenever it is possible. Different sets of these constraints are considered to get maximum entropy estimates of the distributions functions: the Dirac, exponential, Gamma, inverse Gaussian and reciprocal inverse Gaussian k-distribution functions. Analytical formulas are provided for each of these distributions and for their associated transmission function, as a function of the mean and variance of the absorption coefficient. The methodology can be applied considering any spectral interval: narrow, wide, the full spectrum, continuous or not. Thus the resulting associated transmission and cumulative k-distribution functions can be utilized in the frame of a large variety of gas radiation models. Hence the k-moment method using the maximum entropy principle is assessed in the frame of the NBKM and full spectrum SLMB gas radiation models. A series of test cases implying comparisons with reference Line-by-Line results exhibits which maximum entropy k-distributions are likely to give the best estimations of narrow band or total emitted intensities, curves-of-growth of the total emission function and full spectrum cumulative k-distribution functions. In particular, the inverse Gaussian and Gamma k-distributions seem most of the time to perform very well.

Approximate maximum entropy on the mean for instrumental variable regression

We want to estimate an unknown finite measure μX from a noisy observation of generalized moments of μX, defined as the integral of a continuous function Φ with respect to μX. Assuming that only a quadratic approximation Φm is available, we define an approximate maximum entropy solution as a minimizer of a convex functional subject to a sequence of convex constraints. We establish asymptotic properties of the approximate solution under regularity assumptions on the convex functional, and we study an application of this result to instrumental variable estimation.

Convergent meshfree approximation schemes of arbitrary order and smoothness

Local Maximum-Entropy (LME) approximation schemes are meshfree approximation schemes that satisfy consistency conditions of order one, i.e., they approximate affine functions exactly. In addition, LME approximation schemes converge in the Sobolev space W1,p, i.e., they are C0-continuous in the conventional terminology of finite-element interpolation. Here we present a generalization of the Local Max-Ent approximation schemes that are consistent to arbitrary order, i.e., interpolate polynomials of arbitrary degree exactly, and which converge in Wk,p, i.e., they are Ck-continuous to arbitrary order k. We refer to these approximation schemes as High Order Local Maximum-Entropy Approximation Schemes (HOLMES). We prove uniform error bounds for the HOLMES approximates and their derivatives up to order k. Moreover, we show that the HOLMES of order k is dense in the Sobolev space Wk,p, for any 1⩽p<∞. The good performance of HOLMES relative to other meshfree schemes in selected test cases is also critically appraised.

A Job Satisfaction Structural Equation Model Obtained Combining Rasch Analysis and Generalized Maximum Entropy Estimation

The objective of this paper is to describe and analyze with appropriate statistical models the links between the work quality latent factors using data from a recent Survey on the Italian Social Cooperatives known as ICSI2007.Due to the task complexity, the analysis is carry out by a two-step approach: In the first step we construct some multidimensional measures of the subjective quality of work, using the Nonlinear Principal Component Analysis (NPCA) and the Rasch Analysis with the Rating Scale Model (NPCA-RSM); In the second step we adopt a Structural Equation Model based on Generalized Maximum Entropy (SEM-GME) to integrate the measures achieved with the previous step and to evaluate the relationships between the subjective work quality latent factors.Therefore, the novelty aspects of this paper are the following: (i) The integration between the NPCA-RSM and SEM-GME, that allows the reduction of the variables analyzed and the evaluation of the measurement errors; (ii) The formalization of a job satisfaction model for the study of the relationships between the subjective work quality latent factors in the Italian social services sector.

Temporal and spatial variation of extreme precipitation events in flood season over Three Gorges Reservoir area in recent 50 a

利用三峡库区35个台站1961-2010年汛期(5-9月)的逐日降水量资料,首先定义不同台站的极端降水量阈值,统计各站近50 a逐年汛期极端降水事件的发生频次,进而分析其时空变化特征.结果表明:三峡库区汛期极端降水事件发生频次的最主要空间模态是主体一致性,同时存在东西和南北相反变化的差异.三峡库区汛期极端降水事件发生频次具有较大的空间差异,可分为具有不同变化特点的5个主要异常区.滑动t检验表明,三峡库区西南部区代表站巴南的极端降水事件在1974年后发生了一次由偏多转为偏少的突变,北部区代表站北碚在1981年后和1993年后分别发生了由偏少转为偏多和由偏多到偏少的突变,中部区代表站武隆在1984年后发生了一次由偏多转为偏少的突变.结合最大熵谱和功率谱分析表明,近50 a来各分区汛期极端降水事件发生频次的周期振荡不太一致,三峡库区东北部区代表站宜昌、北部区代表站北碚和中部区代表站武隆分别存在5、2.4和8.3 a的显著周期.;Based on the daily precipitation data in flood season(from May to September) in 1961-2010 from 35 stations over Three Gorges Reservoir area, the extreme precipitation threshold value for all stations are determined firstly, then the frequency of extreme precipitation event in the flood season are counted and their temporal and spatial characteristics are analyzed. Consistent anomaly distribution is the main spatial model of extreme precipitation event frequency in flood season over Three Gorges Reservoir area. The spatial distribution of extreme precipitation event frequency is complex with anomaly difference between in the south and west, and in the east and west of Three Gorges Reservoir area, which can be divided into five main regions. The moving t-test analysis shows that there are abrupt changes of Banan in 1974, Beibei in 1981 and 1993, and Wulong in 1984 for the extreme precipitation event frequency. The Maximum Entropy Spectral and Estimation Power Spectral analysis shows that the periodic oscillations of these regions are not consistent in recent 50 a. The distinct periodic oscillation of the extreme precipitation event frequency of Yichang, Beibei and Wulong is 5, 2.4 and 8.3 a, respectively.

A Generalized Maximum Entropy Stochastic Frontier Measuring Productivity Accounting for Spatial Dependency

In this paper, a stochastic frontier model accounting for spatial dependency is developed using generalized maximum entropy estimation. An application is made for measuring total factor productivity in European agriculture. The empirical results show that agricultural productivity growth in Europe is driven by upward movements of technology over time through technological developments. Results are then compared for a situation in which spatial dependency in the technical inefficiency effects is not accounted.

Maximum-entropy Monte Carlo method for the inversion of the structure factor in simple classical systems

We present a method for the evaluation of the interaction potential of an equilibrium classical system starting from the (partial) knowledge of its structure factor. The procedure is divided into two phases, both of which are based on the maximum entropy principle of information theory. First we determine the maximum entropy estimate of the radial distribution function constrained by the information contained in the structure factor. Next we invert the pair function and extract the interaction potential. The method is tested on a Lennard-Jones fluid at high density and the reliability of its results with respect to the missing information in the structure factor data are discussed. Finally, it is applied to the experimental data of liquid sodium at 100 °C.

A Maximum Entropy Estimator for the Aggregate Hierarchical Logit Model

A new approach for estimating the aggregate hierarchical logit model is presented. Though usually derived from random utility theory assuming correlated stochastic errors, the model can also be derived as a solution to a maximum entropy problem. Under the latter approach, the Lagrange multipliers of the optimization problem can be understood as parameter estimators of the model. Based on theoretical analysis and Monte Carlo simulations of a transportation demand model, it is demonstrated that the maximum entropy estimators have statistical properties that are superior to classical maximum likelihood estimators, particularly for small or medium-size samples. The simulations also generated reduced bias in the estimates of the subjective value of time and consumer surplus.

Information theoretic approach for constructing a super data envelopment analysis

PurposeThe purpose of this paper is to provide a robust statistical procedure for evaluating and measuring the relative efficiency of multiple decision‐making units. This robust approach is based on the generalized maximum entropy principle.Design/methodology/approachInformation‐theoretic estimation approach is employed in this paper and a comparison is made with the classical relative efficiency (CCR) by using a non‐parametric bootstrap simulation. A real data application on the research performance of faculty members at Yarmouk University is presented.FindingsResults indicate that the relative efficiency based on the generalized maximum entropy estimation approach is more accurate, costs less and is more efficient than the CCR relative efficiency.Research limitations/implicationsOwing to use of Shannon's entropy formulation, it is still critical whether the results also hold with cross entropy or a higher order entropy formulation for modeling additive, multiplicative or partial relative efficiency.Originality/valueA super data envelopment analysis has been introduced for finding superior decision‐making units (DMU) by solving only one nonlinear programming system, which could be considered as a flexible tool for modeling multiple input‐output DMU.

Generalized Maximum Entropy Estimators: Applications to the Portland Cement Dataset

Consider the linear regression model y= X+ u in the usual notation. In many applications the design matrix X is frequently subject to severe multicollinearity. In this paper an alternative estimation methodology, maximum entropy is given and used to estimate the parameters in a linear regression model when the basic data are ill-conditioned. We de- scribed the generalized maximum entropy (GME) estimator, imposing sign restrictions of parameters and imposing cross parameter restrictions for GME. Mean squared error (mse) values of the estimators are estimated by the bootstrap method. We compared the generalized maximum entropy (GME) estimator, least squares and inequality restricted least squares (IRLS) estimator on the widely analyzed dataset on Portland cement.

Climate variability and maize yield in the Limpopo region of South Africa: Results from GME and MELE methods

This paper investigates the impact of climate variability on maize yield in the Limpopo Basin of South Africa using the generalized maximum entropy (GME) estimator and maximum entropy leuven estimator (MELE). Precipitation and temperature were used as proxies for climate variability, which were combined with traditional input variables (i.e. labour, fertilizer, seed and irrigation). Based on pseudo R-squared, we found that the GME fits the data better than MELE. In addition, increased precipitation, increased temperature and irrigation have a positive impact on yield. Furthermore, the results of the GME show that the impact of precipitation on maize yield is weaker than that of temperature. However, the impact of climate variability on maize yield could be negative if it increases temperature marginally but reduces precipitation to a very large extent simultaneously. Moreover, the impact of irrigation on yield is positive and with a higher elasticity coefficient than that of precipitation, which supposes that the present system of irrigation could mitigate the impact of reduced precipitation on yield.