Maximum Entropy Estimation Research Articles

Comparison of Feature Selection Methods—Modelling COPD Outcomes

Selecting features associated with patient-centered outcomes is of major relevance yet the importance given depends on the method. We aimed to compare stepwise selection, least absolute shrinkage and selection operator, random forest, Boruta, extreme gradient boosting and generalized maximum entropy estimation and suggest an aggregated evaluation. We also aimed to describe outcomes in people with chronic obstructive pulmonary disease (COPD). Data from 42 patients were collected at baseline and at 5 months. Acute exacerbations were the aggregated most important feature in predicting the difference in the handgrip muscle strength (dHMS) and the COVID-19 lockdown group had an increased dHMS of 3.08 kg (CI95 ≈ [0.04, 6.11]). Pack-years achieved the highest importance in predicting the difference in the one-minute sit-to-stand test and no clinical change during lockdown was detected. Charlson comorbidity index was the most important feature in predicting the difference in the COPD assessment test (dCAT) and participants with severe values are expected to have a decreased dCAT of 6.51 points (CI95 ≈ [2.52, 10.50]). Feature selection methods yield inconsistent results, particularly extreme gradient boosting and random forest with the remaining. Models with features ordered by median importance had a meaningful clinical interpretation. Lockdown seem to have had a negative impact in the upper-limb muscle strength.

Indeterminate Hamburger moment problem: Entropy convergence

The indeterminate Hamburger moment problem is considered, jointly with all its real axis supported probability density functions. As a consequence of entropy functional concavity, out of such densities there is one which has largest entropy and that plays a fundamental role: we call it fhmax. It is proved that the approximate Maximum Entropy (MaxEnt) densities constrained by an increasing number of moments converge in entropy to fhmax where the value of its entropy can be finite or −∞.

The Simulation Technique to Estimate the Parameters of Generalized Exponential Rayleigh Model

The paper shows how to estimate the three parameters of the generalized exponential Rayleigh distribution by utilizing the three estimation methods, namely, the moment employing estimation method (MEM), ordinary least squares estimation method (OLSEM), and maximum entropy estimation method (MEEM). The simulation technique is used for all these estimation methods to find the parameters for the generalized exponential Rayleigh distribution. In order to find the best method, we use the mean squares error criterion. Finally, in order to extract the experimental results, one of object oriented programming languages visual basic. net was used

Visual identification of some regularities in packet network traffic

Modern heterogeneous packet networks generate network traffic with a complex structure. In this article, the object of study is a time series. The total number of User Datagram Protocol (UDP) packets has reached 250242. According to analysts, the growth trend of traffic, including real-time applications, will continue and the volume of data will grow, which may lead to the formation of packet queues when processed by network devices. In this case, there may be losses in case of long queues. To solve this problem, a power spectrum assessment was carried out. The AR maximum entropy estimator has been shown to be more sensitive than the auxiliary Fourier estimator. Accounting for non-stationarity by spectral methods is possible only through estimation in a sliding time window. Nine diagrams of spectral-temporal analysis of the original series, its increments, and the mixed series of increments were obtained: with default parameters, with small and large windows. Diagrams related to the original series reflect the dynamics of changes in data transmission intensity in the network; they show higher temporal resolution, indicating the presence of high-frequency components (noise) and the presence of low-frequency components (trend). Diagrams with increments describe signals of periodic components; changing the length of the window did not reflect the presence of noise or trend signs. Diagrams with mixed increments show that frequency components are uniformly distributed. The uniqueness of this work lies in the real measured data, and a distinctive feature of the obtained results is the visual examination of the complex traffic structure, allowing for the resolution of the investigated problem. Practical application of the results obtained can be applied in Quality of Service (QoS) management, resource planning, and network performance optimization

A maximum-entropy length-orientation closure for short-fiber reinforced composites

We describe an algorithm for generating fiber-filled volume elements for use in computational homogenization schemes which accounts for a coupling of the fiber-length and the fiber-orientation. For prescribed fiber-length distribution and fiber-orientation tensor of second order, a maximum-entropy estimate is used to produce a fiber-length-orientation distribution which mimics real injection molded specimens, where longer fibers show a stronger alignment than shorter fibers. We derive the length-orientation closure from scratch, discuss its integration into the sequential addition and migration algorithm for generating fiber-filled microstructures for industrial volume fractions and investigate the resulting effective elastic properties. We demonstrate that accounting for the length-orientation coupling permits to match the measured Young’s moduli in principal fiber direction and transverse to it more accurately than for closure approximations ignoring the length-orientation coupling.

Analysis of Meshfree Galerkin Methods Based on Moving Least Squares and Local Maximum-Entropy Approximation Schemes

Over the last two decades, meshfree Galerkin methods have become increasingly popular in solid and fluid mechanics applications. A variety of these methods have been developed, each incorporating unique meshfree approximation schemes to enhance their performance. In this study, we examine the application of the Moving Least Squares and Local Maximum-Entropy (LME) approximations within the framework of Optimal Transportation Meshfree for solving Galerkin boundary-value problems. We focus on how the choice of basis order and the non-negativity, as well as the weak Kronecker-delta properties of shape functions, influence the performance of numerical solutions. Through comparative numerical experiments, we evaluate the efficiency, accuracy, and capabilities of these two approximation schemes. The decision to use one method over the other often hinges on factors like computational efficiency and resource management, underscoring the importance of carefully considering the specific attributes of the data and the intrinsic nature of the problem being addressed.

A maximum entropy estimate of a wine rating’s distribution: results of tests using large samples of blind triplicates

ABSTRACT An acute difficulty in wine-ratings-related research, and in calculating consensus among judges, is that each rating is one observation drawn from a latent probability distribution that is wine- and judge-specific. One observation is almost no data at all. Minimizing a weighted sum of cross entropies has been proposed as a method of estimating the shape of the latent distribution using one observation. That method was tested on 30 blind triplicate wine ratings. This article replicates that test, presents results of tests at hypothetical boundary conditions, and then presents test results for a set of 30 blind triplicate ratings published by Cicchetti [(2014, August). Blind tasting of South African wines: A tale of two methodologies (AAWE Working Paper No. 164, 15 pages)] and a set of 1599 blind triplicate ratings from the California State Fair. The test results show that the minimum cross entropy solution is substantially more accurate than the analysis of a single rating.

Probability Distributions Approximation via Fractional Moments and Maximum Entropy: Theoretical and Computational Aspects

In the literature, the use of fractional moments to express the available information in the framework of maximum entropy (MaxEnt) approximation of a distribution F having finite or unbounded positive support, has been essentially considered as a computational tool to improve the performance of the analogous procedure based on integer moments. No attention has been paid to two formal aspects concerning fractional moments, such as conditions for the existence of the maximum entropy approximation based on them or convergence in entropy of this approximation to F. This paper aims to fill this gap by providing proofs of these two fundamental results. In fact, convergence in entropy can be involved in the optimal selection of the order of fractional moments for accelerating the convergence of the MaxEnt approximation to F, to clarify the entailment relationships of this type of convergence with other types of convergence useful in statistical applications, and to preserve some important prior features of the underlying F distribution.

An efficient uncertainty propagation analysis method of non-parameterized P-boxes based on dimension-reduction integral and maximum entropy estimation

An efficient uncertainty propagation analysis method of non-parameterized P-boxes based on dimension-reduction integral and maximum entropy estimation

An information–Theoretic approach to partially identified auction models

We consider a situation in which we have data from ascending auctions with symmetric bidders, independent private values, and exogenous entry in which the bidders’ value distribution is partially identified. Focusing on the case in which the seller intends to use a second price auction, we discuss how to determine an optimal reserve price. We justify the use of maximum entropy, explore the properties of the estimand, determine the asymptotic properties of our maximum entropy estimator, evaluate its behavior in a simulation study, and demonstrate its use in a modest application. As an extension, we propose a maxmin decision rule with entropy regularization, which includes Aryal and Kim (2013) and the maximum entropy solution as extreme cases.

Accelerated Maximum Entropy Method for Time Series Models Estimation

The work is devoted to the development of a maximum entropy estimation method with soft randomization for restoring the parameters of probabilistic mathematical models from the available observations. Soft randomization refers to the technique of adding regularization to the functional of information entropy in order to simplify the optimization problem and speed up the learning process compared to the classical maximum entropy method. Entropic estimation makes it possible to restore probability distribution functions for model parameters without introducing additional assumptions about the likelihood function; thus, this estimation method can be used in problems with an unspecified type of measurement noise, such as analysis and forecasting of time series.

Investigating CO2 emissions from aviation in oil producing countries using a two-stage maximum entropy approach

The transportation sector plays a major role in the rapid economic growth of developing countries. Since this sector requires large amounts of fossil energy resources, it has a very important impact on global warming and greenhouse gas emissions. As aviation has a significant contribution to carbon dioxide (CO2) emissions among different type of transportation, there is considerable interest in policies, regulations and research to mitigate the adverse effects of aviation on climate [1]. This work investigates the role of air passengers and air freight on CO2 emissions from aviation for eight oil-producing countries in the middle east for the years 2013–2019. Maximum entropy bootstrap for time series and generalized maximum entropy estimation are used in this work, which represent two powerful techniques that lead to reliable and stable results in ill-posed models. The empirical results confirm the existence of a positive impact of air passengers on CO2 emissions from air transport in all the sample countries. But the connection between air freight and CO2 emissions from aviation is only confirmed for Iran and Qatar. The findings for the other countries regarding the impact of air freight on CO2 emissions from aviation depend on the significance levels considered.

Practical considerations on data patterns in Bayesian Maximum Entropy Estimation: A systematic and critical review

Objective: It is well known that some data features (sample size, skewness, among others) may determine method performance. The choice of those features depends on the researcher’s level of awareness on the statistical method. In this study, the level of awareness on the influence of spatial data key characteristics (sample size, skewness, spatial dependency and variogram model) in Bayesian Maximum Entropy (BME) was analyzed. Methodology: A systematic review was conducted that covers the period from 1990 (year of BME introduction) to 2019. Two main keywords “Bayesian Maximum Entropy” and “BME” were used for literature search. Publications which only mentioned the keywords without applying BME were excluded while those with application and/or BME theory discussion were considered. Six of the world’s leading Open Access sources of scientific literature were considered, namely: Science Direct, African Journals Online, Springer, Google Scholar, MPDI and Academic Journals. A total of 118 research articles from 62 journals were identified. The sample sizes screened shows that 25.4% of the published articles used few samples (less than 100), which implies the variogram might not yield accurate results. The analysis of the use of skewness showed that most researchers do not apply transformation on skewed data (82.2%) nor consider skewness in their descriptive statistics (90.7%). Even though 11% of theoretical papers have mentioned about spatial dependency level, 92.4% of them failed to consider it. Most researchers (68.64%) do not specify the variogram models but when they do, they mostly use exponential model (12.7%). It clearly appears in this review that most researchers do not consider the effect of sample size, skewness, and spatial dependency level when applying BME. Yet very few research works have focused on these aspects. This therefore calls for more in-depth studies on the effect of data characteristics on BME’s performance. Keywords: Bayesian Maximum Entropy, sample size, skewness, spatial dependency.

Building Height Reconstruction Based on Propagator and Generalized Maximum Entropy-TomoSAR

In recent years, the synthetic aperture radar tomography (TomoSAR) technique is becoming an increasingly hot research topic in the three-dimensional (3D) reconstruction of the structure of urban buildings. The robustness of the many advanced non-parametric spectral estimation methods in use is based on a high number of observations. This letter aims to evaluate the ability of the five non-parametric estimator techniques, including propagator, generalized maximum entropy (GME), minimum norm (MN), singular value decomposition (SVD), and capon in the separation of scattering contributions with a small number of SAR images. The performance comparison of the employed algorithms is carried out on the simulated and real datasets. The study results show that the novel high-resolution propagator and generalized maximum entropy estimators, respectively, based on the partitioning and extrapolation of the covariance matrix, can discriminate the number of scatterers in one azimuth-range resolution and satisfactorily prevent the appearance of undesirable sidelobes. The experimental results on real SAR images acquired by the TerraSAR-X satellite show the proposed propagator spectral estimation can significantly improve the 3D building reconstruction in the urban environment. Also, the estimated height of the scatters using the propagator technique is similar to the data obtained from ground observations.

Local maximum-entropy approximation based stabilization methods for the convection diffusion problems

Local Maximum Entropy (LME) approximation in the field of elliptic partial differential equations is more advanced compared to convection dominated flow problems. This paper presents a comparative study on the LME based meshfree stabilization methods for convection dominated convection-diffusion problems. LME approximants account for the disposal of the elements and direct imposition of boundary conditions, whereas the stabilization methods are meant to deal with the issues associated with the non-self-adjoint convective term. A new meshfree stabilization scheme, residual free bubbles based multiscale element free Galerkin method is also presented and compared with streamline upwind Petrov Galerkin meshfree method and variational multiscale element free Galerkin Method. The effect of different priors and radius of support which define the LME basis are studied to test their importance in the three methods with respect to standard benchmark problems. Cubic spline prior based LME is found to be enough to produce better stability and accuracy for all the stabilization methods when compared to quartic spline and Gaussian priors. LME based variational multiscale element free Galerkin method is observed to yield better stability and accuracy for all the priors. The convergence rate for the three stabilization methods is observed to be in good agreement with the classical FEM-SUPG.

A maximum entropy estimate of uncertainty about a wine rating

AbstractMuch research shows that the ratings that judges assign to the same wine are uncertain. And while the ratings may be independent, research also shows that they are not identically distributed. Thus, an acute difficulty in ratings-related research and in calculating consensus among judges is that each rating is one observation drawn from a latent distribution that is wine- and judge-specific. What can be deduced about the shape of a latent distribution from one observation? A simple maximum entropy estimator is proposed to describe the distribution of a rating observed. The estimator can express the implications of zero, one, a few blind replicates, and many observations. Several tests of the estimator show that results are consistent with the results of experiments with blind replicates and that results are more accurate than results based on observed ratings alone.

Ridge estimation in linear mixed measurement error models using generalized maximum entropy

In this article, we concentrate on the generalized maximum entropy (GME) estimators and their asymptotic properties in linear mixed models (LMMs) with measurement error in the fixed effects (MEFE) variables. Moreover, we obtain the Ridge-GME estimator in these models as a remedy to collinearity. Finally, a simulation study and an example of real data are given to characterize the superiority of the Ridge-GME estimator over the corrected score estimator (CSE) and ridge estimator (RE), using the mean squared error matrix (MSEM).

Adaptive Local Maximum-Entropy Surrogate Model and Its Application to Turbine Disk Reliability Analysis

The emerging Local Maximum-Entropy (LME) approximation, which combines the advantages of global and local approximations, has an unsolved issue wherein it cannot adaptively change the morphology of the basis function according to the local characteristics of the sample, which greatly limits its highly nonlinear approximation ability. In this research, a novel Adaptive Local Maximum-Entropy Surrogate Model (ALMESM) is proposed by constructing an algorithm that adaptively changes the LME basis function and introduces Particle Swarm Optimization to ensure the optimality of the adaptively changed basis function. The performance of the ALMESM is systematically investigated by comparison with the LME approximation, a Radial basis function, and the Kriging model in two explicit highly nonlinear mathematical functions. The results show that the ALMESM has the highest accuracy and stability of all the compared models. The ALMESM is further validated by a highly nonlinear engineering case, consisting of a turbine disk reliability analysis under geometrical uncertainty, and achieves a desirable result. Compared with the direct Monte Carlo method, the relative error of the ALMESM is less than 1%, which indicates that the ALMESM has considerable potential for highly nonlinear problems and structural reliability analysis.

Female labor force participation in Egypt and Palestine: An age–period–cohort analysis*

AbstractThis paper considers female labor force participation (FLFP) behavior over the past two decades in two MENA (Middle East and North Africa) countries, namely Egypt and Palestine. Low FLFP rates in these countries, as in other MENA countries, are well documented. We conduct synthetic panel analysis using age–period–cohort (APC) methodology and decompose the FLFP rates into age, period, and cohort effects. We present our results with Hanoch–Honig–Deaton–Paxson normalization and maximum entropy estimation approaches to the APC method to observe the robustness of our results. The analysis is carried out by aggregate FLFP as well as by stratification by rural–urban location and educational attainments. Most of our results are similar in Egypt and Palestine with some exceptions. The two salient findings are as follows. The year effects display an initial U‐shape followed by an inverted U‐shape in both countries. The declining arm of the inverted U‐shape is compatible with the “MENA paradox” observed in both countries. The increase in the participation of the younger cohorts of the below‐secondary educated women, which is a low‐educated group, may be an indication of the wearing away of the social norms in Palestine that are considered to hinder the participation of women.

Neural networks for quantum inverse problems

Quantum inverse problem (QIP) is the problem of estimating an unknown quantum system from a set of measurements, whereas the classical counterpart is the inverse problem of estimating a distribution from a set of observations. In this paper, we present a neural-network-based method for QIPs, which has been widely explored for its classical counterpart. The proposed method utilizes the quantumness of the QIPs and takes advantage of the computational power of neural networks to achieve remarkable efficiency for the quantum state estimation. We test the method on the problem of maximum entropy estimation of an unknown state ρ from partial information both numerically and experimentally. Our method yields high fidelity, efficiency and robustness for both numerical experiments and quantum optical experiments.