Maximum Entropy Density Research Articles

Indeterminate Hamburger moment problem: Entropy convergence

The indeterminate Hamburger moment problem is considered, jointly with all its real axis supported probability density functions. As a consequence of entropy functional concavity, out of such densities there is one which has largest entropy and that plays a fundamental role: we call it fhmax. It is proved that the approximate Maximum Entropy (MaxEnt) densities constrained by an increasing number of moments converge in entropy to fhmax where the value of its entropy can be finite or −∞.

Indeterminate Stieltjes Moment Problem: Entropy Convergence

The aim of this paper is to consider the indeterminate Stieltjes moment problem together with all its probability density functions that have the positive real or the entire real axis as support. As a consequence of the concavity of the entropy function in both cases, there is one such density that has the largest entropy: we call it fhmax, the largest entropy density. We will prove that the Jaynes maximum entropy density (MaxEnt), constrained by an increasing number of integer moments, converges in entropy to the largest entropy density fhmax. Note that this kind of convergence implies convergence almost everywhere, with remarkable consequences in real applications in terms of the reliability of the results obtained by the MaxEnt approximation of the underlying unknown distribution, both for the determinate and the indeterminate case.

The Residual ISI for Which the Convolutional Noise Probability Density Function Associated with the Blind Adaptive Deconvolution Problem Turns Approximately Gaussian.

In a blind adaptive deconvolution problem, the convolutional noise observed at the output of the deconvolution process, in addition to the required source signal, is—according to the literature—assumed to be a Gaussian process when the deconvolution process (the blind adaptive equalizer) is deep in its convergence state. Namely, when the convolutional noise sequence or, equivalently, the residual inter-symbol interference (ISI) is considered small. Up to now, no closed-form approximated expression is given for the residual ISI, where the Gaussian model can be used to describe the convolutional noise probability density function (pdf). In this paper, we use the Maximum Entropy density technique, Lagrange’s Integral method, and quasi-moment truncation technique to obtain an approximated closed-form equation for the residual ISI where the Gaussian model can be used to approximately describe the convolutional noise pdf. We will show, based on this approximated closed-form equation for the residual ISI, that the Gaussian model can be used to approximately describe the convolutional noise pdf just before the equalizer has converged, even at a residual ISI level where the “eye diagram” is still very closed, namely, where the residual ISI can not be considered as small.

Option pricing with maximum entropy densities: The inclusion of higher‐order moments

AbstractEntropy pricing applies notions of information theory to derive the theoretical value of options. This paper employs the maximum entropy (ME) formulation of option pricing, given risk‐neutral moment constraints computed directly from the observed prices. First, higher‐order moments are used to generate option prices. Then a generalization of Shannon entropy, known as Renyi entropy, is studied to account for extreme events. This ME problem provides a class of heavy‐tailed distributions. Examples and Monte Carlo simulations are provided to examine the effects of moment constraints on option prices. The call option values are then constructed using daily Standard and Poor's 500 index options. The findings suggest that entropy pricing with higher‐order moment constraints provides higher forecasting accuracy.

Option Pricing with Maximum Entropy Densities: The Inclusion of Higher-Order Moments

Option Pricing with Maximum Entropy Densities: The Inclusion of Higher-Order Moments

Density Forecast of Financial Returns Using Decomposition and Maximum Entropy

Abstract We consider a multiplicative decomposition of the financial returns to improve the density forecasts of financial returns. The multiplicative decomposition is based on the identity that financial return is the product of its absolute value and its sign. Advantages of modeling the two components are discussed. To reduce the effect of the estimation error due to the multiplicative decomposition in estimation of the density forecast model, we impose a moment constraint that the conditional mean forecast is set to match with the sample mean. Imposing such a moment constraint operates a shrinkage and tilts the density forecast of the decomposition model to produce the improved maximum entropy density forecast. An empirical application to forecasting density of the daily stock returns demonstrates the benefits of using the decomposition and imposing the moment constraint to obtain the improved density forecast. We evaluate the density forecast by comparing the logarithmic score (LS), the quantile score (QS), and the continuous ranked probability score (CRPS). We contribute to the literature on the density forecast and the decomposition models by showing that the density forecast of the decomposition model can be improved by imposing a sensible constraint in the maximum entropy framework.

Modeling time‐varying higher‐order conditional moments: A survey

AbstractSince the introduction of the Autoregressive Conditional Heteroscedasticity (ARCH) model, the literature on modeling the time‐varying second‐order conditional moment has become increasingly popular in the last four decades. Its popularity is partly due to its success in capturing volatility in financial time series, which is useful for modeling and predicting risk for financial assets. A natural extension of this is to model time variation in higher‐order conditional moments, such as the third and fourth moments, which are related to skewness and kurtosis (tail risk). This leads to an emerging literature on time‐varying higher‐order conditional moments in the last two decades. This paper outlines recent developments in modeling time‐varying higher‐order conditional moments in the economics and finance literature.Using the Generalized Autoregressive Conditional Heteroscedasticity (GARCH) framework as a foundation, this paper provides an overview of the two most common approaches for modeling time‐varying higher‐order conditional moments: autoregressive conditional density (ARCD) and autoregressive conditional moment (ARCM). The discussion covers both the theoretical and empirical aspects of the literature. This includes the identification of the associated skewness–kurtosis domain by using the solutions to the classical moment problems, the structural and statistical properties of the models used to model the higher‐order conditional moments and the computational challenges in estimating these models. We also advocate the use of a maximum entropy density (MED) as an alternative method, which circumvents some of the issues prevalent in these common approaches.

Re-constructing the interbank links using machine learning techniques. An application to the Greek interbank market

• We forecast the probability of a pair of banks entering into an interbank market borrower - lender relationship considering their financial characteristics and their past observed behavior. • Our purpose is to depart from agnostic assumptions usually employed in interbank matrix allocation algorithms and take into account the financial features of the banks when assigning prior link probabilities. • Our main finding is that machine learning algorithms outperforms the benchmark logistic regression model in interbank link forecasting. We propose an innovative approach to model the probability of interlinkages in an interbank network with the use of Machine Learning techniques. More precisely we forecast the probability of a pair of banks entering into an interbank market borrower - lender relationship considering their financial characteristics and their past observed behavior. In this framework we examine a new method that employs machine learning in order to increase the accuracy of agnostic algorithms in reconstructing a financial network. The XGBOOST method is combined with both Maximum Entropy (MAXE) and Minimum Density (ANAN). The main contribution of this paper is that we enrich the information generally available for financial networks with variables that are available for the publication of banks financial statements (ensemble method). A set of agnostic models, i.e. models that the exposure allocation algorithm does not include prior information, are used as a benchmark to measure the additional benefit for applying machine learning in estimating prior network probabilities. By comparing the results between the agnostic algorithms and the ensemble method we see an increase in the accuracy and a decrease in the MAE of the financial networks on average. Our purpose is to depart from agnostic assumptions usually employed in interbank matrix allocation algorithms and take into account the financial features of the banks when assigning prior link probabilities. Our main finding is that machine learning algorithms outperforms the benchmark Logistic Regression model in interbank link forecasting and this is also reflected in the enhanced performance when overall network similarity measures are performed.

Classical Chaos Described by a Density Matrix

In this paper, a reference to the semiclassical model, in which quantum degrees of freedom interact with classical ones, is considered. The classical limit of a maximum-entropy density matrix that describes the temporal evolution of such a system is analyzed. Here, it is analytically shown that, in the classical limit, it is possible to reproduce classical results. An example is classical chaos. This is done by means a pure-state density matrix, a rather unexpected result. It is shown that this is possible only if the quantum part of the system is in a special class of states.

Entropy Scaling Law and the Quantum Marginal Problem

Quantum many-body states that frequently appear in physics often obey an entropy scaling law, meaning that an entanglement entropy of a subsystem can be expressed as a sum of terms that scale linearly with its volume and area, plus a correction term that is independent of its size. We conjecture that these states have an efficient dual description in terms of a set of marginal density matrices on bounded regions, obeying the same entropy scaling law locally. We prove a restricted version of this conjecture for translationally invariant systems in two spatial dimensions. Specifically, we prove that a translationally invariant marginal obeying three non-linear constraints -- all of which follow from the entropy scaling law straightforwardly -- must be consistent with some global state on an infinite lattice. Moreover, we derive a closed-form expression for the maximum entropy density compatible with those marginals, deriving a variational upper bound on the thermodynamic free energy. Our construction's main assumptions are satisfied exactly by solvable models of topological order and approximately by finite-temperature Gibbs states of certain quantum spin Hamiltonians.

Improved Approach for the Maximum Entropy Deconvolution Problem.

The probability density function (pdf) valid for the Gaussian case is often applied for describing the convolutional noise pdf in the blind adaptive deconvolution problem, although it is known that it can be applied only at the latter stages of the deconvolution process, where the convolutional noise pdf tends to be approximately Gaussian. Recently, the deconvolutional noise pdf was approximated with the Edgeworth Expansion and with the Maximum Entropy density function for the 16 Quadrature Amplitude Modulation (QAM) input but no equalization performance improvement was seen for the hard channel case with the equalization algorithm based on the Maximum Entropy density function approach for the convolutional noise pdf compared with the original Maximum Entropy algorithm, while for the Edgeworth Expansion approximation technique, additional predefined parameters were needed in the algorithm. In this paper, the Generalized Gaussian density (GGD) function and the Edgeworth Expansion are applied for approximating the convolutional noise pdf for the 16 QAM input case, with no need for additional predefined parameters in the obtained equalization method. Simulation results indicate that improved equalization performance is obtained from the convergence time point of view of approximately 15,000 symbols for the hard channel case with our new proposed equalization method based on the new model for the convolutional noise pdf compared to the original Maximum Entropy algorithm. By convergence time, we mean the number of symbols required to reach a residual inter-symbol-interference (ISI) for which reliable decisions can be made on the equalized output sequence.

Maximum Entropy Analysis of Consumption-based Capital Asset Pricing Model and Volatility

Abstract Based on the maximum entropy (ME) method, we introduce an information theoretic approach to estimating conditional moment functions with incorporating a theoretical constraint implied from the consumption-based capital asset pricing model (CCAPM). Using the ME conditional mean/variance functions obtained from the ME density, we analyze the relationship between asset returns and consumption growth under the theoretical constraint of the CCAPM. We evaluate the predictability of asset return using consumption growth through in-sample estimation and out-of-sample prediction in the ME mean regression function. We also examine the ME variance regression function for the asset return volatility as a function of the consumption growth. Our findings suggest that incorporating the CCAPM constraint can capture the nonlinear predictability of asset returns in mean especially in tails, and that the consumption growth has an effect on reducing stock return volatility, indicating the counter-cyclical variation of stock market volatility.

A Novel Technique for Achieving the Approximated ISI at the Receiver for a 16QAM Signal Sent via a FIR Channel Based Only on the Received Information and Statistical Techniques.

A single-input-multiple-output (SIMO) channel is obtained from the use of an array of antennas in the receiver where the same information is transmitted through different sub-channels, and all received sequences are distinctly distorted versions of the same message. The inter-symbol-interference (ISI) level from each sub-channel is presently unknown to the receiver. Thus, even when one or more sub-channels cause heavy ISI, all the information from all the sub-channels was still considered in the receiver. Obviously, if we know the approximated ISI of each sub-channel, we will use in the receiver only those sub-channels with the lowest ISI level to get improved system performance. In this paper, we present a systematic way for obtaining the approximated ISI from each sub-channel modelled as a finite-impulse-response (FIR) channel with real-valued coefficients for a 16QAM (16 quadrature amplitude modulation) source signal transmission. The approximated ISI is based on the maximum entropy density approximation technique, on the Edgeworth expansion up to order six, on the Laplace integral method and on the generalized Gaussian distribution (GGD). Although the approximated ISI was derived for the noiseless case, it was successfully tested for signal to noise ratio (SNR) down to 20 dB.

Lie Group Statistics and Lie Group Machine Learning Based on Souriau Lie Groups Thermodynamics & Koszul-Souriau-Fisher Metric: New Entropy Definition as Generalized Casimir Invariant Function in Coadjoint Representation.

In 1969, Jean-Marie Souriau introduced a “Lie Groups Thermodynamics” in Statistical Mechanics in the framework of Geometric Mechanics. This Souriau’s model considers the statistical mechanics of dynamic systems in their “space of evolution” associated to a homogeneous symplectic manifold by a Lagrange 2-form, and defines in case of non null cohomology (non equivariance of the coadjoint action on the moment map with appearance of an additional cocyle) a Gibbs density (of maximum entropy) that is covariant under the action of dynamic groups of physics (e.g., Galileo’s group in classical physics). Souriau Lie Group Thermodynamics was also addressed 30 years after Souriau by R.F. Streater in the framework of Quantum Physics by Information Geometry for some Lie algebras, but only in the case of null cohomology. Souriau method could then be applied on Lie groups to define a covariant maximum entropy density by Kirillov representation theory. We will illustrate this method for homogeneous Siegel domains and more especially for Poincaré unit disk by considering SU(1,1) group coadjoint orbit and by using its Souriau’s moment map. For this case, the coadjoint action on moment map is equivariant. For non-null cohomology, we give the case of Lie group SE(2). Finally, we will propose a new geometric definition of Entropy that could be built as a generalized Casimir invariant function in coadjoint representation, and Massieu characteristic function, dual of Entropy by Legendre transform, as a generalized Casimir invariant function in adjoint representation, where Souriau cocycle is a measure of the lack of equivariance of the moment mapping.

Charge density of 4-methyl-3-[(tetrahydro-2H-pyran-2-yl)oxy]thiazole-2(3H)-thione. A comprehensive multipole refinement, maximum entropy method and density functional theory study.

The structure of 4-methyl-3-[(tetrahydro-2H-pyran-2-yl)oxy]thiazole-2(3H)-thione (MTTOTHP) was investigated using X-ray diffraction and computational chemistry methods for determining properties of the nitrogen-oxygen bond, which is the least stable entity upon photochemical excitation. Experimentally measured structure factors have been used to determine and characterize charge density via the multipole model (MM) and the maximum entropy method (MEM). Theoretical investigation of the electron density and the electronic structure has been performed in the finite basis set density functional theory (DFT) framework. Quantum Theory of Atoms In Molecules (QTAIM), deformation densities and Laplacians maps have been used to compare theoretical and experimental results. MM experimental results and predictions from theory differ with respect to the sign and/or magnitude of the Laplacian at the N-O bond critical point (BCP), depending on the treatment of n values of the MM radial functions. Such Laplacian differences in the N-O bond case are discussed with respect to a lack of flexibility in the MM radial functions also reported by Rykounov et al. [Acta Cryst. (2011), B67, 425-436]. BCP Hessian eigenvalues show qualitatively matching results between MM and DFT. In addition, the theoretical analysis used domain-averaged fermi holes (DAFH), natural bond orbital (NBO) analysis and localized (LOC) orbitals to characterize the N-O bond as a single σ bond with marginal π character. Hirshfeld atom refinement (HAR) has been employed to compare to the MM refinement results and/or neutron dataset C-H bond lengths and to crystal or single molecule geometry optimizations, including considerations of anisotropy of H atoms. Our findings help to understand properties of molecules like MTTOTHP as progenitors of free oxygen radicals.

Stress Testing the Unknown - The Impact of Network Reconstruction on Systemic Risk Estimates

Banks connect through a complex network of interbank credit exposures that determine contagion and hence the systemic stability of the network. Yet, the bilateral linkages of these institutions and thus the true structure of the network are typically unobservable, with leading reconstruction methods, such as maximum entropy or minimum density, potentially underestimating the true contagion risk of the network. Based on publicly available data from large European institutions, this paper analyses different network reconstruction techniques and the contagion risk impact of different network structures in a stress testing context. My findings suggest that uncertainty about the true interbank network leads to substantially varying risk assessments, where a Bayesian reconstruction approach can be used to identify contagion risk boundaries. In addition, network structure metrics that explain the variation in risk assessments are identified.

Maximum entropy distribution with fractional moments for reliability analysis

An important task in reliability engineering is the approximation of the probability distribution function from limited sample data, both in terms of modeling of input random variables and response quantities from numerical simulations. Good estimation of the distribution tails is crucial, but challenging. In principle, the maximum entropy density with prescribed moments produces the least biased distribution, but it suffers from several drawbacks. Recently, maximum entropy with fractional moments has become popular as the fractions can be optimized, allowing a reduced number of moments to accurately characterize the distribution. However, the optimization problem is difficult to solve efficiently because the objective function is non-convex and non-continuous. This paper outlines a new method based on maximum entropy and fractional moments. Genetic algorithm is found to be a robust method for finding the global optimal solution for the fractional orders. The number of moments, which controls the bias-variance trade-off, is also optimized using the Akaike information criterion to avoid an overcomplex model. A new location variable is introduced, thus allowing negative data to be modeled. The proposed method is tested on several examples, including theoretical distributions, a truss system, and wind turbine loads. To investigate the bias and variance errors of the exceedance probabilities predicted by the proposed method, for each application, many datasets collected from the same population are analyzed. The speedup over Monte Carlo simulation is calculated from the variance error. The proposed method is shown to have very low bias errors and high speedups for the examples examined.

Some physical features of the Burr-type-XII distribution.

Aspects related to driven nonequilibrium systems considered in Beck-Cohen superstatistics (BCS) formalism and, in addition, the unbiased maximum entropy density estimation, show the pertinence of considering the three-parameter Burr-type-XII distribution as a suitable way to describe stationary states of complex and nonequilibrium systems. The above is shown following a variant to the procedure presented in the formulation of BCS, which consists of the incorporation of an expanded Boltzmann factor through the Mittag-Leffler function. On the other hand, maximization of Shannon-Boltzmann-Gibbs entropy and other generalized forms of entropy show that the Burr-type-XII distribution may emerge as the least informative distribution for the mentioned systems.

A New Efficient Expression for the Conditional Expectation of the Blind Adaptive Deconvolution Problem Valid for the Entire Range ofSignal-to-Noise Ratio.

In the literature, we can find several blind adaptive deconvolution algorithms based on closed-form approximated expressions for the conditional expectation (the expectation of the source input given the equalized or deconvolutional output), involving the maximum entropy density approximation technique. The main drawback of these algorithms is the heavy computational burden involved in calculating the expression for the conditional expectation. In addition, none of these techniques are applicable for signal-to-noise ratios lower than 7 dB. In this paper, I propose a new closed-form approximated expression for the conditional expectation based on a previously obtained expression where the equalized output probability density function is calculated via the approximated input probability density function which itself is approximated with the maximum entropy density approximation technique. This newly proposed expression has a reduced computational burden compared with the previously obtained expressions for the conditional expectation based on the maximum entropy approximation technique. The simulation results indicate that the newly proposed algorithm with the newly proposed Lagrange multipliers is suitable for signal-to-noise ratio values down to 0 dB and has an improved equalization performance from the residual inter-symbol-interference point of view compared to the previously obtained algorithms based on the conditional expectation obtained via the maximum entropy technique.

Methods of Moment and Maximum Entropy for Solving Nonlinear Expectation

In this paper, we consider a special nonlinear expectation problem on the special parameter space and give a necessary and sufficient condition for the existence of the solution. Meanwhile, we generalize the necessary and sufficient condition to the two-dimensional moment problem. Moreover, we use the maximum entropy method to carry out a kind of concrete solution and analyze the convergence for the maximum entropy solution. Numerical experiments are presented to compute the maximum entropy density functions.